Cargando…
The validity and reliability of a digital Ruff Figural Fluency Test (RFFT)
BACKGROUND: The Ruff Figural Fluency Test (RFFT) is a valid but time-consuming and labour-intensive cognitive paper-and-pencil test. A digital RFFT was developed that can be conducted independently using an iPad and Apple Pencil and RFFT scores are computed automatically. We investigated the validit...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8080381/ https://www.ncbi.nlm.nih.gov/pubmed/33910642 http://dx.doi.org/10.1186/s40359-021-00566-x |
_version_ | 1783685413835112448 |
---|---|
author | Vrijsen, J. van Erpecum, C. L. de Rooij, S. E. Niebuur, J. Smidt, N. |
author_facet | Vrijsen, J. van Erpecum, C. L. de Rooij, S. E. Niebuur, J. Smidt, N. |
author_sort | Vrijsen, J. |
collection | PubMed |
description | BACKGROUND: The Ruff Figural Fluency Test (RFFT) is a valid but time-consuming and labour-intensive cognitive paper-and-pencil test. A digital RFFT was developed that can be conducted independently using an iPad and Apple Pencil and RFFT scores are computed automatically. We investigated the validity and reliability of this digital RFFT. METHODS: We randomly allocated participants to the digital or paper-and-pencil RFFT. After the first test, the other test was performed immediately (cross-over). Participants were invited for a second digital RFFT 1 week later. For the digital RFFT, an (automatic) algorithm and two independent raters (criterion standard) assessed the number of unique designs (UD) and perseverative errors (PE). These raters also assessed the paper-and-pencil RFFT. We used Intraclass correlation coefficients (ICC), sensitivity, specificity, %-agreement, Kappa, and Bland–Altman plots. RESULTS: We included 94 participants (mean (SD) age 39.9 (14.8), 73.4% follow-up). Mean (SD) UD and median (IQR) PE of the digital RFFT were 84.2 (26.0) and 4 (2–7.3), respectively. Agreement between manual and automatic scoring of the digital RFFT was high for UD (ICC = 0.99, 95% CI 0.98, 0.99, sensitivity = 0.98; specificity = 0.96) and PE (ICC = 0.99, 95% CI 0.98, 0.99; sensitivity = 0.90, specificity = 1.00), indicating excellent criterion validity. Small but significant differences in UD were found between the automatic and manual scoring (mean difference: − 1.12, 95% CI − 1.92, − 0.33). Digital and paper-and-pencil RFFT had moderate agreement for UD (ICC = 0.73, 95% CI 0.34, 0.87) and poor agreement for PE (ICC = 0.47, 95% CI 0.30, 0.62). Participants had fewer UD on the digital than paper-and-pencil RFFT (mean difference: − 7.09, 95% CI − 11.80, − 2.38). The number of UD on the digital RFFT was associated with higher education (Spearman’s r = 0.43, p < 0.001), and younger age (Pearson’s r = − 0.36, p < 0.001), showing its ability to discriminate between different age categories and levels of education. Test–retest reliability was moderate (ICC = 0.74, 95% CI 0.61, 0.83). CONCLUSIONS: The automatic scoring of the digital RFFT has good criterion and convergent validity. There was low agreement between the digital RFFT and paper-and-pencil RFFT and moderate test–retest reliability, which can be explained by learning effects. The digital RFFT is a valid and reliable instrument to measure executive cognitive function among the general population and is a feasible alternative to the paper-and-pencil RFFT in large-scale studies. However, its scores cannot be used interchangeably with the paper-and-pencil RFFT scores. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s40359-021-00566-x. |
format | Online Article Text |
id | pubmed-8080381 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-80803812021-04-29 The validity and reliability of a digital Ruff Figural Fluency Test (RFFT) Vrijsen, J. van Erpecum, C. L. de Rooij, S. E. Niebuur, J. Smidt, N. BMC Psychol Research BACKGROUND: The Ruff Figural Fluency Test (RFFT) is a valid but time-consuming and labour-intensive cognitive paper-and-pencil test. A digital RFFT was developed that can be conducted independently using an iPad and Apple Pencil and RFFT scores are computed automatically. We investigated the validity and reliability of this digital RFFT. METHODS: We randomly allocated participants to the digital or paper-and-pencil RFFT. After the first test, the other test was performed immediately (cross-over). Participants were invited for a second digital RFFT 1 week later. For the digital RFFT, an (automatic) algorithm and two independent raters (criterion standard) assessed the number of unique designs (UD) and perseverative errors (PE). These raters also assessed the paper-and-pencil RFFT. We used Intraclass correlation coefficients (ICC), sensitivity, specificity, %-agreement, Kappa, and Bland–Altman plots. RESULTS: We included 94 participants (mean (SD) age 39.9 (14.8), 73.4% follow-up). Mean (SD) UD and median (IQR) PE of the digital RFFT were 84.2 (26.0) and 4 (2–7.3), respectively. Agreement between manual and automatic scoring of the digital RFFT was high for UD (ICC = 0.99, 95% CI 0.98, 0.99, sensitivity = 0.98; specificity = 0.96) and PE (ICC = 0.99, 95% CI 0.98, 0.99; sensitivity = 0.90, specificity = 1.00), indicating excellent criterion validity. Small but significant differences in UD were found between the automatic and manual scoring (mean difference: − 1.12, 95% CI − 1.92, − 0.33). Digital and paper-and-pencil RFFT had moderate agreement for UD (ICC = 0.73, 95% CI 0.34, 0.87) and poor agreement for PE (ICC = 0.47, 95% CI 0.30, 0.62). Participants had fewer UD on the digital than paper-and-pencil RFFT (mean difference: − 7.09, 95% CI − 11.80, − 2.38). The number of UD on the digital RFFT was associated with higher education (Spearman’s r = 0.43, p < 0.001), and younger age (Pearson’s r = − 0.36, p < 0.001), showing its ability to discriminate between different age categories and levels of education. Test–retest reliability was moderate (ICC = 0.74, 95% CI 0.61, 0.83). CONCLUSIONS: The automatic scoring of the digital RFFT has good criterion and convergent validity. There was low agreement between the digital RFFT and paper-and-pencil RFFT and moderate test–retest reliability, which can be explained by learning effects. The digital RFFT is a valid and reliable instrument to measure executive cognitive function among the general population and is a feasible alternative to the paper-and-pencil RFFT in large-scale studies. However, its scores cannot be used interchangeably with the paper-and-pencil RFFT scores. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s40359-021-00566-x. BioMed Central 2021-04-28 /pmc/articles/PMC8080381/ /pubmed/33910642 http://dx.doi.org/10.1186/s40359-021-00566-x Text en © The Author(s) 2021 https://creativecommons.org/licenses/by/4.0/Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/ (https://creativecommons.org/publicdomain/zero/1.0/) ) applies to the data made available in this article, unless otherwise stated in a credit line to the data. |
spellingShingle | Research Vrijsen, J. van Erpecum, C. L. de Rooij, S. E. Niebuur, J. Smidt, N. The validity and reliability of a digital Ruff Figural Fluency Test (RFFT) |
title | The validity and reliability of a digital Ruff Figural Fluency Test (RFFT) |
title_full | The validity and reliability of a digital Ruff Figural Fluency Test (RFFT) |
title_fullStr | The validity and reliability of a digital Ruff Figural Fluency Test (RFFT) |
title_full_unstemmed | The validity and reliability of a digital Ruff Figural Fluency Test (RFFT) |
title_short | The validity and reliability of a digital Ruff Figural Fluency Test (RFFT) |
title_sort | validity and reliability of a digital ruff figural fluency test (rfft) |
topic | Research |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8080381/ https://www.ncbi.nlm.nih.gov/pubmed/33910642 http://dx.doi.org/10.1186/s40359-021-00566-x |
work_keys_str_mv | AT vrijsenj thevalidityandreliabilityofadigitalrufffiguralfluencytestrfft AT vanerpecumcl thevalidityandreliabilityofadigitalrufffiguralfluencytestrfft AT derooijse thevalidityandreliabilityofadigitalrufffiguralfluencytestrfft AT niebuurj thevalidityandreliabilityofadigitalrufffiguralfluencytestrfft AT smidtn thevalidityandreliabilityofadigitalrufffiguralfluencytestrfft AT vrijsenj validityandreliabilityofadigitalrufffiguralfluencytestrfft AT vanerpecumcl validityandreliabilityofadigitalrufffiguralfluencytestrfft AT derooijse validityandreliabilityofadigitalrufffiguralfluencytestrfft AT niebuurj validityandreliabilityofadigitalrufffiguralfluencytestrfft AT smidtn validityandreliabilityofadigitalrufffiguralfluencytestrfft |