Cargando…

The validity and reliability of a digital Ruff Figural Fluency Test (RFFT)

BACKGROUND: The Ruff Figural Fluency Test (RFFT) is a valid but time-consuming and labour-intensive cognitive paper-and-pencil test. A digital RFFT was developed that can be conducted independently using an iPad and Apple Pencil and RFFT scores are computed automatically. We investigated the validit...

Descripción completa

Detalles Bibliográficos
Autores principales: Vrijsen, J., van Erpecum, C. L., de Rooij, S. E., Niebuur, J., Smidt, N.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8080381/
https://www.ncbi.nlm.nih.gov/pubmed/33910642
http://dx.doi.org/10.1186/s40359-021-00566-x
_version_ 1783685413835112448
author Vrijsen, J.
van Erpecum, C. L.
de Rooij, S. E.
Niebuur, J.
Smidt, N.
author_facet Vrijsen, J.
van Erpecum, C. L.
de Rooij, S. E.
Niebuur, J.
Smidt, N.
author_sort Vrijsen, J.
collection PubMed
description BACKGROUND: The Ruff Figural Fluency Test (RFFT) is a valid but time-consuming and labour-intensive cognitive paper-and-pencil test. A digital RFFT was developed that can be conducted independently using an iPad and Apple Pencil and RFFT scores are computed automatically. We investigated the validity and reliability of this digital RFFT. METHODS: We randomly allocated participants to the digital or paper-and-pencil RFFT. After the first test, the other test was performed immediately (cross-over). Participants were invited for a second digital RFFT 1 week later. For the digital RFFT, an (automatic) algorithm and two independent raters (criterion standard) assessed the number of unique designs (UD) and perseverative errors (PE). These raters also assessed the paper-and-pencil RFFT. We used Intraclass correlation coefficients (ICC), sensitivity, specificity, %-agreement, Kappa, and Bland–Altman plots. RESULTS: We included 94 participants (mean (SD) age 39.9 (14.8), 73.4% follow-up). Mean (SD) UD and median (IQR) PE of the digital RFFT were 84.2 (26.0) and 4 (2–7.3), respectively. Agreement between manual and automatic scoring of the digital RFFT was high for UD (ICC = 0.99, 95% CI 0.98, 0.99, sensitivity = 0.98; specificity = 0.96) and PE (ICC = 0.99, 95% CI 0.98, 0.99; sensitivity = 0.90, specificity = 1.00), indicating excellent criterion validity. Small but significant differences in UD were found between the automatic and manual scoring (mean difference: − 1.12, 95% CI − 1.92, − 0.33). Digital and paper-and-pencil RFFT had moderate agreement for UD (ICC = 0.73, 95% CI 0.34, 0.87) and poor agreement for PE (ICC = 0.47, 95% CI 0.30, 0.62). Participants had fewer UD on the digital than paper-and-pencil RFFT (mean difference: − 7.09, 95% CI − 11.80, − 2.38). The number of UD on the digital RFFT was associated with higher education (Spearman’s r = 0.43, p < 0.001), and younger age (Pearson’s r = − 0.36, p < 0.001), showing its ability to discriminate between different age categories and levels of education. Test–retest reliability was moderate (ICC = 0.74, 95% CI 0.61, 0.83). CONCLUSIONS: The automatic scoring of the digital RFFT has good criterion and convergent validity. There was low agreement between the digital RFFT and paper-and-pencil RFFT and moderate test–retest reliability, which can be explained by learning effects. The digital RFFT is a valid and reliable instrument to measure executive cognitive function among the general population and is a feasible alternative to the paper-and-pencil RFFT in large-scale studies. However, its scores cannot be used interchangeably with the paper-and-pencil RFFT scores. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s40359-021-00566-x.
format Online
Article
Text
id pubmed-8080381
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-80803812021-04-29 The validity and reliability of a digital Ruff Figural Fluency Test (RFFT) Vrijsen, J. van Erpecum, C. L. de Rooij, S. E. Niebuur, J. Smidt, N. BMC Psychol Research BACKGROUND: The Ruff Figural Fluency Test (RFFT) is a valid but time-consuming and labour-intensive cognitive paper-and-pencil test. A digital RFFT was developed that can be conducted independently using an iPad and Apple Pencil and RFFT scores are computed automatically. We investigated the validity and reliability of this digital RFFT. METHODS: We randomly allocated participants to the digital or paper-and-pencil RFFT. After the first test, the other test was performed immediately (cross-over). Participants were invited for a second digital RFFT 1 week later. For the digital RFFT, an (automatic) algorithm and two independent raters (criterion standard) assessed the number of unique designs (UD) and perseverative errors (PE). These raters also assessed the paper-and-pencil RFFT. We used Intraclass correlation coefficients (ICC), sensitivity, specificity, %-agreement, Kappa, and Bland–Altman plots. RESULTS: We included 94 participants (mean (SD) age 39.9 (14.8), 73.4% follow-up). Mean (SD) UD and median (IQR) PE of the digital RFFT were 84.2 (26.0) and 4 (2–7.3), respectively. Agreement between manual and automatic scoring of the digital RFFT was high for UD (ICC = 0.99, 95% CI 0.98, 0.99, sensitivity = 0.98; specificity = 0.96) and PE (ICC = 0.99, 95% CI 0.98, 0.99; sensitivity = 0.90, specificity = 1.00), indicating excellent criterion validity. Small but significant differences in UD were found between the automatic and manual scoring (mean difference: − 1.12, 95% CI − 1.92, − 0.33). Digital and paper-and-pencil RFFT had moderate agreement for UD (ICC = 0.73, 95% CI 0.34, 0.87) and poor agreement for PE (ICC = 0.47, 95% CI 0.30, 0.62). Participants had fewer UD on the digital than paper-and-pencil RFFT (mean difference: − 7.09, 95% CI − 11.80, − 2.38). The number of UD on the digital RFFT was associated with higher education (Spearman’s r = 0.43, p < 0.001), and younger age (Pearson’s r = − 0.36, p < 0.001), showing its ability to discriminate between different age categories and levels of education. Test–retest reliability was moderate (ICC = 0.74, 95% CI 0.61, 0.83). CONCLUSIONS: The automatic scoring of the digital RFFT has good criterion and convergent validity. There was low agreement between the digital RFFT and paper-and-pencil RFFT and moderate test–retest reliability, which can be explained by learning effects. The digital RFFT is a valid and reliable instrument to measure executive cognitive function among the general population and is a feasible alternative to the paper-and-pencil RFFT in large-scale studies. However, its scores cannot be used interchangeably with the paper-and-pencil RFFT scores. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s40359-021-00566-x. BioMed Central 2021-04-28 /pmc/articles/PMC8080381/ /pubmed/33910642 http://dx.doi.org/10.1186/s40359-021-00566-x Text en © The Author(s) 2021 https://creativecommons.org/licenses/by/4.0/Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/ (https://creativecommons.org/publicdomain/zero/1.0/) ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
spellingShingle Research
Vrijsen, J.
van Erpecum, C. L.
de Rooij, S. E.
Niebuur, J.
Smidt, N.
The validity and reliability of a digital Ruff Figural Fluency Test (RFFT)
title The validity and reliability of a digital Ruff Figural Fluency Test (RFFT)
title_full The validity and reliability of a digital Ruff Figural Fluency Test (RFFT)
title_fullStr The validity and reliability of a digital Ruff Figural Fluency Test (RFFT)
title_full_unstemmed The validity and reliability of a digital Ruff Figural Fluency Test (RFFT)
title_short The validity and reliability of a digital Ruff Figural Fluency Test (RFFT)
title_sort validity and reliability of a digital ruff figural fluency test (rfft)
topic Research
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8080381/
https://www.ncbi.nlm.nih.gov/pubmed/33910642
http://dx.doi.org/10.1186/s40359-021-00566-x
work_keys_str_mv AT vrijsenj thevalidityandreliabilityofadigitalrufffiguralfluencytestrfft
AT vanerpecumcl thevalidityandreliabilityofadigitalrufffiguralfluencytestrfft
AT derooijse thevalidityandreliabilityofadigitalrufffiguralfluencytestrfft
AT niebuurj thevalidityandreliabilityofadigitalrufffiguralfluencytestrfft
AT smidtn thevalidityandreliabilityofadigitalrufffiguralfluencytestrfft
AT vrijsenj validityandreliabilityofadigitalrufffiguralfluencytestrfft
AT vanerpecumcl validityandreliabilityofadigitalrufffiguralfluencytestrfft
AT derooijse validityandreliabilityofadigitalrufffiguralfluencytestrfft
AT niebuurj validityandreliabilityofadigitalrufffiguralfluencytestrfft
AT smidtn validityandreliabilityofadigitalrufffiguralfluencytestrfft