Cargando…

Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions

BACKGROUND: The perspective of users should be taken into account in the evaluation of Web-based health interventions. Assessing the users’ satisfaction with the intervention they receive could enhance the evidence for the intervention effects. Thus, there is a need for valid and reliable measures t...

Descripción completa

Detalles Bibliográficos
Autores principales: Boß, Leif, Lehr, Dirk, Reis, Dorota, Vis, Christiaan, Riper, Heleen, Berking, Matthias, Ebert, David Daniel
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2016
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5023944/
https://www.ncbi.nlm.nih.gov/pubmed/27582341
http://dx.doi.org/10.2196/jmir.5952
_version_ 1782453716432977920
author Boß, Leif
Lehr, Dirk
Reis, Dorota
Vis, Christiaan
Riper, Heleen
Berking, Matthias
Ebert, David Daniel
author_facet Boß, Leif
Lehr, Dirk
Reis, Dorota
Vis, Christiaan
Riper, Heleen
Berking, Matthias
Ebert, David Daniel
author_sort Boß, Leif
collection PubMed
description BACKGROUND: The perspective of users should be taken into account in the evaluation of Web-based health interventions. Assessing the users’ satisfaction with the intervention they receive could enhance the evidence for the intervention effects. Thus, there is a need for valid and reliable measures to assess satisfaction with Web-based health interventions. OBJECTIVE: The objective of this study was to analyze the reliability, factorial structure, and construct validity of the Client Satisfaction Questionnaire adapted to Internet-based interventions (CSQ-I). METHODS: The psychometric quality of the CSQ-I was analyzed in user samples from 2 separate randomized controlled trials evaluating Web-based health interventions, one from a depression prevention intervention (sample 1, N=174) and the other from a stress management intervention (sample 2, N=111). At first, the underlying measurement model of the CSQ-I was analyzed to determine the internal consistency. The factorial structure of the scale and the measurement invariance across groups were tested by multigroup confirmatory factor analyses. Additionally, the construct validity of the scale was examined by comparing satisfaction scores with the primary clinical outcome. RESULTS: Multigroup confirmatory analyses on the scale yielded a one-factorial structure with a good fit (root-mean-square error of approximation =.09, comparative fit index =.96, standardized root-mean-square residual =.05) that showed partial strong invariance across the 2 samples. The scale showed very good reliability, indicated by McDonald omegas of .95 in sample 1 and .93 in sample 2. Significant correlations with change in depressive symptoms (r=−.35, P<.001) and perceived stress (r=−.48, P<.001) demonstrated the construct validity of the scale. CONCLUSIONS: The proven internal consistency, factorial structure, and construct validity of the CSQ-I indicate a good overall psychometric quality of the measure to assess the user’s general satisfaction with Web-based interventions for depression and stress management. Multigroup analyses indicate its robustness across different samples. Thus, the CSQ-I seems to be a suitable measure to consider the user’s perspective in the overall evaluation of Web-based health interventions.
format Online
Article
Text
id pubmed-5023944
institution National Center for Biotechnology Information
language English
publishDate 2016
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-50239442016-10-03 Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions Boß, Leif Lehr, Dirk Reis, Dorota Vis, Christiaan Riper, Heleen Berking, Matthias Ebert, David Daniel J Med Internet Res Original Paper BACKGROUND: The perspective of users should be taken into account in the evaluation of Web-based health interventions. Assessing the users’ satisfaction with the intervention they receive could enhance the evidence for the intervention effects. Thus, there is a need for valid and reliable measures to assess satisfaction with Web-based health interventions. OBJECTIVE: The objective of this study was to analyze the reliability, factorial structure, and construct validity of the Client Satisfaction Questionnaire adapted to Internet-based interventions (CSQ-I). METHODS: The psychometric quality of the CSQ-I was analyzed in user samples from 2 separate randomized controlled trials evaluating Web-based health interventions, one from a depression prevention intervention (sample 1, N=174) and the other from a stress management intervention (sample 2, N=111). At first, the underlying measurement model of the CSQ-I was analyzed to determine the internal consistency. The factorial structure of the scale and the measurement invariance across groups were tested by multigroup confirmatory factor analyses. Additionally, the construct validity of the scale was examined by comparing satisfaction scores with the primary clinical outcome. RESULTS: Multigroup confirmatory analyses on the scale yielded a one-factorial structure with a good fit (root-mean-square error of approximation =.09, comparative fit index =.96, standardized root-mean-square residual =.05) that showed partial strong invariance across the 2 samples. The scale showed very good reliability, indicated by McDonald omegas of .95 in sample 1 and .93 in sample 2. Significant correlations with change in depressive symptoms (r=−.35, P<.001) and perceived stress (r=−.48, P<.001) demonstrated the construct validity of the scale. CONCLUSIONS: The proven internal consistency, factorial structure, and construct validity of the CSQ-I indicate a good overall psychometric quality of the measure to assess the user’s general satisfaction with Web-based interventions for depression and stress management. Multigroup analyses indicate its robustness across different samples. Thus, the CSQ-I seems to be a suitable measure to consider the user’s perspective in the overall evaluation of Web-based health interventions. JMIR Publications 2016-08-31 /pmc/articles/PMC5023944/ /pubmed/27582341 http://dx.doi.org/10.2196/jmir.5952 Text en ©Leif Boß, Dirk Lehr, Dorota Reis, Christiaan Vis, Heleen Riper, Matthias Berking, David Daniel Ebert. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 31.08.2016. https://creativecommons.org/licenses/by/2.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/ (https://creativecommons.org/licenses/by/2.0/) ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
spellingShingle Original Paper
Boß, Leif
Lehr, Dirk
Reis, Dorota
Vis, Christiaan
Riper, Heleen
Berking, Matthias
Ebert, David Daniel
Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions
title Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions
title_full Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions
title_fullStr Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions
title_full_unstemmed Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions
title_short Reliability and Validity of Assessing User Satisfaction With Web-Based Health Interventions
title_sort reliability and validity of assessing user satisfaction with web-based health interventions
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5023944/
https://www.ncbi.nlm.nih.gov/pubmed/27582341
http://dx.doi.org/10.2196/jmir.5952
work_keys_str_mv AT boßleif reliabilityandvalidityofassessingusersatisfactionwithwebbasedhealthinterventions
AT lehrdirk reliabilityandvalidityofassessingusersatisfactionwithwebbasedhealthinterventions
AT reisdorota reliabilityandvalidityofassessingusersatisfactionwithwebbasedhealthinterventions
AT vischristiaan reliabilityandvalidityofassessingusersatisfactionwithwebbasedhealthinterventions
AT riperheleen reliabilityandvalidityofassessingusersatisfactionwithwebbasedhealthinterventions
AT berkingmatthias reliabilityandvalidityofassessingusersatisfactionwithwebbasedhealthinterventions
AT ebertdaviddaniel reliabilityandvalidityofassessingusersatisfactionwithwebbasedhealthinterventions