Cargando…

Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis

BACKGROUND: Significant resources are being invested into eHealth technology to improve health care. Few resources have focused on evaluating the impact of use on patient outcomes A standardized set of metrics used across health systems and research will enable aggregation of data to inform improved...

Descripción completa

Detalles Bibliográficos
Autores principales: Wakefield, Bonnie J, Turvey, Carolyn L, Nazi, Kim M, Holman, John E, Hogan, Timothy P, Shimada, Stephanie L, Kennedy, Diana R
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2017
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5656774/
https://www.ncbi.nlm.nih.gov/pubmed/29021128
http://dx.doi.org/10.2196/jmir.7638
_version_ 1783273759864520704
author Wakefield, Bonnie J
Turvey, Carolyn L
Nazi, Kim M
Holman, John E
Hogan, Timothy P
Shimada, Stephanie L
Kennedy, Diana R
author_facet Wakefield, Bonnie J
Turvey, Carolyn L
Nazi, Kim M
Holman, John E
Hogan, Timothy P
Shimada, Stephanie L
Kennedy, Diana R
author_sort Wakefield, Bonnie J
collection PubMed
description BACKGROUND: Significant resources are being invested into eHealth technology to improve health care. Few resources have focused on evaluating the impact of use on patient outcomes A standardized set of metrics used across health systems and research will enable aggregation of data to inform improved implementation, clinical practice, and ultimately health outcomes associated with use of patient-facing eHealth technologies. OBJECTIVE: The objective of this project was to conduct a systematic review to (1) identify existing instruments for eHealth research and implementation evaluation from the patient’s point of view, (2) characterize measurement components, and (3) assess psychometrics. METHODS: Concepts from existing models and published studies of technology use and adoption were identified and used to inform a search strategy. Search terms were broadly categorized as platforms (eg, email), measurement (eg, survey), function/information use (eg, self-management), health care occupations (eg, nurse), and eHealth/telemedicine (eg, mHealth). A computerized database search was conducted through June 2014. Included articles (1) described development of an instrument, or (2) used an instrument that could be traced back to its original publication, or (3) modified an instrument, and (4) with full text in English language, and (5) focused on the patient perspective on technology, including patient preferences and satisfaction, engagement with technology, usability, competency and fluency with technology, computer literacy, and trust in and acceptance of technology. The review was limited to instruments that reported at least one psychometric property. Excluded were investigator-developed measures, disease-specific assessments delivered via technology or telephone (eg, a cancer-coping measure delivered via computer survey), and measures focused primarily on clinician use (eg, the electronic health record). RESULTS: The search strategy yielded 47,320 articles. Following elimination of duplicates and non-English language publications (n=14,550) and books (n=27), another 31,647 articles were excluded through review of titles. Following a review of the abstracts of the remaining 1096 articles, 68 were retained for full-text review. Of these, 16 described an instrument and six used an instrument; one instrument was drawn from the GEM database, resulting in 23 articles for inclusion. None included a complete psychometric evaluation. The most frequently assessed property was internal consistency (21/23, 91%). Testing for aspects of validity ranged from 48% (11/23) to 78% (18/23). Approximately half (13/23, 57%) reported how to score the instrument. Only six (26%) assessed the readability of the instrument for end users, although all the measures rely on self-report. CONCLUSIONS: Although most measures identified in this review were published after the year 2000, rapidly changing technology makes instrument development challenging. Platform-agnostic measures need to be developed that focus on concepts important for use of any type of eHealth innovation. At present, there are important gaps in the availability of psychometrically sound measures to evaluate eHealth technologies.
format Online
Article
Text
id pubmed-5656774
institution National Center for Biotechnology Information
language English
publishDate 2017
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-56567742017-11-03 Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis Wakefield, Bonnie J Turvey, Carolyn L Nazi, Kim M Holman, John E Hogan, Timothy P Shimada, Stephanie L Kennedy, Diana R J Med Internet Res Original Paper BACKGROUND: Significant resources are being invested into eHealth technology to improve health care. Few resources have focused on evaluating the impact of use on patient outcomes A standardized set of metrics used across health systems and research will enable aggregation of data to inform improved implementation, clinical practice, and ultimately health outcomes associated with use of patient-facing eHealth technologies. OBJECTIVE: The objective of this project was to conduct a systematic review to (1) identify existing instruments for eHealth research and implementation evaluation from the patient’s point of view, (2) characterize measurement components, and (3) assess psychometrics. METHODS: Concepts from existing models and published studies of technology use and adoption were identified and used to inform a search strategy. Search terms were broadly categorized as platforms (eg, email), measurement (eg, survey), function/information use (eg, self-management), health care occupations (eg, nurse), and eHealth/telemedicine (eg, mHealth). A computerized database search was conducted through June 2014. Included articles (1) described development of an instrument, or (2) used an instrument that could be traced back to its original publication, or (3) modified an instrument, and (4) with full text in English language, and (5) focused on the patient perspective on technology, including patient preferences and satisfaction, engagement with technology, usability, competency and fluency with technology, computer literacy, and trust in and acceptance of technology. The review was limited to instruments that reported at least one psychometric property. Excluded were investigator-developed measures, disease-specific assessments delivered via technology or telephone (eg, a cancer-coping measure delivered via computer survey), and measures focused primarily on clinician use (eg, the electronic health record). RESULTS: The search strategy yielded 47,320 articles. Following elimination of duplicates and non-English language publications (n=14,550) and books (n=27), another 31,647 articles were excluded through review of titles. Following a review of the abstracts of the remaining 1096 articles, 68 were retained for full-text review. Of these, 16 described an instrument and six used an instrument; one instrument was drawn from the GEM database, resulting in 23 articles for inclusion. None included a complete psychometric evaluation. The most frequently assessed property was internal consistency (21/23, 91%). Testing for aspects of validity ranged from 48% (11/23) to 78% (18/23). Approximately half (13/23, 57%) reported how to score the instrument. Only six (26%) assessed the readability of the instrument for end users, although all the measures rely on self-report. CONCLUSIONS: Although most measures identified in this review were published after the year 2000, rapidly changing technology makes instrument development challenging. Platform-agnostic measures need to be developed that focus on concepts important for use of any type of eHealth innovation. At present, there are important gaps in the availability of psychometrically sound measures to evaluate eHealth technologies. JMIR Publications 2017-10-11 /pmc/articles/PMC5656774/ /pubmed/29021128 http://dx.doi.org/10.2196/jmir.7638 Text en ©Bonnie J Wakefield, Carolyn L Turvey, Kim M Nazi, John E Holman, Timothy P Hogan, Stephanie L Shimada, Diana R Kennedy. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 11.10.2017. https://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
spellingShingle Original Paper
Wakefield, Bonnie J
Turvey, Carolyn L
Nazi, Kim M
Holman, John E
Hogan, Timothy P
Shimada, Stephanie L
Kennedy, Diana R
Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis
title Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis
title_full Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis
title_fullStr Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis
title_full_unstemmed Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis
title_short Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis
title_sort psychometric properties of patient-facing ehealth evaluation measures: systematic review and analysis
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5656774/
https://www.ncbi.nlm.nih.gov/pubmed/29021128
http://dx.doi.org/10.2196/jmir.7638
work_keys_str_mv AT wakefieldbonniej psychometricpropertiesofpatientfacingehealthevaluationmeasuressystematicreviewandanalysis
AT turveycarolynl psychometricpropertiesofpatientfacingehealthevaluationmeasuressystematicreviewandanalysis
AT nazikimm psychometricpropertiesofpatientfacingehealthevaluationmeasuressystematicreviewandanalysis
AT holmanjohne psychometricpropertiesofpatientfacingehealthevaluationmeasuressystematicreviewandanalysis
AT hogantimothyp psychometricpropertiesofpatientfacingehealthevaluationmeasuressystematicreviewandanalysis
AT shimadastephaniel psychometricpropertiesofpatientfacingehealthevaluationmeasuressystematicreviewandanalysis
AT kennedydianar psychometricpropertiesofpatientfacingehealthevaluationmeasuressystematicreviewandanalysis