Cargando…
Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology
BACKGROUND: Developers, designers, and researchers use rapid prototyping methods to project the adoption and acceptability of their health intervention technology (HIT) before the technology becomes mature enough to be deployed. Although these methods are useful for gathering feedback that advances...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
JMIR Publications
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8764610/ https://www.ncbi.nlm.nih.gov/pubmed/34982038 http://dx.doi.org/10.2196/30474 |
_version_ | 1784634202982973440 |
---|---|
author | Mariakakis, Alex Karkar, Ravi Patel, Shwetak N Kientz, Julie A Fogarty, James Munson, Sean A |
author_facet | Mariakakis, Alex Karkar, Ravi Patel, Shwetak N Kientz, Julie A Fogarty, James Munson, Sean A |
author_sort | Mariakakis, Alex |
collection | PubMed |
description | BACKGROUND: Developers, designers, and researchers use rapid prototyping methods to project the adoption and acceptability of their health intervention technology (HIT) before the technology becomes mature enough to be deployed. Although these methods are useful for gathering feedback that advances the development of HITs, they rarely provide usable evidence that can contribute to our broader understanding of HITs. OBJECTIVE: In this research, we aim to develop and demonstrate a variation of vignette testing that supports developers and designers in evaluating early-stage HIT designs while generating usable evidence for the broader research community. METHODS: We proposed a method called health concept surveying for untangling the causal relationships that people develop around conceptual HITs. In health concept surveying, investigators gather reactions to design concepts through a scenario-based survey instrument. As the investigator manipulates characteristics related to their HIT, the survey instrument also measures proximal cognitive factors according to a health behavior change model to project how HIT design decisions may affect the adoption and acceptability of an HIT. Responses to the survey instrument were analyzed using path analysis to untangle the causal effects of these factors on the outcome variables. RESULTS: We demonstrated health concept surveying in 3 case studies of sensor-based health-screening apps. Our first study (N=54) showed that a wait time incentive could influence more people to go see a dermatologist after a positive test for skin cancer. Our second study (N=54), evaluating a similar application design, showed that although visual explanations of algorithmic decisions could increase participant trust in negative test results, the trust would not have been enough to affect people’s decision-making. Our third study (N=263) showed that people might prioritize test specificity or sensitivity depending on the nature of the medical condition. CONCLUSIONS: Beyond the findings from our 3 case studies, our research uses the framing of the Health Belief Model to elicit and understand the intrinsic and extrinsic factors that may affect the adoption and acceptability of an HIT without having to build a working prototype. We have made our survey instrument publicly available so that others can leverage it for their own investigations. |
format | Online Article Text |
id | pubmed-8764610 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | JMIR Publications |
record_format | MEDLINE/PubMed |
spelling | pubmed-87646102022-02-03 Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology Mariakakis, Alex Karkar, Ravi Patel, Shwetak N Kientz, Julie A Fogarty, James Munson, Sean A JMIR Hum Factors Original Paper BACKGROUND: Developers, designers, and researchers use rapid prototyping methods to project the adoption and acceptability of their health intervention technology (HIT) before the technology becomes mature enough to be deployed. Although these methods are useful for gathering feedback that advances the development of HITs, they rarely provide usable evidence that can contribute to our broader understanding of HITs. OBJECTIVE: In this research, we aim to develop and demonstrate a variation of vignette testing that supports developers and designers in evaluating early-stage HIT designs while generating usable evidence for the broader research community. METHODS: We proposed a method called health concept surveying for untangling the causal relationships that people develop around conceptual HITs. In health concept surveying, investigators gather reactions to design concepts through a scenario-based survey instrument. As the investigator manipulates characteristics related to their HIT, the survey instrument also measures proximal cognitive factors according to a health behavior change model to project how HIT design decisions may affect the adoption and acceptability of an HIT. Responses to the survey instrument were analyzed using path analysis to untangle the causal effects of these factors on the outcome variables. RESULTS: We demonstrated health concept surveying in 3 case studies of sensor-based health-screening apps. Our first study (N=54) showed that a wait time incentive could influence more people to go see a dermatologist after a positive test for skin cancer. Our second study (N=54), evaluating a similar application design, showed that although visual explanations of algorithmic decisions could increase participant trust in negative test results, the trust would not have been enough to affect people’s decision-making. Our third study (N=263) showed that people might prioritize test specificity or sensitivity depending on the nature of the medical condition. CONCLUSIONS: Beyond the findings from our 3 case studies, our research uses the framing of the Health Belief Model to elicit and understand the intrinsic and extrinsic factors that may affect the adoption and acceptability of an HIT without having to build a working prototype. We have made our survey instrument publicly available so that others can leverage it for their own investigations. JMIR Publications 2022-01-03 /pmc/articles/PMC8764610/ /pubmed/34982038 http://dx.doi.org/10.2196/30474 Text en ©Alex Mariakakis, Ravi Karkar, Shwetak N Patel, Julie A Kientz, James Fogarty, Sean A Munson. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 03.01.2022. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included. |
spellingShingle | Original Paper Mariakakis, Alex Karkar, Ravi Patel, Shwetak N Kientz, Julie A Fogarty, James Munson, Sean A Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology |
title | Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology |
title_full | Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology |
title_fullStr | Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology |
title_full_unstemmed | Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology |
title_short | Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology |
title_sort | using health concept surveying to elicit usable evidence: case studies of a novel evaluation methodology |
topic | Original Paper |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8764610/ https://www.ncbi.nlm.nih.gov/pubmed/34982038 http://dx.doi.org/10.2196/30474 |
work_keys_str_mv | AT mariakakisalex usinghealthconceptsurveyingtoelicitusableevidencecasestudiesofanovelevaluationmethodology AT karkarravi usinghealthconceptsurveyingtoelicitusableevidencecasestudiesofanovelevaluationmethodology AT patelshwetakn usinghealthconceptsurveyingtoelicitusableevidencecasestudiesofanovelevaluationmethodology AT kientzjuliea usinghealthconceptsurveyingtoelicitusableevidencecasestudiesofanovelevaluationmethodology AT fogartyjames usinghealthconceptsurveyingtoelicitusableevidencecasestudiesofanovelevaluationmethodology AT munsonseana usinghealthconceptsurveyingtoelicitusableevidencecasestudiesofanovelevaluationmethodology |