Cargando…

Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology

BACKGROUND: Developers, designers, and researchers use rapid prototyping methods to project the adoption and acceptability of their health intervention technology (HIT) before the technology becomes mature enough to be deployed. Although these methods are useful for gathering feedback that advances...

Descripción completa

Detalles Bibliográficos
Autores principales: Mariakakis, Alex, Karkar, Ravi, Patel, Shwetak N, Kientz, Julie A, Fogarty, James, Munson, Sean A
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8764610/
https://www.ncbi.nlm.nih.gov/pubmed/34982038
http://dx.doi.org/10.2196/30474
_version_ 1784634202982973440
author Mariakakis, Alex
Karkar, Ravi
Patel, Shwetak N
Kientz, Julie A
Fogarty, James
Munson, Sean A
author_facet Mariakakis, Alex
Karkar, Ravi
Patel, Shwetak N
Kientz, Julie A
Fogarty, James
Munson, Sean A
author_sort Mariakakis, Alex
collection PubMed
description BACKGROUND: Developers, designers, and researchers use rapid prototyping methods to project the adoption and acceptability of their health intervention technology (HIT) before the technology becomes mature enough to be deployed. Although these methods are useful for gathering feedback that advances the development of HITs, they rarely provide usable evidence that can contribute to our broader understanding of HITs. OBJECTIVE: In this research, we aim to develop and demonstrate a variation of vignette testing that supports developers and designers in evaluating early-stage HIT designs while generating usable evidence for the broader research community. METHODS: We proposed a method called health concept surveying for untangling the causal relationships that people develop around conceptual HITs. In health concept surveying, investigators gather reactions to design concepts through a scenario-based survey instrument. As the investigator manipulates characteristics related to their HIT, the survey instrument also measures proximal cognitive factors according to a health behavior change model to project how HIT design decisions may affect the adoption and acceptability of an HIT. Responses to the survey instrument were analyzed using path analysis to untangle the causal effects of these factors on the outcome variables. RESULTS: We demonstrated health concept surveying in 3 case studies of sensor-based health-screening apps. Our first study (N=54) showed that a wait time incentive could influence more people to go see a dermatologist after a positive test for skin cancer. Our second study (N=54), evaluating a similar application design, showed that although visual explanations of algorithmic decisions could increase participant trust in negative test results, the trust would not have been enough to affect people’s decision-making. Our third study (N=263) showed that people might prioritize test specificity or sensitivity depending on the nature of the medical condition. CONCLUSIONS: Beyond the findings from our 3 case studies, our research uses the framing of the Health Belief Model to elicit and understand the intrinsic and extrinsic factors that may affect the adoption and acceptability of an HIT without having to build a working prototype. We have made our survey instrument publicly available so that others can leverage it for their own investigations.
format Online
Article
Text
id pubmed-8764610
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-87646102022-02-03 Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology Mariakakis, Alex Karkar, Ravi Patel, Shwetak N Kientz, Julie A Fogarty, James Munson, Sean A JMIR Hum Factors Original Paper BACKGROUND: Developers, designers, and researchers use rapid prototyping methods to project the adoption and acceptability of their health intervention technology (HIT) before the technology becomes mature enough to be deployed. Although these methods are useful for gathering feedback that advances the development of HITs, they rarely provide usable evidence that can contribute to our broader understanding of HITs. OBJECTIVE: In this research, we aim to develop and demonstrate a variation of vignette testing that supports developers and designers in evaluating early-stage HIT designs while generating usable evidence for the broader research community. METHODS: We proposed a method called health concept surveying for untangling the causal relationships that people develop around conceptual HITs. In health concept surveying, investigators gather reactions to design concepts through a scenario-based survey instrument. As the investigator manipulates characteristics related to their HIT, the survey instrument also measures proximal cognitive factors according to a health behavior change model to project how HIT design decisions may affect the adoption and acceptability of an HIT. Responses to the survey instrument were analyzed using path analysis to untangle the causal effects of these factors on the outcome variables. RESULTS: We demonstrated health concept surveying in 3 case studies of sensor-based health-screening apps. Our first study (N=54) showed that a wait time incentive could influence more people to go see a dermatologist after a positive test for skin cancer. Our second study (N=54), evaluating a similar application design, showed that although visual explanations of algorithmic decisions could increase participant trust in negative test results, the trust would not have been enough to affect people’s decision-making. Our third study (N=263) showed that people might prioritize test specificity or sensitivity depending on the nature of the medical condition. CONCLUSIONS: Beyond the findings from our 3 case studies, our research uses the framing of the Health Belief Model to elicit and understand the intrinsic and extrinsic factors that may affect the adoption and acceptability of an HIT without having to build a working prototype. We have made our survey instrument publicly available so that others can leverage it for their own investigations. JMIR Publications 2022-01-03 /pmc/articles/PMC8764610/ /pubmed/34982038 http://dx.doi.org/10.2196/30474 Text en ©Alex Mariakakis, Ravi Karkar, Shwetak N Patel, Julie A Kientz, James Fogarty, Sean A Munson. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 03.01.2022. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.
spellingShingle Original Paper
Mariakakis, Alex
Karkar, Ravi
Patel, Shwetak N
Kientz, Julie A
Fogarty, James
Munson, Sean A
Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology
title Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology
title_full Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology
title_fullStr Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology
title_full_unstemmed Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology
title_short Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology
title_sort using health concept surveying to elicit usable evidence: case studies of a novel evaluation methodology
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8764610/
https://www.ncbi.nlm.nih.gov/pubmed/34982038
http://dx.doi.org/10.2196/30474
work_keys_str_mv AT mariakakisalex usinghealthconceptsurveyingtoelicitusableevidencecasestudiesofanovelevaluationmethodology
AT karkarravi usinghealthconceptsurveyingtoelicitusableevidencecasestudiesofanovelevaluationmethodology
AT patelshwetakn usinghealthconceptsurveyingtoelicitusableevidencecasestudiesofanovelevaluationmethodology
AT kientzjuliea usinghealthconceptsurveyingtoelicitusableevidencecasestudiesofanovelevaluationmethodology
AT fogartyjames usinghealthconceptsurveyingtoelicitusableevidencecasestudiesofanovelevaluationmethodology
AT munsonseana usinghealthconceptsurveyingtoelicitusableevidencecasestudiesofanovelevaluationmethodology