Cargando…

The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study

BACKGROUND: The rising adoption of telehealth provides new opportunities for more effective and equitable health care information mediums. The ability of chatbots to provide a conversational, personal, and comprehendible avenue for learning about health care information make them a promising tool fo...

Descripción completa

Detalles Bibliográficos
Autores principales: Biro, Joshua, Linder, Courtney, Neyens, David
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9932873/
https://www.ncbi.nlm.nih.gov/pubmed/36724004
http://dx.doi.org/10.2196/41017
_version_ 1784889554326519808
author Biro, Joshua
Linder, Courtney
Neyens, David
author_facet Biro, Joshua
Linder, Courtney
Neyens, David
author_sort Biro, Joshua
collection PubMed
description BACKGROUND: The rising adoption of telehealth provides new opportunities for more effective and equitable health care information mediums. The ability of chatbots to provide a conversational, personal, and comprehendible avenue for learning about health care information make them a promising tool for addressing health care inequity as health care trends continue toward web-based and remote processes. Although chatbots have been studied in the health care domain for their efficacy for smoking cessation, diet recommendation, and other assistive applications, few studies have examined how specific design characteristics influence the effectiveness of chatbots in providing health information. OBJECTIVE: Our objective was to investigate the influence of different design considerations on the effectiveness of an educational health care chatbot. METHODS: A 2×3 between-subjects study was performed with 2 independent variables: a chatbot’s complexity of responses (eg, technical or nontechnical language) and the presented qualifications of the chatbot’s persona (eg, doctor, nurse, or nursing student). Regression models were used to evaluate the impact of these variables on 3 outcome measures: effectiveness, usability, and trust. A qualitative transcript review was also done to review how participants engaged with the chatbot. RESULTS: Analysis of 71 participants found that participants who received technical language responses were significantly more likely to be in the high effectiveness group, which had higher improvements in test scores (odds ratio [OR] 2.73, 95% CI 1.05-7.41; P=.04). Participants with higher health literacy (OR 2.04, 95% CI 1.11-4.00, P=.03) were significantly more likely to trust the chatbot. The participants engaged with the chatbot in a variety of ways, with some taking a conversational approach and others treating the chatbot more like a search engine. CONCLUSIONS: Given their increasing popularity, it is vital that we consider how chatbots are designed and implemented. This study showed that factors such as chatbots’ persona and language complexity are two design considerations that influence the ability of chatbots to successfully provide health care information.
format Online
Article
Text
id pubmed-9932873
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-99328732023-02-17 The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study Biro, Joshua Linder, Courtney Neyens, David JMIR Hum Factors Original Paper BACKGROUND: The rising adoption of telehealth provides new opportunities for more effective and equitable health care information mediums. The ability of chatbots to provide a conversational, personal, and comprehendible avenue for learning about health care information make them a promising tool for addressing health care inequity as health care trends continue toward web-based and remote processes. Although chatbots have been studied in the health care domain for their efficacy for smoking cessation, diet recommendation, and other assistive applications, few studies have examined how specific design characteristics influence the effectiveness of chatbots in providing health information. OBJECTIVE: Our objective was to investigate the influence of different design considerations on the effectiveness of an educational health care chatbot. METHODS: A 2×3 between-subjects study was performed with 2 independent variables: a chatbot’s complexity of responses (eg, technical or nontechnical language) and the presented qualifications of the chatbot’s persona (eg, doctor, nurse, or nursing student). Regression models were used to evaluate the impact of these variables on 3 outcome measures: effectiveness, usability, and trust. A qualitative transcript review was also done to review how participants engaged with the chatbot. RESULTS: Analysis of 71 participants found that participants who received technical language responses were significantly more likely to be in the high effectiveness group, which had higher improvements in test scores (odds ratio [OR] 2.73, 95% CI 1.05-7.41; P=.04). Participants with higher health literacy (OR 2.04, 95% CI 1.11-4.00, P=.03) were significantly more likely to trust the chatbot. The participants engaged with the chatbot in a variety of ways, with some taking a conversational approach and others treating the chatbot more like a search engine. CONCLUSIONS: Given their increasing popularity, it is vital that we consider how chatbots are designed and implemented. This study showed that factors such as chatbots’ persona and language complexity are two design considerations that influence the ability of chatbots to successfully provide health care information. JMIR Publications 2023-02-01 /pmc/articles/PMC9932873/ /pubmed/36724004 http://dx.doi.org/10.2196/41017 Text en ©Joshua Biro, Courtney Linder, David Neyens. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 01.02.2023. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included.
spellingShingle Original Paper
Biro, Joshua
Linder, Courtney
Neyens, David
The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study
title The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study
title_full The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study
title_fullStr The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study
title_full_unstemmed The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study
title_short The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study
title_sort effects of a health care chatbot’s complexity and persona on user trust, perceived usability, and effectiveness: mixed methods study
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9932873/
https://www.ncbi.nlm.nih.gov/pubmed/36724004
http://dx.doi.org/10.2196/41017
work_keys_str_mv AT birojoshua theeffectsofahealthcarechatbotscomplexityandpersonaonusertrustperceivedusabilityandeffectivenessmixedmethodsstudy
AT lindercourtney theeffectsofahealthcarechatbotscomplexityandpersonaonusertrustperceivedusabilityandeffectivenessmixedmethodsstudy
AT neyensdavid theeffectsofahealthcarechatbotscomplexityandpersonaonusertrustperceivedusabilityandeffectivenessmixedmethodsstudy
AT birojoshua effectsofahealthcarechatbotscomplexityandpersonaonusertrustperceivedusabilityandeffectivenessmixedmethodsstudy
AT lindercourtney effectsofahealthcarechatbotscomplexityandpersonaonusertrustperceivedusabilityandeffectivenessmixedmethodsstudy
AT neyensdavid effectsofahealthcarechatbotscomplexityandpersonaonusertrustperceivedusabilityandeffectivenessmixedmethodsstudy