Cargando…
The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study
BACKGROUND: The rising adoption of telehealth provides new opportunities for more effective and equitable health care information mediums. The ability of chatbots to provide a conversational, personal, and comprehendible avenue for learning about health care information make them a promising tool fo...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
JMIR Publications
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9932873/ https://www.ncbi.nlm.nih.gov/pubmed/36724004 http://dx.doi.org/10.2196/41017 |
_version_ | 1784889554326519808 |
---|---|
author | Biro, Joshua Linder, Courtney Neyens, David |
author_facet | Biro, Joshua Linder, Courtney Neyens, David |
author_sort | Biro, Joshua |
collection | PubMed |
description | BACKGROUND: The rising adoption of telehealth provides new opportunities for more effective and equitable health care information mediums. The ability of chatbots to provide a conversational, personal, and comprehendible avenue for learning about health care information make them a promising tool for addressing health care inequity as health care trends continue toward web-based and remote processes. Although chatbots have been studied in the health care domain for their efficacy for smoking cessation, diet recommendation, and other assistive applications, few studies have examined how specific design characteristics influence the effectiveness of chatbots in providing health information. OBJECTIVE: Our objective was to investigate the influence of different design considerations on the effectiveness of an educational health care chatbot. METHODS: A 2×3 between-subjects study was performed with 2 independent variables: a chatbot’s complexity of responses (eg, technical or nontechnical language) and the presented qualifications of the chatbot’s persona (eg, doctor, nurse, or nursing student). Regression models were used to evaluate the impact of these variables on 3 outcome measures: effectiveness, usability, and trust. A qualitative transcript review was also done to review how participants engaged with the chatbot. RESULTS: Analysis of 71 participants found that participants who received technical language responses were significantly more likely to be in the high effectiveness group, which had higher improvements in test scores (odds ratio [OR] 2.73, 95% CI 1.05-7.41; P=.04). Participants with higher health literacy (OR 2.04, 95% CI 1.11-4.00, P=.03) were significantly more likely to trust the chatbot. The participants engaged with the chatbot in a variety of ways, with some taking a conversational approach and others treating the chatbot more like a search engine. CONCLUSIONS: Given their increasing popularity, it is vital that we consider how chatbots are designed and implemented. This study showed that factors such as chatbots’ persona and language complexity are two design considerations that influence the ability of chatbots to successfully provide health care information. |
format | Online Article Text |
id | pubmed-9932873 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | JMIR Publications |
record_format | MEDLINE/PubMed |
spelling | pubmed-99328732023-02-17 The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study Biro, Joshua Linder, Courtney Neyens, David JMIR Hum Factors Original Paper BACKGROUND: The rising adoption of telehealth provides new opportunities for more effective and equitable health care information mediums. The ability of chatbots to provide a conversational, personal, and comprehendible avenue for learning about health care information make them a promising tool for addressing health care inequity as health care trends continue toward web-based and remote processes. Although chatbots have been studied in the health care domain for their efficacy for smoking cessation, diet recommendation, and other assistive applications, few studies have examined how specific design characteristics influence the effectiveness of chatbots in providing health information. OBJECTIVE: Our objective was to investigate the influence of different design considerations on the effectiveness of an educational health care chatbot. METHODS: A 2×3 between-subjects study was performed with 2 independent variables: a chatbot’s complexity of responses (eg, technical or nontechnical language) and the presented qualifications of the chatbot’s persona (eg, doctor, nurse, or nursing student). Regression models were used to evaluate the impact of these variables on 3 outcome measures: effectiveness, usability, and trust. A qualitative transcript review was also done to review how participants engaged with the chatbot. RESULTS: Analysis of 71 participants found that participants who received technical language responses were significantly more likely to be in the high effectiveness group, which had higher improvements in test scores (odds ratio [OR] 2.73, 95% CI 1.05-7.41; P=.04). Participants with higher health literacy (OR 2.04, 95% CI 1.11-4.00, P=.03) were significantly more likely to trust the chatbot. The participants engaged with the chatbot in a variety of ways, with some taking a conversational approach and others treating the chatbot more like a search engine. CONCLUSIONS: Given their increasing popularity, it is vital that we consider how chatbots are designed and implemented. This study showed that factors such as chatbots’ persona and language complexity are two design considerations that influence the ability of chatbots to successfully provide health care information. JMIR Publications 2023-02-01 /pmc/articles/PMC9932873/ /pubmed/36724004 http://dx.doi.org/10.2196/41017 Text en ©Joshua Biro, Courtney Linder, David Neyens. Originally published in JMIR Human Factors (https://humanfactors.jmir.org), 01.02.2023. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Human Factors, is properly cited. The complete bibliographic information, a link to the original publication on https://humanfactors.jmir.org, as well as this copyright and license information must be included. |
spellingShingle | Original Paper Biro, Joshua Linder, Courtney Neyens, David The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study |
title | The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study |
title_full | The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study |
title_fullStr | The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study |
title_full_unstemmed | The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study |
title_short | The Effects of a Health Care Chatbot’s Complexity and Persona on User Trust, Perceived Usability, and Effectiveness: Mixed Methods Study |
title_sort | effects of a health care chatbot’s complexity and persona on user trust, perceived usability, and effectiveness: mixed methods study |
topic | Original Paper |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9932873/ https://www.ncbi.nlm.nih.gov/pubmed/36724004 http://dx.doi.org/10.2196/41017 |
work_keys_str_mv | AT birojoshua theeffectsofahealthcarechatbotscomplexityandpersonaonusertrustperceivedusabilityandeffectivenessmixedmethodsstudy AT lindercourtney theeffectsofahealthcarechatbotscomplexityandpersonaonusertrustperceivedusabilityandeffectivenessmixedmethodsstudy AT neyensdavid theeffectsofahealthcarechatbotscomplexityandpersonaonusertrustperceivedusabilityandeffectivenessmixedmethodsstudy AT birojoshua effectsofahealthcarechatbotscomplexityandpersonaonusertrustperceivedusabilityandeffectivenessmixedmethodsstudy AT lindercourtney effectsofahealthcarechatbotscomplexityandpersonaonusertrustperceivedusabilityandeffectivenessmixedmethodsstudy AT neyensdavid effectsofahealthcarechatbotscomplexityandpersonaonusertrustperceivedusabilityandeffectivenessmixedmethodsstudy |