Cargando…
Responses of Conversational Agents to Health and Lifestyle Prompts: Investigation of Appropriateness and Presentation Structures
BACKGROUND: Conversational agents (CAs) are systems that mimic human conversations using text or spoken language. Their widely used examples include voice-activated systems such as Apple Siri, Google Assistant, Amazon Alexa, and Microsoft Cortana. The use of CAs in health care has been on the rise,...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
JMIR Publications
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7055771/ https://www.ncbi.nlm.nih.gov/pubmed/32039810 http://dx.doi.org/10.2196/15823 |
_version_ | 1783503415748329472 |
---|---|
author | Kocaballi, Ahmet Baki Quiroz, Juan C Rezazadegan, Dana Berkovsky, Shlomo Magrabi, Farah Coiera, Enrico Laranjo, Liliana |
author_facet | Kocaballi, Ahmet Baki Quiroz, Juan C Rezazadegan, Dana Berkovsky, Shlomo Magrabi, Farah Coiera, Enrico Laranjo, Liliana |
author_sort | Kocaballi, Ahmet Baki |
collection | PubMed |
description | BACKGROUND: Conversational agents (CAs) are systems that mimic human conversations using text or spoken language. Their widely used examples include voice-activated systems such as Apple Siri, Google Assistant, Amazon Alexa, and Microsoft Cortana. The use of CAs in health care has been on the rise, but concerns about their potential safety risks often remain understudied. OBJECTIVE: This study aimed to analyze how commonly available, general-purpose CAs on smartphones and smart speakers respond to health and lifestyle prompts (questions and open-ended statements) by examining their responses in terms of content and structure alike. METHODS: We followed a piloted script to present health- and lifestyle-related prompts to 8 CAs. The CAs’ responses were assessed for their appropriateness on the basis of the prompt type: responses to safety-critical prompts were deemed appropriate if they included a referral to a health professional or service, whereas responses to lifestyle prompts were deemed appropriate if they provided relevant information to address the problem prompted. The response structure was also examined according to information sources (Web search–based or precoded), response content style (informative and/or directive), confirmation of prompt recognition, and empathy. RESULTS: The 8 studied CAs provided in total 240 responses to 30 prompts. They collectively responded appropriately to 41% (46/112) of the safety-critical and 39% (37/96) of the lifestyle prompts. The ratio of appropriate responses deteriorated when safety-critical prompts were rephrased or when the agent used a voice-only interface. The appropriate responses included mostly directive content and empathy statements for the safety-critical prompts and a mix of informative and directive content for the lifestyle prompts. CONCLUSIONS: Our results suggest that the commonly available, general-purpose CAs on smartphones and smart speakers with unconstrained natural language interfaces are limited in their ability to advise on both the safety-critical health prompts and lifestyle prompts. Our study also identified some response structures the CAs employed to present their appropriate responses. Further investigation is needed to establish guidelines for designing suitable response structures for different prompt types. |
format | Online Article Text |
id | pubmed-7055771 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | JMIR Publications |
record_format | MEDLINE/PubMed |
spelling | pubmed-70557712020-03-16 Responses of Conversational Agents to Health and Lifestyle Prompts: Investigation of Appropriateness and Presentation Structures Kocaballi, Ahmet Baki Quiroz, Juan C Rezazadegan, Dana Berkovsky, Shlomo Magrabi, Farah Coiera, Enrico Laranjo, Liliana J Med Internet Res Original Paper BACKGROUND: Conversational agents (CAs) are systems that mimic human conversations using text or spoken language. Their widely used examples include voice-activated systems such as Apple Siri, Google Assistant, Amazon Alexa, and Microsoft Cortana. The use of CAs in health care has been on the rise, but concerns about their potential safety risks often remain understudied. OBJECTIVE: This study aimed to analyze how commonly available, general-purpose CAs on smartphones and smart speakers respond to health and lifestyle prompts (questions and open-ended statements) by examining their responses in terms of content and structure alike. METHODS: We followed a piloted script to present health- and lifestyle-related prompts to 8 CAs. The CAs’ responses were assessed for their appropriateness on the basis of the prompt type: responses to safety-critical prompts were deemed appropriate if they included a referral to a health professional or service, whereas responses to lifestyle prompts were deemed appropriate if they provided relevant information to address the problem prompted. The response structure was also examined according to information sources (Web search–based or precoded), response content style (informative and/or directive), confirmation of prompt recognition, and empathy. RESULTS: The 8 studied CAs provided in total 240 responses to 30 prompts. They collectively responded appropriately to 41% (46/112) of the safety-critical and 39% (37/96) of the lifestyle prompts. The ratio of appropriate responses deteriorated when safety-critical prompts were rephrased or when the agent used a voice-only interface. The appropriate responses included mostly directive content and empathy statements for the safety-critical prompts and a mix of informative and directive content for the lifestyle prompts. CONCLUSIONS: Our results suggest that the commonly available, general-purpose CAs on smartphones and smart speakers with unconstrained natural language interfaces are limited in their ability to advise on both the safety-critical health prompts and lifestyle prompts. Our study also identified some response structures the CAs employed to present their appropriate responses. Further investigation is needed to establish guidelines for designing suitable response structures for different prompt types. JMIR Publications 2020-02-10 /pmc/articles/PMC7055771/ /pubmed/32039810 http://dx.doi.org/10.2196/15823 Text en ©Ahmet Baki Baki Kocaballi, Juan C Quiroz, Dana Rezazadegan, Shlomo Berkovsky, Farah Magrabi, Enrico Coiera, Liliana Laranjo. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 10.02.2020. https://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included. |
spellingShingle | Original Paper Kocaballi, Ahmet Baki Quiroz, Juan C Rezazadegan, Dana Berkovsky, Shlomo Magrabi, Farah Coiera, Enrico Laranjo, Liliana Responses of Conversational Agents to Health and Lifestyle Prompts: Investigation of Appropriateness and Presentation Structures |
title | Responses of Conversational Agents to Health and Lifestyle Prompts: Investigation of Appropriateness and Presentation Structures |
title_full | Responses of Conversational Agents to Health and Lifestyle Prompts: Investigation of Appropriateness and Presentation Structures |
title_fullStr | Responses of Conversational Agents to Health and Lifestyle Prompts: Investigation of Appropriateness and Presentation Structures |
title_full_unstemmed | Responses of Conversational Agents to Health and Lifestyle Prompts: Investigation of Appropriateness and Presentation Structures |
title_short | Responses of Conversational Agents to Health and Lifestyle Prompts: Investigation of Appropriateness and Presentation Structures |
title_sort | responses of conversational agents to health and lifestyle prompts: investigation of appropriateness and presentation structures |
topic | Original Paper |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7055771/ https://www.ncbi.nlm.nih.gov/pubmed/32039810 http://dx.doi.org/10.2196/15823 |
work_keys_str_mv | AT kocaballiahmetbaki responsesofconversationalagentstohealthandlifestylepromptsinvestigationofappropriatenessandpresentationstructures AT quirozjuanc responsesofconversationalagentstohealthandlifestylepromptsinvestigationofappropriatenessandpresentationstructures AT rezazadegandana responsesofconversationalagentstohealthandlifestylepromptsinvestigationofappropriatenessandpresentationstructures AT berkovskyshlomo responsesofconversationalagentstohealthandlifestylepromptsinvestigationofappropriatenessandpresentationstructures AT magrabifarah responsesofconversationalagentstohealthandlifestylepromptsinvestigationofappropriatenessandpresentationstructures AT coieraenrico responsesofconversationalagentstohealthandlifestylepromptsinvestigationofappropriatenessandpresentationstructures AT laranjoliliana responsesofconversationalagentstohealthandlifestylepromptsinvestigationofappropriatenessandpresentationstructures |