Cargando…

Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation

BACKGROUND: Despite the increasing availability of clinical decision support systems (CDSSs) and rising expectation for CDSSs based on artificial intelligence (AI), little is known about the acceptance of AI-based CDSS by physicians and its barriers and facilitators in emergency care settings. OBJEC...

Descripción completa

Detalles Bibliográficos
Autores principales: Fujimori, Ryo, Liu, Keibun, Soeno, Shoko, Naraba, Hiromu, Ogura, Kentaro, Hara, Konan, Sonoo, Tomohiro, Ogura, Takayuki, Nakamura, Kensuke, Goto, Tadahiro
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9237770/
https://www.ncbi.nlm.nih.gov/pubmed/35699995
http://dx.doi.org/10.2196/36501
_version_ 1784736873306914816
author Fujimori, Ryo
Liu, Keibun
Soeno, Shoko
Naraba, Hiromu
Ogura, Kentaro
Hara, Konan
Sonoo, Tomohiro
Ogura, Takayuki
Nakamura, Kensuke
Goto, Tadahiro
author_facet Fujimori, Ryo
Liu, Keibun
Soeno, Shoko
Naraba, Hiromu
Ogura, Kentaro
Hara, Konan
Sonoo, Tomohiro
Ogura, Takayuki
Nakamura, Kensuke
Goto, Tadahiro
author_sort Fujimori, Ryo
collection PubMed
description BACKGROUND: Despite the increasing availability of clinical decision support systems (CDSSs) and rising expectation for CDSSs based on artificial intelligence (AI), little is known about the acceptance of AI-based CDSS by physicians and its barriers and facilitators in emergency care settings. OBJECTIVE: We aimed to evaluate the acceptance, barriers, and facilitators to implementing AI-based CDSSs in the emergency care setting through the opinions of physicians on our newly developed, real-time AI-based CDSS, which alerts ED physicians by predicting aortic dissection based on numeric and text information from medical charts, by using the Unified Theory of Acceptance and Use of Technology (UTAUT; for quantitative evaluation) and the Consolidated Framework for Implementation Research (CFIR; for qualitative evaluation) frameworks. METHODS: This mixed methods study was performed from March to April 2021. Transitional year residents (n=6), emergency medicine residents (n=5), and emergency physicians (n=3) from two community, tertiary care hospitals in Japan were included. We first developed a real-time CDSS for predicting aortic dissection based on numeric and text information from medical charts (eg, chief complaints, medical history, vital signs) with natural language processing. This system was deployed on the internet, and the participants used the system with clinical vignettes of model cases. Participants were then involved in a mixed methods evaluation consisting of a UTAUT-based questionnaire with a 5-point Likert scale (quantitative) and a CFIR-based semistructured interview (qualitative). Cronbach α was calculated as a reliability estimate for UTAUT subconstructs. Interviews were sampled, transcribed, and analyzed using the MaxQDA software. The framework analysis approach was used during the study to determine the relevance of the CFIR constructs. RESULTS: All 14 participants completed the questionnaires and interviews. Quantitative analysis revealed generally positive responses for user acceptance with all scores above the neutral score of 3.0. In addition, the mixed methods analysis identified two significant barriers (System Performance, Compatibility) and two major facilitators (Evidence Strength, Design Quality) for implementation of AI-based CDSSs in emergency care settings. CONCLUSIONS: Our mixed methods evaluation based on theoretically grounded frameworks revealed the acceptance, barriers, and facilitators of implementation of AI-based CDSS. Although the concern of system failure and overtrusting of the system could be barriers to implementation, the locality of the system and designing an intuitive user interface could likely facilitate the use of optimal AI-based CDSS. Alleviating and resolving these factors should be key to achieving good user acceptance of AI-based CDSS.
format Online
Article
Text
id pubmed-9237770
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-92377702022-06-29 Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation Fujimori, Ryo Liu, Keibun Soeno, Shoko Naraba, Hiromu Ogura, Kentaro Hara, Konan Sonoo, Tomohiro Ogura, Takayuki Nakamura, Kensuke Goto, Tadahiro JMIR Form Res Original Paper BACKGROUND: Despite the increasing availability of clinical decision support systems (CDSSs) and rising expectation for CDSSs based on artificial intelligence (AI), little is known about the acceptance of AI-based CDSS by physicians and its barriers and facilitators in emergency care settings. OBJECTIVE: We aimed to evaluate the acceptance, barriers, and facilitators to implementing AI-based CDSSs in the emergency care setting through the opinions of physicians on our newly developed, real-time AI-based CDSS, which alerts ED physicians by predicting aortic dissection based on numeric and text information from medical charts, by using the Unified Theory of Acceptance and Use of Technology (UTAUT; for quantitative evaluation) and the Consolidated Framework for Implementation Research (CFIR; for qualitative evaluation) frameworks. METHODS: This mixed methods study was performed from March to April 2021. Transitional year residents (n=6), emergency medicine residents (n=5), and emergency physicians (n=3) from two community, tertiary care hospitals in Japan were included. We first developed a real-time CDSS for predicting aortic dissection based on numeric and text information from medical charts (eg, chief complaints, medical history, vital signs) with natural language processing. This system was deployed on the internet, and the participants used the system with clinical vignettes of model cases. Participants were then involved in a mixed methods evaluation consisting of a UTAUT-based questionnaire with a 5-point Likert scale (quantitative) and a CFIR-based semistructured interview (qualitative). Cronbach α was calculated as a reliability estimate for UTAUT subconstructs. Interviews were sampled, transcribed, and analyzed using the MaxQDA software. The framework analysis approach was used during the study to determine the relevance of the CFIR constructs. RESULTS: All 14 participants completed the questionnaires and interviews. Quantitative analysis revealed generally positive responses for user acceptance with all scores above the neutral score of 3.0. In addition, the mixed methods analysis identified two significant barriers (System Performance, Compatibility) and two major facilitators (Evidence Strength, Design Quality) for implementation of AI-based CDSSs in emergency care settings. CONCLUSIONS: Our mixed methods evaluation based on theoretically grounded frameworks revealed the acceptance, barriers, and facilitators of implementation of AI-based CDSS. Although the concern of system failure and overtrusting of the system could be barriers to implementation, the locality of the system and designing an intuitive user interface could likely facilitate the use of optimal AI-based CDSS. Alleviating and resolving these factors should be key to achieving good user acceptance of AI-based CDSS. JMIR Publications 2022-06-13 /pmc/articles/PMC9237770/ /pubmed/35699995 http://dx.doi.org/10.2196/36501 Text en ©Ryo Fujimori, Keibun Liu, Shoko Soeno, Hiromu Naraba, Kentaro Ogura, Konan Hara, Tomohiro Sonoo, Takayuki Ogura, Kensuke Nakamura, Tadahiro Goto. Originally published in JMIR Formative Research (https://formative.jmir.org), 13.06.2022. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
spellingShingle Original Paper
Fujimori, Ryo
Liu, Keibun
Soeno, Shoko
Naraba, Hiromu
Ogura, Kentaro
Hara, Konan
Sonoo, Tomohiro
Ogura, Takayuki
Nakamura, Kensuke
Goto, Tadahiro
Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation
title Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation
title_full Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation
title_fullStr Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation
title_full_unstemmed Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation
title_short Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation
title_sort acceptance, barriers, and facilitators to implementing artificial intelligence–based decision support systems in emergency departments: quantitative and qualitative evaluation
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9237770/
https://www.ncbi.nlm.nih.gov/pubmed/35699995
http://dx.doi.org/10.2196/36501
work_keys_str_mv AT fujimoriryo acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation
AT liukeibun acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation
AT soenoshoko acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation
AT narabahiromu acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation
AT ogurakentaro acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation
AT harakonan acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation
AT sonootomohiro acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation
AT oguratakayuki acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation
AT nakamurakensuke acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation
AT gototadahiro acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation