Cargando…
Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation
BACKGROUND: Despite the increasing availability of clinical decision support systems (CDSSs) and rising expectation for CDSSs based on artificial intelligence (AI), little is known about the acceptance of AI-based CDSS by physicians and its barriers and facilitators in emergency care settings. OBJEC...
Autores principales: | , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
JMIR Publications
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9237770/ https://www.ncbi.nlm.nih.gov/pubmed/35699995 http://dx.doi.org/10.2196/36501 |
_version_ | 1784736873306914816 |
---|---|
author | Fujimori, Ryo Liu, Keibun Soeno, Shoko Naraba, Hiromu Ogura, Kentaro Hara, Konan Sonoo, Tomohiro Ogura, Takayuki Nakamura, Kensuke Goto, Tadahiro |
author_facet | Fujimori, Ryo Liu, Keibun Soeno, Shoko Naraba, Hiromu Ogura, Kentaro Hara, Konan Sonoo, Tomohiro Ogura, Takayuki Nakamura, Kensuke Goto, Tadahiro |
author_sort | Fujimori, Ryo |
collection | PubMed |
description | BACKGROUND: Despite the increasing availability of clinical decision support systems (CDSSs) and rising expectation for CDSSs based on artificial intelligence (AI), little is known about the acceptance of AI-based CDSS by physicians and its barriers and facilitators in emergency care settings. OBJECTIVE: We aimed to evaluate the acceptance, barriers, and facilitators to implementing AI-based CDSSs in the emergency care setting through the opinions of physicians on our newly developed, real-time AI-based CDSS, which alerts ED physicians by predicting aortic dissection based on numeric and text information from medical charts, by using the Unified Theory of Acceptance and Use of Technology (UTAUT; for quantitative evaluation) and the Consolidated Framework for Implementation Research (CFIR; for qualitative evaluation) frameworks. METHODS: This mixed methods study was performed from March to April 2021. Transitional year residents (n=6), emergency medicine residents (n=5), and emergency physicians (n=3) from two community, tertiary care hospitals in Japan were included. We first developed a real-time CDSS for predicting aortic dissection based on numeric and text information from medical charts (eg, chief complaints, medical history, vital signs) with natural language processing. This system was deployed on the internet, and the participants used the system with clinical vignettes of model cases. Participants were then involved in a mixed methods evaluation consisting of a UTAUT-based questionnaire with a 5-point Likert scale (quantitative) and a CFIR-based semistructured interview (qualitative). Cronbach α was calculated as a reliability estimate for UTAUT subconstructs. Interviews were sampled, transcribed, and analyzed using the MaxQDA software. The framework analysis approach was used during the study to determine the relevance of the CFIR constructs. RESULTS: All 14 participants completed the questionnaires and interviews. Quantitative analysis revealed generally positive responses for user acceptance with all scores above the neutral score of 3.0. In addition, the mixed methods analysis identified two significant barriers (System Performance, Compatibility) and two major facilitators (Evidence Strength, Design Quality) for implementation of AI-based CDSSs in emergency care settings. CONCLUSIONS: Our mixed methods evaluation based on theoretically grounded frameworks revealed the acceptance, barriers, and facilitators of implementation of AI-based CDSS. Although the concern of system failure and overtrusting of the system could be barriers to implementation, the locality of the system and designing an intuitive user interface could likely facilitate the use of optimal AI-based CDSS. Alleviating and resolving these factors should be key to achieving good user acceptance of AI-based CDSS. |
format | Online Article Text |
id | pubmed-9237770 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | JMIR Publications |
record_format | MEDLINE/PubMed |
spelling | pubmed-92377702022-06-29 Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation Fujimori, Ryo Liu, Keibun Soeno, Shoko Naraba, Hiromu Ogura, Kentaro Hara, Konan Sonoo, Tomohiro Ogura, Takayuki Nakamura, Kensuke Goto, Tadahiro JMIR Form Res Original Paper BACKGROUND: Despite the increasing availability of clinical decision support systems (CDSSs) and rising expectation for CDSSs based on artificial intelligence (AI), little is known about the acceptance of AI-based CDSS by physicians and its barriers and facilitators in emergency care settings. OBJECTIVE: We aimed to evaluate the acceptance, barriers, and facilitators to implementing AI-based CDSSs in the emergency care setting through the opinions of physicians on our newly developed, real-time AI-based CDSS, which alerts ED physicians by predicting aortic dissection based on numeric and text information from medical charts, by using the Unified Theory of Acceptance and Use of Technology (UTAUT; for quantitative evaluation) and the Consolidated Framework for Implementation Research (CFIR; for qualitative evaluation) frameworks. METHODS: This mixed methods study was performed from March to April 2021. Transitional year residents (n=6), emergency medicine residents (n=5), and emergency physicians (n=3) from two community, tertiary care hospitals in Japan were included. We first developed a real-time CDSS for predicting aortic dissection based on numeric and text information from medical charts (eg, chief complaints, medical history, vital signs) with natural language processing. This system was deployed on the internet, and the participants used the system with clinical vignettes of model cases. Participants were then involved in a mixed methods evaluation consisting of a UTAUT-based questionnaire with a 5-point Likert scale (quantitative) and a CFIR-based semistructured interview (qualitative). Cronbach α was calculated as a reliability estimate for UTAUT subconstructs. Interviews were sampled, transcribed, and analyzed using the MaxQDA software. The framework analysis approach was used during the study to determine the relevance of the CFIR constructs. RESULTS: All 14 participants completed the questionnaires and interviews. Quantitative analysis revealed generally positive responses for user acceptance with all scores above the neutral score of 3.0. In addition, the mixed methods analysis identified two significant barriers (System Performance, Compatibility) and two major facilitators (Evidence Strength, Design Quality) for implementation of AI-based CDSSs in emergency care settings. CONCLUSIONS: Our mixed methods evaluation based on theoretically grounded frameworks revealed the acceptance, barriers, and facilitators of implementation of AI-based CDSS. Although the concern of system failure and overtrusting of the system could be barriers to implementation, the locality of the system and designing an intuitive user interface could likely facilitate the use of optimal AI-based CDSS. Alleviating and resolving these factors should be key to achieving good user acceptance of AI-based CDSS. JMIR Publications 2022-06-13 /pmc/articles/PMC9237770/ /pubmed/35699995 http://dx.doi.org/10.2196/36501 Text en ©Ryo Fujimori, Keibun Liu, Shoko Soeno, Hiromu Naraba, Kentaro Ogura, Konan Hara, Tomohiro Sonoo, Takayuki Ogura, Kensuke Nakamura, Tadahiro Goto. Originally published in JMIR Formative Research (https://formative.jmir.org), 13.06.2022. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included. |
spellingShingle | Original Paper Fujimori, Ryo Liu, Keibun Soeno, Shoko Naraba, Hiromu Ogura, Kentaro Hara, Konan Sonoo, Tomohiro Ogura, Takayuki Nakamura, Kensuke Goto, Tadahiro Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation |
title | Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation |
title_full | Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation |
title_fullStr | Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation |
title_full_unstemmed | Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation |
title_short | Acceptance, Barriers, and Facilitators to Implementing Artificial Intelligence–Based Decision Support Systems in Emergency Departments: Quantitative and Qualitative Evaluation |
title_sort | acceptance, barriers, and facilitators to implementing artificial intelligence–based decision support systems in emergency departments: quantitative and qualitative evaluation |
topic | Original Paper |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9237770/ https://www.ncbi.nlm.nih.gov/pubmed/35699995 http://dx.doi.org/10.2196/36501 |
work_keys_str_mv | AT fujimoriryo acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation AT liukeibun acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation AT soenoshoko acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation AT narabahiromu acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation AT ogurakentaro acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation AT harakonan acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation AT sonootomohiro acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation AT oguratakayuki acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation AT nakamurakensuke acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation AT gototadahiro acceptancebarriersandfacilitatorstoimplementingartificialintelligencebaseddecisionsupportsystemsinemergencydepartmentsquantitativeandqualitativeevaluation |