Cargando…

Implementation approaches and barriers for rule-based and machine learning-based sepsis risk prediction tools: a qualitative study

OBJECTIVE: Many options are currently available for sepsis surveillance clinical decision support (CDS) from electronic medical record (EMR) vendors, third party, and homegrown models drawing on rule-based (RB) and machine learning (ML) algorithms. This study explores sepsis CDS implementation from...

Descripción completa

Detalles Bibliográficos
Autores principales: Joshi, Mugdha, Mecklai, Keizra, Rozenblum, Ronen, Samal, Lipika
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Oxford University Press 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9030109/
https://www.ncbi.nlm.nih.gov/pubmed/35474719
http://dx.doi.org/10.1093/jamiaopen/ooac022
_version_ 1784692064063062016
author Joshi, Mugdha
Mecklai, Keizra
Rozenblum, Ronen
Samal, Lipika
author_facet Joshi, Mugdha
Mecklai, Keizra
Rozenblum, Ronen
Samal, Lipika
author_sort Joshi, Mugdha
collection PubMed
description OBJECTIVE: Many options are currently available for sepsis surveillance clinical decision support (CDS) from electronic medical record (EMR) vendors, third party, and homegrown models drawing on rule-based (RB) and machine learning (ML) algorithms. This study explores sepsis CDS implementation from the perspective of implementation leads by describing the motivations, tool choices, and implementation experiences of a diverse group of implementers. MATERIALS AND METHODS: Semi-structured interviews were conducted with and a questionnaire was administered to 21 hospital leaders overseeing CDS implementation at 15 US medical centers. Participants were recruited via convenience sampling. Responses were coded by 2 coders with consensus approach and inductively analyzed for themes. RESULTS: Use of sepsis CDS is motivated in part by quality metrics for sepsis patients. Choice of tool is driven by ease of integration, customization capability, and perceived predictive potential. Implementation processes for these CDS tools are complex, time-consuming, interdisciplinary undertakings resulting in heterogeneous choice of tools and workflow integration. To improve clinician acceptance, implementers addressed both optimization of the alerts as well as clinician understanding and buy in. More distrust and confusion was reported for ML models, as compared to RB models. Respondents described a variety of approaches to overcome implementation barriers; these approaches related to alert firing, content, integration, and buy-in. DISCUSSION: While there are shared socio-technical challenges of implementing CDS for both RB and ML models, attention to user education, support, expectation management, and dissemination of effective practices may improve feasibility and effectiveness of ML models in quality improvement efforts. CONCLUSION: Further implementation science research is needed to determine real world efficacy of these tools. Clinician acceptance is a significant barrier to sepsis CDS implementation. Successful implementation of less clinically intuitive ML models may require additional attention to user confusion and distrust.
format Online
Article
Text
id pubmed-9030109
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Oxford University Press
record_format MEDLINE/PubMed
spelling pubmed-90301092022-04-25 Implementation approaches and barriers for rule-based and machine learning-based sepsis risk prediction tools: a qualitative study Joshi, Mugdha Mecklai, Keizra Rozenblum, Ronen Samal, Lipika JAMIA Open Research and Applications OBJECTIVE: Many options are currently available for sepsis surveillance clinical decision support (CDS) from electronic medical record (EMR) vendors, third party, and homegrown models drawing on rule-based (RB) and machine learning (ML) algorithms. This study explores sepsis CDS implementation from the perspective of implementation leads by describing the motivations, tool choices, and implementation experiences of a diverse group of implementers. MATERIALS AND METHODS: Semi-structured interviews were conducted with and a questionnaire was administered to 21 hospital leaders overseeing CDS implementation at 15 US medical centers. Participants were recruited via convenience sampling. Responses were coded by 2 coders with consensus approach and inductively analyzed for themes. RESULTS: Use of sepsis CDS is motivated in part by quality metrics for sepsis patients. Choice of tool is driven by ease of integration, customization capability, and perceived predictive potential. Implementation processes for these CDS tools are complex, time-consuming, interdisciplinary undertakings resulting in heterogeneous choice of tools and workflow integration. To improve clinician acceptance, implementers addressed both optimization of the alerts as well as clinician understanding and buy in. More distrust and confusion was reported for ML models, as compared to RB models. Respondents described a variety of approaches to overcome implementation barriers; these approaches related to alert firing, content, integration, and buy-in. DISCUSSION: While there are shared socio-technical challenges of implementing CDS for both RB and ML models, attention to user education, support, expectation management, and dissemination of effective practices may improve feasibility and effectiveness of ML models in quality improvement efforts. CONCLUSION: Further implementation science research is needed to determine real world efficacy of these tools. Clinician acceptance is a significant barrier to sepsis CDS implementation. Successful implementation of less clinically intuitive ML models may require additional attention to user confusion and distrust. Oxford University Press 2022-04-18 /pmc/articles/PMC9030109/ /pubmed/35474719 http://dx.doi.org/10.1093/jamiaopen/ooac022 Text en © The Author(s) 2022. Published by Oxford University Press on behalf of the American Medical Informatics Association. https://creativecommons.org/licenses/by-nc/4.0/This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial License (https://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com
spellingShingle Research and Applications
Joshi, Mugdha
Mecklai, Keizra
Rozenblum, Ronen
Samal, Lipika
Implementation approaches and barriers for rule-based and machine learning-based sepsis risk prediction tools: a qualitative study
title Implementation approaches and barriers for rule-based and machine learning-based sepsis risk prediction tools: a qualitative study
title_full Implementation approaches and barriers for rule-based and machine learning-based sepsis risk prediction tools: a qualitative study
title_fullStr Implementation approaches and barriers for rule-based and machine learning-based sepsis risk prediction tools: a qualitative study
title_full_unstemmed Implementation approaches and barriers for rule-based and machine learning-based sepsis risk prediction tools: a qualitative study
title_short Implementation approaches and barriers for rule-based and machine learning-based sepsis risk prediction tools: a qualitative study
title_sort implementation approaches and barriers for rule-based and machine learning-based sepsis risk prediction tools: a qualitative study
topic Research and Applications
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9030109/
https://www.ncbi.nlm.nih.gov/pubmed/35474719
http://dx.doi.org/10.1093/jamiaopen/ooac022
work_keys_str_mv AT joshimugdha implementationapproachesandbarriersforrulebasedandmachinelearningbasedsepsisriskpredictiontoolsaqualitativestudy
AT mecklaikeizra implementationapproachesandbarriersforrulebasedandmachinelearningbasedsepsisriskpredictiontoolsaqualitativestudy
AT rozenblumronen implementationapproachesandbarriersforrulebasedandmachinelearningbasedsepsisriskpredictiontoolsaqualitativestudy
AT samallipika implementationapproachesandbarriersforrulebasedandmachinelearningbasedsepsisriskpredictiontoolsaqualitativestudy