Cargando…
Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study
BACKGROUND: Machine learning (ML)–driven clinical decision support (CDS) continues to draw wide interest and investment as a means of improving care quality and value, despite mixed real-world implementation outcomes. OBJECTIVE: This study aimed to explore the factors that influence the integration...
Autores principales: | , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
JMIR Publications
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10557008/ https://www.ncbi.nlm.nih.gov/pubmed/37733427 http://dx.doi.org/10.2196/43963 |
_version_ | 1785116994987622400 |
---|---|
author | Wang, Sabrina M Hogg, H D Jeffry Sangvai, Devdutta Patel, Manesh R Weissler, E Hope Kellogg, Katherine C Ratliff, William Balu, Suresh Sendak, Mark |
author_facet | Wang, Sabrina M Hogg, H D Jeffry Sangvai, Devdutta Patel, Manesh R Weissler, E Hope Kellogg, Katherine C Ratliff, William Balu, Suresh Sendak, Mark |
author_sort | Wang, Sabrina M |
collection | PubMed |
description | BACKGROUND: Machine learning (ML)–driven clinical decision support (CDS) continues to draw wide interest and investment as a means of improving care quality and value, despite mixed real-world implementation outcomes. OBJECTIVE: This study aimed to explore the factors that influence the integration of a peripheral arterial disease (PAD) identification algorithm to implement timely guideline-based care. METHODS: A total of 12 semistructured interviews were conducted with individuals from 3 stakeholder groups during the first 4 weeks of integration of an ML-driven CDS. The stakeholder groups included technical, administrative, and clinical members of the team interacting with the ML-driven CDS. The ML-driven CDS identified patients with a high probability of having PAD, and these patients were then reviewed by an interdisciplinary team that developed a recommended action plan and sent recommendations to the patient’s primary care provider. Pseudonymized transcripts were coded, and thematic analysis was conducted by a multidisciplinary research team. RESULTS: Three themes were identified: positive factors translating in silico performance to real-world efficacy, organizational factors and data structure factors affecting clinical impact, and potential challenges to advancing equity. Our study found that the factors that led to successful translation of in silico algorithm performance to real-world impact were largely nontechnical, given adequate efficacy in retrospective validation, including strong clinical leadership, trustworthy workflows, early consideration of end-user needs, and ensuring that the CDS addresses an actionable problem. Negative factors of integration included failure to incorporate the on-the-ground context, the lack of feedback loops, and data silos limiting the ML-driven CDS. The success criteria for each stakeholder group were also characterized to better understand how teams work together to integrate ML-driven CDS and to understand the varying needs across stakeholder groups. CONCLUSIONS: Longitudinal and multidisciplinary stakeholder engagement in the development and integration of ML-driven CDS underpins its effective translation into real-world care. Although previous studies have focused on the technical elements of ML-driven CDS, our study demonstrates the importance of including administrative and operational leaders as well as an early consideration of clinicians’ needs. Seeing how different stakeholder groups have this more holistic perspective also permits more effective detection of context-driven health care inequities, which are uncovered or exacerbated via ML-driven CDS integration through structural and organizational challenges. Many of the solutions to these inequities lie outside the scope of ML and require coordinated systematic solutions for mitigation to help reduce disparities in the care of patients with PAD. |
format | Online Article Text |
id | pubmed-10557008 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | JMIR Publications |
record_format | MEDLINE/PubMed |
spelling | pubmed-105570082023-10-07 Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study Wang, Sabrina M Hogg, H D Jeffry Sangvai, Devdutta Patel, Manesh R Weissler, E Hope Kellogg, Katherine C Ratliff, William Balu, Suresh Sendak, Mark JMIR Form Res Original Paper BACKGROUND: Machine learning (ML)–driven clinical decision support (CDS) continues to draw wide interest and investment as a means of improving care quality and value, despite mixed real-world implementation outcomes. OBJECTIVE: This study aimed to explore the factors that influence the integration of a peripheral arterial disease (PAD) identification algorithm to implement timely guideline-based care. METHODS: A total of 12 semistructured interviews were conducted with individuals from 3 stakeholder groups during the first 4 weeks of integration of an ML-driven CDS. The stakeholder groups included technical, administrative, and clinical members of the team interacting with the ML-driven CDS. The ML-driven CDS identified patients with a high probability of having PAD, and these patients were then reviewed by an interdisciplinary team that developed a recommended action plan and sent recommendations to the patient’s primary care provider. Pseudonymized transcripts were coded, and thematic analysis was conducted by a multidisciplinary research team. RESULTS: Three themes were identified: positive factors translating in silico performance to real-world efficacy, organizational factors and data structure factors affecting clinical impact, and potential challenges to advancing equity. Our study found that the factors that led to successful translation of in silico algorithm performance to real-world impact were largely nontechnical, given adequate efficacy in retrospective validation, including strong clinical leadership, trustworthy workflows, early consideration of end-user needs, and ensuring that the CDS addresses an actionable problem. Negative factors of integration included failure to incorporate the on-the-ground context, the lack of feedback loops, and data silos limiting the ML-driven CDS. The success criteria for each stakeholder group were also characterized to better understand how teams work together to integrate ML-driven CDS and to understand the varying needs across stakeholder groups. CONCLUSIONS: Longitudinal and multidisciplinary stakeholder engagement in the development and integration of ML-driven CDS underpins its effective translation into real-world care. Although previous studies have focused on the technical elements of ML-driven CDS, our study demonstrates the importance of including administrative and operational leaders as well as an early consideration of clinicians’ needs. Seeing how different stakeholder groups have this more holistic perspective also permits more effective detection of context-driven health care inequities, which are uncovered or exacerbated via ML-driven CDS integration through structural and organizational challenges. Many of the solutions to these inequities lie outside the scope of ML and require coordinated systematic solutions for mitigation to help reduce disparities in the care of patients with PAD. JMIR Publications 2023-09-21 /pmc/articles/PMC10557008/ /pubmed/37733427 http://dx.doi.org/10.2196/43963 Text en ©Sabrina M Wang, H D Jeffry Hogg, Devdutta Sangvai, Manesh R Patel, E Hope Weissler, Katherine C Kellogg, William Ratliff, Suresh Balu, Mark Sendak. Originally published in JMIR Formative Research (https://formative.jmir.org), 21.09.2023. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included. |
spellingShingle | Original Paper Wang, Sabrina M Hogg, H D Jeffry Sangvai, Devdutta Patel, Manesh R Weissler, E Hope Kellogg, Katherine C Ratliff, William Balu, Suresh Sendak, Mark Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study |
title | Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study |
title_full | Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study |
title_fullStr | Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study |
title_full_unstemmed | Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study |
title_short | Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study |
title_sort | development and integration of machine learning algorithm to identify peripheral arterial disease: multistakeholder qualitative study |
topic | Original Paper |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10557008/ https://www.ncbi.nlm.nih.gov/pubmed/37733427 http://dx.doi.org/10.2196/43963 |
work_keys_str_mv | AT wangsabrinam developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy AT hogghdjeffry developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy AT sangvaidevdutta developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy AT patelmaneshr developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy AT weisslerehope developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy AT kelloggkatherinec developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy AT ratliffwilliam developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy AT balusuresh developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy AT sendakmark developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy |