Cargando…

Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study

BACKGROUND: Machine learning (ML)–driven clinical decision support (CDS) continues to draw wide interest and investment as a means of improving care quality and value, despite mixed real-world implementation outcomes. OBJECTIVE: This study aimed to explore the factors that influence the integration...

Descripción completa

Detalles Bibliográficos
Autores principales: Wang, Sabrina M, Hogg, H D Jeffry, Sangvai, Devdutta, Patel, Manesh R, Weissler, E Hope, Kellogg, Katherine C, Ratliff, William, Balu, Suresh, Sendak, Mark
Formato: Online Artículo Texto
Lenguaje:English
Publicado: JMIR Publications 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10557008/
https://www.ncbi.nlm.nih.gov/pubmed/37733427
http://dx.doi.org/10.2196/43963
_version_ 1785116994987622400
author Wang, Sabrina M
Hogg, H D Jeffry
Sangvai, Devdutta
Patel, Manesh R
Weissler, E Hope
Kellogg, Katherine C
Ratliff, William
Balu, Suresh
Sendak, Mark
author_facet Wang, Sabrina M
Hogg, H D Jeffry
Sangvai, Devdutta
Patel, Manesh R
Weissler, E Hope
Kellogg, Katherine C
Ratliff, William
Balu, Suresh
Sendak, Mark
author_sort Wang, Sabrina M
collection PubMed
description BACKGROUND: Machine learning (ML)–driven clinical decision support (CDS) continues to draw wide interest and investment as a means of improving care quality and value, despite mixed real-world implementation outcomes. OBJECTIVE: This study aimed to explore the factors that influence the integration of a peripheral arterial disease (PAD) identification algorithm to implement timely guideline-based care. METHODS: A total of 12 semistructured interviews were conducted with individuals from 3 stakeholder groups during the first 4 weeks of integration of an ML-driven CDS. The stakeholder groups included technical, administrative, and clinical members of the team interacting with the ML-driven CDS. The ML-driven CDS identified patients with a high probability of having PAD, and these patients were then reviewed by an interdisciplinary team that developed a recommended action plan and sent recommendations to the patient’s primary care provider. Pseudonymized transcripts were coded, and thematic analysis was conducted by a multidisciplinary research team. RESULTS: Three themes were identified: positive factors translating in silico performance to real-world efficacy, organizational factors and data structure factors affecting clinical impact, and potential challenges to advancing equity. Our study found that the factors that led to successful translation of in silico algorithm performance to real-world impact were largely nontechnical, given adequate efficacy in retrospective validation, including strong clinical leadership, trustworthy workflows, early consideration of end-user needs, and ensuring that the CDS addresses an actionable problem. Negative factors of integration included failure to incorporate the on-the-ground context, the lack of feedback loops, and data silos limiting the ML-driven CDS. The success criteria for each stakeholder group were also characterized to better understand how teams work together to integrate ML-driven CDS and to understand the varying needs across stakeholder groups. CONCLUSIONS: Longitudinal and multidisciplinary stakeholder engagement in the development and integration of ML-driven CDS underpins its effective translation into real-world care. Although previous studies have focused on the technical elements of ML-driven CDS, our study demonstrates the importance of including administrative and operational leaders as well as an early consideration of clinicians’ needs. Seeing how different stakeholder groups have this more holistic perspective also permits more effective detection of context-driven health care inequities, which are uncovered or exacerbated via ML-driven CDS integration through structural and organizational challenges. Many of the solutions to these inequities lie outside the scope of ML and require coordinated systematic solutions for mitigation to help reduce disparities in the care of patients with PAD.
format Online
Article
Text
id pubmed-10557008
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher JMIR Publications
record_format MEDLINE/PubMed
spelling pubmed-105570082023-10-07 Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study Wang, Sabrina M Hogg, H D Jeffry Sangvai, Devdutta Patel, Manesh R Weissler, E Hope Kellogg, Katherine C Ratliff, William Balu, Suresh Sendak, Mark JMIR Form Res Original Paper BACKGROUND: Machine learning (ML)–driven clinical decision support (CDS) continues to draw wide interest and investment as a means of improving care quality and value, despite mixed real-world implementation outcomes. OBJECTIVE: This study aimed to explore the factors that influence the integration of a peripheral arterial disease (PAD) identification algorithm to implement timely guideline-based care. METHODS: A total of 12 semistructured interviews were conducted with individuals from 3 stakeholder groups during the first 4 weeks of integration of an ML-driven CDS. The stakeholder groups included technical, administrative, and clinical members of the team interacting with the ML-driven CDS. The ML-driven CDS identified patients with a high probability of having PAD, and these patients were then reviewed by an interdisciplinary team that developed a recommended action plan and sent recommendations to the patient’s primary care provider. Pseudonymized transcripts were coded, and thematic analysis was conducted by a multidisciplinary research team. RESULTS: Three themes were identified: positive factors translating in silico performance to real-world efficacy, organizational factors and data structure factors affecting clinical impact, and potential challenges to advancing equity. Our study found that the factors that led to successful translation of in silico algorithm performance to real-world impact were largely nontechnical, given adequate efficacy in retrospective validation, including strong clinical leadership, trustworthy workflows, early consideration of end-user needs, and ensuring that the CDS addresses an actionable problem. Negative factors of integration included failure to incorporate the on-the-ground context, the lack of feedback loops, and data silos limiting the ML-driven CDS. The success criteria for each stakeholder group were also characterized to better understand how teams work together to integrate ML-driven CDS and to understand the varying needs across stakeholder groups. CONCLUSIONS: Longitudinal and multidisciplinary stakeholder engagement in the development and integration of ML-driven CDS underpins its effective translation into real-world care. Although previous studies have focused on the technical elements of ML-driven CDS, our study demonstrates the importance of including administrative and operational leaders as well as an early consideration of clinicians’ needs. Seeing how different stakeholder groups have this more holistic perspective also permits more effective detection of context-driven health care inequities, which are uncovered or exacerbated via ML-driven CDS integration through structural and organizational challenges. Many of the solutions to these inequities lie outside the scope of ML and require coordinated systematic solutions for mitigation to help reduce disparities in the care of patients with PAD. JMIR Publications 2023-09-21 /pmc/articles/PMC10557008/ /pubmed/37733427 http://dx.doi.org/10.2196/43963 Text en ©Sabrina M Wang, H D Jeffry Hogg, Devdutta Sangvai, Manesh R Patel, E Hope Weissler, Katherine C Kellogg, William Ratliff, Suresh Balu, Mark Sendak. Originally published in JMIR Formative Research (https://formative.jmir.org), 21.09.2023. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included.
spellingShingle Original Paper
Wang, Sabrina M
Hogg, H D Jeffry
Sangvai, Devdutta
Patel, Manesh R
Weissler, E Hope
Kellogg, Katherine C
Ratliff, William
Balu, Suresh
Sendak, Mark
Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study
title Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study
title_full Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study
title_fullStr Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study
title_full_unstemmed Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study
title_short Development and Integration of Machine Learning Algorithm to Identify Peripheral Arterial Disease: Multistakeholder Qualitative Study
title_sort development and integration of machine learning algorithm to identify peripheral arterial disease: multistakeholder qualitative study
topic Original Paper
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10557008/
https://www.ncbi.nlm.nih.gov/pubmed/37733427
http://dx.doi.org/10.2196/43963
work_keys_str_mv AT wangsabrinam developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy
AT hogghdjeffry developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy
AT sangvaidevdutta developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy
AT patelmaneshr developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy
AT weisslerehope developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy
AT kelloggkatherinec developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy
AT ratliffwilliam developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy
AT balusuresh developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy
AT sendakmark developmentandintegrationofmachinelearningalgorithmtoidentifyperipheralarterialdiseasemultistakeholderqualitativestudy