Cargando…
Implementing a Machine Learning Screening Tool for Malnutrition: Insights From Qualitative Research Applicable to Other Machine Learning–Based Clinical Decision Support Systems
BACKGROUND: Machine learning (ML)–based clinical decision support systems (CDSS) are popular in clinical practice settings but are often criticized for being limited in usability, interpretability, and effectiveness. Evaluating the implementation of ML-based CDSS is critical to ensure CDSS is accept...
Autores principales: | , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
JMIR Publications
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10375393/ https://www.ncbi.nlm.nih.gov/pubmed/37440303 http://dx.doi.org/10.2196/42262 |
_version_ | 1785079028039811072 |
---|---|
author | Besculides, Melanie Mazumdar, Madhu Phlegar, Sydney Freeman, Robert Wilson, Sara Joshi, Himanshu Kia, Arash Gorbenko, Ksenia |
author_facet | Besculides, Melanie Mazumdar, Madhu Phlegar, Sydney Freeman, Robert Wilson, Sara Joshi, Himanshu Kia, Arash Gorbenko, Ksenia |
author_sort | Besculides, Melanie |
collection | PubMed |
description | BACKGROUND: Machine learning (ML)–based clinical decision support systems (CDSS) are popular in clinical practice settings but are often criticized for being limited in usability, interpretability, and effectiveness. Evaluating the implementation of ML-based CDSS is critical to ensure CDSS is acceptable and useful to clinicians and helps them deliver high-quality health care. Malnutrition is a common and underdiagnosed condition among hospital patients, which can have serious adverse impacts. Early identification and treatment of malnutrition are important. OBJECTIVE: This study aims to evaluate the implementation of an ML tool, Malnutrition Universal Screening Tool (MUST)–Plus, that predicts hospital patients at high risk for malnutrition and identify best implementation practices applicable to this and other ML-based CDSS. METHODS: We conducted a qualitative postimplementation evaluation using in-depth interviews with registered dietitians (RDs) who use MUST-Plus output in their everyday work. After coding the data, we mapped emergent themes onto select domains of the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework. RESULTS: We interviewed 17 of the 24 RDs approached (71%), representing 37% of those who use MUST-Plus output. Several themes emerged: (1) enhancements to the tool were made to improve accuracy and usability; (2) MUST-Plus helped identify patients that would not otherwise be seen; perceived usefulness was highest in the original site; (3) perceived accuracy varied by respondent and site; (4) RDs valued autonomy in prioritizing patients; (5) depth of tool understanding varied by hospital and level; (6) MUST-Plus was integrated into workflows and electronic health records; and (7) RDs expressed a desire to eventually have 1 automated screener. CONCLUSIONS: Our findings suggest that continuous involvement of stakeholders at new sites given staff turnover is vital to ensure buy-in. Qualitative research can help identify the potential bias of ML tools and should be widely used to ensure health equity. Ongoing collaboration among CDSS developers, data scientists, and clinical providers may help refine CDSS for optimal use and improve the acceptability of CDSS in the clinical context. |
format | Online Article Text |
id | pubmed-10375393 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | JMIR Publications |
record_format | MEDLINE/PubMed |
spelling | pubmed-103753932023-07-29 Implementing a Machine Learning Screening Tool for Malnutrition: Insights From Qualitative Research Applicable to Other Machine Learning–Based Clinical Decision Support Systems Besculides, Melanie Mazumdar, Madhu Phlegar, Sydney Freeman, Robert Wilson, Sara Joshi, Himanshu Kia, Arash Gorbenko, Ksenia JMIR Form Res Original Paper BACKGROUND: Machine learning (ML)–based clinical decision support systems (CDSS) are popular in clinical practice settings but are often criticized for being limited in usability, interpretability, and effectiveness. Evaluating the implementation of ML-based CDSS is critical to ensure CDSS is acceptable and useful to clinicians and helps them deliver high-quality health care. Malnutrition is a common and underdiagnosed condition among hospital patients, which can have serious adverse impacts. Early identification and treatment of malnutrition are important. OBJECTIVE: This study aims to evaluate the implementation of an ML tool, Malnutrition Universal Screening Tool (MUST)–Plus, that predicts hospital patients at high risk for malnutrition and identify best implementation practices applicable to this and other ML-based CDSS. METHODS: We conducted a qualitative postimplementation evaluation using in-depth interviews with registered dietitians (RDs) who use MUST-Plus output in their everyday work. After coding the data, we mapped emergent themes onto select domains of the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework. RESULTS: We interviewed 17 of the 24 RDs approached (71%), representing 37% of those who use MUST-Plus output. Several themes emerged: (1) enhancements to the tool were made to improve accuracy and usability; (2) MUST-Plus helped identify patients that would not otherwise be seen; perceived usefulness was highest in the original site; (3) perceived accuracy varied by respondent and site; (4) RDs valued autonomy in prioritizing patients; (5) depth of tool understanding varied by hospital and level; (6) MUST-Plus was integrated into workflows and electronic health records; and (7) RDs expressed a desire to eventually have 1 automated screener. CONCLUSIONS: Our findings suggest that continuous involvement of stakeholders at new sites given staff turnover is vital to ensure buy-in. Qualitative research can help identify the potential bias of ML tools and should be widely used to ensure health equity. Ongoing collaboration among CDSS developers, data scientists, and clinical providers may help refine CDSS for optimal use and improve the acceptability of CDSS in the clinical context. JMIR Publications 2023-07-13 /pmc/articles/PMC10375393/ /pubmed/37440303 http://dx.doi.org/10.2196/42262 Text en ©Melanie Besculides, Madhu Mazumdar, Sydney Phlegar, Robert Freeman, Sara Wilson, Himanshu Joshi, Arash Kia, Ksenia Gorbenko. Originally published in JMIR Formative Research (https://formative.jmir.org), 13.07.2023. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license information must be included. |
spellingShingle | Original Paper Besculides, Melanie Mazumdar, Madhu Phlegar, Sydney Freeman, Robert Wilson, Sara Joshi, Himanshu Kia, Arash Gorbenko, Ksenia Implementing a Machine Learning Screening Tool for Malnutrition: Insights From Qualitative Research Applicable to Other Machine Learning–Based Clinical Decision Support Systems |
title | Implementing a Machine Learning Screening Tool for Malnutrition: Insights From Qualitative Research Applicable to Other Machine Learning–Based Clinical Decision Support Systems |
title_full | Implementing a Machine Learning Screening Tool for Malnutrition: Insights From Qualitative Research Applicable to Other Machine Learning–Based Clinical Decision Support Systems |
title_fullStr | Implementing a Machine Learning Screening Tool for Malnutrition: Insights From Qualitative Research Applicable to Other Machine Learning–Based Clinical Decision Support Systems |
title_full_unstemmed | Implementing a Machine Learning Screening Tool for Malnutrition: Insights From Qualitative Research Applicable to Other Machine Learning–Based Clinical Decision Support Systems |
title_short | Implementing a Machine Learning Screening Tool for Malnutrition: Insights From Qualitative Research Applicable to Other Machine Learning–Based Clinical Decision Support Systems |
title_sort | implementing a machine learning screening tool for malnutrition: insights from qualitative research applicable to other machine learning–based clinical decision support systems |
topic | Original Paper |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10375393/ https://www.ncbi.nlm.nih.gov/pubmed/37440303 http://dx.doi.org/10.2196/42262 |
work_keys_str_mv | AT besculidesmelanie implementingamachinelearningscreeningtoolformalnutritioninsightsfromqualitativeresearchapplicabletoothermachinelearningbasedclinicaldecisionsupportsystems AT mazumdarmadhu implementingamachinelearningscreeningtoolformalnutritioninsightsfromqualitativeresearchapplicabletoothermachinelearningbasedclinicaldecisionsupportsystems AT phlegarsydney implementingamachinelearningscreeningtoolformalnutritioninsightsfromqualitativeresearchapplicabletoothermachinelearningbasedclinicaldecisionsupportsystems AT freemanrobert implementingamachinelearningscreeningtoolformalnutritioninsightsfromqualitativeresearchapplicabletoothermachinelearningbasedclinicaldecisionsupportsystems AT wilsonsara implementingamachinelearningscreeningtoolformalnutritioninsightsfromqualitativeresearchapplicabletoothermachinelearningbasedclinicaldecisionsupportsystems AT joshihimanshu implementingamachinelearningscreeningtoolformalnutritioninsightsfromqualitativeresearchapplicabletoothermachinelearningbasedclinicaldecisionsupportsystems AT kiaarash implementingamachinelearningscreeningtoolformalnutritioninsightsfromqualitativeresearchapplicabletoothermachinelearningbasedclinicaldecisionsupportsystems AT gorbenkoksenia implementingamachinelearningscreeningtoolformalnutritioninsightsfromqualitativeresearchapplicabletoothermachinelearningbasedclinicaldecisionsupportsystems |