Cargando…
Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults
BACKGROUND: The clinical narrative in electronic health records (EHRs) carries valuable information for predictive analytics; however, its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focuse...
Autores principales: | , , , , , , , , , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
JMIR Publications
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10160938/ https://www.ncbi.nlm.nih.gov/pubmed/37079367 http://dx.doi.org/10.2196/44977 |
_version_ | 1785037384084094976 |
---|---|
author | Afshar, Majid Adelaine, Sabrina Resnik, Felice Mundt, Marlon P Long, John Leaf, Margaret Ampian, Theodore Wills, Graham J Schnapp, Benjamin Chao, Michael Brown, Randy Joyce, Cara Sharma, Brihat Dligach, Dmitriy Burnside, Elizabeth S Mahoney, Jane Churpek, Matthew M Patterson, Brian W Liao, Frank |
author_facet | Afshar, Majid Adelaine, Sabrina Resnik, Felice Mundt, Marlon P Long, John Leaf, Margaret Ampian, Theodore Wills, Graham J Schnapp, Benjamin Chao, Michael Brown, Randy Joyce, Cara Sharma, Brihat Dligach, Dmitriy Burnside, Elizabeth S Mahoney, Jane Churpek, Matthew M Patterson, Brian W Liao, Frank |
author_sort | Afshar, Majid |
collection | PubMed |
description | BACKGROUND: The clinical narrative in electronic health records (EHRs) carries valuable information for predictive analytics; however, its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focused on data warehouse applications for retrospective research efforts. There remains a paucity of evidence for implementing NLP pipelines at the bedside for health care delivery. OBJECTIVE: We aimed to detail a hospital-wide, operational pipeline to implement a real-time NLP-driven CDS tool and describe a protocol for an implementation framework with a user-centered design of the CDS tool. METHODS: The pipeline integrated a previously trained open-source convolutional neural network model for screening opioid misuse that leveraged EHR notes mapped to standardized medical vocabularies in the Unified Medical Language System. A sample of 100 adult encounters were reviewed by a physician informaticist for silent testing of the deep learning algorithm before deployment. An end user interview survey was developed to examine the user acceptability of a best practice alert (BPA) to provide the screening results with recommendations. The planned implementation also included a human-centered design with user feedback on the BPA, an implementation framework with cost-effectiveness, and a noninferiority patient outcome analysis plan. RESULTS: The pipeline was a reproducible workflow with a shared pseudocode for a cloud service to ingest, process, and store clinical notes as Health Level 7 messages from a major EHR vendor in an elastic cloud computing environment. Feature engineering of the notes used an open-source NLP engine, and the features were fed into the deep learning algorithm, with the results returned as a BPA in the EHR. On-site silent testing of the deep learning algorithm demonstrated a sensitivity of 93% (95% CI 66%-99%) and specificity of 92% (95% CI 84%-96%), similar to published validation studies. Before deployment, approvals were received across hospital committees for inpatient operations. Five interviews were conducted; they informed the development of an educational flyer and further modified the BPA to exclude certain patients and allow the refusal of recommendations. The longest delay in pipeline development was because of cybersecurity approvals, especially because of the exchange of protected health information between the Microsoft (Microsoft Corp) and Epic (Epic Systems Corp) cloud vendors. In silent testing, the resultant pipeline provided a BPA to the bedside within minutes of a provider entering a note in the EHR. CONCLUSIONS: The components of the real-time NLP pipeline were detailed with open-source tools and pseudocode for other health systems to benchmark. The deployment of medical artificial intelligence systems in routine clinical care presents an important yet unfulfilled opportunity, and our protocol aimed to close the gap in the implementation of artificial intelligence–driven CDS. TRIAL REGISTRATION: ClinicalTrials.gov NCT05745480; https://www.clinicaltrials.gov/ct2/show/NCT05745480 |
format | Online Article Text |
id | pubmed-10160938 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | JMIR Publications |
record_format | MEDLINE/PubMed |
spelling | pubmed-101609382023-05-06 Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults Afshar, Majid Adelaine, Sabrina Resnik, Felice Mundt, Marlon P Long, John Leaf, Margaret Ampian, Theodore Wills, Graham J Schnapp, Benjamin Chao, Michael Brown, Randy Joyce, Cara Sharma, Brihat Dligach, Dmitriy Burnside, Elizabeth S Mahoney, Jane Churpek, Matthew M Patterson, Brian W Liao, Frank JMIR Med Inform Original Paper BACKGROUND: The clinical narrative in electronic health records (EHRs) carries valuable information for predictive analytics; however, its free-text form is difficult to mine and analyze for clinical decision support (CDS). Large-scale clinical natural language processing (NLP) pipelines have focused on data warehouse applications for retrospective research efforts. There remains a paucity of evidence for implementing NLP pipelines at the bedside for health care delivery. OBJECTIVE: We aimed to detail a hospital-wide, operational pipeline to implement a real-time NLP-driven CDS tool and describe a protocol for an implementation framework with a user-centered design of the CDS tool. METHODS: The pipeline integrated a previously trained open-source convolutional neural network model for screening opioid misuse that leveraged EHR notes mapped to standardized medical vocabularies in the Unified Medical Language System. A sample of 100 adult encounters were reviewed by a physician informaticist for silent testing of the deep learning algorithm before deployment. An end user interview survey was developed to examine the user acceptability of a best practice alert (BPA) to provide the screening results with recommendations. The planned implementation also included a human-centered design with user feedback on the BPA, an implementation framework with cost-effectiveness, and a noninferiority patient outcome analysis plan. RESULTS: The pipeline was a reproducible workflow with a shared pseudocode for a cloud service to ingest, process, and store clinical notes as Health Level 7 messages from a major EHR vendor in an elastic cloud computing environment. Feature engineering of the notes used an open-source NLP engine, and the features were fed into the deep learning algorithm, with the results returned as a BPA in the EHR. On-site silent testing of the deep learning algorithm demonstrated a sensitivity of 93% (95% CI 66%-99%) and specificity of 92% (95% CI 84%-96%), similar to published validation studies. Before deployment, approvals were received across hospital committees for inpatient operations. Five interviews were conducted; they informed the development of an educational flyer and further modified the BPA to exclude certain patients and allow the refusal of recommendations. The longest delay in pipeline development was because of cybersecurity approvals, especially because of the exchange of protected health information between the Microsoft (Microsoft Corp) and Epic (Epic Systems Corp) cloud vendors. In silent testing, the resultant pipeline provided a BPA to the bedside within minutes of a provider entering a note in the EHR. CONCLUSIONS: The components of the real-time NLP pipeline were detailed with open-source tools and pseudocode for other health systems to benchmark. The deployment of medical artificial intelligence systems in routine clinical care presents an important yet unfulfilled opportunity, and our protocol aimed to close the gap in the implementation of artificial intelligence–driven CDS. TRIAL REGISTRATION: ClinicalTrials.gov NCT05745480; https://www.clinicaltrials.gov/ct2/show/NCT05745480 JMIR Publications 2023-04-20 /pmc/articles/PMC10160938/ /pubmed/37079367 http://dx.doi.org/10.2196/44977 Text en ©Majid Afshar, Sabrina Adelaine, Felice Resnik, Marlon P Mundt, John Long, Margaret Leaf, Theodore Ampian, Graham J Wills, Benjamin Schnapp, Michael Chao, Randy Brown, Cara Joyce, Brihat Sharma, Dmitriy Dligach, Elizabeth S Burnside, Jane Mahoney, Matthew M Churpek, Brian W Patterson, Frank Liao. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 20.04.2023. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included. |
spellingShingle | Original Paper Afshar, Majid Adelaine, Sabrina Resnik, Felice Mundt, Marlon P Long, John Leaf, Margaret Ampian, Theodore Wills, Graham J Schnapp, Benjamin Chao, Michael Brown, Randy Joyce, Cara Sharma, Brihat Dligach, Dmitriy Burnside, Elizabeth S Mahoney, Jane Churpek, Matthew M Patterson, Brian W Liao, Frank Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults |
title | Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults |
title_full | Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults |
title_fullStr | Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults |
title_full_unstemmed | Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults |
title_short | Deployment of Real-time Natural Language Processing and Deep Learning Clinical Decision Support in the Electronic Health Record: Pipeline Implementation for an Opioid Misuse Screener in Hospitalized Adults |
title_sort | deployment of real-time natural language processing and deep learning clinical decision support in the electronic health record: pipeline implementation for an opioid misuse screener in hospitalized adults |
topic | Original Paper |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10160938/ https://www.ncbi.nlm.nih.gov/pubmed/37079367 http://dx.doi.org/10.2196/44977 |
work_keys_str_mv | AT afsharmajid deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT adelainesabrina deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT resnikfelice deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT mundtmarlonp deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT longjohn deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT leafmargaret deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT ampiantheodore deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT willsgrahamj deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT schnappbenjamin deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT chaomichael deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT brownrandy deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT joycecara deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT sharmabrihat deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT dligachdmitriy deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT burnsideelizabeths deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT mahoneyjane deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT churpekmatthewm deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT pattersonbrianw deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults AT liaofrank deploymentofrealtimenaturallanguageprocessinganddeeplearningclinicaldecisionsupportintheelectronichealthrecordpipelineimplementationforanopioidmisusescreenerinhospitalizedadults |