Cargando…
Event Prediction Model Considering Time and Input Error Using Electronic Medical Records in the Intensive Care Unit: Retrospective Study
BACKGROUND: In the era of artificial intelligence, event prediction models are abundant. However, considering the limitation of the electronic medical record–based model, including the temporally skewed prediction and the record itself, these models could be delayed or could yield errors. OBJECTIVE:...
Autores principales: | , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
JMIR Publications
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8603167/ https://www.ncbi.nlm.nih.gov/pubmed/34734837 http://dx.doi.org/10.2196/26426 |
_version_ | 1784601716470054912 |
---|---|
author | Sung, MinDong Hahn, Sangchul Han, Chang Hoon Lee, Jung Mo Lee, Jayoung Yoo, Jinkyu Heo, Jay Kim, Young Sam Chung, Kyung Soo |
author_facet | Sung, MinDong Hahn, Sangchul Han, Chang Hoon Lee, Jung Mo Lee, Jayoung Yoo, Jinkyu Heo, Jay Kim, Young Sam Chung, Kyung Soo |
author_sort | Sung, MinDong |
collection | PubMed |
description | BACKGROUND: In the era of artificial intelligence, event prediction models are abundant. However, considering the limitation of the electronic medical record–based model, including the temporally skewed prediction and the record itself, these models could be delayed or could yield errors. OBJECTIVE: In this study, we aim to develop multiple event prediction models in intensive care units to overcome their temporal skewness and evaluate their robustness against delayed and erroneous input. METHODS: A total of 21,738 patients were included in the development cohort. Three events—death, sepsis, and acute kidney injury—were predicted. To overcome the temporal skewness, we developed three models for each event, which predicted the events in advance of three prespecified timepoints. Additionally, to evaluate the robustness against input error and delays, we added simulated errors and delayed input and calculated changes in the area under the receiver operating characteristic curve (AUROC) values. RESULTS: Most of the AUROC and area under the precision-recall curve values of each model were higher than those of the conventional scores, as well as other machine learning models previously used. In the error input experiment, except for our proposed model, an increase in the noise added to the model lowered the resulting AUROC value. However, the delayed input did not show the performance decreased in this experiment. CONCLUSIONS: For a prediction model that was applicable in the real world, we considered not only performance but also temporal skewness, delayed input, and input error. |
format | Online Article Text |
id | pubmed-8603167 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | JMIR Publications |
record_format | MEDLINE/PubMed |
spelling | pubmed-86031672021-12-09 Event Prediction Model Considering Time and Input Error Using Electronic Medical Records in the Intensive Care Unit: Retrospective Study Sung, MinDong Hahn, Sangchul Han, Chang Hoon Lee, Jung Mo Lee, Jayoung Yoo, Jinkyu Heo, Jay Kim, Young Sam Chung, Kyung Soo JMIR Med Inform Original Paper BACKGROUND: In the era of artificial intelligence, event prediction models are abundant. However, considering the limitation of the electronic medical record–based model, including the temporally skewed prediction and the record itself, these models could be delayed or could yield errors. OBJECTIVE: In this study, we aim to develop multiple event prediction models in intensive care units to overcome their temporal skewness and evaluate their robustness against delayed and erroneous input. METHODS: A total of 21,738 patients were included in the development cohort. Three events—death, sepsis, and acute kidney injury—were predicted. To overcome the temporal skewness, we developed three models for each event, which predicted the events in advance of three prespecified timepoints. Additionally, to evaluate the robustness against input error and delays, we added simulated errors and delayed input and calculated changes in the area under the receiver operating characteristic curve (AUROC) values. RESULTS: Most of the AUROC and area under the precision-recall curve values of each model were higher than those of the conventional scores, as well as other machine learning models previously used. In the error input experiment, except for our proposed model, an increase in the noise added to the model lowered the resulting AUROC value. However, the delayed input did not show the performance decreased in this experiment. CONCLUSIONS: For a prediction model that was applicable in the real world, we considered not only performance but also temporal skewness, delayed input, and input error. JMIR Publications 2021-11-04 /pmc/articles/PMC8603167/ /pubmed/34734837 http://dx.doi.org/10.2196/26426 Text en ©MinDong Sung, Sangchul Hahn, Chang Hoon Han, Jung Mo Lee, Jayoung Lee, Jinkyu Yoo, Jay Heo, Young Sam Kim, Kyung Soo Chung. Originally published in JMIR Medical Informatics (https://medinform.jmir.org), 04.11.2021. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Medical Informatics, is properly cited. The complete bibliographic information, a link to the original publication on https://medinform.jmir.org/, as well as this copyright and license information must be included. |
spellingShingle | Original Paper Sung, MinDong Hahn, Sangchul Han, Chang Hoon Lee, Jung Mo Lee, Jayoung Yoo, Jinkyu Heo, Jay Kim, Young Sam Chung, Kyung Soo Event Prediction Model Considering Time and Input Error Using Electronic Medical Records in the Intensive Care Unit: Retrospective Study |
title | Event Prediction Model Considering Time and Input Error Using Electronic Medical Records in the Intensive Care Unit: Retrospective Study |
title_full | Event Prediction Model Considering Time and Input Error Using Electronic Medical Records in the Intensive Care Unit: Retrospective Study |
title_fullStr | Event Prediction Model Considering Time and Input Error Using Electronic Medical Records in the Intensive Care Unit: Retrospective Study |
title_full_unstemmed | Event Prediction Model Considering Time and Input Error Using Electronic Medical Records in the Intensive Care Unit: Retrospective Study |
title_short | Event Prediction Model Considering Time and Input Error Using Electronic Medical Records in the Intensive Care Unit: Retrospective Study |
title_sort | event prediction model considering time and input error using electronic medical records in the intensive care unit: retrospective study |
topic | Original Paper |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8603167/ https://www.ncbi.nlm.nih.gov/pubmed/34734837 http://dx.doi.org/10.2196/26426 |
work_keys_str_mv | AT sungmindong eventpredictionmodelconsideringtimeandinputerrorusingelectronicmedicalrecordsintheintensivecareunitretrospectivestudy AT hahnsangchul eventpredictionmodelconsideringtimeandinputerrorusingelectronicmedicalrecordsintheintensivecareunitretrospectivestudy AT hanchanghoon eventpredictionmodelconsideringtimeandinputerrorusingelectronicmedicalrecordsintheintensivecareunitretrospectivestudy AT leejungmo eventpredictionmodelconsideringtimeandinputerrorusingelectronicmedicalrecordsintheintensivecareunitretrospectivestudy AT leejayoung eventpredictionmodelconsideringtimeandinputerrorusingelectronicmedicalrecordsintheintensivecareunitretrospectivestudy AT yoojinkyu eventpredictionmodelconsideringtimeandinputerrorusingelectronicmedicalrecordsintheintensivecareunitretrospectivestudy AT heojay eventpredictionmodelconsideringtimeandinputerrorusingelectronicmedicalrecordsintheintensivecareunitretrospectivestudy AT kimyoungsam eventpredictionmodelconsideringtimeandinputerrorusingelectronicmedicalrecordsintheintensivecareunitretrospectivestudy AT chungkyungsoo eventpredictionmodelconsideringtimeandinputerrorusingelectronicmedicalrecordsintheintensivecareunitretrospectivestudy |