Multimodal temporal-clinical note network for mortality prediction

BACKGROUND: Mortality prediction is an important task to achieve smart healthcare, especially for the management of intensive care unit. It can provide a reference for doctors to quickly predict the course of disease and customize early intervention programs for the patients in need. With the development of the electronic medical records, deep learning methods are introduced to deal with the prediction task. In the electronic medical records, clinical notes always contain rich and diverse medical information, including the clinical histories and reports during admission. Mortality prediction methods mostly rely on the temporal events such as medical examinations and ignore the related reports and history information in the clinical notes. We hope that we can utilize both temporal events and clinical notes information to get better mortality prediction results. RESULTS: We propose a multimodal temporal-clinical note network to model both temporal and clinical notes. Specifically, the clinical text are further processed for differentiating the chronic illness patients in the historical information of clinical notes from non-chronic illness patients. In order to further mine the information related to the mortality in the text, we learn the time series embedding with Long Short Term Memory networks and the clinical notes embedding with a label aware convolutional neural network. We also propose a scoring function to measure the importance of clinical note sections. Our approach achieved a better AUCPR and AUCROC than competing methods and visual explanations for word importance showed the interpretability improvement of the model. CONCLUSIONS: We have tested our methodology on the MIMIC-III dataset. Contributions of different clinical note sections were uncovered by visualization methods. Our work demonstrates that the introduction of the medical history related information can improve the performance of the mortality prediction. Using label aware convolutional neural networks can further improve the results.