Cargando…
A multistage multimodal deep learning model for disease severity assessment and early warnings of high-risk patients of COVID-19
The outbreak of coronavirus disease 2019 (COVID-19) has caused massive infections and large death tolls worldwide. Despite many studies on the clinical characteristics and the treatment plans of COVID-19, they rarely conduct in-depth prognostic research on leveraging consecutive rounds of multimodal...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9723232/ https://www.ncbi.nlm.nih.gov/pubmed/36483265 http://dx.doi.org/10.3389/fpubh.2022.982289 |
_version_ | 1784844122608107520 |
---|---|
author | Li, Zhuo Xu, Ruiqing Shen, Yifei Cao, Jiannong Wang, Ben Zhang, Ying Li, Shikang |
author_facet | Li, Zhuo Xu, Ruiqing Shen, Yifei Cao, Jiannong Wang, Ben Zhang, Ying Li, Shikang |
author_sort | Li, Zhuo |
collection | PubMed |
description | The outbreak of coronavirus disease 2019 (COVID-19) has caused massive infections and large death tolls worldwide. Despite many studies on the clinical characteristics and the treatment plans of COVID-19, they rarely conduct in-depth prognostic research on leveraging consecutive rounds of multimodal clinical examination and laboratory test data to facilitate clinical decision-making for the treatment of COVID-19. To address this issue, we propose a multistage multimodal deep learning (MMDL) model to (1) first assess the patient's current condition (i.e., the mild and severe symptoms), then (2) give early warnings to patients with mild symptoms who are at high risk to develop severe illness. In MMDL, we build a sequential stage-wise learning architecture whose design philosophy embodies the model's predicted outcome and does not only depend on the current situation but also the history. Concretely, we meticulously combine the latest round of multimodal clinical data and the decayed past information to make assessments and predictions. In each round (stage), we design a two-layer multimodal feature extractor to extract the latent feature representation across different modalities of clinical data, including patient demographics, clinical manifestation, and 11 modalities of laboratory test results. We conduct experiments on a clinical dataset consisting of 216 COVID-19 patients that have passed the ethical review of the medical ethics committee. Experimental results validate our assumption that sequential stage-wise learning outperforms single-stage learning, but history long ago has little influence on the learning outcome. Also, comparison tests show the advantage of multimodal learning. MMDL with multimodal inputs can beat any reduced model with single-modal inputs only. In addition, we have deployed the prototype of MMDL in a hospital for clinical comparison tests and to assist doctors in clinical diagnosis. |
format | Online Article Text |
id | pubmed-9723232 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-97232322022-12-07 A multistage multimodal deep learning model for disease severity assessment and early warnings of high-risk patients of COVID-19 Li, Zhuo Xu, Ruiqing Shen, Yifei Cao, Jiannong Wang, Ben Zhang, Ying Li, Shikang Front Public Health Public Health The outbreak of coronavirus disease 2019 (COVID-19) has caused massive infections and large death tolls worldwide. Despite many studies on the clinical characteristics and the treatment plans of COVID-19, they rarely conduct in-depth prognostic research on leveraging consecutive rounds of multimodal clinical examination and laboratory test data to facilitate clinical decision-making for the treatment of COVID-19. To address this issue, we propose a multistage multimodal deep learning (MMDL) model to (1) first assess the patient's current condition (i.e., the mild and severe symptoms), then (2) give early warnings to patients with mild symptoms who are at high risk to develop severe illness. In MMDL, we build a sequential stage-wise learning architecture whose design philosophy embodies the model's predicted outcome and does not only depend on the current situation but also the history. Concretely, we meticulously combine the latest round of multimodal clinical data and the decayed past information to make assessments and predictions. In each round (stage), we design a two-layer multimodal feature extractor to extract the latent feature representation across different modalities of clinical data, including patient demographics, clinical manifestation, and 11 modalities of laboratory test results. We conduct experiments on a clinical dataset consisting of 216 COVID-19 patients that have passed the ethical review of the medical ethics committee. Experimental results validate our assumption that sequential stage-wise learning outperforms single-stage learning, but history long ago has little influence on the learning outcome. Also, comparison tests show the advantage of multimodal learning. MMDL with multimodal inputs can beat any reduced model with single-modal inputs only. In addition, we have deployed the prototype of MMDL in a hospital for clinical comparison tests and to assist doctors in clinical diagnosis. Frontiers Media S.A. 2022-11-22 /pmc/articles/PMC9723232/ /pubmed/36483265 http://dx.doi.org/10.3389/fpubh.2022.982289 Text en Copyright © 2022 Li, Xu, Shen, Cao, Wang, Zhang and Li. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Public Health Li, Zhuo Xu, Ruiqing Shen, Yifei Cao, Jiannong Wang, Ben Zhang, Ying Li, Shikang A multistage multimodal deep learning model for disease severity assessment and early warnings of high-risk patients of COVID-19 |
title | A multistage multimodal deep learning model for disease severity assessment and early warnings of high-risk patients of COVID-19 |
title_full | A multistage multimodal deep learning model for disease severity assessment and early warnings of high-risk patients of COVID-19 |
title_fullStr | A multistage multimodal deep learning model for disease severity assessment and early warnings of high-risk patients of COVID-19 |
title_full_unstemmed | A multistage multimodal deep learning model for disease severity assessment and early warnings of high-risk patients of COVID-19 |
title_short | A multistage multimodal deep learning model for disease severity assessment and early warnings of high-risk patients of COVID-19 |
title_sort | multistage multimodal deep learning model for disease severity assessment and early warnings of high-risk patients of covid-19 |
topic | Public Health |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9723232/ https://www.ncbi.nlm.nih.gov/pubmed/36483265 http://dx.doi.org/10.3389/fpubh.2022.982289 |
work_keys_str_mv | AT lizhuo amultistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT xuruiqing amultistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT shenyifei amultistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT caojiannong amultistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT wangben amultistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT zhangying amultistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT lishikang amultistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT lizhuo multistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT xuruiqing multistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT shenyifei multistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT caojiannong multistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT wangben multistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT zhangying multistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 AT lishikang multistagemultimodaldeeplearningmodelfordiseaseseverityassessmentandearlywarningsofhighriskpatientsofcovid19 |