Cargando…
Deep learning with multimodal representation for pancancer prognosis prediction
MOTIVATION: Estimating the future course of patients with cancer lesions is invaluable to physicians; however, current clinical methods fail to effectively use the vast amount of multimodal data that is available for cancer patients. To tackle this problem, we constructed a multimodal neural network...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Oxford University Press
2019
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6612862/ https://www.ncbi.nlm.nih.gov/pubmed/31510656 http://dx.doi.org/10.1093/bioinformatics/btz342 |
_version_ | 1783432953848659968 |
---|---|
author | Cheerla, Anika Gevaert, Olivier |
author_facet | Cheerla, Anika Gevaert, Olivier |
author_sort | Cheerla, Anika |
collection | PubMed |
description | MOTIVATION: Estimating the future course of patients with cancer lesions is invaluable to physicians; however, current clinical methods fail to effectively use the vast amount of multimodal data that is available for cancer patients. To tackle this problem, we constructed a multimodal neural network-based model to predict the survival of patients for 20 different cancer types using clinical data, mRNA expression data, microRNA expression data and histopathology whole slide images (WSIs). We developed an unsupervised encoder to compress these four data modalities into a single feature vector for each patient, handling missing data through a resilient, multimodal dropout method. Encoding methods were tailored to each data type—using deep highway networks to extract features from clinical and genomic data, and convolutional neural networks to extract features from WSIs. RESULTS: We used pancancer data to train these feature encodings and predict single cancer and pancancer overall survival, achieving a C-index of 0.78 overall. This work shows that it is possible to build a pancancer model for prognosis that also predicts prognosis in single cancer sites. Furthermore, our model handles multiple data modalities, efficiently analyzes WSIs and represents patient multimodal data flexibly into an unsupervised, informative representation. We thus present a powerful automated tool to accurately determine prognosis, a key step towards personalized treatment for cancer patients. AVAILABILITY AND IMPLEMENTATION: https://github.com/gevaertlab/MultimodalPrognosis |
format | Online Article Text |
id | pubmed-6612862 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2019 |
publisher | Oxford University Press |
record_format | MEDLINE/PubMed |
spelling | pubmed-66128622019-07-12 Deep learning with multimodal representation for pancancer prognosis prediction Cheerla, Anika Gevaert, Olivier Bioinformatics Ismb/Eccb 2019 Conference Proceedings MOTIVATION: Estimating the future course of patients with cancer lesions is invaluable to physicians; however, current clinical methods fail to effectively use the vast amount of multimodal data that is available for cancer patients. To tackle this problem, we constructed a multimodal neural network-based model to predict the survival of patients for 20 different cancer types using clinical data, mRNA expression data, microRNA expression data and histopathology whole slide images (WSIs). We developed an unsupervised encoder to compress these four data modalities into a single feature vector for each patient, handling missing data through a resilient, multimodal dropout method. Encoding methods were tailored to each data type—using deep highway networks to extract features from clinical and genomic data, and convolutional neural networks to extract features from WSIs. RESULTS: We used pancancer data to train these feature encodings and predict single cancer and pancancer overall survival, achieving a C-index of 0.78 overall. This work shows that it is possible to build a pancancer model for prognosis that also predicts prognosis in single cancer sites. Furthermore, our model handles multiple data modalities, efficiently analyzes WSIs and represents patient multimodal data flexibly into an unsupervised, informative representation. We thus present a powerful automated tool to accurately determine prognosis, a key step towards personalized treatment for cancer patients. AVAILABILITY AND IMPLEMENTATION: https://github.com/gevaertlab/MultimodalPrognosis Oxford University Press 2019-07 2019-07-05 /pmc/articles/PMC6612862/ /pubmed/31510656 http://dx.doi.org/10.1093/bioinformatics/btz342 Text en © The Author(s) 2019. Published by Oxford University Press. http://creativecommons.org/licenses/by-nc/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.com |
spellingShingle | Ismb/Eccb 2019 Conference Proceedings Cheerla, Anika Gevaert, Olivier Deep learning with multimodal representation for pancancer prognosis prediction |
title | Deep learning with multimodal representation for pancancer prognosis prediction |
title_full | Deep learning with multimodal representation for pancancer prognosis prediction |
title_fullStr | Deep learning with multimodal representation for pancancer prognosis prediction |
title_full_unstemmed | Deep learning with multimodal representation for pancancer prognosis prediction |
title_short | Deep learning with multimodal representation for pancancer prognosis prediction |
title_sort | deep learning with multimodal representation for pancancer prognosis prediction |
topic | Ismb/Eccb 2019 Conference Proceedings |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6612862/ https://www.ncbi.nlm.nih.gov/pubmed/31510656 http://dx.doi.org/10.1093/bioinformatics/btz342 |
work_keys_str_mv | AT cheerlaanika deeplearningwithmultimodalrepresentationforpancancerprognosisprediction AT gevaertolivier deeplearningwithmultimodalrepresentationforpancancerprognosisprediction |