Cargando…

Prediction of Stroke Outcome Using Natural Language Processing-Based Machine Learning of Radiology Report of Brain MRI

Brain magnetic resonance imaging (MRI) is useful for predicting the outcome of patients with acute ischemic stroke (AIS). Although deep learning (DL) using brain MRI with certain image biomarkers has shown satisfactory results in predicting poor outcomes, no study has assessed the usefulness of natu...

Descripción completa

Detalles Bibliográficos
Autores principales: Heo, Tak Sung, Kim, Yu Seop, Choi, Jeong Myeong, Jeong, Yeong Seok, Seo, Soo Young, Lee, Jun Ho, Jeon, Jin Pyeong, Kim, Chulho
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7766032/
https://www.ncbi.nlm.nih.gov/pubmed/33339385
http://dx.doi.org/10.3390/jpm10040286
_version_ 1783628622090731520
author Heo, Tak Sung
Kim, Yu Seop
Choi, Jeong Myeong
Jeong, Yeong Seok
Seo, Soo Young
Lee, Jun Ho
Jeon, Jin Pyeong
Kim, Chulho
author_facet Heo, Tak Sung
Kim, Yu Seop
Choi, Jeong Myeong
Jeong, Yeong Seok
Seo, Soo Young
Lee, Jun Ho
Jeon, Jin Pyeong
Kim, Chulho
author_sort Heo, Tak Sung
collection PubMed
description Brain magnetic resonance imaging (MRI) is useful for predicting the outcome of patients with acute ischemic stroke (AIS). Although deep learning (DL) using brain MRI with certain image biomarkers has shown satisfactory results in predicting poor outcomes, no study has assessed the usefulness of natural language processing (NLP)-based machine learning (ML) algorithms using brain MRI free-text reports of AIS patients. Therefore, we aimed to assess whether NLP-based ML algorithms using brain MRI text reports could predict poor outcomes in AIS patients. This study included only English text reports of brain MRIs examined during admission of AIS patients. Poor outcome was defined as a modified Rankin Scale score of 3–6, and the data were captured by trained nurses and physicians. We only included MRI text report of the first MRI scan during the admission. The text dataset was randomly divided into a training and test dataset with a 7:3 ratio. Text was vectorized to word, sentence, and document levels. In the word level approach, which did not consider the sequence of words, and the “bag-of-words” model was used to reflect the number of repetitions of text token. The “sent2vec” method was used in the sensation-level approach considering the sequence of words, and the word embedding was used in the document level approach. In addition to conventional ML algorithms, DL algorithms such as the convolutional neural network (CNN), long short-term memory, and multilayer perceptron were used to predict poor outcomes using 5-fold cross-validation and grid search techniques. The performance of each ML classifier was compared with the area under the receiver operating characteristic (AUROC) curve. Among 1840 subjects with AIS, 645 patients (35.1%) had a poor outcome 3 months after the stroke onset. Random forest was the best classifier (0.782 of AUROC) using a word-level approach. Overall, the document-level approach exhibited better performance than did the word- or sentence-level approaches. Among all the ML classifiers, the multi-CNN algorithm demonstrated the best classification performance (0.805), followed by the CNN (0.799) algorithm. When predicting future clinical outcomes using NLP-based ML of radiology free-text reports of brain MRI, DL algorithms showed superior performance over the other ML algorithms. In particular, the prediction of poor outcomes in document-level NLP DL was improved more by multi-CNN and CNN than by recurrent neural network-based algorithms. NLP-based DL algorithms can be used as an important digital marker for unstructured electronic health record data DL prediction.
format Online
Article
Text
id pubmed-7766032
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-77660322020-12-28 Prediction of Stroke Outcome Using Natural Language Processing-Based Machine Learning of Radiology Report of Brain MRI Heo, Tak Sung Kim, Yu Seop Choi, Jeong Myeong Jeong, Yeong Seok Seo, Soo Young Lee, Jun Ho Jeon, Jin Pyeong Kim, Chulho J Pers Med Article Brain magnetic resonance imaging (MRI) is useful for predicting the outcome of patients with acute ischemic stroke (AIS). Although deep learning (DL) using brain MRI with certain image biomarkers has shown satisfactory results in predicting poor outcomes, no study has assessed the usefulness of natural language processing (NLP)-based machine learning (ML) algorithms using brain MRI free-text reports of AIS patients. Therefore, we aimed to assess whether NLP-based ML algorithms using brain MRI text reports could predict poor outcomes in AIS patients. This study included only English text reports of brain MRIs examined during admission of AIS patients. Poor outcome was defined as a modified Rankin Scale score of 3–6, and the data were captured by trained nurses and physicians. We only included MRI text report of the first MRI scan during the admission. The text dataset was randomly divided into a training and test dataset with a 7:3 ratio. Text was vectorized to word, sentence, and document levels. In the word level approach, which did not consider the sequence of words, and the “bag-of-words” model was used to reflect the number of repetitions of text token. The “sent2vec” method was used in the sensation-level approach considering the sequence of words, and the word embedding was used in the document level approach. In addition to conventional ML algorithms, DL algorithms such as the convolutional neural network (CNN), long short-term memory, and multilayer perceptron were used to predict poor outcomes using 5-fold cross-validation and grid search techniques. The performance of each ML classifier was compared with the area under the receiver operating characteristic (AUROC) curve. Among 1840 subjects with AIS, 645 patients (35.1%) had a poor outcome 3 months after the stroke onset. Random forest was the best classifier (0.782 of AUROC) using a word-level approach. Overall, the document-level approach exhibited better performance than did the word- or sentence-level approaches. Among all the ML classifiers, the multi-CNN algorithm demonstrated the best classification performance (0.805), followed by the CNN (0.799) algorithm. When predicting future clinical outcomes using NLP-based ML of radiology free-text reports of brain MRI, DL algorithms showed superior performance over the other ML algorithms. In particular, the prediction of poor outcomes in document-level NLP DL was improved more by multi-CNN and CNN than by recurrent neural network-based algorithms. NLP-based DL algorithms can be used as an important digital marker for unstructured electronic health record data DL prediction. MDPI 2020-12-16 /pmc/articles/PMC7766032/ /pubmed/33339385 http://dx.doi.org/10.3390/jpm10040286 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Heo, Tak Sung
Kim, Yu Seop
Choi, Jeong Myeong
Jeong, Yeong Seok
Seo, Soo Young
Lee, Jun Ho
Jeon, Jin Pyeong
Kim, Chulho
Prediction of Stroke Outcome Using Natural Language Processing-Based Machine Learning of Radiology Report of Brain MRI
title Prediction of Stroke Outcome Using Natural Language Processing-Based Machine Learning of Radiology Report of Brain MRI
title_full Prediction of Stroke Outcome Using Natural Language Processing-Based Machine Learning of Radiology Report of Brain MRI
title_fullStr Prediction of Stroke Outcome Using Natural Language Processing-Based Machine Learning of Radiology Report of Brain MRI
title_full_unstemmed Prediction of Stroke Outcome Using Natural Language Processing-Based Machine Learning of Radiology Report of Brain MRI
title_short Prediction of Stroke Outcome Using Natural Language Processing-Based Machine Learning of Radiology Report of Brain MRI
title_sort prediction of stroke outcome using natural language processing-based machine learning of radiology report of brain mri
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7766032/
https://www.ncbi.nlm.nih.gov/pubmed/33339385
http://dx.doi.org/10.3390/jpm10040286
work_keys_str_mv AT heotaksung predictionofstrokeoutcomeusingnaturallanguageprocessingbasedmachinelearningofradiologyreportofbrainmri
AT kimyuseop predictionofstrokeoutcomeusingnaturallanguageprocessingbasedmachinelearningofradiologyreportofbrainmri
AT choijeongmyeong predictionofstrokeoutcomeusingnaturallanguageprocessingbasedmachinelearningofradiologyreportofbrainmri
AT jeongyeongseok predictionofstrokeoutcomeusingnaturallanguageprocessingbasedmachinelearningofradiologyreportofbrainmri
AT seosooyoung predictionofstrokeoutcomeusingnaturallanguageprocessingbasedmachinelearningofradiologyreportofbrainmri
AT leejunho predictionofstrokeoutcomeusingnaturallanguageprocessingbasedmachinelearningofradiologyreportofbrainmri
AT jeonjinpyeong predictionofstrokeoutcomeusingnaturallanguageprocessingbasedmachinelearningofradiologyreportofbrainmri
AT kimchulho predictionofstrokeoutcomeusingnaturallanguageprocessingbasedmachinelearningofradiologyreportofbrainmri