Cargando…

Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data

Transformer is the latest deep neural network (DNN) architecture for sequence data learning that has revolutionized the field of natural language processing. This success has motivated researchers to explore its application in the healthcare domain. Despite the similarities between longitudinal clin...

Descripción completa

Detalles Bibliográficos
Autores principales: Shao, Yijun, Cheng, Yan, Nelson, Stuart J., Kokkinos, Peter, Zamrini, Edward Y., Ahmed, Ali, Zeng-Treitler, Qing
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Cold Spring Harbor Laboratory 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10055462/
https://www.ncbi.nlm.nih.gov/pubmed/36993767
http://dx.doi.org/10.1101/2023.03.09.23287046
_version_ 1785015879464910848
author Shao, Yijun
Cheng, Yan
Nelson, Stuart J.
Kokkinos, Peter
Zamrini, Edward Y.
Ahmed, Ali
Zeng-Treitler, Qing
author_facet Shao, Yijun
Cheng, Yan
Nelson, Stuart J.
Kokkinos, Peter
Zamrini, Edward Y.
Ahmed, Ali
Zeng-Treitler, Qing
author_sort Shao, Yijun
collection PubMed
description Transformer is the latest deep neural network (DNN) architecture for sequence data learning that has revolutionized the field of natural language processing. This success has motivated researchers to explore its application in the healthcare domain. Despite the similarities between longitudinal clinical data and natural language data, clinical data presents unique complexities that make adapting Transformer to this domain challenging. To address this issue, we have designed a new Transformer-based DNN architecture, referred to as Hybrid Value-Aware Transformer (HVAT), which can jointly learn from longitudinal and non-longitudinal clinical data. HVAT is unique in the ability to learn from the numerical values associated with clinical codes/concepts such as labs, and also the use of a flexible longitudinal data representation called clinical tokens. We trained a prototype HVAT model on a case-control dataset, achieving high performance in predicting Alzheimer’s disease and related dementias as the patient outcome. The result demonstrates the potential of HVAT for broader clinical data learning tasks.
format Online
Article
Text
id pubmed-10055462
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Cold Spring Harbor Laboratory
record_format MEDLINE/PubMed
spelling pubmed-100554622023-03-30 Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data Shao, Yijun Cheng, Yan Nelson, Stuart J. Kokkinos, Peter Zamrini, Edward Y. Ahmed, Ali Zeng-Treitler, Qing medRxiv Article Transformer is the latest deep neural network (DNN) architecture for sequence data learning that has revolutionized the field of natural language processing. This success has motivated researchers to explore its application in the healthcare domain. Despite the similarities between longitudinal clinical data and natural language data, clinical data presents unique complexities that make adapting Transformer to this domain challenging. To address this issue, we have designed a new Transformer-based DNN architecture, referred to as Hybrid Value-Aware Transformer (HVAT), which can jointly learn from longitudinal and non-longitudinal clinical data. HVAT is unique in the ability to learn from the numerical values associated with clinical codes/concepts such as labs, and also the use of a flexible longitudinal data representation called clinical tokens. We trained a prototype HVAT model on a case-control dataset, achieving high performance in predicting Alzheimer’s disease and related dementias as the patient outcome. The result demonstrates the potential of HVAT for broader clinical data learning tasks. Cold Spring Harbor Laboratory 2023-03-14 /pmc/articles/PMC10055462/ /pubmed/36993767 http://dx.doi.org/10.1101/2023.03.09.23287046 Text en https://creativecommons.org/licenses/by-nc/4.0/This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (https://creativecommons.org/licenses/by-nc/4.0/) , which allows reusers to distribute, remix, adapt, and build upon the material in any medium or format for noncommercial purposes only, and only so long as attribution is given to the creator.
spellingShingle Article
Shao, Yijun
Cheng, Yan
Nelson, Stuart J.
Kokkinos, Peter
Zamrini, Edward Y.
Ahmed, Ali
Zeng-Treitler, Qing
Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data
title Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data
title_full Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data
title_fullStr Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data
title_full_unstemmed Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data
title_short Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data
title_sort hybrid value-aware transformer architecture for joint learning from longitudinal and non-longitudinal clinical data
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10055462/
https://www.ncbi.nlm.nih.gov/pubmed/36993767
http://dx.doi.org/10.1101/2023.03.09.23287046
work_keys_str_mv AT shaoyijun hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata
AT chengyan hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata
AT nelsonstuartj hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata
AT kokkinospeter hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata
AT zamriniedwardy hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata
AT ahmedali hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata
AT zengtreitlerqing hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata