Cargando…
Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data
Transformer is the latest deep neural network (DNN) architecture for sequence data learning that has revolutionized the field of natural language processing. This success has motivated researchers to explore its application in the healthcare domain. Despite the similarities between longitudinal clin...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Cold Spring Harbor Laboratory
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10055462/ https://www.ncbi.nlm.nih.gov/pubmed/36993767 http://dx.doi.org/10.1101/2023.03.09.23287046 |
_version_ | 1785015879464910848 |
---|---|
author | Shao, Yijun Cheng, Yan Nelson, Stuart J. Kokkinos, Peter Zamrini, Edward Y. Ahmed, Ali Zeng-Treitler, Qing |
author_facet | Shao, Yijun Cheng, Yan Nelson, Stuart J. Kokkinos, Peter Zamrini, Edward Y. Ahmed, Ali Zeng-Treitler, Qing |
author_sort | Shao, Yijun |
collection | PubMed |
description | Transformer is the latest deep neural network (DNN) architecture for sequence data learning that has revolutionized the field of natural language processing. This success has motivated researchers to explore its application in the healthcare domain. Despite the similarities between longitudinal clinical data and natural language data, clinical data presents unique complexities that make adapting Transformer to this domain challenging. To address this issue, we have designed a new Transformer-based DNN architecture, referred to as Hybrid Value-Aware Transformer (HVAT), which can jointly learn from longitudinal and non-longitudinal clinical data. HVAT is unique in the ability to learn from the numerical values associated with clinical codes/concepts such as labs, and also the use of a flexible longitudinal data representation called clinical tokens. We trained a prototype HVAT model on a case-control dataset, achieving high performance in predicting Alzheimer’s disease and related dementias as the patient outcome. The result demonstrates the potential of HVAT for broader clinical data learning tasks. |
format | Online Article Text |
id | pubmed-10055462 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Cold Spring Harbor Laboratory |
record_format | MEDLINE/PubMed |
spelling | pubmed-100554622023-03-30 Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data Shao, Yijun Cheng, Yan Nelson, Stuart J. Kokkinos, Peter Zamrini, Edward Y. Ahmed, Ali Zeng-Treitler, Qing medRxiv Article Transformer is the latest deep neural network (DNN) architecture for sequence data learning that has revolutionized the field of natural language processing. This success has motivated researchers to explore its application in the healthcare domain. Despite the similarities between longitudinal clinical data and natural language data, clinical data presents unique complexities that make adapting Transformer to this domain challenging. To address this issue, we have designed a new Transformer-based DNN architecture, referred to as Hybrid Value-Aware Transformer (HVAT), which can jointly learn from longitudinal and non-longitudinal clinical data. HVAT is unique in the ability to learn from the numerical values associated with clinical codes/concepts such as labs, and also the use of a flexible longitudinal data representation called clinical tokens. We trained a prototype HVAT model on a case-control dataset, achieving high performance in predicting Alzheimer’s disease and related dementias as the patient outcome. The result demonstrates the potential of HVAT for broader clinical data learning tasks. Cold Spring Harbor Laboratory 2023-03-14 /pmc/articles/PMC10055462/ /pubmed/36993767 http://dx.doi.org/10.1101/2023.03.09.23287046 Text en https://creativecommons.org/licenses/by-nc/4.0/This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License (https://creativecommons.org/licenses/by-nc/4.0/) , which allows reusers to distribute, remix, adapt, and build upon the material in any medium or format for noncommercial purposes only, and only so long as attribution is given to the creator. |
spellingShingle | Article Shao, Yijun Cheng, Yan Nelson, Stuart J. Kokkinos, Peter Zamrini, Edward Y. Ahmed, Ali Zeng-Treitler, Qing Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data |
title | Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data |
title_full | Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data |
title_fullStr | Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data |
title_full_unstemmed | Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data |
title_short | Hybrid Value-Aware Transformer Architecture for Joint Learning from Longitudinal and Non-Longitudinal Clinical Data |
title_sort | hybrid value-aware transformer architecture for joint learning from longitudinal and non-longitudinal clinical data |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10055462/ https://www.ncbi.nlm.nih.gov/pubmed/36993767 http://dx.doi.org/10.1101/2023.03.09.23287046 |
work_keys_str_mv | AT shaoyijun hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata AT chengyan hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata AT nelsonstuartj hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata AT kokkinospeter hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata AT zamriniedwardy hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata AT ahmedali hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata AT zengtreitlerqing hybridvalueawaretransformerarchitectureforjointlearningfromlongitudinalandnonlongitudinalclinicaldata |