Cargando…

Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks

Multi-task learning has shown promising results in many applications of machine learning: given several related tasks, it aims to generalize better on the original tasks, by leveraging the knowledge among tasks. The knowledge transfer mainly depends on task relationships. Most of existing multi-task...

Descripción completa

Detalles Bibliográficos
Autores principales: Ma, Tao, Tan, Ying
Formato: Online Artículo Texto
Lenguaje:English
Publicado: 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7351683/
http://dx.doi.org/10.1007/978-981-15-7205-0_1
_version_ 1783557487245393920
author Ma, Tao
Tan, Ying
author_facet Ma, Tao
Tan, Ying
author_sort Ma, Tao
collection PubMed
description Multi-task learning has shown promising results in many applications of machine learning: given several related tasks, it aims to generalize better on the original tasks, by leveraging the knowledge among tasks. The knowledge transfer mainly depends on task relationships. Most of existing multi-task learning methods guide learning processes based on predefined task relationships. However, the associated relationships have not been fully exploited in these methods. Replacing predefined task relationships with the adaptively learned ones may lead to superior performance as it can avoid the misguiding of improper pre-definition. Therefore, in this paper, we propose Task Relation Attention Networks to adaptively model the task relationships and dynamically control the positive and negative knowledge transfer for different samples in multi-task learning. To evaluate the effectiveness of the proposed method, experiments on various datasets are conducted. The experimental results demonstrate that the proposed method outperforms both classical and state-of-the-art multi-task learning baselines.
format Online
Article
Text
id pubmed-7351683
institution National Center for Biotechnology Information
language English
publishDate 2020
record_format MEDLINE/PubMed
spelling pubmed-73516832020-07-13 Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks Ma, Tao Tan, Ying Data Mining and Big Data Article Multi-task learning has shown promising results in many applications of machine learning: given several related tasks, it aims to generalize better on the original tasks, by leveraging the knowledge among tasks. The knowledge transfer mainly depends on task relationships. Most of existing multi-task learning methods guide learning processes based on predefined task relationships. However, the associated relationships have not been fully exploited in these methods. Replacing predefined task relationships with the adaptively learned ones may lead to superior performance as it can avoid the misguiding of improper pre-definition. Therefore, in this paper, we propose Task Relation Attention Networks to adaptively model the task relationships and dynamically control the positive and negative knowledge transfer for different samples in multi-task learning. To evaluate the effectiveness of the proposed method, experiments on various datasets are conducted. The experimental results demonstrate that the proposed method outperforms both classical and state-of-the-art multi-task learning baselines. 2020-07-11 /pmc/articles/PMC7351683/ http://dx.doi.org/10.1007/978-981-15-7205-0_1 Text en © Springer Nature Singapore Pte Ltd. 2020 This article is made available via the PMC Open Access Subset for unrestricted research re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic.
spellingShingle Article
Ma, Tao
Tan, Ying
Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks
title Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks
title_full Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks
title_fullStr Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks
title_full_unstemmed Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks
title_short Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks
title_sort adaptive and dynamic knowledge transfer in multi-task learning with attention networks
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7351683/
http://dx.doi.org/10.1007/978-981-15-7205-0_1
work_keys_str_mv AT matao adaptiveanddynamicknowledgetransferinmultitasklearningwithattentionnetworks
AT tanying adaptiveanddynamicknowledgetransferinmultitasklearningwithattentionnetworks