Cargando…
Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks
Multi-task learning has shown promising results in many applications of machine learning: given several related tasks, it aims to generalize better on the original tasks, by leveraging the knowledge among tasks. The knowledge transfer mainly depends on task relationships. Most of existing multi-task...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7351683/ http://dx.doi.org/10.1007/978-981-15-7205-0_1 |
_version_ | 1783557487245393920 |
---|---|
author | Ma, Tao Tan, Ying |
author_facet | Ma, Tao Tan, Ying |
author_sort | Ma, Tao |
collection | PubMed |
description | Multi-task learning has shown promising results in many applications of machine learning: given several related tasks, it aims to generalize better on the original tasks, by leveraging the knowledge among tasks. The knowledge transfer mainly depends on task relationships. Most of existing multi-task learning methods guide learning processes based on predefined task relationships. However, the associated relationships have not been fully exploited in these methods. Replacing predefined task relationships with the adaptively learned ones may lead to superior performance as it can avoid the misguiding of improper pre-definition. Therefore, in this paper, we propose Task Relation Attention Networks to adaptively model the task relationships and dynamically control the positive and negative knowledge transfer for different samples in multi-task learning. To evaluate the effectiveness of the proposed method, experiments on various datasets are conducted. The experimental results demonstrate that the proposed method outperforms both classical and state-of-the-art multi-task learning baselines. |
format | Online Article Text |
id | pubmed-7351683 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
record_format | MEDLINE/PubMed |
spelling | pubmed-73516832020-07-13 Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks Ma, Tao Tan, Ying Data Mining and Big Data Article Multi-task learning has shown promising results in many applications of machine learning: given several related tasks, it aims to generalize better on the original tasks, by leveraging the knowledge among tasks. The knowledge transfer mainly depends on task relationships. Most of existing multi-task learning methods guide learning processes based on predefined task relationships. However, the associated relationships have not been fully exploited in these methods. Replacing predefined task relationships with the adaptively learned ones may lead to superior performance as it can avoid the misguiding of improper pre-definition. Therefore, in this paper, we propose Task Relation Attention Networks to adaptively model the task relationships and dynamically control the positive and negative knowledge transfer for different samples in multi-task learning. To evaluate the effectiveness of the proposed method, experiments on various datasets are conducted. The experimental results demonstrate that the proposed method outperforms both classical and state-of-the-art multi-task learning baselines. 2020-07-11 /pmc/articles/PMC7351683/ http://dx.doi.org/10.1007/978-981-15-7205-0_1 Text en © Springer Nature Singapore Pte Ltd. 2020 This article is made available via the PMC Open Access Subset for unrestricted research re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic. |
spellingShingle | Article Ma, Tao Tan, Ying Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks |
title | Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks |
title_full | Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks |
title_fullStr | Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks |
title_full_unstemmed | Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks |
title_short | Adaptive and Dynamic Knowledge Transfer in Multi-task Learning with Attention Networks |
title_sort | adaptive and dynamic knowledge transfer in multi-task learning with attention networks |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7351683/ http://dx.doi.org/10.1007/978-981-15-7205-0_1 |
work_keys_str_mv | AT matao adaptiveanddynamicknowledgetransferinmultitasklearningwithattentionnetworks AT tanying adaptiveanddynamicknowledgetransferinmultitasklearningwithattentionnetworks |