Cargando…

Multilingual multi-aspect explainability analyses on machine reading comprehension models

Achieving human-level performance on some of the machine reading comprehension (MRC) datasets is no longer challenging with the help of powerful pre-trained language models (PLMs). However, the internal mechanism of these artifacts remains unclear, placing an obstacle to further understand these mod...

Descripción completa

Detalles Bibliográficos
Autores principales: Cui, Yiming, Zhang, Wei-Nan, Che, Wanxiang, Liu, Ting, Chen, Zhigang, Wang, Shijin
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Elsevier 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9019247/
https://www.ncbi.nlm.nih.gov/pubmed/35465050
http://dx.doi.org/10.1016/j.isci.2022.104176
_version_ 1784689215996428288
author Cui, Yiming
Zhang, Wei-Nan
Che, Wanxiang
Liu, Ting
Chen, Zhigang
Wang, Shijin
author_facet Cui, Yiming
Zhang, Wei-Nan
Che, Wanxiang
Liu, Ting
Chen, Zhigang
Wang, Shijin
author_sort Cui, Yiming
collection PubMed
description Achieving human-level performance on some of the machine reading comprehension (MRC) datasets is no longer challenging with the help of powerful pre-trained language models (PLMs). However, the internal mechanism of these artifacts remains unclear, placing an obstacle to further understand these models. This paper focuses on conducting a series of analytical experiments to examine the relations between the multi-head self-attention and the final MRC system performance, revealing the potential explainability in PLM-based MRC models. To ensure the robustness of the analyses, we perform our experiments in a multilingual way on top of various PLMs. We discover that passage-to-question and passage understanding attentions are the most important ones in the question answering process, showing strong correlations to the final performance than other parts. Through comprehensive visualizations and case studies, we also observe several general findings on the attention maps, which can be helpful to understand how these models solve the questions.
format Online
Article
Text
id pubmed-9019247
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Elsevier
record_format MEDLINE/PubMed
spelling pubmed-90192472022-04-21 Multilingual multi-aspect explainability analyses on machine reading comprehension models Cui, Yiming Zhang, Wei-Nan Che, Wanxiang Liu, Ting Chen, Zhigang Wang, Shijin iScience Article Achieving human-level performance on some of the machine reading comprehension (MRC) datasets is no longer challenging with the help of powerful pre-trained language models (PLMs). However, the internal mechanism of these artifacts remains unclear, placing an obstacle to further understand these models. This paper focuses on conducting a series of analytical experiments to examine the relations between the multi-head self-attention and the final MRC system performance, revealing the potential explainability in PLM-based MRC models. To ensure the robustness of the analyses, we perform our experiments in a multilingual way on top of various PLMs. We discover that passage-to-question and passage understanding attentions are the most important ones in the question answering process, showing strong correlations to the final performance than other parts. Through comprehensive visualizations and case studies, we also observe several general findings on the attention maps, which can be helpful to understand how these models solve the questions. Elsevier 2022-03-31 /pmc/articles/PMC9019247/ /pubmed/35465050 http://dx.doi.org/10.1016/j.isci.2022.104176 Text en © 2022 The Author(s) https://creativecommons.org/licenses/by/4.0/This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Cui, Yiming
Zhang, Wei-Nan
Che, Wanxiang
Liu, Ting
Chen, Zhigang
Wang, Shijin
Multilingual multi-aspect explainability analyses on machine reading comprehension models
title Multilingual multi-aspect explainability analyses on machine reading comprehension models
title_full Multilingual multi-aspect explainability analyses on machine reading comprehension models
title_fullStr Multilingual multi-aspect explainability analyses on machine reading comprehension models
title_full_unstemmed Multilingual multi-aspect explainability analyses on machine reading comprehension models
title_short Multilingual multi-aspect explainability analyses on machine reading comprehension models
title_sort multilingual multi-aspect explainability analyses on machine reading comprehension models
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9019247/
https://www.ncbi.nlm.nih.gov/pubmed/35465050
http://dx.doi.org/10.1016/j.isci.2022.104176
work_keys_str_mv AT cuiyiming multilingualmultiaspectexplainabilityanalysesonmachinereadingcomprehensionmodels
AT zhangweinan multilingualmultiaspectexplainabilityanalysesonmachinereadingcomprehensionmodels
AT chewanxiang multilingualmultiaspectexplainabilityanalysesonmachinereadingcomprehensionmodels
AT liuting multilingualmultiaspectexplainabilityanalysesonmachinereadingcomprehensionmodels
AT chenzhigang multilingualmultiaspectexplainabilityanalysesonmachinereadingcomprehensionmodels
AT wangshijin multilingualmultiaspectexplainabilityanalysesonmachinereadingcomprehensionmodels