Cargando…
Multilingual multi-aspect explainability analyses on machine reading comprehension models
Achieving human-level performance on some of the machine reading comprehension (MRC) datasets is no longer challenging with the help of powerful pre-trained language models (PLMs). However, the internal mechanism of these artifacts remains unclear, placing an obstacle to further understand these mod...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Elsevier
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9019247/ https://www.ncbi.nlm.nih.gov/pubmed/35465050 http://dx.doi.org/10.1016/j.isci.2022.104176 |
_version_ | 1784689215996428288 |
---|---|
author | Cui, Yiming Zhang, Wei-Nan Che, Wanxiang Liu, Ting Chen, Zhigang Wang, Shijin |
author_facet | Cui, Yiming Zhang, Wei-Nan Che, Wanxiang Liu, Ting Chen, Zhigang Wang, Shijin |
author_sort | Cui, Yiming |
collection | PubMed |
description | Achieving human-level performance on some of the machine reading comprehension (MRC) datasets is no longer challenging with the help of powerful pre-trained language models (PLMs). However, the internal mechanism of these artifacts remains unclear, placing an obstacle to further understand these models. This paper focuses on conducting a series of analytical experiments to examine the relations between the multi-head self-attention and the final MRC system performance, revealing the potential explainability in PLM-based MRC models. To ensure the robustness of the analyses, we perform our experiments in a multilingual way on top of various PLMs. We discover that passage-to-question and passage understanding attentions are the most important ones in the question answering process, showing strong correlations to the final performance than other parts. Through comprehensive visualizations and case studies, we also observe several general findings on the attention maps, which can be helpful to understand how these models solve the questions. |
format | Online Article Text |
id | pubmed-9019247 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | Elsevier |
record_format | MEDLINE/PubMed |
spelling | pubmed-90192472022-04-21 Multilingual multi-aspect explainability analyses on machine reading comprehension models Cui, Yiming Zhang, Wei-Nan Che, Wanxiang Liu, Ting Chen, Zhigang Wang, Shijin iScience Article Achieving human-level performance on some of the machine reading comprehension (MRC) datasets is no longer challenging with the help of powerful pre-trained language models (PLMs). However, the internal mechanism of these artifacts remains unclear, placing an obstacle to further understand these models. This paper focuses on conducting a series of analytical experiments to examine the relations between the multi-head self-attention and the final MRC system performance, revealing the potential explainability in PLM-based MRC models. To ensure the robustness of the analyses, we perform our experiments in a multilingual way on top of various PLMs. We discover that passage-to-question and passage understanding attentions are the most important ones in the question answering process, showing strong correlations to the final performance than other parts. Through comprehensive visualizations and case studies, we also observe several general findings on the attention maps, which can be helpful to understand how these models solve the questions. Elsevier 2022-03-31 /pmc/articles/PMC9019247/ /pubmed/35465050 http://dx.doi.org/10.1016/j.isci.2022.104176 Text en © 2022 The Author(s) https://creativecommons.org/licenses/by/4.0/This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Cui, Yiming Zhang, Wei-Nan Che, Wanxiang Liu, Ting Chen, Zhigang Wang, Shijin Multilingual multi-aspect explainability analyses on machine reading comprehension models |
title | Multilingual multi-aspect explainability analyses on machine reading comprehension models |
title_full | Multilingual multi-aspect explainability analyses on machine reading comprehension models |
title_fullStr | Multilingual multi-aspect explainability analyses on machine reading comprehension models |
title_full_unstemmed | Multilingual multi-aspect explainability analyses on machine reading comprehension models |
title_short | Multilingual multi-aspect explainability analyses on machine reading comprehension models |
title_sort | multilingual multi-aspect explainability analyses on machine reading comprehension models |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9019247/ https://www.ncbi.nlm.nih.gov/pubmed/35465050 http://dx.doi.org/10.1016/j.isci.2022.104176 |
work_keys_str_mv | AT cuiyiming multilingualmultiaspectexplainabilityanalysesonmachinereadingcomprehensionmodels AT zhangweinan multilingualmultiaspectexplainabilityanalysesonmachinereadingcomprehensionmodels AT chewanxiang multilingualmultiaspectexplainabilityanalysesonmachinereadingcomprehensionmodels AT liuting multilingualmultiaspectexplainabilityanalysesonmachinereadingcomprehensionmodels AT chenzhigang multilingualmultiaspectexplainabilityanalysesonmachinereadingcomprehensionmodels AT wangshijin multilingualmultiaspectexplainabilityanalysesonmachinereadingcomprehensionmodels |