Cargando…
Chemical–protein interaction extraction via contextualized word representations and multihead attention
A rich source of chemical–protein interactions (CPIs) is locked in the exponentially growing biomedical literature. Automatic extraction of CPIs is a crucial task in biomedical natural language processing (NLP), which has great benefits for pharmacological and clinical research. Deep context represe...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Oxford University Press
2019
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6534182/ https://www.ncbi.nlm.nih.gov/pubmed/31125403 http://dx.doi.org/10.1093/database/baz054 |
_version_ | 1783421356360073216 |
---|---|
author | Zhang, Yijia Lin, Hongfei Yang, Zhihao Wang, Jian Sun, Yuanyuan |
author_facet | Zhang, Yijia Lin, Hongfei Yang, Zhihao Wang, Jian Sun, Yuanyuan |
author_sort | Zhang, Yijia |
collection | PubMed |
description | A rich source of chemical–protein interactions (CPIs) is locked in the exponentially growing biomedical literature. Automatic extraction of CPIs is a crucial task in biomedical natural language processing (NLP), which has great benefits for pharmacological and clinical research. Deep context representation and multihead attention are recent developments in deep learning and have shown their potential in some NLP tasks. Unlike traditional word embedding, deep context representation has the ability to generate comprehensive sentence representation based on the sentence context. The multihead attention mechanism can effectively learn the important features from different heads and emphasize the relatively important features. Integrating deep context representation and multihead attention with a neural network-based model may improve CPI extraction. We present a deep neural model for CPI extraction based on deep context representation and multihead attention. Our model mainly consists of the following three parts: a deep context representation layer, a bidirectional long short-term memory networks (Bi-LSTMs) layer and a multihead attention layer. The deep context representation is employed to provide more comprehensive feature input for Bi-LSTMs. The multihead attention can effectively emphasize the important part of the Bi-LSTMs output. We evaluated our method on the public ChemProt corpus. These experimental results show that both deep context representation and multihead attention are helpful in CPI extraction. Our method can compete with other state-of-the-art methods on ChemProt corpus. |
format | Online Article Text |
id | pubmed-6534182 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2019 |
publisher | Oxford University Press |
record_format | MEDLINE/PubMed |
spelling | pubmed-65341822019-05-28 Chemical–protein interaction extraction via contextualized word representations and multihead attention Zhang, Yijia Lin, Hongfei Yang, Zhihao Wang, Jian Sun, Yuanyuan Database (Oxford) Original Article A rich source of chemical–protein interactions (CPIs) is locked in the exponentially growing biomedical literature. Automatic extraction of CPIs is a crucial task in biomedical natural language processing (NLP), which has great benefits for pharmacological and clinical research. Deep context representation and multihead attention are recent developments in deep learning and have shown their potential in some NLP tasks. Unlike traditional word embedding, deep context representation has the ability to generate comprehensive sentence representation based on the sentence context. The multihead attention mechanism can effectively learn the important features from different heads and emphasize the relatively important features. Integrating deep context representation and multihead attention with a neural network-based model may improve CPI extraction. We present a deep neural model for CPI extraction based on deep context representation and multihead attention. Our model mainly consists of the following three parts: a deep context representation layer, a bidirectional long short-term memory networks (Bi-LSTMs) layer and a multihead attention layer. The deep context representation is employed to provide more comprehensive feature input for Bi-LSTMs. The multihead attention can effectively emphasize the important part of the Bi-LSTMs output. We evaluated our method on the public ChemProt corpus. These experimental results show that both deep context representation and multihead attention are helpful in CPI extraction. Our method can compete with other state-of-the-art methods on ChemProt corpus. Oxford University Press 2019-05-24 /pmc/articles/PMC6534182/ /pubmed/31125403 http://dx.doi.org/10.1093/database/baz054 Text en © The Author(s) 2019. Published by Oxford University Press. http://creativecommons.org/licenses/by/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. |
spellingShingle | Original Article Zhang, Yijia Lin, Hongfei Yang, Zhihao Wang, Jian Sun, Yuanyuan Chemical–protein interaction extraction via contextualized word representations and multihead attention |
title | Chemical–protein interaction extraction via contextualized word representations and multihead attention |
title_full | Chemical–protein interaction extraction via contextualized word representations and multihead attention |
title_fullStr | Chemical–protein interaction extraction via contextualized word representations and multihead attention |
title_full_unstemmed | Chemical–protein interaction extraction via contextualized word representations and multihead attention |
title_short | Chemical–protein interaction extraction via contextualized word representations and multihead attention |
title_sort | chemical–protein interaction extraction via contextualized word representations and multihead attention |
topic | Original Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6534182/ https://www.ncbi.nlm.nih.gov/pubmed/31125403 http://dx.doi.org/10.1093/database/baz054 |
work_keys_str_mv | AT zhangyijia chemicalproteininteractionextractionviacontextualizedwordrepresentationsandmultiheadattention AT linhongfei chemicalproteininteractionextractionviacontextualizedwordrepresentationsandmultiheadattention AT yangzhihao chemicalproteininteractionextractionviacontextualizedwordrepresentationsandmultiheadattention AT wangjian chemicalproteininteractionextractionviacontextualizedwordrepresentationsandmultiheadattention AT sunyuanyuan chemicalproteininteractionextractionviacontextualizedwordrepresentationsandmultiheadattention |