Cargando…
Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering
The increasing expansion of biomedical documents has increased the number of natural language textual resources related to the current applications. Meanwhile, there has been a great interest in extracting useful information from meaningful coherent groupings of textual content documents in the last...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9141535/ https://www.ncbi.nlm.nih.gov/pubmed/35627429 http://dx.doi.org/10.3390/ijerph19105893 |
_version_ | 1784715369179512832 |
---|---|
author | Davagdorj, Khishigsuren Wang, Ling Li, Meijing Pham, Van-Huy Ryu, Keun Ho Theera-Umpon, Nipon |
author_facet | Davagdorj, Khishigsuren Wang, Ling Li, Meijing Pham, Van-Huy Ryu, Keun Ho Theera-Umpon, Nipon |
author_sort | Davagdorj, Khishigsuren |
collection | PubMed |
description | The increasing expansion of biomedical documents has increased the number of natural language textual resources related to the current applications. Meanwhile, there has been a great interest in extracting useful information from meaningful coherent groupings of textual content documents in the last decade. However, it is challenging to discover informative representations and define relevant articles from the rapidly growing biomedical literature due to the unsupervised nature of document clustering. Moreover, empirical investigations demonstrated that traditional text clustering methods produce unsatisfactory results in terms of non-contextualized vector space representations because that neglect the semantic relationship between biomedical texts. Recently, pre-trained language models have emerged as successful in a wide range of natural language processing applications. In this paper, we propose the Gaussian Mixture Model-based efficient clustering framework that incorporates substantially pre-trained (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) BioBERT domain-specific language representations to enhance the clustering accuracy. Our proposed framework consists of main three phases. First, classic text pre-processing techniques are used biomedical document data, which crawled from the PubMed repository. Second, representative vectors are extracted from a pre-trained BioBERT language model for biomedical text mining. Third, we employ the Gaussian Mixture Model as a clustering algorithm, which allows us to assign labels for each biomedical document. In order to prove the efficiency of our proposed model, we conducted a comprehensive experimental analysis utilizing several clustering algorithms while combining diverse embedding techniques. Consequently, the experimental results show that the proposed model outperforms the benchmark models by reaching performance measures of Fowlkes mallows score, silhouette coefficient, adjusted rand index, Davies-Bouldin score of 0.7817, 0.3765, 0.4478, 1.6849, respectively. We expect the outcomes of this study will assist domain specialists in comprehending thematically cohesive documents in the healthcare field. |
format | Online Article Text |
id | pubmed-9141535 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-91415352022-05-28 Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering Davagdorj, Khishigsuren Wang, Ling Li, Meijing Pham, Van-Huy Ryu, Keun Ho Theera-Umpon, Nipon Int J Environ Res Public Health Article The increasing expansion of biomedical documents has increased the number of natural language textual resources related to the current applications. Meanwhile, there has been a great interest in extracting useful information from meaningful coherent groupings of textual content documents in the last decade. However, it is challenging to discover informative representations and define relevant articles from the rapidly growing biomedical literature due to the unsupervised nature of document clustering. Moreover, empirical investigations demonstrated that traditional text clustering methods produce unsatisfactory results in terms of non-contextualized vector space representations because that neglect the semantic relationship between biomedical texts. Recently, pre-trained language models have emerged as successful in a wide range of natural language processing applications. In this paper, we propose the Gaussian Mixture Model-based efficient clustering framework that incorporates substantially pre-trained (Bidirectional Encoder Representations from Transformers for Biomedical Text Mining) BioBERT domain-specific language representations to enhance the clustering accuracy. Our proposed framework consists of main three phases. First, classic text pre-processing techniques are used biomedical document data, which crawled from the PubMed repository. Second, representative vectors are extracted from a pre-trained BioBERT language model for biomedical text mining. Third, we employ the Gaussian Mixture Model as a clustering algorithm, which allows us to assign labels for each biomedical document. In order to prove the efficiency of our proposed model, we conducted a comprehensive experimental analysis utilizing several clustering algorithms while combining diverse embedding techniques. Consequently, the experimental results show that the proposed model outperforms the benchmark models by reaching performance measures of Fowlkes mallows score, silhouette coefficient, adjusted rand index, Davies-Bouldin score of 0.7817, 0.3765, 0.4478, 1.6849, respectively. We expect the outcomes of this study will assist domain specialists in comprehending thematically cohesive documents in the healthcare field. MDPI 2022-05-12 /pmc/articles/PMC9141535/ /pubmed/35627429 http://dx.doi.org/10.3390/ijerph19105893 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Davagdorj, Khishigsuren Wang, Ling Li, Meijing Pham, Van-Huy Ryu, Keun Ho Theera-Umpon, Nipon Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering |
title | Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering |
title_full | Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering |
title_fullStr | Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering |
title_full_unstemmed | Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering |
title_short | Discovering Thematically Coherent Biomedical Documents Using Contextualized Bidirectional Encoder Representations from Transformers-Based Clustering |
title_sort | discovering thematically coherent biomedical documents using contextualized bidirectional encoder representations from transformers-based clustering |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9141535/ https://www.ncbi.nlm.nih.gov/pubmed/35627429 http://dx.doi.org/10.3390/ijerph19105893 |
work_keys_str_mv | AT davagdorjkhishigsuren discoveringthematicallycoherentbiomedicaldocumentsusingcontextualizedbidirectionalencoderrepresentationsfromtransformersbasedclustering AT wangling discoveringthematicallycoherentbiomedicaldocumentsusingcontextualizedbidirectionalencoderrepresentationsfromtransformersbasedclustering AT limeijing discoveringthematicallycoherentbiomedicaldocumentsusingcontextualizedbidirectionalencoderrepresentationsfromtransformersbasedclustering AT phamvanhuy discoveringthematicallycoherentbiomedicaldocumentsusingcontextualizedbidirectionalencoderrepresentationsfromtransformersbasedclustering AT ryukeunho discoveringthematicallycoherentbiomedicaldocumentsusingcontextualizedbidirectionalencoderrepresentationsfromtransformersbasedclustering AT theeraumponnipon discoveringthematicallycoherentbiomedicaldocumentsusingcontextualizedbidirectionalencoderrepresentationsfromtransformersbasedclustering |