Cargando…
Benchmarking infrastructure for mutation text mining
BACKGROUND: Experimental research on the automatic extraction of information about mutations from texts is greatly hindered by the lack of consensus evaluation infrastructure for the testing and benchmarking of mutation text mining systems. RESULTS: We propose a community-oriented annotation and ben...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2014
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3939821/ https://www.ncbi.nlm.nih.gov/pubmed/24568600 http://dx.doi.org/10.1186/2041-1480-5-11 |
_version_ | 1782305745145954304 |
---|---|
author | Klein, Artjom Riazanov, Alexandre Hindle, Matthew M Baker, Christopher JO |
author_facet | Klein, Artjom Riazanov, Alexandre Hindle, Matthew M Baker, Christopher JO |
author_sort | Klein, Artjom |
collection | PubMed |
description | BACKGROUND: Experimental research on the automatic extraction of information about mutations from texts is greatly hindered by the lack of consensus evaluation infrastructure for the testing and benchmarking of mutation text mining systems. RESULTS: We propose a community-oriented annotation and benchmarking infrastructure to support development, testing, benchmarking, and comparison of mutation text mining systems. The design is based on semantic standards, where RDF is used to represent annotations, an OWL ontology provides an extensible schema for the data and SPARQL is used to compute various performance metrics, so that in many cases no programming is needed to analyze results from a text mining system. While large benchmark corpora for biological entity and relation extraction are focused mostly on genes, proteins, diseases, and species, our benchmarking infrastructure fills the gap for mutation information. The core infrastructure comprises (1) an ontology for modelling annotations, (2) SPARQL queries for computing performance metrics, and (3) a sizeable collection of manually curated documents, that can support mutation grounding and mutation impact extraction experiments. CONCLUSION: We have developed the principal infrastructure for the benchmarking of mutation text mining tasks. The use of RDF and OWL as the representation for corpora ensures extensibility. The infrastructure is suitable for out-of-the-box use in several important scenarios and is ready, in its current state, for initial community adoption. |
format | Online Article Text |
id | pubmed-3939821 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2014 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-39398212014-03-04 Benchmarking infrastructure for mutation text mining Klein, Artjom Riazanov, Alexandre Hindle, Matthew M Baker, Christopher JO J Biomed Semantics Research BACKGROUND: Experimental research on the automatic extraction of information about mutations from texts is greatly hindered by the lack of consensus evaluation infrastructure for the testing and benchmarking of mutation text mining systems. RESULTS: We propose a community-oriented annotation and benchmarking infrastructure to support development, testing, benchmarking, and comparison of mutation text mining systems. The design is based on semantic standards, where RDF is used to represent annotations, an OWL ontology provides an extensible schema for the data and SPARQL is used to compute various performance metrics, so that in many cases no programming is needed to analyze results from a text mining system. While large benchmark corpora for biological entity and relation extraction are focused mostly on genes, proteins, diseases, and species, our benchmarking infrastructure fills the gap for mutation information. The core infrastructure comprises (1) an ontology for modelling annotations, (2) SPARQL queries for computing performance metrics, and (3) a sizeable collection of manually curated documents, that can support mutation grounding and mutation impact extraction experiments. CONCLUSION: We have developed the principal infrastructure for the benchmarking of mutation text mining tasks. The use of RDF and OWL as the representation for corpora ensures extensibility. The infrastructure is suitable for out-of-the-box use in several important scenarios and is ready, in its current state, for initial community adoption. BioMed Central 2014-02-25 /pmc/articles/PMC3939821/ /pubmed/24568600 http://dx.doi.org/10.1186/2041-1480-5-11 Text en Copyright © 2014 Klein et al.; licensee BioMed Central Ltd. http://creativecommons.org/licenses/by/2.0 This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |
spellingShingle | Research Klein, Artjom Riazanov, Alexandre Hindle, Matthew M Baker, Christopher JO Benchmarking infrastructure for mutation text mining |
title | Benchmarking infrastructure for mutation text mining |
title_full | Benchmarking infrastructure for mutation text mining |
title_fullStr | Benchmarking infrastructure for mutation text mining |
title_full_unstemmed | Benchmarking infrastructure for mutation text mining |
title_short | Benchmarking infrastructure for mutation text mining |
title_sort | benchmarking infrastructure for mutation text mining |
topic | Research |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3939821/ https://www.ncbi.nlm.nih.gov/pubmed/24568600 http://dx.doi.org/10.1186/2041-1480-5-11 |
work_keys_str_mv | AT kleinartjom benchmarkinginfrastructureformutationtextmining AT riazanovalexandre benchmarkinginfrastructureformutationtextmining AT hindlematthewm benchmarkinginfrastructureformutationtextmining AT bakerchristopherjo benchmarkinginfrastructureformutationtextmining |