Cargando…
12 years on – Is the NLM medical text indexer still useful and relevant?
BACKGROUND: Facing a growing workload and dwindling resources, the US National Library of Medicine (NLM) created the Indexing Initiative project in 1996. This cross-library team’s mission is to explore indexing methodologies for ensuring quality and currency of NLM document collections. The NLM Medi...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2017
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5324252/ https://www.ncbi.nlm.nih.gov/pubmed/28231809 http://dx.doi.org/10.1186/s13326-017-0113-5 |
_version_ | 1782510188018794496 |
---|---|
author | Mork, James Aronson, Alan Demner-Fushman, Dina |
author_facet | Mork, James Aronson, Alan Demner-Fushman, Dina |
author_sort | Mork, James |
collection | PubMed |
description | BACKGROUND: Facing a growing workload and dwindling resources, the US National Library of Medicine (NLM) created the Indexing Initiative project in 1996. This cross-library team’s mission is to explore indexing methodologies for ensuring quality and currency of NLM document collections. The NLM Medical Text Indexer (MTI) is the main product of this project and has been providing automated indexing recommendations since 2002. After all of this time, the questions arise whether MTI is still useful and relevant. METHODS: To answer the question about MTI usefulness, we track a wide variety of statistics related to how frequently MEDLINE indexers refer to MTI recommendations, how well MTI performs against human indexing, and how often MTI is used. To answer the question of MTI relevancy compared to other available tools, we have participated in the 2013 and 2014 BioASQ Challenges. The BioASQ Challenges have provided us with an unbiased comparison between the MTI system and other systems performing the same task. RESULTS: Indexers have continually increased their use of MTI recommendations over the years from 15.75% of the articles they index in 2002 to 62.44% in 2014 showing that the indexers find MTI to be increasingly useful. The MTI performance statistics show significant improvement in Precision (+0.2992) and F(1) (+0.1997) with modest gains in Recall (+0.0454) over the years. MTI consistency is comparable to the available indexer consistency studies. MTI performed well in both of the BioASQ Challenges ranking within the top tier teams. CONCLUSIONS: Based on our findings, yes, MTI is still relevant and useful, and needs to be improved and expanded. The BioASQ Challenge results have shown that we need to incorporate more machine learning into MTI while still retaining the indexing rules that have earned MTI the indexers’ trust over the years. We also need to expand MTI through the use of full text, when and where it is available, to provide coverage of indexing terms that are typically only found in the full text. The role of MTI at NLM is also expanding into new areas, further reinforcing the idea that MTI is increasingly useful and relevant. |
format | Online Article Text |
id | pubmed-5324252 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2017 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-53242522017-03-01 12 years on – Is the NLM medical text indexer still useful and relevant? Mork, James Aronson, Alan Demner-Fushman, Dina J Biomed Semantics Research BACKGROUND: Facing a growing workload and dwindling resources, the US National Library of Medicine (NLM) created the Indexing Initiative project in 1996. This cross-library team’s mission is to explore indexing methodologies for ensuring quality and currency of NLM document collections. The NLM Medical Text Indexer (MTI) is the main product of this project and has been providing automated indexing recommendations since 2002. After all of this time, the questions arise whether MTI is still useful and relevant. METHODS: To answer the question about MTI usefulness, we track a wide variety of statistics related to how frequently MEDLINE indexers refer to MTI recommendations, how well MTI performs against human indexing, and how often MTI is used. To answer the question of MTI relevancy compared to other available tools, we have participated in the 2013 and 2014 BioASQ Challenges. The BioASQ Challenges have provided us with an unbiased comparison between the MTI system and other systems performing the same task. RESULTS: Indexers have continually increased their use of MTI recommendations over the years from 15.75% of the articles they index in 2002 to 62.44% in 2014 showing that the indexers find MTI to be increasingly useful. The MTI performance statistics show significant improvement in Precision (+0.2992) and F(1) (+0.1997) with modest gains in Recall (+0.0454) over the years. MTI consistency is comparable to the available indexer consistency studies. MTI performed well in both of the BioASQ Challenges ranking within the top tier teams. CONCLUSIONS: Based on our findings, yes, MTI is still relevant and useful, and needs to be improved and expanded. The BioASQ Challenge results have shown that we need to incorporate more machine learning into MTI while still retaining the indexing rules that have earned MTI the indexers’ trust over the years. We also need to expand MTI through the use of full text, when and where it is available, to provide coverage of indexing terms that are typically only found in the full text. The role of MTI at NLM is also expanding into new areas, further reinforcing the idea that MTI is increasingly useful and relevant. BioMed Central 2017-02-23 /pmc/articles/PMC5324252/ /pubmed/28231809 http://dx.doi.org/10.1186/s13326-017-0113-5 Text en © The Author(s) 2017 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. |
spellingShingle | Research Mork, James Aronson, Alan Demner-Fushman, Dina 12 years on – Is the NLM medical text indexer still useful and relevant? |
title | 12 years on – Is the NLM medical text indexer still useful and relevant? |
title_full | 12 years on – Is the NLM medical text indexer still useful and relevant? |
title_fullStr | 12 years on – Is the NLM medical text indexer still useful and relevant? |
title_full_unstemmed | 12 years on – Is the NLM medical text indexer still useful and relevant? |
title_short | 12 years on – Is the NLM medical text indexer still useful and relevant? |
title_sort | 12 years on – is the nlm medical text indexer still useful and relevant? |
topic | Research |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5324252/ https://www.ncbi.nlm.nih.gov/pubmed/28231809 http://dx.doi.org/10.1186/s13326-017-0113-5 |
work_keys_str_mv | AT morkjames 12yearsonisthenlmmedicaltextindexerstillusefulandrelevant AT aronsonalan 12yearsonisthenlmmedicaltextindexerstillusefulandrelevant AT demnerfushmandina 12yearsonisthenlmmedicaltextindexerstillusefulandrelevant |