Cargando…

12 years on – Is the NLM medical text indexer still useful and relevant?

BACKGROUND: Facing a growing workload and dwindling resources, the US National Library of Medicine (NLM) created the Indexing Initiative project in 1996. This cross-library team’s mission is to explore indexing methodologies for ensuring quality and currency of NLM document collections. The NLM Medi...

Descripción completa

Detalles Bibliográficos
Autores principales: Mork, James, Aronson, Alan, Demner-Fushman, Dina
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2017
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5324252/
https://www.ncbi.nlm.nih.gov/pubmed/28231809
http://dx.doi.org/10.1186/s13326-017-0113-5
_version_ 1782510188018794496
author Mork, James
Aronson, Alan
Demner-Fushman, Dina
author_facet Mork, James
Aronson, Alan
Demner-Fushman, Dina
author_sort Mork, James
collection PubMed
description BACKGROUND: Facing a growing workload and dwindling resources, the US National Library of Medicine (NLM) created the Indexing Initiative project in 1996. This cross-library team’s mission is to explore indexing methodologies for ensuring quality and currency of NLM document collections. The NLM Medical Text Indexer (MTI) is the main product of this project and has been providing automated indexing recommendations since 2002. After all of this time, the questions arise whether MTI is still useful and relevant. METHODS: To answer the question about MTI usefulness, we track a wide variety of statistics related to how frequently MEDLINE indexers refer to MTI recommendations, how well MTI performs against human indexing, and how often MTI is used. To answer the question of MTI relevancy compared to other available tools, we have participated in the 2013 and 2014 BioASQ Challenges. The BioASQ Challenges have provided us with an unbiased comparison between the MTI system and other systems performing the same task. RESULTS: Indexers have continually increased their use of MTI recommendations over the years from 15.75% of the articles they index in 2002 to 62.44% in 2014 showing that the indexers find MTI to be increasingly useful. The MTI performance statistics show significant improvement in Precision (+0.2992) and F(1) (+0.1997) with modest gains in Recall (+0.0454) over the years. MTI consistency is comparable to the available indexer consistency studies. MTI performed well in both of the BioASQ Challenges ranking within the top tier teams. CONCLUSIONS: Based on our findings, yes, MTI is still relevant and useful, and needs to be improved and expanded. The BioASQ Challenge results have shown that we need to incorporate more machine learning into MTI while still retaining the indexing rules that have earned MTI the indexers’ trust over the years. We also need to expand MTI through the use of full text, when and where it is available, to provide coverage of indexing terms that are typically only found in the full text. The role of MTI at NLM is also expanding into new areas, further reinforcing the idea that MTI is increasingly useful and relevant.
format Online
Article
Text
id pubmed-5324252
institution National Center for Biotechnology Information
language English
publishDate 2017
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-53242522017-03-01 12 years on – Is the NLM medical text indexer still useful and relevant? Mork, James Aronson, Alan Demner-Fushman, Dina J Biomed Semantics Research BACKGROUND: Facing a growing workload and dwindling resources, the US National Library of Medicine (NLM) created the Indexing Initiative project in 1996. This cross-library team’s mission is to explore indexing methodologies for ensuring quality and currency of NLM document collections. The NLM Medical Text Indexer (MTI) is the main product of this project and has been providing automated indexing recommendations since 2002. After all of this time, the questions arise whether MTI is still useful and relevant. METHODS: To answer the question about MTI usefulness, we track a wide variety of statistics related to how frequently MEDLINE indexers refer to MTI recommendations, how well MTI performs against human indexing, and how often MTI is used. To answer the question of MTI relevancy compared to other available tools, we have participated in the 2013 and 2014 BioASQ Challenges. The BioASQ Challenges have provided us with an unbiased comparison between the MTI system and other systems performing the same task. RESULTS: Indexers have continually increased their use of MTI recommendations over the years from 15.75% of the articles they index in 2002 to 62.44% in 2014 showing that the indexers find MTI to be increasingly useful. The MTI performance statistics show significant improvement in Precision (+0.2992) and F(1) (+0.1997) with modest gains in Recall (+0.0454) over the years. MTI consistency is comparable to the available indexer consistency studies. MTI performed well in both of the BioASQ Challenges ranking within the top tier teams. CONCLUSIONS: Based on our findings, yes, MTI is still relevant and useful, and needs to be improved and expanded. The BioASQ Challenge results have shown that we need to incorporate more machine learning into MTI while still retaining the indexing rules that have earned MTI the indexers’ trust over the years. We also need to expand MTI through the use of full text, when and where it is available, to provide coverage of indexing terms that are typically only found in the full text. The role of MTI at NLM is also expanding into new areas, further reinforcing the idea that MTI is increasingly useful and relevant. BioMed Central 2017-02-23 /pmc/articles/PMC5324252/ /pubmed/28231809 http://dx.doi.org/10.1186/s13326-017-0113-5 Text en © The Author(s) 2017 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License(http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Research
Mork, James
Aronson, Alan
Demner-Fushman, Dina
12 years on – Is the NLM medical text indexer still useful and relevant?
title 12 years on – Is the NLM medical text indexer still useful and relevant?
title_full 12 years on – Is the NLM medical text indexer still useful and relevant?
title_fullStr 12 years on – Is the NLM medical text indexer still useful and relevant?
title_full_unstemmed 12 years on – Is the NLM medical text indexer still useful and relevant?
title_short 12 years on – Is the NLM medical text indexer still useful and relevant?
title_sort 12 years on – is the nlm medical text indexer still useful and relevant?
topic Research
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5324252/
https://www.ncbi.nlm.nih.gov/pubmed/28231809
http://dx.doi.org/10.1186/s13326-017-0113-5
work_keys_str_mv AT morkjames 12yearsonisthenlmmedicaltextindexerstillusefulandrelevant
AT aronsonalan 12yearsonisthenlmmedicaltextindexerstillusefulandrelevant
AT demnerfushmandina 12yearsonisthenlmmedicaltextindexerstillusefulandrelevant