Cargando…

Using text mining for study identification in systematic reviews: a systematic review of current approaches

BACKGROUND: The large and growing number of published studies, and their increasing rate of publication, makes the task of identifying relevant studies in an unbiased way for inclusion in systematic reviews both complex and time consuming. Text mining has been offered as a potential solution: throug...

Descripción completa

Detalles Bibliográficos
Autores principales: O’Mara-Eves, Alison, Thomas, James, McNaught, John, Miwa, Makoto, Ananiadou, Sophia
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2015
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4320539/
https://www.ncbi.nlm.nih.gov/pubmed/25588314
http://dx.doi.org/10.1186/2046-4053-4-5
_version_ 1782356136604729344
author O’Mara-Eves, Alison
Thomas, James
McNaught, John
Miwa, Makoto
Ananiadou, Sophia
author_facet O’Mara-Eves, Alison
Thomas, James
McNaught, John
Miwa, Makoto
Ananiadou, Sophia
author_sort O’Mara-Eves, Alison
collection PubMed
description BACKGROUND: The large and growing number of published studies, and their increasing rate of publication, makes the task of identifying relevant studies in an unbiased way for inclusion in systematic reviews both complex and time consuming. Text mining has been offered as a potential solution: through automating some of the screening process, reviewer time can be saved. The evidence base around the use of text mining for screening has not yet been pulled together systematically; this systematic review fills that research gap. Focusing mainly on non-technical issues, the review aims to increase awareness of the potential of these technologies and promote further collaborative research between the computer science and systematic review communities. METHODS: Five research questions led our review: what is the state of the evidence base; how has workload reduction been evaluated; what are the purposes of semi-automation and how effective are they; how have key contextual problems of applying text mining to the systematic review field been addressed; and what challenges to implementation have emerged? We answered these questions using standard systematic review methods: systematic and exhaustive searching, quality-assured data extraction and a narrative synthesis to synthesise findings. RESULTS: The evidence base is active and diverse; there is almost no replication between studies or collaboration between research teams and, whilst it is difficult to establish any overall conclusions about best approaches, it is clear that efficiencies and reductions in workload are potentially achievable. On the whole, most suggested that a saving in workload of between 30% and 70% might be possible, though sometimes the saving in workload is accompanied by the loss of 5% of relevant studies (i.e. a 95% recall). CONCLUSIONS: Using text mining to prioritise the order in which items are screened should be considered safe and ready for use in ‘live’ reviews. The use of text mining as a ‘second screener’ may also be used cautiously. The use of text mining to eliminate studies automatically should be considered promising, but not yet fully proven. In highly technical/clinical areas, it may be used with a high degree of confidence; but more developmental and evaluative work is needed in other disciplines. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/2046-4053-4-5) contains supplementary material, which is available to authorized users.
format Online
Article
Text
id pubmed-4320539
institution National Center for Biotechnology Information
language English
publishDate 2015
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-43205392015-02-08 Using text mining for study identification in systematic reviews: a systematic review of current approaches O’Mara-Eves, Alison Thomas, James McNaught, John Miwa, Makoto Ananiadou, Sophia Syst Rev Research BACKGROUND: The large and growing number of published studies, and their increasing rate of publication, makes the task of identifying relevant studies in an unbiased way for inclusion in systematic reviews both complex and time consuming. Text mining has been offered as a potential solution: through automating some of the screening process, reviewer time can be saved. The evidence base around the use of text mining for screening has not yet been pulled together systematically; this systematic review fills that research gap. Focusing mainly on non-technical issues, the review aims to increase awareness of the potential of these technologies and promote further collaborative research between the computer science and systematic review communities. METHODS: Five research questions led our review: what is the state of the evidence base; how has workload reduction been evaluated; what are the purposes of semi-automation and how effective are they; how have key contextual problems of applying text mining to the systematic review field been addressed; and what challenges to implementation have emerged? We answered these questions using standard systematic review methods: systematic and exhaustive searching, quality-assured data extraction and a narrative synthesis to synthesise findings. RESULTS: The evidence base is active and diverse; there is almost no replication between studies or collaboration between research teams and, whilst it is difficult to establish any overall conclusions about best approaches, it is clear that efficiencies and reductions in workload are potentially achievable. On the whole, most suggested that a saving in workload of between 30% and 70% might be possible, though sometimes the saving in workload is accompanied by the loss of 5% of relevant studies (i.e. a 95% recall). CONCLUSIONS: Using text mining to prioritise the order in which items are screened should be considered safe and ready for use in ‘live’ reviews. The use of text mining as a ‘second screener’ may also be used cautiously. The use of text mining to eliminate studies automatically should be considered promising, but not yet fully proven. In highly technical/clinical areas, it may be used with a high degree of confidence; but more developmental and evaluative work is needed in other disciplines. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/2046-4053-4-5) contains supplementary material, which is available to authorized users. BioMed Central 2015-01-14 /pmc/articles/PMC4320539/ /pubmed/25588314 http://dx.doi.org/10.1186/2046-4053-4-5 Text en © O’Mara-Eves et al.; licensee BioMed Central. 2015 This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Research
O’Mara-Eves, Alison
Thomas, James
McNaught, John
Miwa, Makoto
Ananiadou, Sophia
Using text mining for study identification in systematic reviews: a systematic review of current approaches
title Using text mining for study identification in systematic reviews: a systematic review of current approaches
title_full Using text mining for study identification in systematic reviews: a systematic review of current approaches
title_fullStr Using text mining for study identification in systematic reviews: a systematic review of current approaches
title_full_unstemmed Using text mining for study identification in systematic reviews: a systematic review of current approaches
title_short Using text mining for study identification in systematic reviews: a systematic review of current approaches
title_sort using text mining for study identification in systematic reviews: a systematic review of current approaches
topic Research
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4320539/
https://www.ncbi.nlm.nih.gov/pubmed/25588314
http://dx.doi.org/10.1186/2046-4053-4-5
work_keys_str_mv AT omaraevesalison usingtextminingforstudyidentificationinsystematicreviewsasystematicreviewofcurrentapproaches
AT thomasjames usingtextminingforstudyidentificationinsystematicreviewsasystematicreviewofcurrentapproaches
AT mcnaughtjohn usingtextminingforstudyidentificationinsystematicreviewsasystematicreviewofcurrentapproaches
AT miwamakoto usingtextminingforstudyidentificationinsystematicreviewsasystematicreviewofcurrentapproaches
AT ananiadousophia usingtextminingforstudyidentificationinsystematicreviewsasystematicreviewofcurrentapproaches