Cargando…
Performance of a rule-based semi-automated method to optimize chart abstraction for surveillance imaging among patients treated for non-small cell lung cancer
BACKGROUND: We aim to develop and test performance of a semi-automated method (computerized query combined with manual review) for chart abstraction in the identification and characterization of surveillance radiology imaging for post-treatment non-small cell lung cancer patients. METHODS: A gold st...
Autores principales: | , , , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9166440/ https://www.ncbi.nlm.nih.gov/pubmed/35659230 http://dx.doi.org/10.1186/s12911-022-01863-0 |
_version_ | 1784720603422392320 |
---|---|
author | Byrd, Catherine Ajawara, Ureka Laundry, Ryan Radin, John Bhandari, Prasha Leung, Ann Han, Summer Asch, Stephen M. Zeliadt, Steven Harris, Alex H. S. Backhus, Leah |
author_facet | Byrd, Catherine Ajawara, Ureka Laundry, Ryan Radin, John Bhandari, Prasha Leung, Ann Han, Summer Asch, Stephen M. Zeliadt, Steven Harris, Alex H. S. Backhus, Leah |
author_sort | Byrd, Catherine |
collection | PubMed |
description | BACKGROUND: We aim to develop and test performance of a semi-automated method (computerized query combined with manual review) for chart abstraction in the identification and characterization of surveillance radiology imaging for post-treatment non-small cell lung cancer patients. METHODS: A gold standard dataset consisting of 3011 radiology reports from 361 lung cancer patients treated at the Veterans Health Administration from 2008 to 2016 was manually created by an abstractor coding image type, image indication, and image findings. Computerized queries using a text search tool were performed to code reports. The primary endpoint of query performance was evaluated by sensitivity, positive predictive value (PPV), and F1 score. The secondary endpoint of efficiency compared semi-automated abstraction time to manual abstraction time using a separate dataset and the Wilcoxon rank-sum test. RESULTS: Query for image type demonstrated the highest sensitivity of 85%, PPV 95%, and F1 score 0.90. Query for image indication demonstrated sensitivity 72%, PPV 70%, and F1 score 0.71. The image findings queries ranged from sensitivity 75–85%, PPV 23–25%, and F1 score 0.36–0.37. Semi-automated abstraction with our best performing query (image type) improved abstraction times by 68% per patient compared to manual abstraction alone (from median 21.5 min (interquartile range 16.0) to 6.9 min (interquartile range 9.5), p < 0.005). CONCLUSIONS: Semi-automated abstraction using the best performing query of image type improved abstraction efficiency while preserving data accuracy. The computerized query acts as a pre-processing tool for manual abstraction by restricting effort to relevant images. Determining image indication and findings requires the addition of manual review for a semi-automatic abstraction approach in order to ensure data accuracy. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12911-022-01863-0. |
format | Online Article Text |
id | pubmed-9166440 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-91664402022-06-05 Performance of a rule-based semi-automated method to optimize chart abstraction for surveillance imaging among patients treated for non-small cell lung cancer Byrd, Catherine Ajawara, Ureka Laundry, Ryan Radin, John Bhandari, Prasha Leung, Ann Han, Summer Asch, Stephen M. Zeliadt, Steven Harris, Alex H. S. Backhus, Leah BMC Med Inform Decis Mak Research BACKGROUND: We aim to develop and test performance of a semi-automated method (computerized query combined with manual review) for chart abstraction in the identification and characterization of surveillance radiology imaging for post-treatment non-small cell lung cancer patients. METHODS: A gold standard dataset consisting of 3011 radiology reports from 361 lung cancer patients treated at the Veterans Health Administration from 2008 to 2016 was manually created by an abstractor coding image type, image indication, and image findings. Computerized queries using a text search tool were performed to code reports. The primary endpoint of query performance was evaluated by sensitivity, positive predictive value (PPV), and F1 score. The secondary endpoint of efficiency compared semi-automated abstraction time to manual abstraction time using a separate dataset and the Wilcoxon rank-sum test. RESULTS: Query for image type demonstrated the highest sensitivity of 85%, PPV 95%, and F1 score 0.90. Query for image indication demonstrated sensitivity 72%, PPV 70%, and F1 score 0.71. The image findings queries ranged from sensitivity 75–85%, PPV 23–25%, and F1 score 0.36–0.37. Semi-automated abstraction with our best performing query (image type) improved abstraction times by 68% per patient compared to manual abstraction alone (from median 21.5 min (interquartile range 16.0) to 6.9 min (interquartile range 9.5), p < 0.005). CONCLUSIONS: Semi-automated abstraction using the best performing query of image type improved abstraction efficiency while preserving data accuracy. The computerized query acts as a pre-processing tool for manual abstraction by restricting effort to relevant images. Determining image indication and findings requires the addition of manual review for a semi-automatic abstraction approach in order to ensure data accuracy. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12911-022-01863-0. BioMed Central 2022-06-03 /pmc/articles/PMC9166440/ /pubmed/35659230 http://dx.doi.org/10.1186/s12911-022-01863-0 Text en © The Author(s) 2022 https://creativecommons.org/licenses/by/4.0/Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/ (https://creativecommons.org/publicdomain/zero/1.0/) ) applies to the data made available in this article, unless otherwise stated in a credit line to the data. |
spellingShingle | Research Byrd, Catherine Ajawara, Ureka Laundry, Ryan Radin, John Bhandari, Prasha Leung, Ann Han, Summer Asch, Stephen M. Zeliadt, Steven Harris, Alex H. S. Backhus, Leah Performance of a rule-based semi-automated method to optimize chart abstraction for surveillance imaging among patients treated for non-small cell lung cancer |
title | Performance of a rule-based semi-automated method to optimize chart abstraction for surveillance imaging among patients treated for non-small cell lung cancer |
title_full | Performance of a rule-based semi-automated method to optimize chart abstraction for surveillance imaging among patients treated for non-small cell lung cancer |
title_fullStr | Performance of a rule-based semi-automated method to optimize chart abstraction for surveillance imaging among patients treated for non-small cell lung cancer |
title_full_unstemmed | Performance of a rule-based semi-automated method to optimize chart abstraction for surveillance imaging among patients treated for non-small cell lung cancer |
title_short | Performance of a rule-based semi-automated method to optimize chart abstraction for surveillance imaging among patients treated for non-small cell lung cancer |
title_sort | performance of a rule-based semi-automated method to optimize chart abstraction for surveillance imaging among patients treated for non-small cell lung cancer |
topic | Research |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9166440/ https://www.ncbi.nlm.nih.gov/pubmed/35659230 http://dx.doi.org/10.1186/s12911-022-01863-0 |
work_keys_str_mv | AT byrdcatherine performanceofarulebasedsemiautomatedmethodtooptimizechartabstractionforsurveillanceimagingamongpatientstreatedfornonsmallcelllungcancer AT ajawaraureka performanceofarulebasedsemiautomatedmethodtooptimizechartabstractionforsurveillanceimagingamongpatientstreatedfornonsmallcelllungcancer AT laundryryan performanceofarulebasedsemiautomatedmethodtooptimizechartabstractionforsurveillanceimagingamongpatientstreatedfornonsmallcelllungcancer AT radinjohn performanceofarulebasedsemiautomatedmethodtooptimizechartabstractionforsurveillanceimagingamongpatientstreatedfornonsmallcelllungcancer AT bhandariprasha performanceofarulebasedsemiautomatedmethodtooptimizechartabstractionforsurveillanceimagingamongpatientstreatedfornonsmallcelllungcancer AT leungann performanceofarulebasedsemiautomatedmethodtooptimizechartabstractionforsurveillanceimagingamongpatientstreatedfornonsmallcelllungcancer AT hansummer performanceofarulebasedsemiautomatedmethodtooptimizechartabstractionforsurveillanceimagingamongpatientstreatedfornonsmallcelllungcancer AT aschstephenm performanceofarulebasedsemiautomatedmethodtooptimizechartabstractionforsurveillanceimagingamongpatientstreatedfornonsmallcelllungcancer AT zeliadtsteven performanceofarulebasedsemiautomatedmethodtooptimizechartabstractionforsurveillanceimagingamongpatientstreatedfornonsmallcelllungcancer AT harrisalexhs performanceofarulebasedsemiautomatedmethodtooptimizechartabstractionforsurveillanceimagingamongpatientstreatedfornonsmallcelllungcancer AT backhusleah performanceofarulebasedsemiautomatedmethodtooptimizechartabstractionforsurveillanceimagingamongpatientstreatedfornonsmallcelllungcancer |