Cargando…
Automated Recognition of Visual Acuity Measurements in Ophthalmology Clinical Notes Using Deep Learning
PURPOSE: Visual acuity (VA) is a critical component of the eye examination but is often only documented in electronic health records (EHRs) as unstructured free-text notes, making it challenging to use in research. This study aimed to improve on existing rule-based algorithms by developing and evalu...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Elsevier
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10587603/ https://www.ncbi.nlm.nih.gov/pubmed/37868799 http://dx.doi.org/10.1016/j.xops.2023.100371 |
_version_ | 1785123402110992384 |
---|---|
author | Bernstein, Isaac A. Koornwinder, Abigail Hwang, Hannah H. Wang, Sophia Y. |
author_facet | Bernstein, Isaac A. Koornwinder, Abigail Hwang, Hannah H. Wang, Sophia Y. |
author_sort | Bernstein, Isaac A. |
collection | PubMed |
description | PURPOSE: Visual acuity (VA) is a critical component of the eye examination but is often only documented in electronic health records (EHRs) as unstructured free-text notes, making it challenging to use in research. This study aimed to improve on existing rule-based algorithms by developing and evaluating deep learning models to perform named entity recognition of different types of VA measurements and their lateralities from free-text ophthalmology notes: VA for each of the right and left eyes, with and without glasses correction, and with and without pinhole. DESIGN: Cross-sectional study. SUBJECTS: A total of 319 756 clinical notes with documented VA measurements from approximately 90 000 patients were included. METHODS: The notes were split into train, validation, and test sets. Bidirectional Encoder Representations from Transformers (BERT) models were fine-tuned to identify VA measurements from the progress notes and included BERT models pretrained on biomedical literature (BioBERT), critical care EHR notes (ClinicalBERT), both (BlueBERT), and a lighter version of BERT with 40% fewer parameters (DistilBERT). A baseline rule-based algorithm was created to recognize the same VA entities to compare against BERT models. MAIN OUTCOME MEASURES: Model performance was evaluated on a held-out test set using microaveraged precision, recall, and F1 score for all entities. RESULTS: On the human-annotated subset, BlueBERT achieved the best microaveraged F1 score (F1 = 0.92), followed by ClinicalBERT (F1 = 0.91), DistilBERT (F1 = 0.90), BioBERT (F1 = 0.84), and the baseline model (F1 = 0.83). Common errors included labeling VA in sections outside of the examination portion of the note, difficulties labeling current VA alongside a series of past VAs, and missing nonnumeric VAs. CONCLUSIONS: This study demonstrates that deep learning models are capable of identifying VA measurements from free-text ophthalmology notes with high precision and recall, achieving significant performance improvements over a rule-based algorithm. The ability to recognize VA from free-text notes would enable a more detailed characterization of ophthalmology patient cohorts and enhance the development of models to predict ophthalmology outcomes. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article. |
format | Online Article Text |
id | pubmed-10587603 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Elsevier |
record_format | MEDLINE/PubMed |
spelling | pubmed-105876032023-10-21 Automated Recognition of Visual Acuity Measurements in Ophthalmology Clinical Notes Using Deep Learning Bernstein, Isaac A. Koornwinder, Abigail Hwang, Hannah H. Wang, Sophia Y. Ophthalmol Sci Original Article PURPOSE: Visual acuity (VA) is a critical component of the eye examination but is often only documented in electronic health records (EHRs) as unstructured free-text notes, making it challenging to use in research. This study aimed to improve on existing rule-based algorithms by developing and evaluating deep learning models to perform named entity recognition of different types of VA measurements and their lateralities from free-text ophthalmology notes: VA for each of the right and left eyes, with and without glasses correction, and with and without pinhole. DESIGN: Cross-sectional study. SUBJECTS: A total of 319 756 clinical notes with documented VA measurements from approximately 90 000 patients were included. METHODS: The notes were split into train, validation, and test sets. Bidirectional Encoder Representations from Transformers (BERT) models were fine-tuned to identify VA measurements from the progress notes and included BERT models pretrained on biomedical literature (BioBERT), critical care EHR notes (ClinicalBERT), both (BlueBERT), and a lighter version of BERT with 40% fewer parameters (DistilBERT). A baseline rule-based algorithm was created to recognize the same VA entities to compare against BERT models. MAIN OUTCOME MEASURES: Model performance was evaluated on a held-out test set using microaveraged precision, recall, and F1 score for all entities. RESULTS: On the human-annotated subset, BlueBERT achieved the best microaveraged F1 score (F1 = 0.92), followed by ClinicalBERT (F1 = 0.91), DistilBERT (F1 = 0.90), BioBERT (F1 = 0.84), and the baseline model (F1 = 0.83). Common errors included labeling VA in sections outside of the examination portion of the note, difficulties labeling current VA alongside a series of past VAs, and missing nonnumeric VAs. CONCLUSIONS: This study demonstrates that deep learning models are capable of identifying VA measurements from free-text ophthalmology notes with high precision and recall, achieving significant performance improvements over a rule-based algorithm. The ability to recognize VA from free-text notes would enable a more detailed characterization of ophthalmology patient cohorts and enhance the development of models to predict ophthalmology outcomes. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article. Elsevier 2023-07-19 /pmc/articles/PMC10587603/ /pubmed/37868799 http://dx.doi.org/10.1016/j.xops.2023.100371 Text en © 2023 by the American Academy of Ophthalmology. https://creativecommons.org/licenses/by-nc-nd/4.0/This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). |
spellingShingle | Original Article Bernstein, Isaac A. Koornwinder, Abigail Hwang, Hannah H. Wang, Sophia Y. Automated Recognition of Visual Acuity Measurements in Ophthalmology Clinical Notes Using Deep Learning |
title | Automated Recognition of Visual Acuity Measurements in Ophthalmology Clinical Notes Using Deep Learning |
title_full | Automated Recognition of Visual Acuity Measurements in Ophthalmology Clinical Notes Using Deep Learning |
title_fullStr | Automated Recognition of Visual Acuity Measurements in Ophthalmology Clinical Notes Using Deep Learning |
title_full_unstemmed | Automated Recognition of Visual Acuity Measurements in Ophthalmology Clinical Notes Using Deep Learning |
title_short | Automated Recognition of Visual Acuity Measurements in Ophthalmology Clinical Notes Using Deep Learning |
title_sort | automated recognition of visual acuity measurements in ophthalmology clinical notes using deep learning |
topic | Original Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10587603/ https://www.ncbi.nlm.nih.gov/pubmed/37868799 http://dx.doi.org/10.1016/j.xops.2023.100371 |
work_keys_str_mv | AT bernsteinisaaca automatedrecognitionofvisualacuitymeasurementsinophthalmologyclinicalnotesusingdeeplearning AT koornwinderabigail automatedrecognitionofvisualacuitymeasurementsinophthalmologyclinicalnotesusingdeeplearning AT hwanghannahh automatedrecognitionofvisualacuitymeasurementsinophthalmologyclinicalnotesusingdeeplearning AT wangsophiay automatedrecognitionofvisualacuitymeasurementsinophthalmologyclinicalnotesusingdeeplearning |