Multicenter Validation of Deep Learning Algorithm ROP.AI for the Automated Diagnosis of Plus Disease in ROP
PURPOSE: Retinopathy of prematurity (ROP) is a sight-threatening vasoproliferative retinal disease affecting premature infants. The detection of plus disease, a severe form of ROP requiring treatment, remains challenging owing to subjectivity, frequency, and time intensity of retinal examinations. R...
Autores principales: | , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
The Association for Research in Vision and Ophthalmology
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10431208/ https://www.ncbi.nlm.nih.gov/pubmed/37578427 http://dx.doi.org/10.1167/tvst.12.8.13 |
_version_ | 1785091146745118720 |
---|---|
author | Bai, Amelia Dai, Shuan Hung, Jacky Kirpalani, Aditi Russell, Heather Elder, James Shah, Shaheen Carty, Christopher Tan, Zachary |
author_facet | Bai, Amelia Dai, Shuan Hung, Jacky Kirpalani, Aditi Russell, Heather Elder, James Shah, Shaheen Carty, Christopher Tan, Zachary |
author_sort | Bai, Amelia |
collection | PubMed |
description | PURPOSE: Retinopathy of prematurity (ROP) is a sight-threatening vasoproliferative retinal disease affecting premature infants. The detection of plus disease, a severe form of ROP requiring treatment, remains challenging owing to subjectivity, frequency, and time intensity of retinal examinations. Recent artificial intelligence (AI) algorithms developed to detect plus disease aims to alleviate these challenges; however, they have not been tested against a diverse neonatal population. Our study aims to validate ROP.AI, an AI algorithm developed from a single cohort, against a multicenter Australian cohort to determine its performance in detecting plus disease. METHODS: Retinal images captured during routine ROP screening from May 2021 to February 2022 across five major tertiary centers throughout Australia were collected and uploaded to ROP.AI. AI diagnostic output was compared with one of five ROP experts. Sensitivity, specificity, negative predictive value, and area under the receiver operator curve were determined. RESULTS: We collected 8052 images. The area under the receiver operator curve for the diagnosis of plus disease was 0.75. ROP.AI achieved 84% sensitivity, 43% specificity, and 96% negative predictive value for the detection of plus disease after operating point optimization. CONCLUSIONS: ROP.AI was able to detect plus disease in an external, multicenter cohort despite being trained from a single center. Algorithm performance was demonstrated without preprocessing or augmentation, simulating real-world clinical applicability. Further training may improve generalizability for clinical implementation. TRANSLATIONAL RELEVANCE: These results demonstrate ROP.AI's potential as a screening tool for the detection of plus disease in future clinical practice and provides a solution to overcome current diagnostic challenges. |
format | Online Article Text |
id | pubmed-10431208 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | The Association for Research in Vision and Ophthalmology |
record_format | MEDLINE/PubMed |
spelling | pubmed-104312082023-08-17 Multicenter Validation of Deep Learning Algorithm ROP.AI for the Automated Diagnosis of Plus Disease in ROP Bai, Amelia Dai, Shuan Hung, Jacky Kirpalani, Aditi Russell, Heather Elder, James Shah, Shaheen Carty, Christopher Tan, Zachary Transl Vis Sci Technol Artificial Intelligence PURPOSE: Retinopathy of prematurity (ROP) is a sight-threatening vasoproliferative retinal disease affecting premature infants. The detection of plus disease, a severe form of ROP requiring treatment, remains challenging owing to subjectivity, frequency, and time intensity of retinal examinations. Recent artificial intelligence (AI) algorithms developed to detect plus disease aims to alleviate these challenges; however, they have not been tested against a diverse neonatal population. Our study aims to validate ROP.AI, an AI algorithm developed from a single cohort, against a multicenter Australian cohort to determine its performance in detecting plus disease. METHODS: Retinal images captured during routine ROP screening from May 2021 to February 2022 across five major tertiary centers throughout Australia were collected and uploaded to ROP.AI. AI diagnostic output was compared with one of five ROP experts. Sensitivity, specificity, negative predictive value, and area under the receiver operator curve were determined. RESULTS: We collected 8052 images. The area under the receiver operator curve for the diagnosis of plus disease was 0.75. ROP.AI achieved 84% sensitivity, 43% specificity, and 96% negative predictive value for the detection of plus disease after operating point optimization. CONCLUSIONS: ROP.AI was able to detect plus disease in an external, multicenter cohort despite being trained from a single center. Algorithm performance was demonstrated without preprocessing or augmentation, simulating real-world clinical applicability. Further training may improve generalizability for clinical implementation. TRANSLATIONAL RELEVANCE: These results demonstrate ROP.AI's potential as a screening tool for the detection of plus disease in future clinical practice and provides a solution to overcome current diagnostic challenges. The Association for Research in Vision and Ophthalmology 2023-08-14 /pmc/articles/PMC10431208/ /pubmed/37578427 http://dx.doi.org/10.1167/tvst.12.8.13 Text en Copyright 2023 The Authors https://creativecommons.org/licenses/by-nc-nd/4.0/This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. |
spellingShingle | Artificial Intelligence Bai, Amelia Dai, Shuan Hung, Jacky Kirpalani, Aditi Russell, Heather Elder, James Shah, Shaheen Carty, Christopher Tan, Zachary Multicenter Validation of Deep Learning Algorithm ROP.AI for the Automated Diagnosis of Plus Disease in ROP |
title | Multicenter Validation of Deep Learning Algorithm ROP.AI for the Automated Diagnosis of Plus Disease in ROP |
title_full | Multicenter Validation of Deep Learning Algorithm ROP.AI for the Automated Diagnosis of Plus Disease in ROP |
title_fullStr | Multicenter Validation of Deep Learning Algorithm ROP.AI for the Automated Diagnosis of Plus Disease in ROP |
title_full_unstemmed | Multicenter Validation of Deep Learning Algorithm ROP.AI for the Automated Diagnosis of Plus Disease in ROP |
title_short | Multicenter Validation of Deep Learning Algorithm ROP.AI for the Automated Diagnosis of Plus Disease in ROP |
title_sort | multicenter validation of deep learning algorithm rop.ai for the automated diagnosis of plus disease in rop |
topic | Artificial Intelligence |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10431208/ https://www.ncbi.nlm.nih.gov/pubmed/37578427 http://dx.doi.org/10.1167/tvst.12.8.13 |
work_keys_str_mv | AT baiamelia multicentervalidationofdeeplearningalgorithmropaifortheautomateddiagnosisofplusdiseaseinrop AT daishuan multicentervalidationofdeeplearningalgorithmropaifortheautomateddiagnosisofplusdiseaseinrop AT hungjacky multicentervalidationofdeeplearningalgorithmropaifortheautomateddiagnosisofplusdiseaseinrop AT kirpalaniaditi multicentervalidationofdeeplearningalgorithmropaifortheautomateddiagnosisofplusdiseaseinrop AT russellheather multicentervalidationofdeeplearningalgorithmropaifortheautomateddiagnosisofplusdiseaseinrop AT elderjames multicentervalidationofdeeplearningalgorithmropaifortheautomateddiagnosisofplusdiseaseinrop AT shahshaheen multicentervalidationofdeeplearningalgorithmropaifortheautomateddiagnosisofplusdiseaseinrop AT cartychristopher multicentervalidationofdeeplearningalgorithmropaifortheautomateddiagnosisofplusdiseaseinrop AT tanzachary multicentervalidationofdeeplearningalgorithmropaifortheautomateddiagnosisofplusdiseaseinrop |