Cargando…

A cascade eye diseases screening system with interpretability and expandability in ultra-wide field fundus images: A multicentre diagnostic accuracy study

BACKGROUND: Clinical application of artificial intelligence is limited due to the lack of interpretability and expandability in complex clinical settings. We aimed to develop an eye diseases screening system with improved interpretability and expandability based on a lesion-level dissection and test...

Descripción completa

Detalles Bibliográficos
Autores principales: Cao, Jing, You, Kun, Zhou, Jingxin, Xu, Mingyu, Xu, Peifang, Wen, Lei, Wang, Shengzhan, Jin, Kai, Lou, Lixia, Wang, Yao, Ye, Juan
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Elsevier 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9468501/
https://www.ncbi.nlm.nih.gov/pubmed/36110868
http://dx.doi.org/10.1016/j.eclinm.2022.101633
_version_ 1784788425151348736
author Cao, Jing
You, Kun
Zhou, Jingxin
Xu, Mingyu
Xu, Peifang
Wen, Lei
Wang, Shengzhan
Jin, Kai
Lou, Lixia
Wang, Yao
Ye, Juan
author_facet Cao, Jing
You, Kun
Zhou, Jingxin
Xu, Mingyu
Xu, Peifang
Wen, Lei
Wang, Shengzhan
Jin, Kai
Lou, Lixia
Wang, Yao
Ye, Juan
author_sort Cao, Jing
collection PubMed
description BACKGROUND: Clinical application of artificial intelligence is limited due to the lack of interpretability and expandability in complex clinical settings. We aimed to develop an eye diseases screening system with improved interpretability and expandability based on a lesion-level dissection and tested the clinical expandability and auxiliary ability of the system. METHODS: The four-hierarchical interpretable eye diseases screening system (IEDSS) based on a novel structural pattern named lesion atlas was developed to identify 30 eye diseases and conditions using a total of 32,026 ultra-wide field images collected from the Second Affiliated Hospital of Zhejiang University, School of Medicine (SAHZU), the First Affiliated Hospital of University of Science and Technology of China (FAHUSTC), and the Affiliated People's Hospital of Ningbo University (APHNU) in China between November 1, 2016 to February 28, 2022. The performance of IEDSS was compared with ophthalmologists and classic models trained with image-level labels. We further evaluated IEDSS in two external datasets, and tested it in a real-world scenario and an extended dataset with new phenotypes beyond the training categories. The accuracy (ACC), F1 score and confusion matrix were calculated to assess the performance of IEDSS. FINDINGS: IEDSS reached average ACCs (aACC) of 0·9781 (95%CI 0·9739-0·9824), 0·9660 (95%CI 0·9591-0·9730) and 0·9709 (95%CI 0·9655-0·9763), frequency-weighted average F1 scores of 0·9042 (95%CI 0·8957-0·9127), 0·8837 (95%CI 0·8714-0·8960) and 0·8874 (95%CI 0·8772-0·8972) in datasets of SAHZU, APHNU and FAHUSTC, respectively. IEDSS reached a higher aACC (0·9781, 95%CI 0·9739-0·9824) compared with a multi-class image-level model (0·9398, 95%CI 0·9329-0·9467), a classic multi-label image-level model (0·9278, 95%CI 0·9189-0·9366), a novel multi-label image-level model (0·9241, 95%CI 0·9151-0·9331) and a lesion-level model without Adaboost (0·9381, 95%CI 0·9299-0·9463). In the real-world scenario, the aACC of IEDSS (0·9872, 95%CI 0·9828-0·9915) was higher than that of the senior ophthalmologist (SO) (0·9413, 95%CI 0·9321-0·9504, p = 0·000) and the junior ophthalmologist (JO) (0·8846, 95%CI 0·8722-0·8971, p = 0·000). IEDSS remained strong performance (ACC = 0·8560, 95%CI 0·8252-0·8868) compared with JO (ACC = 0·784, 95%CI 0·7479-0·8201, p= 0·003) and SO (ACC = 0·8500, 95%CI 0·8187-0·8813, p = 0·789) in the extended dataset. INTERPRETATION: IEDSS showed excellent and stable performance in identifying common eye conditions and conditions beyond the training categories. The transparency and expandability of IEDSS could tremendously increase the clinical application range and the practical clinical value of it. It would enhance the efficiency and reliability of clinical practice, especially in remote areas with a lack of experienced specialists. FUNDING: National Natural Science Foundation Regional Innovation and Development Joint Fund (U20A20386), Key research and development program of Zhejiang Province (2019C03020), Clinical Medical Research Centre for Eye Diseases of Zhejiang Province (2021E50007).
format Online
Article
Text
id pubmed-9468501
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Elsevier
record_format MEDLINE/PubMed
spelling pubmed-94685012022-09-14 A cascade eye diseases screening system with interpretability and expandability in ultra-wide field fundus images: A multicentre diagnostic accuracy study Cao, Jing You, Kun Zhou, Jingxin Xu, Mingyu Xu, Peifang Wen, Lei Wang, Shengzhan Jin, Kai Lou, Lixia Wang, Yao Ye, Juan eClinicalMedicine Articles BACKGROUND: Clinical application of artificial intelligence is limited due to the lack of interpretability and expandability in complex clinical settings. We aimed to develop an eye diseases screening system with improved interpretability and expandability based on a lesion-level dissection and tested the clinical expandability and auxiliary ability of the system. METHODS: The four-hierarchical interpretable eye diseases screening system (IEDSS) based on a novel structural pattern named lesion atlas was developed to identify 30 eye diseases and conditions using a total of 32,026 ultra-wide field images collected from the Second Affiliated Hospital of Zhejiang University, School of Medicine (SAHZU), the First Affiliated Hospital of University of Science and Technology of China (FAHUSTC), and the Affiliated People's Hospital of Ningbo University (APHNU) in China between November 1, 2016 to February 28, 2022. The performance of IEDSS was compared with ophthalmologists and classic models trained with image-level labels. We further evaluated IEDSS in two external datasets, and tested it in a real-world scenario and an extended dataset with new phenotypes beyond the training categories. The accuracy (ACC), F1 score and confusion matrix were calculated to assess the performance of IEDSS. FINDINGS: IEDSS reached average ACCs (aACC) of 0·9781 (95%CI 0·9739-0·9824), 0·9660 (95%CI 0·9591-0·9730) and 0·9709 (95%CI 0·9655-0·9763), frequency-weighted average F1 scores of 0·9042 (95%CI 0·8957-0·9127), 0·8837 (95%CI 0·8714-0·8960) and 0·8874 (95%CI 0·8772-0·8972) in datasets of SAHZU, APHNU and FAHUSTC, respectively. IEDSS reached a higher aACC (0·9781, 95%CI 0·9739-0·9824) compared with a multi-class image-level model (0·9398, 95%CI 0·9329-0·9467), a classic multi-label image-level model (0·9278, 95%CI 0·9189-0·9366), a novel multi-label image-level model (0·9241, 95%CI 0·9151-0·9331) and a lesion-level model without Adaboost (0·9381, 95%CI 0·9299-0·9463). In the real-world scenario, the aACC of IEDSS (0·9872, 95%CI 0·9828-0·9915) was higher than that of the senior ophthalmologist (SO) (0·9413, 95%CI 0·9321-0·9504, p = 0·000) and the junior ophthalmologist (JO) (0·8846, 95%CI 0·8722-0·8971, p = 0·000). IEDSS remained strong performance (ACC = 0·8560, 95%CI 0·8252-0·8868) compared with JO (ACC = 0·784, 95%CI 0·7479-0·8201, p= 0·003) and SO (ACC = 0·8500, 95%CI 0·8187-0·8813, p = 0·789) in the extended dataset. INTERPRETATION: IEDSS showed excellent and stable performance in identifying common eye conditions and conditions beyond the training categories. The transparency and expandability of IEDSS could tremendously increase the clinical application range and the practical clinical value of it. It would enhance the efficiency and reliability of clinical practice, especially in remote areas with a lack of experienced specialists. FUNDING: National Natural Science Foundation Regional Innovation and Development Joint Fund (U20A20386), Key research and development program of Zhejiang Province (2019C03020), Clinical Medical Research Centre for Eye Diseases of Zhejiang Province (2021E50007). Elsevier 2022-09-05 /pmc/articles/PMC9468501/ /pubmed/36110868 http://dx.doi.org/10.1016/j.eclinm.2022.101633 Text en © 2022 The Author(s) https://creativecommons.org/licenses/by-nc-nd/4.0/This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
spellingShingle Articles
Cao, Jing
You, Kun
Zhou, Jingxin
Xu, Mingyu
Xu, Peifang
Wen, Lei
Wang, Shengzhan
Jin, Kai
Lou, Lixia
Wang, Yao
Ye, Juan
A cascade eye diseases screening system with interpretability and expandability in ultra-wide field fundus images: A multicentre diagnostic accuracy study
title A cascade eye diseases screening system with interpretability and expandability in ultra-wide field fundus images: A multicentre diagnostic accuracy study
title_full A cascade eye diseases screening system with interpretability and expandability in ultra-wide field fundus images: A multicentre diagnostic accuracy study
title_fullStr A cascade eye diseases screening system with interpretability and expandability in ultra-wide field fundus images: A multicentre diagnostic accuracy study
title_full_unstemmed A cascade eye diseases screening system with interpretability and expandability in ultra-wide field fundus images: A multicentre diagnostic accuracy study
title_short A cascade eye diseases screening system with interpretability and expandability in ultra-wide field fundus images: A multicentre diagnostic accuracy study
title_sort cascade eye diseases screening system with interpretability and expandability in ultra-wide field fundus images: a multicentre diagnostic accuracy study
topic Articles
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9468501/
https://www.ncbi.nlm.nih.gov/pubmed/36110868
http://dx.doi.org/10.1016/j.eclinm.2022.101633
work_keys_str_mv AT caojing acascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT youkun acascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT zhoujingxin acascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT xumingyu acascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT xupeifang acascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT wenlei acascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT wangshengzhan acascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT jinkai acascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT loulixia acascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT wangyao acascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT yejuan acascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT caojing cascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT youkun cascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT zhoujingxin cascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT xumingyu cascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT xupeifang cascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT wenlei cascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT wangshengzhan cascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT jinkai cascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT loulixia cascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT wangyao cascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy
AT yejuan cascadeeyediseasesscreeningsystemwithinterpretabilityandexpandabilityinultrawidefieldfundusimagesamulticentrediagnosticaccuracystudy