Development of a Cloud-Based Clinical Decision Support System for Ophthalmology Triage Using Decision Tree Artificial Intelligence

PURPOSE: Clinical decision support systems (CDSS) are an emerging frontier in teleophthalmology, drawing on heuristic decision making to augment processes such as triage and referral. We describe the development and implementation of a novel cloud-based decision tree CDSS for on-call ophthalmology consults. The objective was to standardize the triage and referral process while providing a more accurate provisional diagnosis and urgency. DESIGN: Prospective comparative cohort study. SUBJECTS: On-call referrals to a Canadian community ophthalmology clinic. METHODS: A web-based decision tree algorithm was developed using current guidelines and expert opinion. The algorithm collected tailored information on the patient’s ophthalmic concern, and outputted a provisional diagnosis and urgency before sending an electronic referral to the on-call ophthalmology clinic. Data were described using descriptive statistics. Spearman-rho correlations and Cohen’s kappa coefficient were used to characterize the observed relationships. Post hoc analysis was conducted using analysis of contingency tables and adjusted residuals. MAIN OUTCOME MEASURES: Diagnostic category, provisional diagnosis, and urgency for the referring provider, CDSS, and ophthalmologist. RESULTS: Ninety-six referrals were processed. Referring providers included medical doctors (76.0%, n = 73), optometrists (20.8%, n = 20), and nurse practitioners (3.1%, n = 3). The CDSS (κ = 0.5898; 95% confidence interval [CI], 0.4868–0.6928; P < 0.0001) performed equally well with 66.7% agreement in determining category when compared with referring providers (κ = 0.5880; 95% CI, 0.4798–0.6961; P < 0.0001). The CDSS (agreement = 53.1%; κ = 0.4999; 95% CI, 0.4021–0.5978; P < 0.0001) performed better than referring providers (agreement = 43.8%; κ = 0.4191; 95% CI, 0.3194–0.5188; P < 0.0001) in determining a diagnosis. The CDSS (ρ = 0.5014; 95% CI, 0.3092–0.6935; P < 0.0001) also performed better than referring providers (ρ = 0.4035; 95% CI, 0.2406–0.5665; P < 0.0001) in determining urgency. The CDSS assigned a lower level of urgency in 22 cases (22.9%) compared with referring providers in 6 cases (6.3%). CONCLUSIONS: To our knowledge, this is the first cloud-based CDSS in ophthalmology designed to augment the triage and referral process. The CDSS achieves a more accurate diagnosis and urgency, standardizes information collection, and overcomes antiquated paper-based consults. Future directions include developing a random forest model or integrating convolutional neural network-based machine learning to refine the speed and accuracy of triage and referral processes, with emphasis on increasing sensitivity of the CDSS.