Reliability and Efficiency of the CAPRI-3 Metastatic Prostate Cancer Registry Driven by Artificial Intelligence

SIMPLE SUMMARY: CAPRI-3 is an observational registry on metastatic prostate cancer that uses artificial intelligence (AI) for patient identification and data collection. The aim of this study is to demonstrate the reliability and efficiency of this method. Our deliberate effort to maximize the negative predictive value of our patient-identification algorithm to rule out unsuitable candidates without manual screening was successful and reached 94.8%. Completeness and accuracy of data extraction were 92.3% or higher but were lower (up to 10%) for date fields and inaccessible data (images/pdf). The AI-driven approach, including additional manual quality control, was much faster than full manual data collection (105 vs. 300 min per patient). In conclusion, the AI-driven approach of the CAPRI-3 registry is largely reliable and timesaving but manual quality control is needed for the less reliable and inaccessible data. ABSTRACT: Background: Manual data collection is still the gold standard for disease-specific patient registries. However, CAPRI-3 uses text mining (an artificial intelligence (AI) technology) for patient identification and data collection. The aim of this study is to demonstrate the reliability and efficiency of this AI-driven approach. Methods: CAPRI-3 is an observational retrospective multicenter cohort registry on metastatic prostate cancer. We tested the patient-identification algorithm and automated data extraction through manual validation of the same patients in two pilots in 2019 and 2022. Results: Pilot one identified 2030 patients and pilot two 9464 patients. The negative predictive value of the algorithm was maximized to prevent false exclusions and reached 94.8%. The completeness and accuracy of the automated data extraction were 92.3% or higher, except for date fields and inaccessible data (images/pdf) (10–88.9%). Additional manual quality control took over 3 h less time per patient than the original fully manual CAPRI registry (105 vs. 300 min). Conclusions: The CAPRI-3 patient-identification algorithm is a sound replacement for excluding ineligible candidates. The AI-driven data extraction is largely accurate and complete, but manual quality control is needed for less reliable and inaccessible data. Overall, the AI-driven approach of the CAPRI-3 registry is reliable and timesaving.