External Validation of the Prostate Biopsy Collaborative Group Risk Calculator and the Rotterdam Prostate Cancer Risk Calculator in a Swedish Population-based Screening Cohort

BACKGROUND: External validation of risk calculators (RCs) is necessary to determine their clinical applicability beyond the setting in which these were developed. OBJECTIVE: To assess the performance of the Rotterdam Prostate Cancer RC (RPCRC) and the Prostate Biopsy Collaborative Group RC (PBCG-RC). DESIGN, SETTING, AND PARTICIPANTS: We used data from the prospective, population-based STHLM3 screening study, performed in 2012–2015. Participants with prostate-specific antigen ≥3 ng/ml who underwent systematic prostate biopsies were included. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Probabilities for clinically significant prostate cancer (csPCa), defined as International Society of Urological Pathology grade ≥2, were calculated for each participant. External validity was assessed by calibration, discrimination, and clinical usefulness for both original and recalibrated models. RESULTS AND LIMITATIONS: Out of 5841 men, 1054 (18%) had csPCa. Distribution of risk predictions differed between RCs; median risks for csPCa using the RPCRC and PBCG-RC were 3.3% (interquartile range [IQR] 2.1–7.1%) and 20% (IQR 15–28%), respectively. The correlation between RC risk estimates on individual level was moderate (Spearman’s r = 0.55). Using the RPCRC’s recommended risk threshold of ≥4% for finding csPCa, 36% of participants would get concordant biopsy recommendations. At 10% risk cut-off, RCs agreed in 23% of cases. Both RCs showed good discrimination, with areas under the curves for the RPCRC of 0.74 (95% confidence interval [CI] 0.72–0.76) and the PBCG-RC of 0.70 (95% CI 0.68–0.72). Calibration was adequate using the PBCG-RC (calibration slope: 1.13 [95% CI 1.03–1.23]), but the RPCRC underestimated the risk of csPCa (calibration slope: 0.73 [0.68–0.79]). The PBCG-RC showed a net benefit in a decision curve analysis, whereas the RPCRC showed no net benefit at clinically relevant risk threshold levels. Recalibration improved clinical benefit, and differences between RCs decreased. CONCLUSIONS: Assessment of calibration is essential to ensure the clinical value of risk prediction tools. The PBCG-RC provided clinical benefit in its current version online. On the contrary, the RPCRC cannot be recommended in this setting. PATIENT SUMMARY: Predicting the probability of finding prostate cancer on biopsy differed between two assessed risk calculators. After recalibration, the agreement of the models improved, and both were shown to be clinically useful.