Integrating economic considerations into cutpoint selection may help align clinical decision support toward value-based healthcare

OBJECTIVE: Clinical prediction models providing binary categorizations for clinical decision support require the selection of a probability threshold, or “cutpoint,” to classify individuals. Existing cutpoint selection approaches typically optimize test-specific metrics, including sensitivity and specificity, but overlook the consequences of correct or incorrect classification. We introduce a new cutpoint selection approach considering downstream consequences using net monetary benefit (NMB) and through simulations compared it with alternative approaches in 2 use-cases: (i) preventing intensive care unit readmission and (ii) preventing inpatient falls. MATERIALS AND METHODS: Parameter estimates for costs and effectiveness from prior studies were included in Monte Carlo simulations. For each use-case, we simulated the expected NMB resulting from the model-guided decision using a range of cutpoint selection approaches, including our new value-optimizing approach. Sensitivity analyses applied alternative event rates, model discrimination, and calibration performance. RESULTS: The proposed approach that considered expected downstream consequences was frequently NMB-maximizing compared with other methods. Sensitivity analysis demonstrated that it was or closely tracked the optimal strategy under a range of scenarios. Under scenarios of relatively low event rates and discrimination that may be considered realistic for intensive care (prevalence = 0.025, area under the receiver operating characteristic curve [AUC] = 0.70) and falls (prevalence = 0.036, AUC = 0.70), our proposed cutpoint method was either the best or similar to the best of the compared methods regarding NMB, and was robust to model miscalibration. DISCUSSION: Our results highlight the potential value of conditioning cutpoints on the implementation setting, particularly for rare and costly events, which are often the target of prediction model development research. CONCLUSIONS: This study proposes a cutpoint selection method that may optimize clinical decision support systems toward value-based care.