Required efficacy for novel therapies in BCG‐unresponsive non‐muscle invasive bladder cancer: Do current recommendations really reflect clinically meaningful outcomes?

BACKGROUND: Single‐arm trials are currently an accepted study design to investigate the efficacy of novel therapies (NT) in non‐muscle invasive bladder cancer (NMIBC) unresponsive to intravesical Bacillus Calmette‐Guérin (BCG) immunotherapy as randomized controlled trials are either unfeasible (comparator: early radical cystectomy; ERC), or unethical (comparator: placebo). To guide the design of such single‐arm trials, expert groups published recommendations for clinically meaningful outcomes. The aim of this study was to quantitatively verify the appropriateness of these recommendations. METHODS: We used a discrete event simulation framework in combination with a supercomputer to find the required efficacy at which a NT can compete with ERC when it comes to quality‐adjusted life expectancy (QALE). In total, 24 different efficacy thresholds (including the recommendations) were investigated. RESULTS: After ascertaining face validity with content experts, repeated verification, external validation, and calibration we considered our model valid. Both recommendations rarely showed an incremental benefit of the NT over ERC. In the most optimistic scenario, an increase in the IBCG recommendation by 10% and an increase in the FDA/AUA recommendation by 5% would yield results at which a NT could compete with ERC from a QALE perspective. CONCLUSIONS: This simulation study demonstrated that the current recommendations regarding clinically meaningful outcomes for single‐arm trials evaluating the efficacy of NT in BCG‐unresponsive NMIBC may be too low. Based on our quantitative approach, we propose increasing these thresholds to at least 45%‐55% at 6 months and 35% at 18‐24 months (complete response rates/recurrence‐free survival) to promote the development of clinically truly meaningful NT.