Design, power, and alpha levels in randomized phase II oncology trials

BACKGROUND: The statistical plan of a phase II trial should balance minimizing the premature termination of potentially beneficial therapies (i.e. false negatives) and the further, costly testing of ineffective drugs (i.e. false positives). We sought to examine the methodology, reporting, and bias in the interpretation of outcomes of phase II oncology trials in recent years. MATERIALS AND METHODS: In a retrospective cross-sectional analysis, we reviewed all full-length articles published on PubMed from 1 January 2021 to 20 June 2022. We searched for data regarding the sample size calculation (number, α value, power, and expected effect size), the primary and secondary outcomes and results, and the authors’ conclusion of the study. RESULTS: About 5.4% of studies (n = 10) used a statistical power that was inferior to 80%, and 16.7% (n = 34) did not indicate the level of power for the sample size calculation. Approximately 16.7% (n = 31) of studies used a one-sided α level of ≤0.025; 17.7% (n = 33) of studies used a predefined threshold (no comparator effect size or difference between groups) to determine the sample size for efficacy. The percentage of studies with a positive authors’ conclusion but not meeting the primary endpoint, or the endpoint was equivocal, was 27.4% (n = 51). CONCLUSION: Many randomized phase II studies in oncology failed to report essential data for determining sample size calculations, many did not actually use a comparator to determine efficacy even though the studies were randomized, and many had positive conclusions even though the results were indeterminate or the primary endpoint was not met.