Permutation criteria to evaluate multiple clinical endpoints in a proof-of-concept study: lessons from Pre-RELAX-AHF

BACKGROUND: Clinically relevant endpoints cannot be routinely targeted with reasonable power in a small study. Hence, proof-of-concept studies are often powered to a primary surrogate endpoint. However, in acute heart failure (AHF) effects on surrogates have not translated into clinical benefit in confirmatory studies. Although observing an effect on one of many endpoints due to chance is likely, observing concurrent positive trends across several outcomes by chance is usually unlikely. METHODS: Pre-RELAX-AHF, which compared 4 relaxin doses with placebo in AHF, has shown favourable trends versus placebo (one-sided P < 0.10) on six of nine clinical endpoints in the 30 μg/kg/day group. To illustrate evaluation of multiple, correlated clinical endpoints for evidence of efficacy and for dose selection, a permutation method was applied retrospectively. By randomly re-assigning the treatment group to the actual data for each of the 229 subjects, 20,000 permutation samples were constructed. RESULTS: The permutation P value for at least six favourable trends among nine endpoints in any dose groups was 0.0073 (99.9% CI 0.0053–0.0093). This is higher than would be expected if the endpoints were uncorrelated (0.00026), but much lower than the probability of observing one of nine comparisons significant at the traditional two-sided P < 0.05 (0.74). Thus, the result was unlikely due to correlated endpoints or to chance. CONCLUSIONS: Examining consistency of effect across multiple clinical endpoints in a proof-of-concept study may identify efficacious therapies and enable dose selection for confirmatory trials. The merit of the approach described requires confirmation through prospective application in designing future studies.