The statistical properties of RCTs and a proposal for shrinkage

We abstract the concept of a randomized controlled trial as a triple [Formula: see text] , where [Formula: see text] is the primary efficacy parameter, b the estimate, and s the standard error ([Formula: see text]). If the parameter [Formula: see text] is either a difference of means, a log odds ratio or a log hazard ratio, then it is reasonable to assume that b is unbiased and normally distributed. This then allows us to estimate the joint distribution of the z‐value [Formula: see text] and the signal‐to‐noise ratio [Formula: see text] from a sample of pairs [Formula: see text]. We have collected 23 551 such pairs from the Cochrane database. We note that there are many statistical quantities that depend on [Formula: see text] only through the pair [Formula: see text]. We start by determining the estimated distribution of the achieved power. In particular, we estimate the median achieved power to be only 13%. We also consider the exaggeration ratio which is the factor by which the magnitude of [Formula: see text] is overestimated. We find that if the estimate is just significant at the 5% level, we would expect it to overestimate the true effect by a factor of 1.7. This exaggeration is sometimes referred to as the winner's curse and it is undoubtedly to a considerable extent responsible for disappointing replication results. For this reason, we believe it is important to shrink the unbiased estimator, and we propose a method for doing so. We show that our shrinkage estimator successfully addresses the exaggeration. As an example, we re‐analyze the ANDROMEDA‐SHOCK trial.