Meta‐analysis of two studies in the presence of heterogeneity with applications in rare diseases

Random‐effects meta‐analyses are used to combine evidence of treatment effects from multiple studies. Since treatment effects may vary across trials due to differences in study characteristics, heterogeneity in treatment effects between studies must be accounted for to achieve valid inference. The standard model for random‐effects meta‐analysis assumes approximately normal effect estimates and a normal random‐effects model. However, standard methods based on this model ignore the uncertainty in estimating the between‐trial heterogeneity. In the special setting of only two studies and in the presence of heterogeneity, we investigate here alternatives such as the Hartung‐Knapp‐Sidik‐Jonkman method (HKSJ), the modified Knapp‐Hartung method (mKH, a variation of the HKSJ method) and Bayesian random‐effects meta‐analyses with priors covering plausible heterogeneity values; [Formula: see text] code to reproduce the examples is presented in an appendix. The properties of these methods are assessed by applying them to five examples from various rare diseases and by a simulation study. Whereas the standard method based on normal quantiles has poor coverage, the HKSJ and mKH generally lead to very long, and therefore inconclusive, confidence intervals. The Bayesian intervals on the whole show satisfying properties and offer a reasonable compromise between these two extremes.