Quantifying heterogeneity in individual participant data meta-analysis with binary outcomes

BACKGROUND: In meta-analyses (MA), effect estimates that are pooled together will often be heterogeneous. Determining how substantial heterogeneity is is an important aspect of MA. METHOD: We consider how best to quantify heterogeneity in the context of individual participant data meta-analysis (IPD-MA) of binary data. Both two- and one-stage approaches are evaluated via simulation study. We consider conventional I (2) and R (2) statistics estimated via a two-stage approach and R (2) estimated via a one-stage approach. We propose a simulation-based intraclass correlation coefficient (ICC) adapted from Goldstein et al. to estimate the I (2), from the one-stage approach. RESULTS: Results show that when there is no effect modification, the estimated I (2) from the two-stage model is underestimated, while in the one-stage model, it is overestimated. In the presence of effect modification, the estimated I (2) from the one-stage model has better performance than that from the two-stage model when the prevalence of the outcome is high. The I (2) from the two-stage model is less sensitive to the strength of effect modification when the number of studies is large and prevalence is low. CONCLUSIONS: The simulation-based I (2) based on a one-stage approach has better performance than the conventional I (2) based on a two-stage approach when there is strong effect modification with high prevalence. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s13643-017-0630-4) contains supplementary material, which is available to authorized users.