Sensitivity and identification quantification by a relative latent model complexity perturbation in Bayesian meta‐analysis

In recent years, Bayesian meta‐analysis expressed by a normal–normal hierarchical model (NNHM) has been widely used for combining evidence from multiple studies. Data provided for the NNHM are frequently based on a small number of studies and on uncertain within‐study standard deviation values. Despite the widespread use of Bayesian NNHM, it has always been unclear to what extent the posterior inference is impacted by the heterogeneity prior (sensitivity [Formula: see text]) and by the uncertainty in the within‐study standard deviation values (identification [Formula: see text]). Thus, to answer this question, we developed a unified method to simultaneously quantify both sensitivity and identification ([Formula: see text] ‐ [Formula: see text]) for all model parameters in a Bayesian NNHM, based on derivatives of the Bhattacharyya coefficient with respect to relative latent model complexity (RLMC) perturbations. Three case studies exemplify the applicability of the method proposed: historical data for a conventional therapy, data from which one large study is first included and then excluded, and two subgroup meta‐analyses specified by their randomization status. We analyzed six scenarios, crossing three RLMC targets with two heterogeneity priors (half‐normal, half‐Cauchy). The results show that [Formula: see text] ‐ [Formula: see text] explicitly reveals which parameters are affected by the heterogeneity prior and by the uncertainty in the within‐study standard deviation values. In addition, we compare the impact of both heterogeneity priors and quantify how [Formula: see text] ‐ [Formula: see text] values are affected by omitting one large study and by the randomization status. Finally, the range of applicability of [Formula: see text] ‐ [Formula: see text] is extended to Bayesian NtHM. A dedicated R package facilitates automatic [Formula: see text] ‐ [Formula: see text] quantification in applied Bayesian meta‐analyses.