Accuracy Evaluation of the Unified P-Value from Combining Correlated P-Values

Meta-analysis methods that combine [Image: see text]-values into a single unified [Image: see text]-value are frequently employed to improve confidence in hypothesis testing. An assumption made by most meta-analysis methods is that the [Image: see text]-values to be combined are independent, which may not always be true. To investigate the accuracy of the unified [Image: see text]-value from combining correlated [Image: see text]-values, we have evaluated a family of statistical methods that combine: independent, weighted independent, correlated, and weighted correlated [Image: see text]-values. Statistical accuracy evaluation by combining simulated correlated [Image: see text]-values showed that correlation among [Image: see text]-values can have a significant effect on the accuracy of the combined [Image: see text]-value obtained. Among the statistical methods evaluated those that weight [Image: see text]-values compute more accurate combined [Image: see text]-values than those that do not. Also, statistical methods that utilize the correlation information have the best performance, producing significantly more accurate combined [Image: see text]-values. In our study we have demonstrated that statistical methods that combine [Image: see text]-values based on the assumption of independence can produce inaccurate [Image: see text]-values when combining correlated [Image: see text]-values, even when the [Image: see text]-values are only weakly correlated. Therefore, to prevent from drawing false conclusions during hypothesis testing, our study advises caution be used when interpreting the [Image: see text]-value obtained from combining [Image: see text]-values of unknown correlation. However, when the correlation information is available, the weighting-capable statistical method, first introduced by Brown and recently modified by Hou, seems to perform the best amongst the methods investigated.