The P value - and its historical underpinnings – pro and con

The derivation and interpretation of P values derived from inferential testing remain somewhat vague and ambiguous in the minds of some researchers/editors/reviewers/readers. The British polymath Fisher famously averred: “the value for which P = 0.05, or 1 in 20, is 1.96 or nearly 2; it is convenient to take this point as a limit in judging whether a deviation is to be considered significant or not. Deviations exceeding twice the standard deviation are thus formally regarded as significant.” This sometimes leads to an almost reductio ad absurdum mindset with an automatic discardment of studies with results where P > 0.05. It must be remembered that results may be negatively impacted by myriad factors that may be out of the researcher/s control, such as small sample sizes, small effects, bias, and random error. This paper briefly reviews the historical events leading to the acceptance of P ≤ 0.05 for statistical significance, the rationale behind the null hypothesis (H(0)), the meaning of P (and the potential for Type 1 and 2 Errors), α, β, the possibility of using non-0.05 cut-offs when studies are “trending toward statistical significance,” and the importance of including confidence intervals (CIs) in results. P values are vital but must be tempered by judicial consideration of CI and study design. P is a probability spectrum and not simply a binary significant/non-significant statistical metric. MESH: 95% confidence interval, biostatistics, P value