Cargando…

Distributions of p-values smaller than .05 in psychology: what is going on?

Previous studies provided mixed findings on pecularities in p-value distributions in psychology. This paper examined 258,050 test results across 30,710 articles from eight high impact journals to investigate the existence of a peculiar prevalence of p-values just below .05 (i.e., a bump) in the psyc...

Descripción completa

Detalles Bibliográficos
Autores principales: Hartgerink, Chris H.J., van Aert, Robbie C.M., Nuijten, Michèle B., Wicherts, Jelte M., van Assen, Marcel A.L.M.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: PeerJ Inc. 2016
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4830257/
https://www.ncbi.nlm.nih.gov/pubmed/27077017
http://dx.doi.org/10.7717/peerj.1935
_version_ 1782426880365821952
author Hartgerink, Chris H.J.
van Aert, Robbie C.M.
Nuijten, Michèle B.
Wicherts, Jelte M.
van Assen, Marcel A.L.M.
author_facet Hartgerink, Chris H.J.
van Aert, Robbie C.M.
Nuijten, Michèle B.
Wicherts, Jelte M.
van Assen, Marcel A.L.M.
author_sort Hartgerink, Chris H.J.
collection PubMed
description Previous studies provided mixed findings on pecularities in p-value distributions in psychology. This paper examined 258,050 test results across 30,710 articles from eight high impact journals to investigate the existence of a peculiar prevalence of p-values just below .05 (i.e., a bump) in the psychological literature, and a potential increase thereof over time. We indeed found evidence for a bump just below .05 in the distribution of exactly reported p-values in the journals Developmental Psychology, Journal of Applied Psychology, and Journal of Personality and Social Psychology, but the bump did not increase over the years and disappeared when using recalculated p-values. We found clear and direct evidence for the QRP “incorrect rounding of p-value” (John, Loewenstein & Prelec, 2012) in all psychology journals. Finally, we also investigated monotonic excess of p-values, an effect of certain QRPs that has been neglected in previous research, and developed two measures to detect this by modeling the distributions of statistically significant p-values. Using simulations and applying the two measures to the retrieved test results, we argue that, although one of the measures suggests the use of QRPs in psychology, it is difficult to draw general conclusions concerning QRPs based on modeling of p-value distributions.
format Online
Article
Text
id pubmed-4830257
institution National Center for Biotechnology Information
language English
publishDate 2016
publisher PeerJ Inc.
record_format MEDLINE/PubMed
spelling pubmed-48302572016-04-13 Distributions of p-values smaller than .05 in psychology: what is going on? Hartgerink, Chris H.J. van Aert, Robbie C.M. Nuijten, Michèle B. Wicherts, Jelte M. van Assen, Marcel A.L.M. PeerJ Psychiatry and Psychology Previous studies provided mixed findings on pecularities in p-value distributions in psychology. This paper examined 258,050 test results across 30,710 articles from eight high impact journals to investigate the existence of a peculiar prevalence of p-values just below .05 (i.e., a bump) in the psychological literature, and a potential increase thereof over time. We indeed found evidence for a bump just below .05 in the distribution of exactly reported p-values in the journals Developmental Psychology, Journal of Applied Psychology, and Journal of Personality and Social Psychology, but the bump did not increase over the years and disappeared when using recalculated p-values. We found clear and direct evidence for the QRP “incorrect rounding of p-value” (John, Loewenstein & Prelec, 2012) in all psychology journals. Finally, we also investigated monotonic excess of p-values, an effect of certain QRPs that has been neglected in previous research, and developed two measures to detect this by modeling the distributions of statistically significant p-values. Using simulations and applying the two measures to the retrieved test results, we argue that, although one of the measures suggests the use of QRPs in psychology, it is difficult to draw general conclusions concerning QRPs based on modeling of p-value distributions. PeerJ Inc. 2016-04-11 /pmc/articles/PMC4830257/ /pubmed/27077017 http://dx.doi.org/10.7717/peerj.1935 Text en ©2016 Hartgerink et al. http://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ) and either DOI or URL of the article must be cited.
spellingShingle Psychiatry and Psychology
Hartgerink, Chris H.J.
van Aert, Robbie C.M.
Nuijten, Michèle B.
Wicherts, Jelte M.
van Assen, Marcel A.L.M.
Distributions of p-values smaller than .05 in psychology: what is going on?
title Distributions of p-values smaller than .05 in psychology: what is going on?
title_full Distributions of p-values smaller than .05 in psychology: what is going on?
title_fullStr Distributions of p-values smaller than .05 in psychology: what is going on?
title_full_unstemmed Distributions of p-values smaller than .05 in psychology: what is going on?
title_short Distributions of p-values smaller than .05 in psychology: what is going on?
title_sort distributions of p-values smaller than .05 in psychology: what is going on?
topic Psychiatry and Psychology
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4830257/
https://www.ncbi.nlm.nih.gov/pubmed/27077017
http://dx.doi.org/10.7717/peerj.1935
work_keys_str_mv AT hartgerinkchrishj distributionsofpvaluessmallerthan05inpsychologywhatisgoingon
AT vanaertrobbiecm distributionsofpvaluessmallerthan05inpsychologywhatisgoingon
AT nuijtenmicheleb distributionsofpvaluessmallerthan05inpsychologywhatisgoingon
AT wichertsjeltem distributionsofpvaluessmallerthan05inpsychologywhatisgoingon
AT vanassenmarcelalm distributionsofpvaluessmallerthan05inpsychologywhatisgoingon