Cargando…
Distributions of p-values smaller than .05 in psychology: what is going on?
Previous studies provided mixed findings on pecularities in p-value distributions in psychology. This paper examined 258,050 test results across 30,710 articles from eight high impact journals to investigate the existence of a peculiar prevalence of p-values just below .05 (i.e., a bump) in the psyc...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
PeerJ Inc.
2016
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4830257/ https://www.ncbi.nlm.nih.gov/pubmed/27077017 http://dx.doi.org/10.7717/peerj.1935 |
_version_ | 1782426880365821952 |
---|---|
author | Hartgerink, Chris H.J. van Aert, Robbie C.M. Nuijten, Michèle B. Wicherts, Jelte M. van Assen, Marcel A.L.M. |
author_facet | Hartgerink, Chris H.J. van Aert, Robbie C.M. Nuijten, Michèle B. Wicherts, Jelte M. van Assen, Marcel A.L.M. |
author_sort | Hartgerink, Chris H.J. |
collection | PubMed |
description | Previous studies provided mixed findings on pecularities in p-value distributions in psychology. This paper examined 258,050 test results across 30,710 articles from eight high impact journals to investigate the existence of a peculiar prevalence of p-values just below .05 (i.e., a bump) in the psychological literature, and a potential increase thereof over time. We indeed found evidence for a bump just below .05 in the distribution of exactly reported p-values in the journals Developmental Psychology, Journal of Applied Psychology, and Journal of Personality and Social Psychology, but the bump did not increase over the years and disappeared when using recalculated p-values. We found clear and direct evidence for the QRP “incorrect rounding of p-value” (John, Loewenstein & Prelec, 2012) in all psychology journals. Finally, we also investigated monotonic excess of p-values, an effect of certain QRPs that has been neglected in previous research, and developed two measures to detect this by modeling the distributions of statistically significant p-values. Using simulations and applying the two measures to the retrieved test results, we argue that, although one of the measures suggests the use of QRPs in psychology, it is difficult to draw general conclusions concerning QRPs based on modeling of p-value distributions. |
format | Online Article Text |
id | pubmed-4830257 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2016 |
publisher | PeerJ Inc. |
record_format | MEDLINE/PubMed |
spelling | pubmed-48302572016-04-13 Distributions of p-values smaller than .05 in psychology: what is going on? Hartgerink, Chris H.J. van Aert, Robbie C.M. Nuijten, Michèle B. Wicherts, Jelte M. van Assen, Marcel A.L.M. PeerJ Psychiatry and Psychology Previous studies provided mixed findings on pecularities in p-value distributions in psychology. This paper examined 258,050 test results across 30,710 articles from eight high impact journals to investigate the existence of a peculiar prevalence of p-values just below .05 (i.e., a bump) in the psychological literature, and a potential increase thereof over time. We indeed found evidence for a bump just below .05 in the distribution of exactly reported p-values in the journals Developmental Psychology, Journal of Applied Psychology, and Journal of Personality and Social Psychology, but the bump did not increase over the years and disappeared when using recalculated p-values. We found clear and direct evidence for the QRP “incorrect rounding of p-value” (John, Loewenstein & Prelec, 2012) in all psychology journals. Finally, we also investigated monotonic excess of p-values, an effect of certain QRPs that has been neglected in previous research, and developed two measures to detect this by modeling the distributions of statistically significant p-values. Using simulations and applying the two measures to the retrieved test results, we argue that, although one of the measures suggests the use of QRPs in psychology, it is difficult to draw general conclusions concerning QRPs based on modeling of p-value distributions. PeerJ Inc. 2016-04-11 /pmc/articles/PMC4830257/ /pubmed/27077017 http://dx.doi.org/10.7717/peerj.1935 Text en ©2016 Hartgerink et al. http://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ) and either DOI or URL of the article must be cited. |
spellingShingle | Psychiatry and Psychology Hartgerink, Chris H.J. van Aert, Robbie C.M. Nuijten, Michèle B. Wicherts, Jelte M. van Assen, Marcel A.L.M. Distributions of p-values smaller than .05 in psychology: what is going on? |
title | Distributions of p-values smaller than .05 in psychology: what is going on? |
title_full | Distributions of p-values smaller than .05 in psychology: what is going on? |
title_fullStr | Distributions of p-values smaller than .05 in psychology: what is going on? |
title_full_unstemmed | Distributions of p-values smaller than .05 in psychology: what is going on? |
title_short | Distributions of p-values smaller than .05 in psychology: what is going on? |
title_sort | distributions of p-values smaller than .05 in psychology: what is going on? |
topic | Psychiatry and Psychology |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4830257/ https://www.ncbi.nlm.nih.gov/pubmed/27077017 http://dx.doi.org/10.7717/peerj.1935 |
work_keys_str_mv | AT hartgerinkchrishj distributionsofpvaluessmallerthan05inpsychologywhatisgoingon AT vanaertrobbiecm distributionsofpvaluessmallerthan05inpsychologywhatisgoingon AT nuijtenmicheleb distributionsofpvaluessmallerthan05inpsychologywhatisgoingon AT wichertsjeltem distributionsofpvaluessmallerthan05inpsychologywhatisgoingon AT vanassenmarcelalm distributionsofpvaluessmallerthan05inpsychologywhatisgoingon |