Cargando…
Too True to be Bad: When Sets of Studies With Significant and Nonsignificant Findings Are Probably True
Psychology journals rarely publish nonsignificant results. At the same time, it is often very unlikely (or “too good to be true”) that a set of studies yields exclusively significant results. Here, we use likelihood ratios to explain when sets of studies that contain a mix of significant and nonsign...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
SAGE Publications
2017
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5734376/ https://www.ncbi.nlm.nih.gov/pubmed/29276574 http://dx.doi.org/10.1177/1948550617693058 |
_version_ | 1783287044666032128 |
---|---|
author | Lakens, Daniël Etz, Alexander J. |
author_facet | Lakens, Daniël Etz, Alexander J. |
author_sort | Lakens, Daniël |
collection | PubMed |
description | Psychology journals rarely publish nonsignificant results. At the same time, it is often very unlikely (or “too good to be true”) that a set of studies yields exclusively significant results. Here, we use likelihood ratios to explain when sets of studies that contain a mix of significant and nonsignificant results are likely to be true or “too true to be bad.” As we show, mixed results are not only likely to be observed in lines of research but also, when observed, often provide evidence for the alternative hypothesis, given reasonable levels of statistical power and an adequately controlled low Type 1 error rate. Researchers should feel comfortable submitting such lines of research with an internal meta-analysis for publication. A better understanding of probabilities, accompanied by more realistic expectations of what real sets of studies look like, might be an important step in mitigating publication bias in the scientific literature. |
format | Online Article Text |
id | pubmed-5734376 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2017 |
publisher | SAGE Publications |
record_format | MEDLINE/PubMed |
spelling | pubmed-57343762017-12-22 Too True to be Bad: When Sets of Studies With Significant and Nonsignificant Findings Are Probably True Lakens, Daniël Etz, Alexander J. Soc Psychol Personal Sci Articles Psychology journals rarely publish nonsignificant results. At the same time, it is often very unlikely (or “too good to be true”) that a set of studies yields exclusively significant results. Here, we use likelihood ratios to explain when sets of studies that contain a mix of significant and nonsignificant results are likely to be true or “too true to be bad.” As we show, mixed results are not only likely to be observed in lines of research but also, when observed, often provide evidence for the alternative hypothesis, given reasonable levels of statistical power and an adequately controlled low Type 1 error rate. Researchers should feel comfortable submitting such lines of research with an internal meta-analysis for publication. A better understanding of probabilities, accompanied by more realistic expectations of what real sets of studies look like, might be an important step in mitigating publication bias in the scientific literature. SAGE Publications 2017-05-05 2017-11 /pmc/articles/PMC5734376/ /pubmed/29276574 http://dx.doi.org/10.1177/1948550617693058 Text en © The Author(s) 2017 http://creativecommons.org/licenses/by/4.0/ This article is distributed under the terms of the Creative Commons Attribution 4.0 License (http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage). |
spellingShingle | Articles Lakens, Daniël Etz, Alexander J. Too True to be Bad: When Sets of Studies With Significant and Nonsignificant Findings Are Probably True |
title | Too True to be Bad: When Sets of Studies With Significant and Nonsignificant Findings Are Probably True |
title_full | Too True to be Bad: When Sets of Studies With Significant and Nonsignificant Findings Are Probably True |
title_fullStr | Too True to be Bad: When Sets of Studies With Significant and Nonsignificant Findings Are Probably True |
title_full_unstemmed | Too True to be Bad: When Sets of Studies With Significant and Nonsignificant Findings Are Probably True |
title_short | Too True to be Bad: When Sets of Studies With Significant and Nonsignificant Findings Are Probably True |
title_sort | too true to be bad: when sets of studies with significant and nonsignificant findings are probably true |
topic | Articles |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5734376/ https://www.ncbi.nlm.nih.gov/pubmed/29276574 http://dx.doi.org/10.1177/1948550617693058 |
work_keys_str_mv | AT lakensdaniel tootruetobebadwhensetsofstudieswithsignificantandnonsignificantfindingsareprobablytrue AT etzalexanderj tootruetobebadwhensetsofstudieswithsignificantandnonsignificantfindingsareprobablytrue |