Cargando…

Underpowered samples, false negatives, and unconscious learning

The scientific community has witnessed growing concern about the high rate of false positives and unreliable results within the psychological literature, but the harmful impact of false negatives has been largely ignored. False negatives are particularly concerning in research areas where demonstrat...

Descripción completa

Detalles Bibliográficos
Autores principales: Vadillo, Miguel A., Konstantinidis, Emmanouil, Shanks, David R.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Springer US 2015
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4742512/
https://www.ncbi.nlm.nih.gov/pubmed/26122896
http://dx.doi.org/10.3758/s13423-015-0892-6
_version_ 1782414205673013248
author Vadillo, Miguel A.
Konstantinidis, Emmanouil
Shanks, David R.
author_facet Vadillo, Miguel A.
Konstantinidis, Emmanouil
Shanks, David R.
author_sort Vadillo, Miguel A.
collection PubMed
description The scientific community has witnessed growing concern about the high rate of false positives and unreliable results within the psychological literature, but the harmful impact of false negatives has been largely ignored. False negatives are particularly concerning in research areas where demonstrating the absence of an effect is crucial, such as studies of unconscious or implicit processing. Research on implicit processes seeks evidence of above-chance performance on some implicit behavioral measure at the same time as chance-level performance (that is, a null result) on an explicit measure of awareness. A systematic review of 73 studies of contextual cuing, a popular implicit learning paradigm, involving 181 statistical analyses of awareness tests, reveals how underpowered studies can lead to failure to reject a false null hypothesis. Among the studies that reported sufficient information, the meta-analytic effect size across awareness tests was d(z) = 0.31 (95 % CI 0.24–0.37), showing that participants’ learning in these experiments was conscious. The unusually large number of positive results in this literature cannot be explained by selective publication. Instead, our analyses demonstrate that these tests are typically insensitive and underpowered to detect medium to small, but true, effects in awareness tests. These findings challenge a widespread and theoretically important claim about the extent of unconscious human cognition.
format Online
Article
Text
id pubmed-4742512
institution National Center for Biotechnology Information
language English
publishDate 2015
publisher Springer US
record_format MEDLINE/PubMed
spelling pubmed-47425122016-02-16 Underpowered samples, false negatives, and unconscious learning Vadillo, Miguel A. Konstantinidis, Emmanouil Shanks, David R. Psychon Bull Rev Theoretical Review The scientific community has witnessed growing concern about the high rate of false positives and unreliable results within the psychological literature, but the harmful impact of false negatives has been largely ignored. False negatives are particularly concerning in research areas where demonstrating the absence of an effect is crucial, such as studies of unconscious or implicit processing. Research on implicit processes seeks evidence of above-chance performance on some implicit behavioral measure at the same time as chance-level performance (that is, a null result) on an explicit measure of awareness. A systematic review of 73 studies of contextual cuing, a popular implicit learning paradigm, involving 181 statistical analyses of awareness tests, reveals how underpowered studies can lead to failure to reject a false null hypothesis. Among the studies that reported sufficient information, the meta-analytic effect size across awareness tests was d(z) = 0.31 (95 % CI 0.24–0.37), showing that participants’ learning in these experiments was conscious. The unusually large number of positive results in this literature cannot be explained by selective publication. Instead, our analyses demonstrate that these tests are typically insensitive and underpowered to detect medium to small, but true, effects in awareness tests. These findings challenge a widespread and theoretically important claim about the extent of unconscious human cognition. Springer US 2015-06-30 2016 /pmc/articles/PMC4742512/ /pubmed/26122896 http://dx.doi.org/10.3758/s13423-015-0892-6 Text en © The Author(s) 2015 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
spellingShingle Theoretical Review
Vadillo, Miguel A.
Konstantinidis, Emmanouil
Shanks, David R.
Underpowered samples, false negatives, and unconscious learning
title Underpowered samples, false negatives, and unconscious learning
title_full Underpowered samples, false negatives, and unconscious learning
title_fullStr Underpowered samples, false negatives, and unconscious learning
title_full_unstemmed Underpowered samples, false negatives, and unconscious learning
title_short Underpowered samples, false negatives, and unconscious learning
title_sort underpowered samples, false negatives, and unconscious learning
topic Theoretical Review
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4742512/
https://www.ncbi.nlm.nih.gov/pubmed/26122896
http://dx.doi.org/10.3758/s13423-015-0892-6
work_keys_str_mv AT vadillomiguela underpoweredsamplesfalsenegativesandunconsciouslearning
AT konstantinidisemmanouil underpoweredsamplesfalsenegativesandunconsciouslearning
AT shanksdavidr underpoweredsamplesfalsenegativesandunconsciouslearning