Cargando…

Reporting and interpreting non-significant results in animal cognition research

How statistically non-significant results are reported and interpreted following null hypothesis significance testing is often criticized. This issue is important for animal cognition research because studies in the field are often underpowered to detect theoretically meaningful effect sizes, i.e.,...

Descripción completa

Detalles Bibliográficos
Autores principales: Farrar, Benjamin G., Vernouillet, Alizée, Garcia-Pelegrin, Elias, Legg, Edward W., Brecht, Katharina F., Lambert, Poppy J., Elsherif, Mahmoud, Francis, Shannon, O’Neill, Laurie, Clayton, Nicola S., Ostojić, Ljerka
Formato: Online Artículo Texto
Lenguaje:English
Publicado: PeerJ Inc. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10008313/
https://www.ncbi.nlm.nih.gov/pubmed/36919170
http://dx.doi.org/10.7717/peerj.14963
_version_ 1784905729960837120
author Farrar, Benjamin G.
Vernouillet, Alizée
Garcia-Pelegrin, Elias
Legg, Edward W.
Brecht, Katharina F.
Lambert, Poppy J.
Elsherif, Mahmoud
Francis, Shannon
O’Neill, Laurie
Clayton, Nicola S.
Ostojić, Ljerka
author_facet Farrar, Benjamin G.
Vernouillet, Alizée
Garcia-Pelegrin, Elias
Legg, Edward W.
Brecht, Katharina F.
Lambert, Poppy J.
Elsherif, Mahmoud
Francis, Shannon
O’Neill, Laurie
Clayton, Nicola S.
Ostojić, Ljerka
author_sort Farrar, Benjamin G.
collection PubMed
description How statistically non-significant results are reported and interpreted following null hypothesis significance testing is often criticized. This issue is important for animal cognition research because studies in the field are often underpowered to detect theoretically meaningful effect sizes, i.e., often produce non-significant p-values even when the null hypothesis is incorrect. Thus, we manually extracted and classified how researchers report and interpret non-significant p-values and examined the p-value distribution of these non-significant results across published articles in animal cognition and related fields. We found a large amount of heterogeneity in how researchers report statistically non-significant p-values in the result sections of articles, and how they interpret them in the titles and abstracts. Reporting of the non-significant results as “No Effect” was common in the titles (84%), abstracts (64%), and results sections (41%) of papers, whereas reporting of the results as “Non-Significant” was less common in the titles (0%) and abstracts (26%), but was present in the results (52%). Discussions of effect sizes were rare (<5% of articles). A p-value distribution analysis was consistent with research being performed with low power of statistical tests to detect effect sizes of interest. These findings suggest that researchers in animal cognition should pay close attention to the evidence used to support claims of absence of effects in the literature, and—in their own work—report statistically non-significant results clearly and formally correct, as well as use more formal methods of assessing evidence against theoretical predictions.
format Online
Article
Text
id pubmed-10008313
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher PeerJ Inc.
record_format MEDLINE/PubMed
spelling pubmed-100083132023-03-13 Reporting and interpreting non-significant results in animal cognition research Farrar, Benjamin G. Vernouillet, Alizée Garcia-Pelegrin, Elias Legg, Edward W. Brecht, Katharina F. Lambert, Poppy J. Elsherif, Mahmoud Francis, Shannon O’Neill, Laurie Clayton, Nicola S. Ostojić, Ljerka PeerJ Animal Behavior How statistically non-significant results are reported and interpreted following null hypothesis significance testing is often criticized. This issue is important for animal cognition research because studies in the field are often underpowered to detect theoretically meaningful effect sizes, i.e., often produce non-significant p-values even when the null hypothesis is incorrect. Thus, we manually extracted and classified how researchers report and interpret non-significant p-values and examined the p-value distribution of these non-significant results across published articles in animal cognition and related fields. We found a large amount of heterogeneity in how researchers report statistically non-significant p-values in the result sections of articles, and how they interpret them in the titles and abstracts. Reporting of the non-significant results as “No Effect” was common in the titles (84%), abstracts (64%), and results sections (41%) of papers, whereas reporting of the results as “Non-Significant” was less common in the titles (0%) and abstracts (26%), but was present in the results (52%). Discussions of effect sizes were rare (<5% of articles). A p-value distribution analysis was consistent with research being performed with low power of statistical tests to detect effect sizes of interest. These findings suggest that researchers in animal cognition should pay close attention to the evidence used to support claims of absence of effects in the literature, and—in their own work—report statistically non-significant results clearly and formally correct, as well as use more formal methods of assessing evidence against theoretical predictions. PeerJ Inc. 2023-03-09 /pmc/articles/PMC10008313/ /pubmed/36919170 http://dx.doi.org/10.7717/peerj.14963 Text en ©2023 Farrar et al. https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. For attribution, the original author(s), title, publication source (PeerJ) and either DOI or URL of the article must be cited.
spellingShingle Animal Behavior
Farrar, Benjamin G.
Vernouillet, Alizée
Garcia-Pelegrin, Elias
Legg, Edward W.
Brecht, Katharina F.
Lambert, Poppy J.
Elsherif, Mahmoud
Francis, Shannon
O’Neill, Laurie
Clayton, Nicola S.
Ostojić, Ljerka
Reporting and interpreting non-significant results in animal cognition research
title Reporting and interpreting non-significant results in animal cognition research
title_full Reporting and interpreting non-significant results in animal cognition research
title_fullStr Reporting and interpreting non-significant results in animal cognition research
title_full_unstemmed Reporting and interpreting non-significant results in animal cognition research
title_short Reporting and interpreting non-significant results in animal cognition research
title_sort reporting and interpreting non-significant results in animal cognition research
topic Animal Behavior
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10008313/
https://www.ncbi.nlm.nih.gov/pubmed/36919170
http://dx.doi.org/10.7717/peerj.14963
work_keys_str_mv AT farrarbenjaming reportingandinterpretingnonsignificantresultsinanimalcognitionresearch
AT vernouilletalizee reportingandinterpretingnonsignificantresultsinanimalcognitionresearch
AT garciapelegrinelias reportingandinterpretingnonsignificantresultsinanimalcognitionresearch
AT leggedwardw reportingandinterpretingnonsignificantresultsinanimalcognitionresearch
AT brechtkatharinaf reportingandinterpretingnonsignificantresultsinanimalcognitionresearch
AT lambertpoppyj reportingandinterpretingnonsignificantresultsinanimalcognitionresearch
AT elsherifmahmoud reportingandinterpretingnonsignificantresultsinanimalcognitionresearch
AT francisshannon reportingandinterpretingnonsignificantresultsinanimalcognitionresearch
AT oneilllaurie reportingandinterpretingnonsignificantresultsinanimalcognitionresearch
AT claytonnicolas reportingandinterpretingnonsignificantresultsinanimalcognitionresearch
AT ostojicljerka reportingandinterpretingnonsignificantresultsinanimalcognitionresearch