Cargando…
Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects
Efficient medical progress requires that we know when a treatment effect is absent. We considered all 207 Original Articles published in the 2015 volume of the New England Journal of Medicine and found that 45 (21.7%) reported a null result for at least one of the primary outcome measures. Unfortuna...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2018
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5919013/ https://www.ncbi.nlm.nih.gov/pubmed/29694370 http://dx.doi.org/10.1371/journal.pone.0195474 |
_version_ | 1783317543372455936 |
---|---|
author | Hoekstra, Rink Monden, Rei van Ravenzwaaij, Don Wagenmakers, Eric-Jan |
author_facet | Hoekstra, Rink Monden, Rei van Ravenzwaaij, Don Wagenmakers, Eric-Jan |
author_sort | Hoekstra, Rink |
collection | PubMed |
description | Efficient medical progress requires that we know when a treatment effect is absent. We considered all 207 Original Articles published in the 2015 volume of the New England Journal of Medicine and found that 45 (21.7%) reported a null result for at least one of the primary outcome measures. Unfortunately, standard statistical analyses are unable to quantify the degree to which these null results actually support the null hypothesis. Such quantification is possible, however, by conducting a Bayesian hypothesis test. Here we reanalyzed a subset of 43 null results from 36 articles using a default Bayesian test for contingency tables. This Bayesian reanalysis revealed that, on average, the reported null results provided strong evidence for the absence of an effect. However, the degree of this evidence is variable and cannot be reliably predicted from the p-value. For null results, sample size is a better (albeit imperfect) predictor for the strength of evidence in favor of the null hypothesis. Together, our findings suggest that (a) the reported null results generally correspond to strong evidence in favor of the null hypothesis; (b) a Bayesian hypothesis test can provide additional information to assist the interpretation of null results. |
format | Online Article Text |
id | pubmed-5919013 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2018 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-59190132018-05-05 Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects Hoekstra, Rink Monden, Rei van Ravenzwaaij, Don Wagenmakers, Eric-Jan PLoS One Research Article Efficient medical progress requires that we know when a treatment effect is absent. We considered all 207 Original Articles published in the 2015 volume of the New England Journal of Medicine and found that 45 (21.7%) reported a null result for at least one of the primary outcome measures. Unfortunately, standard statistical analyses are unable to quantify the degree to which these null results actually support the null hypothesis. Such quantification is possible, however, by conducting a Bayesian hypothesis test. Here we reanalyzed a subset of 43 null results from 36 articles using a default Bayesian test for contingency tables. This Bayesian reanalysis revealed that, on average, the reported null results provided strong evidence for the absence of an effect. However, the degree of this evidence is variable and cannot be reliably predicted from the p-value. For null results, sample size is a better (albeit imperfect) predictor for the strength of evidence in favor of the null hypothesis. Together, our findings suggest that (a) the reported null results generally correspond to strong evidence in favor of the null hypothesis; (b) a Bayesian hypothesis test can provide additional information to assist the interpretation of null results. Public Library of Science 2018-04-25 /pmc/articles/PMC5919013/ /pubmed/29694370 http://dx.doi.org/10.1371/journal.pone.0195474 Text en © 2018 Hoekstra et al http://creativecommons.org/licenses/by/4.0/ This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Research Article Hoekstra, Rink Monden, Rei van Ravenzwaaij, Don Wagenmakers, Eric-Jan Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects |
title | Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects |
title_full | Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects |
title_fullStr | Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects |
title_full_unstemmed | Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects |
title_short | Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects |
title_sort | bayesian reanalysis of null results reported in medicine: strong yet variable evidence for the absence of treatment effects |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5919013/ https://www.ncbi.nlm.nih.gov/pubmed/29694370 http://dx.doi.org/10.1371/journal.pone.0195474 |
work_keys_str_mv | AT hoekstrarink bayesianreanalysisofnullresultsreportedinmedicinestrongyetvariableevidencefortheabsenceoftreatmenteffects AT mondenrei bayesianreanalysisofnullresultsreportedinmedicinestrongyetvariableevidencefortheabsenceoftreatmenteffects AT vanravenzwaaijdon bayesianreanalysisofnullresultsreportedinmedicinestrongyetvariableevidencefortheabsenceoftreatmenteffects AT wagenmakersericjan bayesianreanalysisofnullresultsreportedinmedicinestrongyetvariableevidencefortheabsenceoftreatmenteffects |