Cargando…
The intriguing evolution of effect sizes in biomedical research over time: smaller but more often statistically significant
BACKGROUND: In medicine, effect sizes (ESs) allow the effects of independent variables (including risk/protective factors or treatment interventions) on dependent variables (e.g., health outcomes) to be quantified. Given that many public health decisions and health care policies are based on ES esti...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Oxford University Press
2017
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5765564/ https://www.ncbi.nlm.nih.gov/pubmed/29228281 http://dx.doi.org/10.1093/gigascience/gix121 |
_version_ | 1783292255384109056 |
---|---|
author | Monsarrat, Paul Vergnes, Jean-Noel |
author_facet | Monsarrat, Paul Vergnes, Jean-Noel |
author_sort | Monsarrat, Paul |
collection | PubMed |
description | BACKGROUND: In medicine, effect sizes (ESs) allow the effects of independent variables (including risk/protective factors or treatment interventions) on dependent variables (e.g., health outcomes) to be quantified. Given that many public health decisions and health care policies are based on ES estimates, it is important to assess how ESs are used in the biomedical literature and to investigate potential trends in their reporting over time. RESULTS: Through a big data approach, the text mining process automatically extracted 814 120 ESs from 13 322 754 PubMed abstracts. Eligible ESs were risk ratio, odds ratio, and hazard ratio, along with their confidence intervals. Here we show a remarkable decrease of ES values in PubMed abstracts between 1990 and 2015 while, concomitantly, results become more often statistically significant. Medians of ES values have decreased over time for both “risk” and “protective” values. This trend was found in nearly all fields of biomedical research, with the most marked downward tendency in genetics. Over the same period, the proportion of statistically significant ESs increased regularly: among the abstracts with at least 1 ES, 74% were statistically significant in 1990–1995, vs 85% in 2010–2015. CONCLUSIONS: whereas decreasing ESs could be an intrinsic evolution in biomedical research, the concomitant increase of statistically significant results is more intriguing. Although it is likely that growing sample sizes in biomedical research could explain these results, another explanation may lie in the “publish or perish” context of scientific research, with the probability of a growing orientation toward sensationalism in research reports. Important provisions must be made to improve the credibility of biomedical research and limit waste of resources. |
format | Online Article Text |
id | pubmed-5765564 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2017 |
publisher | Oxford University Press |
record_format | MEDLINE/PubMed |
spelling | pubmed-57655642018-01-16 The intriguing evolution of effect sizes in biomedical research over time: smaller but more often statistically significant Monsarrat, Paul Vergnes, Jean-Noel Gigascience Research BACKGROUND: In medicine, effect sizes (ESs) allow the effects of independent variables (including risk/protective factors or treatment interventions) on dependent variables (e.g., health outcomes) to be quantified. Given that many public health decisions and health care policies are based on ES estimates, it is important to assess how ESs are used in the biomedical literature and to investigate potential trends in their reporting over time. RESULTS: Through a big data approach, the text mining process automatically extracted 814 120 ESs from 13 322 754 PubMed abstracts. Eligible ESs were risk ratio, odds ratio, and hazard ratio, along with their confidence intervals. Here we show a remarkable decrease of ES values in PubMed abstracts between 1990 and 2015 while, concomitantly, results become more often statistically significant. Medians of ES values have decreased over time for both “risk” and “protective” values. This trend was found in nearly all fields of biomedical research, with the most marked downward tendency in genetics. Over the same period, the proportion of statistically significant ESs increased regularly: among the abstracts with at least 1 ES, 74% were statistically significant in 1990–1995, vs 85% in 2010–2015. CONCLUSIONS: whereas decreasing ESs could be an intrinsic evolution in biomedical research, the concomitant increase of statistically significant results is more intriguing. Although it is likely that growing sample sizes in biomedical research could explain these results, another explanation may lie in the “publish or perish” context of scientific research, with the probability of a growing orientation toward sensationalism in research reports. Important provisions must be made to improve the credibility of biomedical research and limit waste of resources. Oxford University Press 2017-12-06 /pmc/articles/PMC5765564/ /pubmed/29228281 http://dx.doi.org/10.1093/gigascience/gix121 Text en © The Author 2017. Published by Oxford University Press. http://creativecommons.org/licenses/by/4.0/ This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. |
spellingShingle | Research Monsarrat, Paul Vergnes, Jean-Noel The intriguing evolution of effect sizes in biomedical research over time: smaller but more often statistically significant |
title | The intriguing evolution of effect sizes in biomedical research over time: smaller but more often statistically significant |
title_full | The intriguing evolution of effect sizes in biomedical research over time: smaller but more often statistically significant |
title_fullStr | The intriguing evolution of effect sizes in biomedical research over time: smaller but more often statistically significant |
title_full_unstemmed | The intriguing evolution of effect sizes in biomedical research over time: smaller but more often statistically significant |
title_short | The intriguing evolution of effect sizes in biomedical research over time: smaller but more often statistically significant |
title_sort | intriguing evolution of effect sizes in biomedical research over time: smaller but more often statistically significant |
topic | Research |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5765564/ https://www.ncbi.nlm.nih.gov/pubmed/29228281 http://dx.doi.org/10.1093/gigascience/gix121 |
work_keys_str_mv | AT monsarratpaul theintriguingevolutionofeffectsizesinbiomedicalresearchovertimesmallerbutmoreoftenstatisticallysignificant AT vergnesjeannoel theintriguingevolutionofeffectsizesinbiomedicalresearchovertimesmallerbutmoreoftenstatisticallysignificant AT monsarratpaul intriguingevolutionofeffectsizesinbiomedicalresearchovertimesmallerbutmoreoftenstatisticallysignificant AT vergnesjeannoel intriguingevolutionofeffectsizesinbiomedicalresearchovertimesmallerbutmoreoftenstatisticallysignificant |