Cargando…
Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review
OBJECTIVE: To examine the time trend of statistical inference, statistical reporting style of results, and effect measures from the abstracts of randomized controlled trials (RCTs). STUDY DESGIN AND SETTINGS: We downloaded 385,867 PubMed abstracts of RCTs from 1975 to 2021. We used text-mining to de...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Springer Netherlands
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10570208/ https://www.ncbi.nlm.nih.gov/pubmed/37715928 http://dx.doi.org/10.1007/s10654-023-01047-8 |
_version_ | 1785119711962333184 |
---|---|
author | Stang, Andreas Rothman, Kenneth J |
author_facet | Stang, Andreas Rothman, Kenneth J |
author_sort | Stang, Andreas |
collection | PubMed |
description | OBJECTIVE: To examine the time trend of statistical inference, statistical reporting style of results, and effect measures from the abstracts of randomized controlled trials (RCTs). STUDY DESGIN AND SETTINGS: We downloaded 385,867 PubMed abstracts of RCTs from 1975 to 2021. We used text-mining to detect reporting of statistical inference (p-values, confidence intervals, significance terminology), statistical reporting style of results, and effect measures for binary outcomes, including time-to-event measures. We validated the text mining algorithms by random samples of abstracts. RESULTS: A total of 320 676 abstracts contained statistical inference. The percentage of abstracts including statistical inference increased from 65% (1975) to 87% (2006) and then decreased slightly. From 1975 to 1990, the sole reporting of language regarding statistical significance was predominant. Since 1990, reporting of p-values without confidence intervals has been the most common reporting style. Reporting of confidence intervals increased from 0.5% (1975) to 29% (2021). The two most common effect measures for binary outcomes were hazard ratios and odds ratios. Number needed to treat and number needed to harm are reported in less than 5% of abstracts with binary endpoints. CONCLUSIONS: Reporting of statistical inference in abstracts of RCTs has increased over time. Increasingly, p-values and confidence intervals are reported rather than just mentioning the presence of “statistical significance”. The reporting of odds ratios comes with the liability that the untrained reader will interpret them as risk ratios, which is often not justified, especially in RCTs. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s10654-023-01047-8. |
format | Online Article Text |
id | pubmed-10570208 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Springer Netherlands |
record_format | MEDLINE/PubMed |
spelling | pubmed-105702082023-10-14 Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review Stang, Andreas Rothman, Kenneth J Eur J Epidemiol Review OBJECTIVE: To examine the time trend of statistical inference, statistical reporting style of results, and effect measures from the abstracts of randomized controlled trials (RCTs). STUDY DESGIN AND SETTINGS: We downloaded 385,867 PubMed abstracts of RCTs from 1975 to 2021. We used text-mining to detect reporting of statistical inference (p-values, confidence intervals, significance terminology), statistical reporting style of results, and effect measures for binary outcomes, including time-to-event measures. We validated the text mining algorithms by random samples of abstracts. RESULTS: A total of 320 676 abstracts contained statistical inference. The percentage of abstracts including statistical inference increased from 65% (1975) to 87% (2006) and then decreased slightly. From 1975 to 1990, the sole reporting of language regarding statistical significance was predominant. Since 1990, reporting of p-values without confidence intervals has been the most common reporting style. Reporting of confidence intervals increased from 0.5% (1975) to 29% (2021). The two most common effect measures for binary outcomes were hazard ratios and odds ratios. Number needed to treat and number needed to harm are reported in less than 5% of abstracts with binary endpoints. CONCLUSIONS: Reporting of statistical inference in abstracts of RCTs has increased over time. Increasingly, p-values and confidence intervals are reported rather than just mentioning the presence of “statistical significance”. The reporting of odds ratios comes with the liability that the untrained reader will interpret them as risk ratios, which is often not justified, especially in RCTs. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s10654-023-01047-8. Springer Netherlands 2023-09-16 2023 /pmc/articles/PMC10570208/ /pubmed/37715928 http://dx.doi.org/10.1007/s10654-023-01047-8 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) . |
spellingShingle | Review Stang, Andreas Rothman, Kenneth J Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review |
title | Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review |
title_full | Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review |
title_fullStr | Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review |
title_full_unstemmed | Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review |
title_short | Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review |
title_sort | statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. a systematic review |
topic | Review |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10570208/ https://www.ncbi.nlm.nih.gov/pubmed/37715928 http://dx.doi.org/10.1007/s10654-023-01047-8 |
work_keys_str_mv | AT stangandreas statisticalinferenceandeffectmeasuresinabstractsofrandomizedcontrolledtrials19752021asystematicreview AT rothmankennethj statisticalinferenceandeffectmeasuresinabstractsofrandomizedcontrolledtrials19752021asystematicreview |