Cargando…

Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review

OBJECTIVE: To examine the time trend of statistical inference, statistical reporting style of results, and effect measures from the abstracts of randomized controlled trials (RCTs). STUDY DESGIN AND SETTINGS: We downloaded 385,867 PubMed abstracts of RCTs from 1975 to 2021. We used text-mining to de...

Descripción completa

Detalles Bibliográficos
Autores principales: Stang, Andreas, Rothman, Kenneth J
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Springer Netherlands 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10570208/
https://www.ncbi.nlm.nih.gov/pubmed/37715928
http://dx.doi.org/10.1007/s10654-023-01047-8
_version_ 1785119711962333184
author Stang, Andreas
Rothman, Kenneth J
author_facet Stang, Andreas
Rothman, Kenneth J
author_sort Stang, Andreas
collection PubMed
description OBJECTIVE: To examine the time trend of statistical inference, statistical reporting style of results, and effect measures from the abstracts of randomized controlled trials (RCTs). STUDY DESGIN AND SETTINGS: We downloaded 385,867 PubMed abstracts of RCTs from 1975 to 2021. We used text-mining to detect reporting of statistical inference (p-values, confidence intervals, significance terminology), statistical reporting style of results, and effect measures for binary outcomes, including time-to-event measures. We validated the text mining algorithms by random samples of abstracts. RESULTS: A total of 320 676 abstracts contained statistical inference. The percentage of abstracts including statistical inference increased from 65% (1975) to 87% (2006) and then decreased slightly. From 1975 to 1990, the sole reporting of language regarding statistical significance was predominant. Since 1990, reporting of p-values without confidence intervals has been the most common reporting style. Reporting of confidence intervals increased from 0.5% (1975) to 29% (2021). The two most common effect measures for binary outcomes were hazard ratios and odds ratios. Number needed to treat and number needed to harm are reported in less than 5% of abstracts with binary endpoints. CONCLUSIONS: Reporting of statistical inference in abstracts of RCTs has increased over time. Increasingly, p-values and confidence intervals are reported rather than just mentioning the presence of “statistical significance”. The reporting of odds ratios comes with the liability that the untrained reader will interpret them as risk ratios, which is often not justified, especially in RCTs. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s10654-023-01047-8.
format Online
Article
Text
id pubmed-10570208
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Springer Netherlands
record_format MEDLINE/PubMed
spelling pubmed-105702082023-10-14 Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review Stang, Andreas Rothman, Kenneth J Eur J Epidemiol Review OBJECTIVE: To examine the time trend of statistical inference, statistical reporting style of results, and effect measures from the abstracts of randomized controlled trials (RCTs). STUDY DESGIN AND SETTINGS: We downloaded 385,867 PubMed abstracts of RCTs from 1975 to 2021. We used text-mining to detect reporting of statistical inference (p-values, confidence intervals, significance terminology), statistical reporting style of results, and effect measures for binary outcomes, including time-to-event measures. We validated the text mining algorithms by random samples of abstracts. RESULTS: A total of 320 676 abstracts contained statistical inference. The percentage of abstracts including statistical inference increased from 65% (1975) to 87% (2006) and then decreased slightly. From 1975 to 1990, the sole reporting of language regarding statistical significance was predominant. Since 1990, reporting of p-values without confidence intervals has been the most common reporting style. Reporting of confidence intervals increased from 0.5% (1975) to 29% (2021). The two most common effect measures for binary outcomes were hazard ratios and odds ratios. Number needed to treat and number needed to harm are reported in less than 5% of abstracts with binary endpoints. CONCLUSIONS: Reporting of statistical inference in abstracts of RCTs has increased over time. Increasingly, p-values and confidence intervals are reported rather than just mentioning the presence of “statistical significance”. The reporting of odds ratios comes with the liability that the untrained reader will interpret them as risk ratios, which is often not justified, especially in RCTs. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s10654-023-01047-8. Springer Netherlands 2023-09-16 2023 /pmc/articles/PMC10570208/ /pubmed/37715928 http://dx.doi.org/10.1007/s10654-023-01047-8 Text en © The Author(s) 2023 https://creativecommons.org/licenses/by/4.0/Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Review
Stang, Andreas
Rothman, Kenneth J
Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review
title Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review
title_full Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review
title_fullStr Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review
title_full_unstemmed Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review
title_short Statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. A systematic review
title_sort statistical inference and effect measures in abstracts of randomized controlled trials, 1975–2021. a systematic review
topic Review
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10570208/
https://www.ncbi.nlm.nih.gov/pubmed/37715928
http://dx.doi.org/10.1007/s10654-023-01047-8
work_keys_str_mv AT stangandreas statisticalinferenceandeffectmeasuresinabstractsofrandomizedcontrolledtrials19752021asystematicreview
AT rothmankennethj statisticalinferenceandeffectmeasuresinabstractsofrandomizedcontrolledtrials19752021asystematicreview