Cargando…

Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial

BACKGROUND: Although peer review is widely considered to be the most credible way of selecting manuscripts and improving the quality of accepted papers in scientific journals, there is little evidence to support its use. Our aim was to estimate the effects on manuscript quality of either adding a st...

Descripción completa

Detalles Bibliográficos
Autores principales: Cobo, Erik, Selva-O'Callagham, Albert, Ribera, Josep-Maria, Cardellach, Francesc, Dominguez, Ruth, Vilardell, Miquel
Formato: Texto
Lenguaje:English
Publicado: Public Library of Science 2007
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1824709/
https://www.ncbi.nlm.nih.gov/pubmed/17389922
http://dx.doi.org/10.1371/journal.pone.0000332
_version_ 1782132689004920832
author Cobo, Erik
Selva-O'Callagham, Albert
Ribera, Josep-Maria
Cardellach, Francesc
Dominguez, Ruth
Vilardell, Miquel
author_facet Cobo, Erik
Selva-O'Callagham, Albert
Ribera, Josep-Maria
Cardellach, Francesc
Dominguez, Ruth
Vilardell, Miquel
author_sort Cobo, Erik
collection PubMed
description BACKGROUND: Although peer review is widely considered to be the most credible way of selecting manuscripts and improving the quality of accepted papers in scientific journals, there is little evidence to support its use. Our aim was to estimate the effects on manuscript quality of either adding a statistical peer reviewer or suggesting the use of checklists such as CONSORT or STARD to clinical reviewers or both. METHODOLOGY AND PRINCIPAL FINDINGS: Interventions were defined as 1) the addition of a statistical reviewer to the clinical peer review process, and 2) suggesting reporting guidelines to reviewers; with “no statistical expert” and “no checklist” as controls. The two interventions were crossed in a 2×2 balanced factorial design including original research articles consecutively selected, between May 2004 and March 2005, by the Medicina Clinica (Barc) editorial committee. We randomized manuscripts to minimize differences in terms of baseline quality and type of study (intervention, longitudinal, cross-sectional, others). Sample-size calculations indicated that 100 papers provide an 80% power to test a 55% standardized difference. We specified the main outcome as the increment in quality of papers as measured on the Goodman Scale. Two blinded evaluators rated the quality of manuscripts at initial submission and final post peer review version. Of the 327 manuscripts submitted to the journal, 131 were accepted for further review, and 129 were randomized. Of those, 14 that were lost to follow-up showed no differences in initial quality to the followed-up papers. Hence, 115 were included in the main analysis, with 16 rejected for publication after peer review. 21 (18.3%) of the 115 included papers were interventions, 46 (40.0%) were longitudinal designs, 28 (24.3%) cross-sectional and 20 (17.4%) others. The 16 (13.9%) rejected papers had a significantly lower initial score on the overall Goodman scale than accepted papers (difference 15.0, 95% CI: 4.6–24.4). The effect of suggesting a guideline to the reviewers had no effect on change in overall quality as measured by the Goodman scale (0.9, 95% CI: −0.3–+2.1). The estimated effect of adding a statistical reviewer was 5.5 (95% CI: 4.3–6.7), showing a significant improvement in quality. CONCLUSIONS AND SIGNIFICANCE: This prospective randomized study shows the positive effect of adding a statistical reviewer to the field-expert peers in improving manuscript quality. We did not find a statistically significant positive effect by suggesting reviewers use reporting guidelines.
format Text
id pubmed-1824709
institution National Center for Biotechnology Information
language English
publishDate 2007
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-18247092007-03-28 Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial Cobo, Erik Selva-O'Callagham, Albert Ribera, Josep-Maria Cardellach, Francesc Dominguez, Ruth Vilardell, Miquel PLoS One Research Article BACKGROUND: Although peer review is widely considered to be the most credible way of selecting manuscripts and improving the quality of accepted papers in scientific journals, there is little evidence to support its use. Our aim was to estimate the effects on manuscript quality of either adding a statistical peer reviewer or suggesting the use of checklists such as CONSORT or STARD to clinical reviewers or both. METHODOLOGY AND PRINCIPAL FINDINGS: Interventions were defined as 1) the addition of a statistical reviewer to the clinical peer review process, and 2) suggesting reporting guidelines to reviewers; with “no statistical expert” and “no checklist” as controls. The two interventions were crossed in a 2×2 balanced factorial design including original research articles consecutively selected, between May 2004 and March 2005, by the Medicina Clinica (Barc) editorial committee. We randomized manuscripts to minimize differences in terms of baseline quality and type of study (intervention, longitudinal, cross-sectional, others). Sample-size calculations indicated that 100 papers provide an 80% power to test a 55% standardized difference. We specified the main outcome as the increment in quality of papers as measured on the Goodman Scale. Two blinded evaluators rated the quality of manuscripts at initial submission and final post peer review version. Of the 327 manuscripts submitted to the journal, 131 were accepted for further review, and 129 were randomized. Of those, 14 that were lost to follow-up showed no differences in initial quality to the followed-up papers. Hence, 115 were included in the main analysis, with 16 rejected for publication after peer review. 21 (18.3%) of the 115 included papers were interventions, 46 (40.0%) were longitudinal designs, 28 (24.3%) cross-sectional and 20 (17.4%) others. The 16 (13.9%) rejected papers had a significantly lower initial score on the overall Goodman scale than accepted papers (difference 15.0, 95% CI: 4.6–24.4). The effect of suggesting a guideline to the reviewers had no effect on change in overall quality as measured by the Goodman scale (0.9, 95% CI: −0.3–+2.1). The estimated effect of adding a statistical reviewer was 5.5 (95% CI: 4.3–6.7), showing a significant improvement in quality. CONCLUSIONS AND SIGNIFICANCE: This prospective randomized study shows the positive effect of adding a statistical reviewer to the field-expert peers in improving manuscript quality. We did not find a statistically significant positive effect by suggesting reviewers use reporting guidelines. Public Library of Science 2007-03-28 /pmc/articles/PMC1824709/ /pubmed/17389922 http://dx.doi.org/10.1371/journal.pone.0000332 Text en Cobo et al. http://creativecommons.org/licenses/by/4.0/ This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are properly credited.
spellingShingle Research Article
Cobo, Erik
Selva-O'Callagham, Albert
Ribera, Josep-Maria
Cardellach, Francesc
Dominguez, Ruth
Vilardell, Miquel
Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial
title Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial
title_full Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial
title_fullStr Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial
title_full_unstemmed Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial
title_short Statistical Reviewers Improve Reporting in Biomedical Articles: A Randomized Trial
title_sort statistical reviewers improve reporting in biomedical articles: a randomized trial
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1824709/
https://www.ncbi.nlm.nih.gov/pubmed/17389922
http://dx.doi.org/10.1371/journal.pone.0000332
work_keys_str_mv AT coboerik statisticalreviewersimprovereportinginbiomedicalarticlesarandomizedtrial
AT selvaocallaghamalbert statisticalreviewersimprovereportinginbiomedicalarticlesarandomizedtrial
AT riberajosepmaria statisticalreviewersimprovereportinginbiomedicalarticlesarandomizedtrial
AT cardellachfrancesc statisticalreviewersimprovereportinginbiomedicalarticlesarandomizedtrial
AT dominguezruth statisticalreviewersimprovereportinginbiomedicalarticlesarandomizedtrial
AT vilardellmiquel statisticalreviewersimprovereportinginbiomedicalarticlesarandomizedtrial