Cargando…

Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial

Objectives To see whether telling peer reviewers that their signed reviews of original research papers might be posted on the BMJ’s website would affect the quality of their reviews. Design Randomised controlled trial. Setting A large international general medical journal based in the United Kingdom...

Descripción completa

Detalles Bibliográficos
Autores principales: van Rooyen, Susan, Delamothe, Tony, Evans, Stephen J W
Formato: Texto
Lenguaje:English
Publicado: BMJ Publishing Group Ltd. 2010
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2982798/
https://www.ncbi.nlm.nih.gov/pubmed/21081600
http://dx.doi.org/10.1136/bmj.c5729
_version_ 1782191761995595776
author van Rooyen, Susan
Delamothe, Tony
Evans, Stephen J W
author_facet van Rooyen, Susan
Delamothe, Tony
Evans, Stephen J W
author_sort van Rooyen, Susan
collection PubMed
description Objectives To see whether telling peer reviewers that their signed reviews of original research papers might be posted on the BMJ’s website would affect the quality of their reviews. Design Randomised controlled trial. Setting A large international general medical journal based in the United Kingdom. Participants 541 authors, 471 peer reviewers, and 12 editors. Intervention Consecutive eligible papers were randomised either to have the reviewer’s signed report made available on the BMJ’s website alongside the published paper (intervention group) or to have the report made available only to the author—the BMJ’s normal procedure (control group). The intervention was the act of revealing to reviewers—after they had agreed to review but before they undertook their review—that their signed report might appear on the website. Main outcome measures The main outcome measure was the quality of the reviews, as independently rated on a scale of 1 to 5 using a validated instrument by two editors and the corresponding author. Authors and editors were blind to the intervention group. Authors rated review quality before the fate of their paper had been decided. Additional outcomes were the time taken to complete the review and the reviewer’s recommendation regarding publication. Results 558 manuscripts were randomised, and 471 manuscripts remained after exclusions. Of the 1039 reviewers approached to take part in the study, 568 (55%) declined. Two editors’ evaluations of the quality of the peer review were obtained for all 471 manuscripts, with the corresponding author’s evaluation obtained for 453. There was no significant difference in review quality between the intervention and control groups (mean difference for editors 0.04, 95% CI −0.09 to 0.17; for authors 0.06, 95% CI −0.09 to 0.20). Any possible difference in favour of the control group was well below the level regarded as editorially significant. Reviewers in the intervention group took significantly longer to review (mean difference 25 minutes, 95% CI 3.0 to 47.0 minutes). Conclusion Telling peer reviewers that their signed reviews might be available in the public domain on the BMJ’s website had no important effect on review quality. Although the possibility of posting reviews online was associated with a high refusal rate among potential peer reviewers and an increase in the amount of time taken to write a review, we believe that the ethical arguments in favour of open peer review more than outweigh these disadvantages.
format Text
id pubmed-2982798
institution National Center for Biotechnology Information
language English
publishDate 2010
publisher BMJ Publishing Group Ltd.
record_format MEDLINE/PubMed
spelling pubmed-29827982010-11-26 Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial van Rooyen, Susan Delamothe, Tony Evans, Stephen J W BMJ Research Objectives To see whether telling peer reviewers that their signed reviews of original research papers might be posted on the BMJ’s website would affect the quality of their reviews. Design Randomised controlled trial. Setting A large international general medical journal based in the United Kingdom. Participants 541 authors, 471 peer reviewers, and 12 editors. Intervention Consecutive eligible papers were randomised either to have the reviewer’s signed report made available on the BMJ’s website alongside the published paper (intervention group) or to have the report made available only to the author—the BMJ’s normal procedure (control group). The intervention was the act of revealing to reviewers—after they had agreed to review but before they undertook their review—that their signed report might appear on the website. Main outcome measures The main outcome measure was the quality of the reviews, as independently rated on a scale of 1 to 5 using a validated instrument by two editors and the corresponding author. Authors and editors were blind to the intervention group. Authors rated review quality before the fate of their paper had been decided. Additional outcomes were the time taken to complete the review and the reviewer’s recommendation regarding publication. Results 558 manuscripts were randomised, and 471 manuscripts remained after exclusions. Of the 1039 reviewers approached to take part in the study, 568 (55%) declined. Two editors’ evaluations of the quality of the peer review were obtained for all 471 manuscripts, with the corresponding author’s evaluation obtained for 453. There was no significant difference in review quality between the intervention and control groups (mean difference for editors 0.04, 95% CI −0.09 to 0.17; for authors 0.06, 95% CI −0.09 to 0.20). Any possible difference in favour of the control group was well below the level regarded as editorially significant. Reviewers in the intervention group took significantly longer to review (mean difference 25 minutes, 95% CI 3.0 to 47.0 minutes). Conclusion Telling peer reviewers that their signed reviews might be available in the public domain on the BMJ’s website had no important effect on review quality. Although the possibility of posting reviews online was associated with a high refusal rate among potential peer reviewers and an increase in the amount of time taken to write a review, we believe that the ethical arguments in favour of open peer review more than outweigh these disadvantages. BMJ Publishing Group Ltd. 2010-11-16 /pmc/articles/PMC2982798/ /pubmed/21081600 http://dx.doi.org/10.1136/bmj.c5729 Text en © van Rooyen et al 2010 This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode.
spellingShingle Research
van Rooyen, Susan
Delamothe, Tony
Evans, Stephen J W
Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial
title Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial
title_full Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial
title_fullStr Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial
title_full_unstemmed Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial
title_short Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial
title_sort effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial
topic Research
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2982798/
https://www.ncbi.nlm.nih.gov/pubmed/21081600
http://dx.doi.org/10.1136/bmj.c5729
work_keys_str_mv AT vanrooyensusan effectonpeerreviewoftellingreviewersthattheirsignedreviewsmightbepostedonthewebrandomisedcontrolledtrial
AT delamothetony effectonpeerreviewoftellingreviewersthattheirsignedreviewsmightbepostedonthewebrandomisedcontrolledtrial
AT evansstephenjw effectonpeerreviewoftellingreviewersthattheirsignedreviewsmightbepostedonthewebrandomisedcontrolledtrial