Cargando…
Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial
Objectives To see whether telling peer reviewers that their signed reviews of original research papers might be posted on the BMJ’s website would affect the quality of their reviews. Design Randomised controlled trial. Setting A large international general medical journal based in the United Kingdom...
Autores principales: | , , |
---|---|
Formato: | Texto |
Lenguaje: | English |
Publicado: |
BMJ Publishing Group Ltd.
2010
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2982798/ https://www.ncbi.nlm.nih.gov/pubmed/21081600 http://dx.doi.org/10.1136/bmj.c5729 |
_version_ | 1782191761995595776 |
---|---|
author | van Rooyen, Susan Delamothe, Tony Evans, Stephen J W |
author_facet | van Rooyen, Susan Delamothe, Tony Evans, Stephen J W |
author_sort | van Rooyen, Susan |
collection | PubMed |
description | Objectives To see whether telling peer reviewers that their signed reviews of original research papers might be posted on the BMJ’s website would affect the quality of their reviews. Design Randomised controlled trial. Setting A large international general medical journal based in the United Kingdom. Participants 541 authors, 471 peer reviewers, and 12 editors. Intervention Consecutive eligible papers were randomised either to have the reviewer’s signed report made available on the BMJ’s website alongside the published paper (intervention group) or to have the report made available only to the author—the BMJ’s normal procedure (control group). The intervention was the act of revealing to reviewers—after they had agreed to review but before they undertook their review—that their signed report might appear on the website. Main outcome measures The main outcome measure was the quality of the reviews, as independently rated on a scale of 1 to 5 using a validated instrument by two editors and the corresponding author. Authors and editors were blind to the intervention group. Authors rated review quality before the fate of their paper had been decided. Additional outcomes were the time taken to complete the review and the reviewer’s recommendation regarding publication. Results 558 manuscripts were randomised, and 471 manuscripts remained after exclusions. Of the 1039 reviewers approached to take part in the study, 568 (55%) declined. Two editors’ evaluations of the quality of the peer review were obtained for all 471 manuscripts, with the corresponding author’s evaluation obtained for 453. There was no significant difference in review quality between the intervention and control groups (mean difference for editors 0.04, 95% CI −0.09 to 0.17; for authors 0.06, 95% CI −0.09 to 0.20). Any possible difference in favour of the control group was well below the level regarded as editorially significant. Reviewers in the intervention group took significantly longer to review (mean difference 25 minutes, 95% CI 3.0 to 47.0 minutes). Conclusion Telling peer reviewers that their signed reviews might be available in the public domain on the BMJ’s website had no important effect on review quality. Although the possibility of posting reviews online was associated with a high refusal rate among potential peer reviewers and an increase in the amount of time taken to write a review, we believe that the ethical arguments in favour of open peer review more than outweigh these disadvantages. |
format | Text |
id | pubmed-2982798 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2010 |
publisher | BMJ Publishing Group Ltd. |
record_format | MEDLINE/PubMed |
spelling | pubmed-29827982010-11-26 Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial van Rooyen, Susan Delamothe, Tony Evans, Stephen J W BMJ Research Objectives To see whether telling peer reviewers that their signed reviews of original research papers might be posted on the BMJ’s website would affect the quality of their reviews. Design Randomised controlled trial. Setting A large international general medical journal based in the United Kingdom. Participants 541 authors, 471 peer reviewers, and 12 editors. Intervention Consecutive eligible papers were randomised either to have the reviewer’s signed report made available on the BMJ’s website alongside the published paper (intervention group) or to have the report made available only to the author—the BMJ’s normal procedure (control group). The intervention was the act of revealing to reviewers—after they had agreed to review but before they undertook their review—that their signed report might appear on the website. Main outcome measures The main outcome measure was the quality of the reviews, as independently rated on a scale of 1 to 5 using a validated instrument by two editors and the corresponding author. Authors and editors were blind to the intervention group. Authors rated review quality before the fate of their paper had been decided. Additional outcomes were the time taken to complete the review and the reviewer’s recommendation regarding publication. Results 558 manuscripts were randomised, and 471 manuscripts remained after exclusions. Of the 1039 reviewers approached to take part in the study, 568 (55%) declined. Two editors’ evaluations of the quality of the peer review were obtained for all 471 manuscripts, with the corresponding author’s evaluation obtained for 453. There was no significant difference in review quality between the intervention and control groups (mean difference for editors 0.04, 95% CI −0.09 to 0.17; for authors 0.06, 95% CI −0.09 to 0.20). Any possible difference in favour of the control group was well below the level regarded as editorially significant. Reviewers in the intervention group took significantly longer to review (mean difference 25 minutes, 95% CI 3.0 to 47.0 minutes). Conclusion Telling peer reviewers that their signed reviews might be available in the public domain on the BMJ’s website had no important effect on review quality. Although the possibility of posting reviews online was associated with a high refusal rate among potential peer reviewers and an increase in the amount of time taken to write a review, we believe that the ethical arguments in favour of open peer review more than outweigh these disadvantages. BMJ Publishing Group Ltd. 2010-11-16 /pmc/articles/PMC2982798/ /pubmed/21081600 http://dx.doi.org/10.1136/bmj.c5729 Text en © van Rooyen et al 2010 This is an open-access article distributed under the terms of the Creative Commons Attribution Non-commercial License, which permits use, distribution, and reproduction in any medium, provided the original work is properly cited, the use is non commercial and is otherwise in compliance with the license. See: http://creativecommons.org/licenses/by-nc/2.0/ and http://creativecommons.org/licenses/by-nc/2.0/legalcode. |
spellingShingle | Research van Rooyen, Susan Delamothe, Tony Evans, Stephen J W Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial |
title | Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial |
title_full | Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial |
title_fullStr | Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial |
title_full_unstemmed | Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial |
title_short | Effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial |
title_sort | effect on peer review of telling reviewers that their signed reviews might be posted on the web: randomised controlled trial |
topic | Research |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2982798/ https://www.ncbi.nlm.nih.gov/pubmed/21081600 http://dx.doi.org/10.1136/bmj.c5729 |
work_keys_str_mv | AT vanrooyensusan effectonpeerreviewoftellingreviewersthattheirsignedreviewsmightbepostedonthewebrandomisedcontrolledtrial AT delamothetony effectonpeerreviewoftellingreviewersthattheirsignedreviewsmightbepostedonthewebrandomisedcontrolledtrial AT evansstephenjw effectonpeerreviewoftellingreviewersthattheirsignedreviewsmightbepostedonthewebrandomisedcontrolledtrial |