Cargando…

Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors

PURPOSE: Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer-review process. Peer-review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) h...

Descripción completa

Detalles Bibliográficos
Autores principales: O’Brien, Bridget C., Artino, Anthony R., Costello, Joseph A., Driessen, Erik, Maggio, Lauren A.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8629260/
https://www.ncbi.nlm.nih.gov/pubmed/34843564
http://dx.doi.org/10.1371/journal.pone.0260558
_version_ 1784607167737757696
author O’Brien, Bridget C.
Artino, Anthony R.
Costello, Joseph A.
Driessen, Erik
Maggio, Lauren A.
author_facet O’Brien, Bridget C.
Artino, Anthony R.
Costello, Joseph A.
Driessen, Erik
Maggio, Lauren A.
author_sort O’Brien, Bridget C.
collection PubMed
description PURPOSE: Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer-review process. Peer-review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone. METHODS: Our dataset included 358 reviews of 168 manuscripts submitted between January 1, 2019 and August 24, 2020 to a health professions education journal with a single blind review process. We first identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consistent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended “reject,” we coded for alignment between reviewers’ comments to the editor and to authors. We report descriptive statistics. RESULTS: 49% of reviews contained comments to the editor (n = 176). Most of these comments summarized the reviewers’ impression of the article (85%), which included explicit reference to their recommended decision (44%) and suitability for the journal (10%). The majority of comments addressed argument quality (56%) or research design/methods/data (51%). The tone of comments tended to be critical (40%) or constructive (34%). For the 86 reviews recommending “reject,” the majority of comments to the editor contained content that also appeared in comments to the authors (80%); additional content tended to be irrelevant to the manuscript. Tone frequently aligned (91%). CONCLUSION: Findings indicate variability in how reviewers use the confidential comments to editor section in online peer-review systems, though generally the way they use them suggests integrity and transparency to authors.
format Online
Article
Text
id pubmed-8629260
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-86292602021-11-30 Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors O’Brien, Bridget C. Artino, Anthony R. Costello, Joseph A. Driessen, Erik Maggio, Lauren A. PLoS One Research Article PURPOSE: Recent calls to improve transparency in peer review have prompted examination of many aspects of the peer-review process. Peer-review systems often allow confidential comments to editors that could reduce transparency to authors, yet this option has escaped scrutiny. Our study explores 1) how reviewers use the confidential comments section and 2) alignment between comments to the editor and comments to authors with respect to content and tone. METHODS: Our dataset included 358 reviews of 168 manuscripts submitted between January 1, 2019 and August 24, 2020 to a health professions education journal with a single blind review process. We first identified reviews containing comments to the editor. Then, for the reviews with comments, we used procedures consistent with conventional and directed qualitative content analysis to develop a coding scheme and code comments for content, tone, and section of the manuscript. For reviews in which the reviewer recommended “reject,” we coded for alignment between reviewers’ comments to the editor and to authors. We report descriptive statistics. RESULTS: 49% of reviews contained comments to the editor (n = 176). Most of these comments summarized the reviewers’ impression of the article (85%), which included explicit reference to their recommended decision (44%) and suitability for the journal (10%). The majority of comments addressed argument quality (56%) or research design/methods/data (51%). The tone of comments tended to be critical (40%) or constructive (34%). For the 86 reviews recommending “reject,” the majority of comments to the editor contained content that also appeared in comments to the authors (80%); additional content tended to be irrelevant to the manuscript. Tone frequently aligned (91%). CONCLUSION: Findings indicate variability in how reviewers use the confidential comments to editor section in online peer-review systems, though generally the way they use them suggests integrity and transparency to authors. Public Library of Science 2021-11-29 /pmc/articles/PMC8629260/ /pubmed/34843564 http://dx.doi.org/10.1371/journal.pone.0260558 Text en https://creativecommons.org/publicdomain/zero/1.0/This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 (https://creativecommons.org/publicdomain/zero/1.0/) public domain dedication.
spellingShingle Research Article
O’Brien, Bridget C.
Artino, Anthony R.
Costello, Joseph A.
Driessen, Erik
Maggio, Lauren A.
Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors
title Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors
title_full Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors
title_fullStr Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors
title_full_unstemmed Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors
title_short Transparency in peer review: Exploring the content and tone of reviewers’ confidential comments to editors
title_sort transparency in peer review: exploring the content and tone of reviewers’ confidential comments to editors
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8629260/
https://www.ncbi.nlm.nih.gov/pubmed/34843564
http://dx.doi.org/10.1371/journal.pone.0260558
work_keys_str_mv AT obrienbridgetc transparencyinpeerreviewexploringthecontentandtoneofreviewersconfidentialcommentstoeditors
AT artinoanthonyr transparencyinpeerreviewexploringthecontentandtoneofreviewersconfidentialcommentstoeditors
AT costellojosepha transparencyinpeerreviewexploringthecontentandtoneofreviewersconfidentialcommentstoeditors
AT driessenerik transparencyinpeerreviewexploringthecontentandtoneofreviewersconfidentialcommentstoeditors
AT maggiolaurena transparencyinpeerreviewexploringthecontentandtoneofreviewersconfidentialcommentstoeditors