Cargando…

Tools used to assess the quality of peer review reports: a methodological systematic review

BACKGROUND: A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing th...

Descripción completa

Detalles Bibliográficos
Autores principales: Superchi, Cecilia, González, José Antonio, Solà, Ivan, Cobo, Erik, Hren, Darko, Boutron, Isabelle
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BioMed Central 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6402095/
https://www.ncbi.nlm.nih.gov/pubmed/30841850
http://dx.doi.org/10.1186/s12874-019-0688-x
_version_ 1783400316392177664
author Superchi, Cecilia
González, José Antonio
Solà, Ivan
Cobo, Erik
Hren, Darko
Boutron, Isabelle
author_facet Superchi, Cecilia
González, José Antonio
Solà, Ivan
Cobo, Erik
Hren, Darko
Boutron, Isabelle
author_sort Superchi, Cecilia
collection PubMed
description BACKGROUND: A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research. METHODS: We conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis. RESULTS: We identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of ‘quality’. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer’s comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18). CONCLUSION: Several tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12874-019-0688-x) contains supplementary material, which is available to authorized users.
format Online
Article
Text
id pubmed-6402095
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher BioMed Central
record_format MEDLINE/PubMed
spelling pubmed-64020952019-03-14 Tools used to assess the quality of peer review reports: a methodological systematic review Superchi, Cecilia González, José Antonio Solà, Ivan Cobo, Erik Hren, Darko Boutron, Isabelle BMC Med Res Methodol Research Article BACKGROUND: A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research. METHODS: We conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis. RESULTS: We identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of ‘quality’. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer’s comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18). CONCLUSION: Several tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12874-019-0688-x) contains supplementary material, which is available to authorized users. BioMed Central 2019-03-06 /pmc/articles/PMC6402095/ /pubmed/30841850 http://dx.doi.org/10.1186/s12874-019-0688-x Text en © The Author(s). 2019 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
spellingShingle Research Article
Superchi, Cecilia
González, José Antonio
Solà, Ivan
Cobo, Erik
Hren, Darko
Boutron, Isabelle
Tools used to assess the quality of peer review reports: a methodological systematic review
title Tools used to assess the quality of peer review reports: a methodological systematic review
title_full Tools used to assess the quality of peer review reports: a methodological systematic review
title_fullStr Tools used to assess the quality of peer review reports: a methodological systematic review
title_full_unstemmed Tools used to assess the quality of peer review reports: a methodological systematic review
title_short Tools used to assess the quality of peer review reports: a methodological systematic review
title_sort tools used to assess the quality of peer review reports: a methodological systematic review
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6402095/
https://www.ncbi.nlm.nih.gov/pubmed/30841850
http://dx.doi.org/10.1186/s12874-019-0688-x
work_keys_str_mv AT superchicecilia toolsusedtoassessthequalityofpeerreviewreportsamethodologicalsystematicreview
AT gonzalezjoseantonio toolsusedtoassessthequalityofpeerreviewreportsamethodologicalsystematicreview
AT solaivan toolsusedtoassessthequalityofpeerreviewreportsamethodologicalsystematicreview
AT coboerik toolsusedtoassessthequalityofpeerreviewreportsamethodologicalsystematicreview
AT hrendarko toolsusedtoassessthequalityofpeerreviewreportsamethodologicalsystematicreview
AT boutronisabelle toolsusedtoassessthequalityofpeerreviewreportsamethodologicalsystematicreview