Cargando…
Tools used to assess the quality of peer review reports: a methodological systematic review
BACKGROUND: A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing th...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
BioMed Central
2019
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6402095/ https://www.ncbi.nlm.nih.gov/pubmed/30841850 http://dx.doi.org/10.1186/s12874-019-0688-x |
_version_ | 1783400316392177664 |
---|---|
author | Superchi, Cecilia González, José Antonio Solà, Ivan Cobo, Erik Hren, Darko Boutron, Isabelle |
author_facet | Superchi, Cecilia González, José Antonio Solà, Ivan Cobo, Erik Hren, Darko Boutron, Isabelle |
author_sort | Superchi, Cecilia |
collection | PubMed |
description | BACKGROUND: A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research. METHODS: We conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis. RESULTS: We identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of ‘quality’. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer’s comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18). CONCLUSION: Several tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12874-019-0688-x) contains supplementary material, which is available to authorized users. |
format | Online Article Text |
id | pubmed-6402095 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2019 |
publisher | BioMed Central |
record_format | MEDLINE/PubMed |
spelling | pubmed-64020952019-03-14 Tools used to assess the quality of peer review reports: a methodological systematic review Superchi, Cecilia González, José Antonio Solà, Ivan Cobo, Erik Hren, Darko Boutron, Isabelle BMC Med Res Methodol Research Article BACKGROUND: A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research. METHODS: We conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis. RESULTS: We identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of ‘quality’. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer’s comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18). CONCLUSION: Several tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (10.1186/s12874-019-0688-x) contains supplementary material, which is available to authorized users. BioMed Central 2019-03-06 /pmc/articles/PMC6402095/ /pubmed/30841850 http://dx.doi.org/10.1186/s12874-019-0688-x Text en © The Author(s). 2019 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. |
spellingShingle | Research Article Superchi, Cecilia González, José Antonio Solà, Ivan Cobo, Erik Hren, Darko Boutron, Isabelle Tools used to assess the quality of peer review reports: a methodological systematic review |
title | Tools used to assess the quality of peer review reports: a methodological systematic review |
title_full | Tools used to assess the quality of peer review reports: a methodological systematic review |
title_fullStr | Tools used to assess the quality of peer review reports: a methodological systematic review |
title_full_unstemmed | Tools used to assess the quality of peer review reports: a methodological systematic review |
title_short | Tools used to assess the quality of peer review reports: a methodological systematic review |
title_sort | tools used to assess the quality of peer review reports: a methodological systematic review |
topic | Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6402095/ https://www.ncbi.nlm.nih.gov/pubmed/30841850 http://dx.doi.org/10.1186/s12874-019-0688-x |
work_keys_str_mv | AT superchicecilia toolsusedtoassessthequalityofpeerreviewreportsamethodologicalsystematicreview AT gonzalezjoseantonio toolsusedtoassessthequalityofpeerreviewreportsamethodologicalsystematicreview AT solaivan toolsusedtoassessthequalityofpeerreviewreportsamethodologicalsystematicreview AT coboerik toolsusedtoassessthequalityofpeerreviewreportsamethodologicalsystematicreview AT hrendarko toolsusedtoassessthequalityofpeerreviewreportsamethodologicalsystematicreview AT boutronisabelle toolsusedtoassessthequalityofpeerreviewreportsamethodologicalsystematicreview |