Cargando…

A scoping review of simulation models of peer review

Peer review is a process used in the selection of manuscripts for journal publication and proposals for research grant funding. Though widely used, peer review is not without flaws and critics. Performing large-scale experiments to evaluate and test correctives and alternatives is difficult, if not...

Descripción completa

Detalles Bibliográficos
Autores principales: Feliciani, Thomas, Luo, Junwen, Ma, Lai, Lucas, Pablo, Squazzoni, Flaminio, Marušić, Ana, Shankar, Kalpana
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Springer International Publishing 2019
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6744516/
https://www.ncbi.nlm.nih.gov/pubmed/31564758
http://dx.doi.org/10.1007/s11192-019-03205-w
_version_ 1783451388253044736
author Feliciani, Thomas
Luo, Junwen
Ma, Lai
Lucas, Pablo
Squazzoni, Flaminio
Marušić, Ana
Shankar, Kalpana
author_facet Feliciani, Thomas
Luo, Junwen
Ma, Lai
Lucas, Pablo
Squazzoni, Flaminio
Marušić, Ana
Shankar, Kalpana
author_sort Feliciani, Thomas
collection PubMed
description Peer review is a process used in the selection of manuscripts for journal publication and proposals for research grant funding. Though widely used, peer review is not without flaws and critics. Performing large-scale experiments to evaluate and test correctives and alternatives is difficult, if not impossible. Thus, many researchers have turned to simulation studies to overcome these difficulties. In the last 10 years this field of research has grown significantly but with only limited attempts to integrate disparate models or build on previous work. Thus, the resulting body of literature consists of a large variety of models, hinging on incompatible assumptions, which have not been compared, and whose predictions have rarely been empirically tested. This scoping review is an attempt to understand the current state of simulation studies of peer review. Based on 46 articles identified through literature searching, we develop a proposed taxonomy of model features that include model type (e.g. formal models vs. ABMs or other) and the type of modeled peer review system (e.g. peer review in grants vs. in journals or other). We classify the models by their features (including some core assumptions) to help distinguish between the modeling approaches. Finally, we summarize the models’ findings around six general themes: decision-making, matching submissions/reviewers, editorial strategies; reviewer behaviors, comparisons of alternative peer review systems, and the identification and addressing of biases. We conclude with some open challenges and promising avenues for future modeling work.
format Online
Article
Text
id pubmed-6744516
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher Springer International Publishing
record_format MEDLINE/PubMed
spelling pubmed-67445162019-09-27 A scoping review of simulation models of peer review Feliciani, Thomas Luo, Junwen Ma, Lai Lucas, Pablo Squazzoni, Flaminio Marušić, Ana Shankar, Kalpana Scientometrics Article Peer review is a process used in the selection of manuscripts for journal publication and proposals for research grant funding. Though widely used, peer review is not without flaws and critics. Performing large-scale experiments to evaluate and test correctives and alternatives is difficult, if not impossible. Thus, many researchers have turned to simulation studies to overcome these difficulties. In the last 10 years this field of research has grown significantly but with only limited attempts to integrate disparate models or build on previous work. Thus, the resulting body of literature consists of a large variety of models, hinging on incompatible assumptions, which have not been compared, and whose predictions have rarely been empirically tested. This scoping review is an attempt to understand the current state of simulation studies of peer review. Based on 46 articles identified through literature searching, we develop a proposed taxonomy of model features that include model type (e.g. formal models vs. ABMs or other) and the type of modeled peer review system (e.g. peer review in grants vs. in journals or other). We classify the models by their features (including some core assumptions) to help distinguish between the modeling approaches. Finally, we summarize the models’ findings around six general themes: decision-making, matching submissions/reviewers, editorial strategies; reviewer behaviors, comparisons of alternative peer review systems, and the identification and addressing of biases. We conclude with some open challenges and promising avenues for future modeling work. Springer International Publishing 2019-08-19 2019 /pmc/articles/PMC6744516/ /pubmed/31564758 http://dx.doi.org/10.1007/s11192-019-03205-w Text en © The Author(s) 2019 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
spellingShingle Article
Feliciani, Thomas
Luo, Junwen
Ma, Lai
Lucas, Pablo
Squazzoni, Flaminio
Marušić, Ana
Shankar, Kalpana
A scoping review of simulation models of peer review
title A scoping review of simulation models of peer review
title_full A scoping review of simulation models of peer review
title_fullStr A scoping review of simulation models of peer review
title_full_unstemmed A scoping review of simulation models of peer review
title_short A scoping review of simulation models of peer review
title_sort scoping review of simulation models of peer review
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6744516/
https://www.ncbi.nlm.nih.gov/pubmed/31564758
http://dx.doi.org/10.1007/s11192-019-03205-w
work_keys_str_mv AT felicianithomas ascopingreviewofsimulationmodelsofpeerreview
AT luojunwen ascopingreviewofsimulationmodelsofpeerreview
AT malai ascopingreviewofsimulationmodelsofpeerreview
AT lucaspablo ascopingreviewofsimulationmodelsofpeerreview
AT squazzoniflaminio ascopingreviewofsimulationmodelsofpeerreview
AT marusicana ascopingreviewofsimulationmodelsofpeerreview
AT shankarkalpana ascopingreviewofsimulationmodelsofpeerreview
AT felicianithomas scopingreviewofsimulationmodelsofpeerreview
AT luojunwen scopingreviewofsimulationmodelsofpeerreview
AT malai scopingreviewofsimulationmodelsofpeerreview
AT lucaspablo scopingreviewofsimulationmodelsofpeerreview
AT squazzoniflaminio scopingreviewofsimulationmodelsofpeerreview
AT marusicana scopingreviewofsimulationmodelsofpeerreview
AT shankarkalpana scopingreviewofsimulationmodelsofpeerreview