Cargando…
A scoping review of simulation models of peer review
Peer review is a process used in the selection of manuscripts for journal publication and proposals for research grant funding. Though widely used, peer review is not without flaws and critics. Performing large-scale experiments to evaluate and test correctives and alternatives is difficult, if not...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Springer International Publishing
2019
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6744516/ https://www.ncbi.nlm.nih.gov/pubmed/31564758 http://dx.doi.org/10.1007/s11192-019-03205-w |
_version_ | 1783451388253044736 |
---|---|
author | Feliciani, Thomas Luo, Junwen Ma, Lai Lucas, Pablo Squazzoni, Flaminio Marušić, Ana Shankar, Kalpana |
author_facet | Feliciani, Thomas Luo, Junwen Ma, Lai Lucas, Pablo Squazzoni, Flaminio Marušić, Ana Shankar, Kalpana |
author_sort | Feliciani, Thomas |
collection | PubMed |
description | Peer review is a process used in the selection of manuscripts for journal publication and proposals for research grant funding. Though widely used, peer review is not without flaws and critics. Performing large-scale experiments to evaluate and test correctives and alternatives is difficult, if not impossible. Thus, many researchers have turned to simulation studies to overcome these difficulties. In the last 10 years this field of research has grown significantly but with only limited attempts to integrate disparate models or build on previous work. Thus, the resulting body of literature consists of a large variety of models, hinging on incompatible assumptions, which have not been compared, and whose predictions have rarely been empirically tested. This scoping review is an attempt to understand the current state of simulation studies of peer review. Based on 46 articles identified through literature searching, we develop a proposed taxonomy of model features that include model type (e.g. formal models vs. ABMs or other) and the type of modeled peer review system (e.g. peer review in grants vs. in journals or other). We classify the models by their features (including some core assumptions) to help distinguish between the modeling approaches. Finally, we summarize the models’ findings around six general themes: decision-making, matching submissions/reviewers, editorial strategies; reviewer behaviors, comparisons of alternative peer review systems, and the identification and addressing of biases. We conclude with some open challenges and promising avenues for future modeling work. |
format | Online Article Text |
id | pubmed-6744516 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2019 |
publisher | Springer International Publishing |
record_format | MEDLINE/PubMed |
spelling | pubmed-67445162019-09-27 A scoping review of simulation models of peer review Feliciani, Thomas Luo, Junwen Ma, Lai Lucas, Pablo Squazzoni, Flaminio Marušić, Ana Shankar, Kalpana Scientometrics Article Peer review is a process used in the selection of manuscripts for journal publication and proposals for research grant funding. Though widely used, peer review is not without flaws and critics. Performing large-scale experiments to evaluate and test correctives and alternatives is difficult, if not impossible. Thus, many researchers have turned to simulation studies to overcome these difficulties. In the last 10 years this field of research has grown significantly but with only limited attempts to integrate disparate models or build on previous work. Thus, the resulting body of literature consists of a large variety of models, hinging on incompatible assumptions, which have not been compared, and whose predictions have rarely been empirically tested. This scoping review is an attempt to understand the current state of simulation studies of peer review. Based on 46 articles identified through literature searching, we develop a proposed taxonomy of model features that include model type (e.g. formal models vs. ABMs or other) and the type of modeled peer review system (e.g. peer review in grants vs. in journals or other). We classify the models by their features (including some core assumptions) to help distinguish between the modeling approaches. Finally, we summarize the models’ findings around six general themes: decision-making, matching submissions/reviewers, editorial strategies; reviewer behaviors, comparisons of alternative peer review systems, and the identification and addressing of biases. We conclude with some open challenges and promising avenues for future modeling work. Springer International Publishing 2019-08-19 2019 /pmc/articles/PMC6744516/ /pubmed/31564758 http://dx.doi.org/10.1007/s11192-019-03205-w Text en © The Author(s) 2019 Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. |
spellingShingle | Article Feliciani, Thomas Luo, Junwen Ma, Lai Lucas, Pablo Squazzoni, Flaminio Marušić, Ana Shankar, Kalpana A scoping review of simulation models of peer review |
title | A scoping review of simulation models of peer review |
title_full | A scoping review of simulation models of peer review |
title_fullStr | A scoping review of simulation models of peer review |
title_full_unstemmed | A scoping review of simulation models of peer review |
title_short | A scoping review of simulation models of peer review |
title_sort | scoping review of simulation models of peer review |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6744516/ https://www.ncbi.nlm.nih.gov/pubmed/31564758 http://dx.doi.org/10.1007/s11192-019-03205-w |
work_keys_str_mv | AT felicianithomas ascopingreviewofsimulationmodelsofpeerreview AT luojunwen ascopingreviewofsimulationmodelsofpeerreview AT malai ascopingreviewofsimulationmodelsofpeerreview AT lucaspablo ascopingreviewofsimulationmodelsofpeerreview AT squazzoniflaminio ascopingreviewofsimulationmodelsofpeerreview AT marusicana ascopingreviewofsimulationmodelsofpeerreview AT shankarkalpana ascopingreviewofsimulationmodelsofpeerreview AT felicianithomas scopingreviewofsimulationmodelsofpeerreview AT luojunwen scopingreviewofsimulationmodelsofpeerreview AT malai scopingreviewofsimulationmodelsofpeerreview AT lucaspablo scopingreviewofsimulationmodelsofpeerreview AT squazzoniflaminio scopingreviewofsimulationmodelsofpeerreview AT marusicana scopingreviewofsimulationmodelsofpeerreview AT shankarkalpana scopingreviewofsimulationmodelsofpeerreview |