Cargando…

Is novel research worth doing? Evidence from peer review at 49 journals

There are long-standing concerns that peer review, which is foundational to scientific institutions like journals and funding agencies, favors conservative ideas over novel ones. We investigate the association between novelty and the acceptance of manuscripts submitted to a large sample of scientifi...

Descripción completa

Detalles Bibliográficos
Autores principales: Teplitskiy, Misha, Peng, Hao, Blasco, Andrea, Lakhani, Karim R.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: National Academy of Sciences 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9704701/
https://www.ncbi.nlm.nih.gov/pubmed/36395142
http://dx.doi.org/10.1073/pnas.2118046119
_version_ 1784840111590998016
author Teplitskiy, Misha
Peng, Hao
Blasco, Andrea
Lakhani, Karim R.
author_facet Teplitskiy, Misha
Peng, Hao
Blasco, Andrea
Lakhani, Karim R.
author_sort Teplitskiy, Misha
collection PubMed
description There are long-standing concerns that peer review, which is foundational to scientific institutions like journals and funding agencies, favors conservative ideas over novel ones. We investigate the association between novelty and the acceptance of manuscripts submitted to a large sample of scientific journals. The data cover 20,538 manuscripts submitted between 2013 and 2018 to the journals Cell and Cell Reports and 6,785 manuscripts submitted in 2018 to 47 journals published by the Institute of Physics Publishing. Following previous work that found that a balance of novel and conventional ideas predicts citation impact, we measure the novelty and conventionality of manuscripts by the atypicality of combinations of journals in their reference lists, taking the 90th percentile most atypical combination as “novelty” and the 50th percentile as “conventionality.” We find that higher novelty is consistently associated with higher acceptance; submissions in the top novelty quintile are 6.5 percentage points more likely than bottom quintile ones to get accepted. Higher conventionality is also associated with acceptance (+16.3% top–bottom quintile difference). Disagreement among peer reviewers was not systematically related to submission novelty or conventionality, and editors select strongly for novelty even conditional on reviewers' recommendations (+7.0% top–bottom quintile difference). Manuscripts exhibiting higher novelty were more highly cited. Overall, the findings suggest that journal peer review favors novel research that is well situated in the existing literature, incentivizing exploration in science and challenging the view that peer review is inherently antinovelty.
format Online
Article
Text
id pubmed-9704701
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher National Academy of Sciences
record_format MEDLINE/PubMed
spelling pubmed-97047012022-11-29 Is novel research worth doing? Evidence from peer review at 49 journals Teplitskiy, Misha Peng, Hao Blasco, Andrea Lakhani, Karim R. Proc Natl Acad Sci U S A Social Sciences There are long-standing concerns that peer review, which is foundational to scientific institutions like journals and funding agencies, favors conservative ideas over novel ones. We investigate the association between novelty and the acceptance of manuscripts submitted to a large sample of scientific journals. The data cover 20,538 manuscripts submitted between 2013 and 2018 to the journals Cell and Cell Reports and 6,785 manuscripts submitted in 2018 to 47 journals published by the Institute of Physics Publishing. Following previous work that found that a balance of novel and conventional ideas predicts citation impact, we measure the novelty and conventionality of manuscripts by the atypicality of combinations of journals in their reference lists, taking the 90th percentile most atypical combination as “novelty” and the 50th percentile as “conventionality.” We find that higher novelty is consistently associated with higher acceptance; submissions in the top novelty quintile are 6.5 percentage points more likely than bottom quintile ones to get accepted. Higher conventionality is also associated with acceptance (+16.3% top–bottom quintile difference). Disagreement among peer reviewers was not systematically related to submission novelty or conventionality, and editors select strongly for novelty even conditional on reviewers' recommendations (+7.0% top–bottom quintile difference). Manuscripts exhibiting higher novelty were more highly cited. Overall, the findings suggest that journal peer review favors novel research that is well situated in the existing literature, incentivizing exploration in science and challenging the view that peer review is inherently antinovelty. National Academy of Sciences 2022-11-17 2022-11-22 /pmc/articles/PMC9704701/ /pubmed/36395142 http://dx.doi.org/10.1073/pnas.2118046119 Text en Copyright © 2022 the Author(s). Published by PNAS. https://creativecommons.org/licenses/by/4.0/This open access article is distributed under Creative Commons Attribution License 4.0 (CC BY) (https://creativecommons.org/licenses/by/4.0/) .
spellingShingle Social Sciences
Teplitskiy, Misha
Peng, Hao
Blasco, Andrea
Lakhani, Karim R.
Is novel research worth doing? Evidence from peer review at 49 journals
title Is novel research worth doing? Evidence from peer review at 49 journals
title_full Is novel research worth doing? Evidence from peer review at 49 journals
title_fullStr Is novel research worth doing? Evidence from peer review at 49 journals
title_full_unstemmed Is novel research worth doing? Evidence from peer review at 49 journals
title_short Is novel research worth doing? Evidence from peer review at 49 journals
title_sort is novel research worth doing? evidence from peer review at 49 journals
topic Social Sciences
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9704701/
https://www.ncbi.nlm.nih.gov/pubmed/36395142
http://dx.doi.org/10.1073/pnas.2118046119
work_keys_str_mv AT teplitskiymisha isnovelresearchworthdoingevidencefrompeerreviewat49journals
AT penghao isnovelresearchworthdoingevidencefrompeerreviewat49journals
AT blascoandrea isnovelresearchworthdoingevidencefrompeerreviewat49journals
AT lakhanikarimr isnovelresearchworthdoingevidencefrompeerreviewat49journals