Cargando…

Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports

OBJECTIVES: To examine whether the gender of applicants and peer reviewers and other factors influence peer review of grant proposals submitted to a national funding agency. SETTING: Swiss National Science Foundation (SNSF). DESIGN: Cross-sectional analysis of peer review reports submitted from 2009...

Descripción completa

Detalles Bibliográficos
Autores principales: Severin, Anna, Martins, Joao, Heyard, Rachel, Delavy, François, Jorstad, Anne, Egger, Matthias
Formato: Online Artículo Texto
Lenguaje:English
Publicado: BMJ Publishing Group 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7440717/
https://www.ncbi.nlm.nih.gov/pubmed/32819934
http://dx.doi.org/10.1136/bmjopen-2019-035058
_version_ 1783573175551918080
author Severin, Anna
Martins, Joao
Heyard, Rachel
Delavy, François
Jorstad, Anne
Egger, Matthias
author_facet Severin, Anna
Martins, Joao
Heyard, Rachel
Delavy, François
Jorstad, Anne
Egger, Matthias
author_sort Severin, Anna
collection PubMed
description OBJECTIVES: To examine whether the gender of applicants and peer reviewers and other factors influence peer review of grant proposals submitted to a national funding agency. SETTING: Swiss National Science Foundation (SNSF). DESIGN: Cross-sectional analysis of peer review reports submitted from 2009 to 2016 using linear mixed effects regression models adjusted for research topic, applicant’s age, nationality, affiliation and calendar period. PARTICIPANTS: External peer reviewers. PRIMARY OUTCOME MEASURE: Overall score on a scale from 1 (worst) to 6 (best). RESULTS: Analyses included 38 250 reports on 12 294 grant applications from medicine, architecture, biology, chemistry, economics, engineering, geology, history, linguistics, mathematics, physics, psychology and sociology submitted by 26 829 unique peer reviewers. In univariable analysis, male applicants received more favourable evaluation scores than female applicants (+0.18 points; 95% CI 0.14 to 0.23), and male reviewers awarded higher scores than female reviewers (+0.11; 95% CI 0.08 to 0.15). Applicant-nominated reviewers awarded higher scores than reviewers nominated by the SNSF (+0.53; 95% CI 0.50 to 0.56), and reviewers from outside of Switzerland more favourable scores than reviewers affiliated with Swiss institutions (+0.53; 95% CI 0.49 to 0.56). In multivariable analysis, differences between male and female applicants were attenuated (+0.08; 95% CI 0.04 to 0.13) whereas results changed little for source of nomination and affiliation of reviewers. The gender difference increased after September 2011, when new evaluation forms were introduced (p=0.033 from test of interaction). CONCLUSIONS: Peer review of grant applications at SNSF might be prone to biases stemming from different applicant and reviewer characteristics. The SNSF abandoned the nomination of peer reviewers by applicants. The new form introduced in 2011 may inadvertently have given more emphasis to the applicant’s track record. We encourage other funders to conduct similar studies, in order to improve the evidence base for rational and fair research funding.
format Online
Article
Text
id pubmed-7440717
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher BMJ Publishing Group
record_format MEDLINE/PubMed
spelling pubmed-74407172020-08-28 Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports Severin, Anna Martins, Joao Heyard, Rachel Delavy, François Jorstad, Anne Egger, Matthias BMJ Open Epidemiology OBJECTIVES: To examine whether the gender of applicants and peer reviewers and other factors influence peer review of grant proposals submitted to a national funding agency. SETTING: Swiss National Science Foundation (SNSF). DESIGN: Cross-sectional analysis of peer review reports submitted from 2009 to 2016 using linear mixed effects regression models adjusted for research topic, applicant’s age, nationality, affiliation and calendar period. PARTICIPANTS: External peer reviewers. PRIMARY OUTCOME MEASURE: Overall score on a scale from 1 (worst) to 6 (best). RESULTS: Analyses included 38 250 reports on 12 294 grant applications from medicine, architecture, biology, chemistry, economics, engineering, geology, history, linguistics, mathematics, physics, psychology and sociology submitted by 26 829 unique peer reviewers. In univariable analysis, male applicants received more favourable evaluation scores than female applicants (+0.18 points; 95% CI 0.14 to 0.23), and male reviewers awarded higher scores than female reviewers (+0.11; 95% CI 0.08 to 0.15). Applicant-nominated reviewers awarded higher scores than reviewers nominated by the SNSF (+0.53; 95% CI 0.50 to 0.56), and reviewers from outside of Switzerland more favourable scores than reviewers affiliated with Swiss institutions (+0.53; 95% CI 0.49 to 0.56). In multivariable analysis, differences between male and female applicants were attenuated (+0.08; 95% CI 0.04 to 0.13) whereas results changed little for source of nomination and affiliation of reviewers. The gender difference increased after September 2011, when new evaluation forms were introduced (p=0.033 from test of interaction). CONCLUSIONS: Peer review of grant applications at SNSF might be prone to biases stemming from different applicant and reviewer characteristics. The SNSF abandoned the nomination of peer reviewers by applicants. The new form introduced in 2011 may inadvertently have given more emphasis to the applicant’s track record. We encourage other funders to conduct similar studies, in order to improve the evidence base for rational and fair research funding. BMJ Publishing Group 2020-08-20 /pmc/articles/PMC7440717/ /pubmed/32819934 http://dx.doi.org/10.1136/bmjopen-2019-035058 Text en © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY. Published by BMJ. https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
spellingShingle Epidemiology
Severin, Anna
Martins, Joao
Heyard, Rachel
Delavy, François
Jorstad, Anne
Egger, Matthias
Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports
title Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports
title_full Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports
title_fullStr Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports
title_full_unstemmed Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports
title_short Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports
title_sort gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports
topic Epidemiology
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7440717/
https://www.ncbi.nlm.nih.gov/pubmed/32819934
http://dx.doi.org/10.1136/bmjopen-2019-035058
work_keys_str_mv AT severinanna genderandotherpotentialbiasesinpeerreviewcrosssectionalanalysisof38250externalpeerreviewreports
AT martinsjoao genderandotherpotentialbiasesinpeerreviewcrosssectionalanalysisof38250externalpeerreviewreports
AT heyardrachel genderandotherpotentialbiasesinpeerreviewcrosssectionalanalysisof38250externalpeerreviewreports
AT delavyfrancois genderandotherpotentialbiasesinpeerreviewcrosssectionalanalysisof38250externalpeerreviewreports
AT jorstadanne genderandotherpotentialbiasesinpeerreviewcrosssectionalanalysisof38250externalpeerreviewreports
AT eggermatthias genderandotherpotentialbiasesinpeerreviewcrosssectionalanalysisof38250externalpeerreviewreports