Cargando…

Conservatism Gets Funded? A Field Experiment on the Role of Negative Information in Novel Project Evaluation

The evaluation and selection of novel projects lies at the heart of scientific and technological innovation, and yet there are persistent concerns about bias, such as conservatism. This paper investigates the role that the format of evaluation, specifically information sharing among expert evaluator...

Descripción completa

Detalles Bibliográficos
Autores principales: Lanei, Jacqueline N., Teplitskiy, Misha, Gray, Gary, Ranu, Hardeep, Menietti, Michael, Guinan, Eva, Lakhani, Karim R.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9531843/
https://www.ncbi.nlm.nih.gov/pubmed/36200060
http://dx.doi.org/10.1287/mnsc.2021.4107
_version_ 1784801988477714432
author Lanei, Jacqueline N.
Teplitskiy, Misha
Gray, Gary
Ranu, Hardeep
Menietti, Michael
Guinan, Eva
Lakhani, Karim R.
author_facet Lanei, Jacqueline N.
Teplitskiy, Misha
Gray, Gary
Ranu, Hardeep
Menietti, Michael
Guinan, Eva
Lakhani, Karim R.
author_sort Lanei, Jacqueline N.
collection PubMed
description The evaluation and selection of novel projects lies at the heart of scientific and technological innovation, and yet there are persistent concerns about bias, such as conservatism. This paper investigates the role that the format of evaluation, specifically information sharing among expert evaluators, plays in generating conservative decisions. We executed two field experiments in two separate grant-funding opportunities at a leading research university, mobilizing 369 evaluators from seven universities to evaluate 97 projects, resulting in 761 proposal-evaluation pairs and more than $250,000 in awards. We exogenously varied the relative valence (positive and negative) of others’ scores and measured how exposures to higher and lower scores affect the focal evaluator’s propensity to change their initial score. We found causal evidence of a negativity bias, where evaluators lower their scores by more points after seeing scores more critical than their own rather than raise them after seeing more favorable scores. Qualitative coding of the evaluators’ justifications for score changes reveals that exposures to lower scores were associated with greater attention to uncovering weaknesses, whereas exposures to neutral or higher scores were associated with increased emphasis on nonevaluation criteria, such as confidence in one’s judgment. The greater power of negative information suggests that information sharing among expert evaluators can lead to more conservative allocation decisions that favor protecting against failure rather than maximizing success.
format Online
Article
Text
id pubmed-9531843
institution National Center for Biotechnology Information
language English
publishDate 2022
record_format MEDLINE/PubMed
spelling pubmed-95318432022-10-04 Conservatism Gets Funded? A Field Experiment on the Role of Negative Information in Novel Project Evaluation Lanei, Jacqueline N. Teplitskiy, Misha Gray, Gary Ranu, Hardeep Menietti, Michael Guinan, Eva Lakhani, Karim R. Manage Sci Article The evaluation and selection of novel projects lies at the heart of scientific and technological innovation, and yet there are persistent concerns about bias, such as conservatism. This paper investigates the role that the format of evaluation, specifically information sharing among expert evaluators, plays in generating conservative decisions. We executed two field experiments in two separate grant-funding opportunities at a leading research university, mobilizing 369 evaluators from seven universities to evaluate 97 projects, resulting in 761 proposal-evaluation pairs and more than $250,000 in awards. We exogenously varied the relative valence (positive and negative) of others’ scores and measured how exposures to higher and lower scores affect the focal evaluator’s propensity to change their initial score. We found causal evidence of a negativity bias, where evaluators lower their scores by more points after seeing scores more critical than their own rather than raise them after seeing more favorable scores. Qualitative coding of the evaluators’ justifications for score changes reveals that exposures to lower scores were associated with greater attention to uncovering weaknesses, whereas exposures to neutral or higher scores were associated with increased emphasis on nonevaluation criteria, such as confidence in one’s judgment. The greater power of negative information suggests that information sharing among expert evaluators can lead to more conservative allocation decisions that favor protecting against failure rather than maximizing success. 2022-06 2021-10-28 /pmc/articles/PMC9531843/ /pubmed/36200060 http://dx.doi.org/10.1287/mnsc.2021.4107 Text en https://creativecommons.org/licenses/by/4.0/Open Access Statement: This work is licensed under a Creative Commons Attribution 4.0 International License. You are free to copy, distribute, transmit and adapt this work, but you must attribute this work as "Management Science. Copyright © 2021 The Author(s). https://doi.org/10.1287/mnsc.2021.4107, used under a Creative Commons Attribution License: https://creativecommons.org/licenses/by/4.0/."
spellingShingle Article
Lanei, Jacqueline N.
Teplitskiy, Misha
Gray, Gary
Ranu, Hardeep
Menietti, Michael
Guinan, Eva
Lakhani, Karim R.
Conservatism Gets Funded? A Field Experiment on the Role of Negative Information in Novel Project Evaluation
title Conservatism Gets Funded? A Field Experiment on the Role of Negative Information in Novel Project Evaluation
title_full Conservatism Gets Funded? A Field Experiment on the Role of Negative Information in Novel Project Evaluation
title_fullStr Conservatism Gets Funded? A Field Experiment on the Role of Negative Information in Novel Project Evaluation
title_full_unstemmed Conservatism Gets Funded? A Field Experiment on the Role of Negative Information in Novel Project Evaluation
title_short Conservatism Gets Funded? A Field Experiment on the Role of Negative Information in Novel Project Evaluation
title_sort conservatism gets funded? a field experiment on the role of negative information in novel project evaluation
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9531843/
https://www.ncbi.nlm.nih.gov/pubmed/36200060
http://dx.doi.org/10.1287/mnsc.2021.4107
work_keys_str_mv AT laneijacquelinen conservatismgetsfundedafieldexperimentontheroleofnegativeinformationinnovelprojectevaluation
AT teplitskiymisha conservatismgetsfundedafieldexperimentontheroleofnegativeinformationinnovelprojectevaluation
AT graygary conservatismgetsfundedafieldexperimentontheroleofnegativeinformationinnovelprojectevaluation
AT ranuhardeep conservatismgetsfundedafieldexperimentontheroleofnegativeinformationinnovelprojectevaluation
AT meniettimichael conservatismgetsfundedafieldexperimentontheroleofnegativeinformationinnovelprojectevaluation
AT guinaneva conservatismgetsfundedafieldexperimentontheroleofnegativeinformationinnovelprojectevaluation
AT lakhanikarimr conservatismgetsfundedafieldexperimentontheroleofnegativeinformationinnovelprojectevaluation