Cargando…

Experiment aversion does not appear to generalize

Over the past decade, governments and organizations around the world have established behavioral insights teams advocating for randomized experiments. However, recent findings by M. N. Meyer et al., Proc. Natl. Acad. Sci. U.S.A. 116, 10723–10728 (2019) and P. R. Heck, C. F. Chabris, D. J. Watts, M....

Descripción completa

Detalles Bibliográficos
Autores principales: Mazar, Nina, Elbaek, Christian T., Mitkidis, Panagiotis
Formato: Online Artículo Texto
Lenguaje:English
Publicado: National Academy of Sciences 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10120089/
https://www.ncbi.nlm.nih.gov/pubmed/37036965
http://dx.doi.org/10.1073/pnas.2217551120
_version_ 1785029124559994880
author Mazar, Nina
Elbaek, Christian T.
Mitkidis, Panagiotis
author_facet Mazar, Nina
Elbaek, Christian T.
Mitkidis, Panagiotis
author_sort Mazar, Nina
collection PubMed
description Over the past decade, governments and organizations around the world have established behavioral insights teams advocating for randomized experiments. However, recent findings by M. N. Meyer et al., Proc. Natl. Acad. Sci. U.S.A. 116, 10723–10728 (2019) and P. R. Heck, C. F. Chabris, D. J. Watts, M. N. Meyer, Proc. Natl. Acad. Sci. U.S.A. 117, 18948–18950 (2020) suggest that people often rate randomized experiments as less appropriate than the policies they contain even when approving the implementation of either policy untested and when none of the individual policies is clearly superior. The authors warn that this could cause policymakers to avoid running large-scale field experiments or being transparent about running them and might contribute to an adverse heterogeneity bias in terms of who is participating in experiments. In one direct and six conceptual preregistered replications (total N = 5,200) of the previously published larger-effect studies, using the same main dependent variable but with variations in scenario wordings, recruitment platforms, and countries, and the addition of further measures to assess people’s views, we test the generalizability and robustness of these findings. Together, we find that the original results do not appear to generalize. That is, our triangulation reveals insufficient evidence to conclude that people exhibit a common pattern of behavior that would be consistent with relative experiment aversion, thereby supporting recent findings by R. Mislavsky, B. Dietvorst, U. Simonsohn, Mark. Sci. 39, 1092–1104 (2020). Thus, policymakers may not need to be concerned about employing evidence-based practices more so than about universally implementing policies.
format Online
Article
Text
id pubmed-10120089
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher National Academy of Sciences
record_format MEDLINE/PubMed
spelling pubmed-101200892023-10-10 Experiment aversion does not appear to generalize Mazar, Nina Elbaek, Christian T. Mitkidis, Panagiotis Proc Natl Acad Sci U S A Social Sciences Over the past decade, governments and organizations around the world have established behavioral insights teams advocating for randomized experiments. However, recent findings by M. N. Meyer et al., Proc. Natl. Acad. Sci. U.S.A. 116, 10723–10728 (2019) and P. R. Heck, C. F. Chabris, D. J. Watts, M. N. Meyer, Proc. Natl. Acad. Sci. U.S.A. 117, 18948–18950 (2020) suggest that people often rate randomized experiments as less appropriate than the policies they contain even when approving the implementation of either policy untested and when none of the individual policies is clearly superior. The authors warn that this could cause policymakers to avoid running large-scale field experiments or being transparent about running them and might contribute to an adverse heterogeneity bias in terms of who is participating in experiments. In one direct and six conceptual preregistered replications (total N = 5,200) of the previously published larger-effect studies, using the same main dependent variable but with variations in scenario wordings, recruitment platforms, and countries, and the addition of further measures to assess people’s views, we test the generalizability and robustness of these findings. Together, we find that the original results do not appear to generalize. That is, our triangulation reveals insufficient evidence to conclude that people exhibit a common pattern of behavior that would be consistent with relative experiment aversion, thereby supporting recent findings by R. Mislavsky, B. Dietvorst, U. Simonsohn, Mark. Sci. 39, 1092–1104 (2020). Thus, policymakers may not need to be concerned about employing evidence-based practices more so than about universally implementing policies. National Academy of Sciences 2023-04-10 2023-04-18 /pmc/articles/PMC10120089/ /pubmed/37036965 http://dx.doi.org/10.1073/pnas.2217551120 Text en Copyright © 2023 the Author(s). Published by PNAS. https://creativecommons.org/licenses/by-nc-nd/4.0/This article is distributed under Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND) (https://creativecommons.org/licenses/by-nc-nd/4.0/) .
spellingShingle Social Sciences
Mazar, Nina
Elbaek, Christian T.
Mitkidis, Panagiotis
Experiment aversion does not appear to generalize
title Experiment aversion does not appear to generalize
title_full Experiment aversion does not appear to generalize
title_fullStr Experiment aversion does not appear to generalize
title_full_unstemmed Experiment aversion does not appear to generalize
title_short Experiment aversion does not appear to generalize
title_sort experiment aversion does not appear to generalize
topic Social Sciences
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10120089/
https://www.ncbi.nlm.nih.gov/pubmed/37036965
http://dx.doi.org/10.1073/pnas.2217551120
work_keys_str_mv AT mazarnina experimentaversiondoesnotappeartogeneralize
AT elbaekchristiant experimentaversiondoesnotappeartogeneralize
AT mitkidispanagiotis experimentaversiondoesnotappeartogeneralize