Cargando…

Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a...

Descripción completa

Detalles Bibliográficos
Autores principales: Christie, Alec P., Abecasis, David, Adjeroud, Mehdi, Alonso, Juan C., Amano, Tatsuya, Anton, Alvaro, Baldigo, Barry P., Barrientos, Rafael, Bicknell, Jake E., Buhl, Deborah A., Cebrian, Just, Ceia, Ricardo S., Cibils-Martina, Luciana, Clarke, Sarah, Claudet, Joachim, Craig, Michael D., Davoult, Dominique, De Backer, Annelies, Donovan, Mary K., Eddy, Tyler D., França, Filipe M., Gardner, Jonathan P. A., Harris, Bradley P., Huusko, Ari, Jones, Ian L., Kelaher, Brendan P., Kotiaho, Janne S., López-Baucells, Adrià, Major, Heather L., Mäki-Petäys, Aki, Martín, Beatriz, Martín, Carlos A., Martin, Philip A., Mateos-Molina, Daniel, McConnaughey, Robert A., Meroni, Michele, Meyer, Christoph F. J., Mills, Kade, Montefalcone, Monica, Noreika, Norbertas, Palacín, Carlos, Pande, Anjali, Pitcher, C. Roland, Ponce, Carlos, Rinella, Matt, Rocha, Ricardo, Ruiz-Delgado, María C., Schmitter-Soto, Juan J., Shaffer, Jill A., Sharma, Shailesh, Sher, Anna A., Stagnol, Doriane, Stanley, Thomas R., Stokesbury, Kevin D. E., Torres, Aurora, Tully, Oliver, Vehanen, Teppo, Watts, Corinne, Zhao, Qingyuan, Sutherland, William J.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7733498/
https://www.ncbi.nlm.nih.gov/pubmed/33311448
http://dx.doi.org/10.1038/s41467-020-20142-y
_version_ 1783622285416988672
author Christie, Alec P.
Abecasis, David
Adjeroud, Mehdi
Alonso, Juan C.
Amano, Tatsuya
Anton, Alvaro
Baldigo, Barry P.
Barrientos, Rafael
Bicknell, Jake E.
Buhl, Deborah A.
Cebrian, Just
Ceia, Ricardo S.
Cibils-Martina, Luciana
Clarke, Sarah
Claudet, Joachim
Craig, Michael D.
Davoult, Dominique
De Backer, Annelies
Donovan, Mary K.
Eddy, Tyler D.
França, Filipe M.
Gardner, Jonathan P. A.
Harris, Bradley P.
Huusko, Ari
Jones, Ian L.
Kelaher, Brendan P.
Kotiaho, Janne S.
López-Baucells, Adrià
Major, Heather L.
Mäki-Petäys, Aki
Martín, Beatriz
Martín, Carlos A.
Martin, Philip A.
Mateos-Molina, Daniel
McConnaughey, Robert A.
Meroni, Michele
Meyer, Christoph F. J.
Mills, Kade
Montefalcone, Monica
Noreika, Norbertas
Palacín, Carlos
Pande, Anjali
Pitcher, C. Roland
Ponce, Carlos
Rinella, Matt
Rocha, Ricardo
Ruiz-Delgado, María C.
Schmitter-Soto, Juan J.
Shaffer, Jill A.
Sharma, Shailesh
Sher, Anna A.
Stagnol, Doriane
Stanley, Thomas R.
Stokesbury, Kevin D. E.
Torres, Aurora
Tully, Oliver
Vehanen, Teppo
Watts, Corinne
Zhao, Qingyuan
Sutherland, William J.
author_facet Christie, Alec P.
Abecasis, David
Adjeroud, Mehdi
Alonso, Juan C.
Amano, Tatsuya
Anton, Alvaro
Baldigo, Barry P.
Barrientos, Rafael
Bicknell, Jake E.
Buhl, Deborah A.
Cebrian, Just
Ceia, Ricardo S.
Cibils-Martina, Luciana
Clarke, Sarah
Claudet, Joachim
Craig, Michael D.
Davoult, Dominique
De Backer, Annelies
Donovan, Mary K.
Eddy, Tyler D.
França, Filipe M.
Gardner, Jonathan P. A.
Harris, Bradley P.
Huusko, Ari
Jones, Ian L.
Kelaher, Brendan P.
Kotiaho, Janne S.
López-Baucells, Adrià
Major, Heather L.
Mäki-Petäys, Aki
Martín, Beatriz
Martín, Carlos A.
Martin, Philip A.
Mateos-Molina, Daniel
McConnaughey, Robert A.
Meroni, Michele
Meyer, Christoph F. J.
Mills, Kade
Montefalcone, Monica
Noreika, Norbertas
Palacín, Carlos
Pande, Anjali
Pitcher, C. Roland
Ponce, Carlos
Rinella, Matt
Rocha, Ricardo
Ruiz-Delgado, María C.
Schmitter-Soto, Juan J.
Shaffer, Jill A.
Sharma, Shailesh
Sher, Anna A.
Stagnol, Doriane
Stanley, Thomas R.
Stokesbury, Kevin D. E.
Torres, Aurora
Tully, Oliver
Vehanen, Teppo
Watts, Corinne
Zhao, Qingyuan
Sutherland, William J.
author_sort Christie, Alec P.
collection PubMed
description Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.
format Online
Article
Text
id pubmed-7733498
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-77334982020-12-17 Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences Christie, Alec P. Abecasis, David Adjeroud, Mehdi Alonso, Juan C. Amano, Tatsuya Anton, Alvaro Baldigo, Barry P. Barrientos, Rafael Bicknell, Jake E. Buhl, Deborah A. Cebrian, Just Ceia, Ricardo S. Cibils-Martina, Luciana Clarke, Sarah Claudet, Joachim Craig, Michael D. Davoult, Dominique De Backer, Annelies Donovan, Mary K. Eddy, Tyler D. França, Filipe M. Gardner, Jonathan P. A. Harris, Bradley P. Huusko, Ari Jones, Ian L. Kelaher, Brendan P. Kotiaho, Janne S. López-Baucells, Adrià Major, Heather L. Mäki-Petäys, Aki Martín, Beatriz Martín, Carlos A. Martin, Philip A. Mateos-Molina, Daniel McConnaughey, Robert A. Meroni, Michele Meyer, Christoph F. J. Mills, Kade Montefalcone, Monica Noreika, Norbertas Palacín, Carlos Pande, Anjali Pitcher, C. Roland Ponce, Carlos Rinella, Matt Rocha, Ricardo Ruiz-Delgado, María C. Schmitter-Soto, Juan J. Shaffer, Jill A. Sharma, Shailesh Sher, Anna A. Stagnol, Doriane Stanley, Thomas R. Stokesbury, Kevin D. E. Torres, Aurora Tully, Oliver Vehanen, Teppo Watts, Corinne Zhao, Qingyuan Sutherland, William J. Nat Commun Article Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs. Nature Publishing Group UK 2020-12-11 /pmc/articles/PMC7733498/ /pubmed/33311448 http://dx.doi.org/10.1038/s41467-020-20142-y Text en © The Author(s) 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
spellingShingle Article
Christie, Alec P.
Abecasis, David
Adjeroud, Mehdi
Alonso, Juan C.
Amano, Tatsuya
Anton, Alvaro
Baldigo, Barry P.
Barrientos, Rafael
Bicknell, Jake E.
Buhl, Deborah A.
Cebrian, Just
Ceia, Ricardo S.
Cibils-Martina, Luciana
Clarke, Sarah
Claudet, Joachim
Craig, Michael D.
Davoult, Dominique
De Backer, Annelies
Donovan, Mary K.
Eddy, Tyler D.
França, Filipe M.
Gardner, Jonathan P. A.
Harris, Bradley P.
Huusko, Ari
Jones, Ian L.
Kelaher, Brendan P.
Kotiaho, Janne S.
López-Baucells, Adrià
Major, Heather L.
Mäki-Petäys, Aki
Martín, Beatriz
Martín, Carlos A.
Martin, Philip A.
Mateos-Molina, Daniel
McConnaughey, Robert A.
Meroni, Michele
Meyer, Christoph F. J.
Mills, Kade
Montefalcone, Monica
Noreika, Norbertas
Palacín, Carlos
Pande, Anjali
Pitcher, C. Roland
Ponce, Carlos
Rinella, Matt
Rocha, Ricardo
Ruiz-Delgado, María C.
Schmitter-Soto, Juan J.
Shaffer, Jill A.
Sharma, Shailesh
Sher, Anna A.
Stagnol, Doriane
Stanley, Thomas R.
Stokesbury, Kevin D. E.
Torres, Aurora
Tully, Oliver
Vehanen, Teppo
Watts, Corinne
Zhao, Qingyuan
Sutherland, William J.
Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences
title Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences
title_full Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences
title_fullStr Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences
title_full_unstemmed Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences
title_short Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences
title_sort quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7733498/
https://www.ncbi.nlm.nih.gov/pubmed/33311448
http://dx.doi.org/10.1038/s41467-020-20142-y
work_keys_str_mv AT christiealecp quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT abecasisdavid quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT adjeroudmehdi quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT alonsojuanc quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT amanotatsuya quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT antonalvaro quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT baldigobarryp quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT barrientosrafael quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT bicknelljakee quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT buhldeboraha quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT cebrianjust quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT ceiaricardos quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT cibilsmartinaluciana quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT clarkesarah quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT claudetjoachim quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT craigmichaeld quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT davoultdominique quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT debackerannelies quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT donovanmaryk quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT eddytylerd quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT francafilipem quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT gardnerjonathanpa quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT harrisbradleyp quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT huuskoari quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT jonesianl quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT kelaherbrendanp quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT kotiahojannes quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT lopezbaucellsadria quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT majorheatherl quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT makipetaysaki quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT martinbeatriz quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT martincarlosa quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT martinphilipa quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT mateosmolinadaniel quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT mcconnaugheyroberta quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT meronimichele quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT meyerchristophfj quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT millskade quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT montefalconemonica quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT noreikanorbertas quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT palacincarlos quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT pandeanjali quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT pitchercroland quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT poncecarlos quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT rinellamatt quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT rocharicardo quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT ruizdelgadomariac quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT schmittersotojuanj quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT shafferjilla quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT sharmashailesh quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT sherannaa quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT stagnoldoriane quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT stanleythomasr quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT stokesburykevinde quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT torresaurora quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT tullyoliver quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT vehanenteppo quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT wattscorinne quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT zhaoqingyuan quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences
AT sutherlandwilliamj quantifyingandaddressingtheprevalenceandbiasofstudydesignsintheenvironmentalandsocialsciences