Cargando…

Do multiple experimenters improve the reproducibility of animal studies?

The credibility of scientific research has been seriously questioned by the widely claimed “reproducibility crisis”. In light of this crisis, there is a growing awareness that the rigorous standardisation of experimental conditions may contribute to poor reproducibility of animal studies. Instead, s...

Descripción completa

Detalles Bibliográficos
Autores principales: von Kortzfleisch, Vanessa Tabea, Ambrée, Oliver, Karp, Natasha A., Meyer, Neele, Novak, Janja, Palme, Rupert, Rosso, Marianna, Touma, Chadi, Würbel, Hanno, Kaiser, Sylvia, Sachser, Norbert, Richter, S. Helene
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9070896/
https://www.ncbi.nlm.nih.gov/pubmed/35511779
http://dx.doi.org/10.1371/journal.pbio.3001564
_version_ 1784700730562576384
author von Kortzfleisch, Vanessa Tabea
Ambrée, Oliver
Karp, Natasha A.
Meyer, Neele
Novak, Janja
Palme, Rupert
Rosso, Marianna
Touma, Chadi
Würbel, Hanno
Kaiser, Sylvia
Sachser, Norbert
Richter, S. Helene
author_facet von Kortzfleisch, Vanessa Tabea
Ambrée, Oliver
Karp, Natasha A.
Meyer, Neele
Novak, Janja
Palme, Rupert
Rosso, Marianna
Touma, Chadi
Würbel, Hanno
Kaiser, Sylvia
Sachser, Norbert
Richter, S. Helene
author_sort von Kortzfleisch, Vanessa Tabea
collection PubMed
description The credibility of scientific research has been seriously questioned by the widely claimed “reproducibility crisis”. In light of this crisis, there is a growing awareness that the rigorous standardisation of experimental conditions may contribute to poor reproducibility of animal studies. Instead, systematic heterogenisation has been proposed as a tool to enhance reproducibility, but a real-life test across multiple independent laboratories is still pending. The aim of this study was therefore to test whether heterogenisation of experimental conditions by using multiple experimenters improves the reproducibility of research findings compared to standardised conditions with only one experimenter. To this end, we replicated the same animal experiment in 3 independent laboratories, each employing both a heterogenised and a standardised design. Whereas in the standardised design, all animals were tested by a single experimenter; in the heterogenised design, 3 different experimenters were involved in testing the animals. In contrast to our expectation, the inclusion of multiple experimenters in the heterogenised design did not improve the reproducibility of the results across the 3 laboratories. Interestingly, however, a variance component analysis indicated that the variation introduced by the different experimenters was not as high as the variation introduced by the laboratories, probably explaining why this heterogenisation strategy did not bring the anticipated success. Even more interestingly, for the majority of outcome measures, the remaining residual variation was identified as an important source of variance accounting for 41% (CI(95) [34%, 49%]) to 72% (CI(95) [58%, 88%]) of the observed total variance. Despite some uncertainty surrounding the estimated numbers, these findings argue for systematically including biological variation rather than eliminating it in animal studies and call for future research on effective improvement strategies.
format Online
Article
Text
id pubmed-9070896
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-90708962022-05-06 Do multiple experimenters improve the reproducibility of animal studies? von Kortzfleisch, Vanessa Tabea Ambrée, Oliver Karp, Natasha A. Meyer, Neele Novak, Janja Palme, Rupert Rosso, Marianna Touma, Chadi Würbel, Hanno Kaiser, Sylvia Sachser, Norbert Richter, S. Helene PLoS Biol Meta-Research Article The credibility of scientific research has been seriously questioned by the widely claimed “reproducibility crisis”. In light of this crisis, there is a growing awareness that the rigorous standardisation of experimental conditions may contribute to poor reproducibility of animal studies. Instead, systematic heterogenisation has been proposed as a tool to enhance reproducibility, but a real-life test across multiple independent laboratories is still pending. The aim of this study was therefore to test whether heterogenisation of experimental conditions by using multiple experimenters improves the reproducibility of research findings compared to standardised conditions with only one experimenter. To this end, we replicated the same animal experiment in 3 independent laboratories, each employing both a heterogenised and a standardised design. Whereas in the standardised design, all animals were tested by a single experimenter; in the heterogenised design, 3 different experimenters were involved in testing the animals. In contrast to our expectation, the inclusion of multiple experimenters in the heterogenised design did not improve the reproducibility of the results across the 3 laboratories. Interestingly, however, a variance component analysis indicated that the variation introduced by the different experimenters was not as high as the variation introduced by the laboratories, probably explaining why this heterogenisation strategy did not bring the anticipated success. Even more interestingly, for the majority of outcome measures, the remaining residual variation was identified as an important source of variance accounting for 41% (CI(95) [34%, 49%]) to 72% (CI(95) [58%, 88%]) of the observed total variance. Despite some uncertainty surrounding the estimated numbers, these findings argue for systematically including biological variation rather than eliminating it in animal studies and call for future research on effective improvement strategies. Public Library of Science 2022-05-05 /pmc/articles/PMC9070896/ /pubmed/35511779 http://dx.doi.org/10.1371/journal.pbio.3001564 Text en © 2022 von Kortzfleisch et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Meta-Research Article
von Kortzfleisch, Vanessa Tabea
Ambrée, Oliver
Karp, Natasha A.
Meyer, Neele
Novak, Janja
Palme, Rupert
Rosso, Marianna
Touma, Chadi
Würbel, Hanno
Kaiser, Sylvia
Sachser, Norbert
Richter, S. Helene
Do multiple experimenters improve the reproducibility of animal studies?
title Do multiple experimenters improve the reproducibility of animal studies?
title_full Do multiple experimenters improve the reproducibility of animal studies?
title_fullStr Do multiple experimenters improve the reproducibility of animal studies?
title_full_unstemmed Do multiple experimenters improve the reproducibility of animal studies?
title_short Do multiple experimenters improve the reproducibility of animal studies?
title_sort do multiple experimenters improve the reproducibility of animal studies?
topic Meta-Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9070896/
https://www.ncbi.nlm.nih.gov/pubmed/35511779
http://dx.doi.org/10.1371/journal.pbio.3001564
work_keys_str_mv AT vonkortzfleischvanessatabea domultipleexperimentersimprovethereproducibilityofanimalstudies
AT ambreeoliver domultipleexperimentersimprovethereproducibilityofanimalstudies
AT karpnatashaa domultipleexperimentersimprovethereproducibilityofanimalstudies
AT meyerneele domultipleexperimentersimprovethereproducibilityofanimalstudies
AT novakjanja domultipleexperimentersimprovethereproducibilityofanimalstudies
AT palmerupert domultipleexperimentersimprovethereproducibilityofanimalstudies
AT rossomarianna domultipleexperimentersimprovethereproducibilityofanimalstudies
AT toumachadi domultipleexperimentersimprovethereproducibilityofanimalstudies
AT wurbelhanno domultipleexperimentersimprovethereproducibilityofanimalstudies
AT kaisersylvia domultipleexperimentersimprovethereproducibilityofanimalstudies
AT sachsernorbert domultipleexperimentersimprovethereproducibilityofanimalstudies
AT richtershelene domultipleexperimentersimprovethereproducibilityofanimalstudies