Cargando…
Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research
The replicability of research results has been a cause of increasing concern to the scientific community. The long-held belief that experimental standardization begets replicability has also been recently challenged, with the observation that the reduction of variability within studies can lead to i...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Public Library of Science
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8168858/ https://www.ncbi.nlm.nih.gov/pubmed/34010281 http://dx.doi.org/10.1371/journal.pbio.3001009 |
_version_ | 1783701943551524864 |
---|---|
author | Usui, Takuji Macleod, Malcolm R. McCann, Sarah K. Senior, Alistair M. Nakagawa, Shinichi |
author_facet | Usui, Takuji Macleod, Malcolm R. McCann, Sarah K. Senior, Alistair M. Nakagawa, Shinichi |
author_sort | Usui, Takuji |
collection | PubMed |
description | The replicability of research results has been a cause of increasing concern to the scientific community. The long-held belief that experimental standardization begets replicability has also been recently challenged, with the observation that the reduction of variability within studies can lead to idiosyncratic, lab-specific results that cannot be replicated. An alternative approach is to, instead, deliberately introduce heterogeneity, known as “heterogenization” of experimental design. Here, we explore a novel perspective in the heterogenization program in a meta-analysis of variability in observed phenotypic outcomes in both control and experimental animal models of ischemic stroke. First, by quantifying interindividual variability across control groups, we illustrate that the amount of heterogeneity in disease state (infarct volume) differs according to methodological approach, for example, in disease induction methods and disease models. We argue that such methods may improve replicability by creating diverse and representative distribution of baseline disease state in the reference group, against which treatment efficacy is assessed. Second, we illustrate how meta-analysis can be used to simultaneously assess efficacy and stability (i.e., mean effect and among-individual variability). We identify treatments that have efficacy and are generalizable to the population level (i.e., low interindividual variability), as well as those where there is high interindividual variability in response; for these, latter treatments translation to a clinical setting may require nuance. We argue that by embracing rather than seeking to minimize variability in phenotypic outcomes, we can motivate the shift toward heterogenization and improve both the replicability and generalizability of preclinical research. |
format | Online Article Text |
id | pubmed-8168858 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | Public Library of Science |
record_format | MEDLINE/PubMed |
spelling | pubmed-81688582021-06-11 Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research Usui, Takuji Macleod, Malcolm R. McCann, Sarah K. Senior, Alistair M. Nakagawa, Shinichi PLoS Biol Meta-Research Article The replicability of research results has been a cause of increasing concern to the scientific community. The long-held belief that experimental standardization begets replicability has also been recently challenged, with the observation that the reduction of variability within studies can lead to idiosyncratic, lab-specific results that cannot be replicated. An alternative approach is to, instead, deliberately introduce heterogeneity, known as “heterogenization” of experimental design. Here, we explore a novel perspective in the heterogenization program in a meta-analysis of variability in observed phenotypic outcomes in both control and experimental animal models of ischemic stroke. First, by quantifying interindividual variability across control groups, we illustrate that the amount of heterogeneity in disease state (infarct volume) differs according to methodological approach, for example, in disease induction methods and disease models. We argue that such methods may improve replicability by creating diverse and representative distribution of baseline disease state in the reference group, against which treatment efficacy is assessed. Second, we illustrate how meta-analysis can be used to simultaneously assess efficacy and stability (i.e., mean effect and among-individual variability). We identify treatments that have efficacy and are generalizable to the population level (i.e., low interindividual variability), as well as those where there is high interindividual variability in response; for these, latter treatments translation to a clinical setting may require nuance. We argue that by embracing rather than seeking to minimize variability in phenotypic outcomes, we can motivate the shift toward heterogenization and improve both the replicability and generalizability of preclinical research. Public Library of Science 2021-05-19 /pmc/articles/PMC8168858/ /pubmed/34010281 http://dx.doi.org/10.1371/journal.pbio.3001009 Text en © 2021 Usui et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. |
spellingShingle | Meta-Research Article Usui, Takuji Macleod, Malcolm R. McCann, Sarah K. Senior, Alistair M. Nakagawa, Shinichi Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research |
title | Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research |
title_full | Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research |
title_fullStr | Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research |
title_full_unstemmed | Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research |
title_short | Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research |
title_sort | meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research |
topic | Meta-Research Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8168858/ https://www.ncbi.nlm.nih.gov/pubmed/34010281 http://dx.doi.org/10.1371/journal.pbio.3001009 |
work_keys_str_mv | AT usuitakuji metaanalysisofvariationsuggeststhatembracingvariabilityimprovesbothreplicabilityandgeneralizabilityinpreclinicalresearch AT macleodmalcolmr metaanalysisofvariationsuggeststhatembracingvariabilityimprovesbothreplicabilityandgeneralizabilityinpreclinicalresearch AT mccannsarahk metaanalysisofvariationsuggeststhatembracingvariabilityimprovesbothreplicabilityandgeneralizabilityinpreclinicalresearch AT senioralistairm metaanalysisofvariationsuggeststhatembracingvariabilityimprovesbothreplicabilityandgeneralizabilityinpreclinicalresearch AT nakagawashinichi metaanalysisofvariationsuggeststhatembracingvariabilityimprovesbothreplicabilityandgeneralizabilityinpreclinicalresearch |