Cargando…
Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data
Inference from limited data requires a notion of measure on parameter space, which is most explicit in the Bayesian framework as a prior distribution. Jeffreys prior is the best-known uninformative choice, the invariant volume element from information geometry, but we demonstrate here that this lead...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10048238/ https://www.ncbi.nlm.nih.gov/pubmed/36981323 http://dx.doi.org/10.3390/e25030434 |
_version_ | 1785014133728477184 |
---|---|
author | Abbott, Michael C. Machta, Benjamin B. |
author_facet | Abbott, Michael C. Machta, Benjamin B. |
author_sort | Abbott, Michael C. |
collection | PubMed |
description | Inference from limited data requires a notion of measure on parameter space, which is most explicit in the Bayesian framework as a prior distribution. Jeffreys prior is the best-known uninformative choice, the invariant volume element from information geometry, but we demonstrate here that this leads to enormous bias in typical high-dimensional models. This is because models found in science typically have an effective dimensionality of accessible behaviors much smaller than the number of microscopic parameters. Any measure which treats all of these parameters equally is far from uniform when projected onto the sub-space of relevant parameters, due to variations in the local co-volume of irrelevant directions. We present results on a principled choice of measure which avoids this issue and leads to unbiased posteriors by focusing on relevant parameters. This optimal prior depends on the quantity of data to be gathered, and approaches Jeffreys prior in the asymptotic limit. However, for typical models, this limit cannot be justified without an impossibly large increase in the quantity of data, exponential in the number of microscopic parameters. |
format | Online Article Text |
id | pubmed-10048238 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-100482382023-03-29 Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data Abbott, Michael C. Machta, Benjamin B. Entropy (Basel) Article Inference from limited data requires a notion of measure on parameter space, which is most explicit in the Bayesian framework as a prior distribution. Jeffreys prior is the best-known uninformative choice, the invariant volume element from information geometry, but we demonstrate here that this leads to enormous bias in typical high-dimensional models. This is because models found in science typically have an effective dimensionality of accessible behaviors much smaller than the number of microscopic parameters. Any measure which treats all of these parameters equally is far from uniform when projected onto the sub-space of relevant parameters, due to variations in the local co-volume of irrelevant directions. We present results on a principled choice of measure which avoids this issue and leads to unbiased posteriors by focusing on relevant parameters. This optimal prior depends on the quantity of data to be gathered, and approaches Jeffreys prior in the asymptotic limit. However, for typical models, this limit cannot be justified without an impossibly large increase in the quantity of data, exponential in the number of microscopic parameters. MDPI 2023-03-01 /pmc/articles/PMC10048238/ /pubmed/36981323 http://dx.doi.org/10.3390/e25030434 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Abbott, Michael C. Machta, Benjamin B. Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data |
title | Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data |
title_full | Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data |
title_fullStr | Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data |
title_full_unstemmed | Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data |
title_short | Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data |
title_sort | far from asymptopia: unbiased high-dimensional inference cannot assume unlimited data |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10048238/ https://www.ncbi.nlm.nih.gov/pubmed/36981323 http://dx.doi.org/10.3390/e25030434 |
work_keys_str_mv | AT abbottmichaelc farfromasymptopiaunbiasedhighdimensionalinferencecannotassumeunlimiteddata AT machtabenjaminb farfromasymptopiaunbiasedhighdimensionalinferencecannotassumeunlimiteddata |