Cargando…

Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data

Inference from limited data requires a notion of measure on parameter space, which is most explicit in the Bayesian framework as a prior distribution. Jeffreys prior is the best-known uninformative choice, the invariant volume element from information geometry, but we demonstrate here that this lead...

Descripción completa

Detalles Bibliográficos
Autores principales: Abbott, Michael C., Machta, Benjamin B.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10048238/
https://www.ncbi.nlm.nih.gov/pubmed/36981323
http://dx.doi.org/10.3390/e25030434
_version_ 1785014133728477184
author Abbott, Michael C.
Machta, Benjamin B.
author_facet Abbott, Michael C.
Machta, Benjamin B.
author_sort Abbott, Michael C.
collection PubMed
description Inference from limited data requires a notion of measure on parameter space, which is most explicit in the Bayesian framework as a prior distribution. Jeffreys prior is the best-known uninformative choice, the invariant volume element from information geometry, but we demonstrate here that this leads to enormous bias in typical high-dimensional models. This is because models found in science typically have an effective dimensionality of accessible behaviors much smaller than the number of microscopic parameters. Any measure which treats all of these parameters equally is far from uniform when projected onto the sub-space of relevant parameters, due to variations in the local co-volume of irrelevant directions. We present results on a principled choice of measure which avoids this issue and leads to unbiased posteriors by focusing on relevant parameters. This optimal prior depends on the quantity of data to be gathered, and approaches Jeffreys prior in the asymptotic limit. However, for typical models, this limit cannot be justified without an impossibly large increase in the quantity of data, exponential in the number of microscopic parameters.
format Online
Article
Text
id pubmed-10048238
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-100482382023-03-29 Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data Abbott, Michael C. Machta, Benjamin B. Entropy (Basel) Article Inference from limited data requires a notion of measure on parameter space, which is most explicit in the Bayesian framework as a prior distribution. Jeffreys prior is the best-known uninformative choice, the invariant volume element from information geometry, but we demonstrate here that this leads to enormous bias in typical high-dimensional models. This is because models found in science typically have an effective dimensionality of accessible behaviors much smaller than the number of microscopic parameters. Any measure which treats all of these parameters equally is far from uniform when projected onto the sub-space of relevant parameters, due to variations in the local co-volume of irrelevant directions. We present results on a principled choice of measure which avoids this issue and leads to unbiased posteriors by focusing on relevant parameters. This optimal prior depends on the quantity of data to be gathered, and approaches Jeffreys prior in the asymptotic limit. However, for typical models, this limit cannot be justified without an impossibly large increase in the quantity of data, exponential in the number of microscopic parameters. MDPI 2023-03-01 /pmc/articles/PMC10048238/ /pubmed/36981323 http://dx.doi.org/10.3390/e25030434 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Abbott, Michael C.
Machta, Benjamin B.
Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data
title Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data
title_full Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data
title_fullStr Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data
title_full_unstemmed Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data
title_short Far from Asymptopia: Unbiased High-Dimensional Inference Cannot Assume Unlimited Data
title_sort far from asymptopia: unbiased high-dimensional inference cannot assume unlimited data
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10048238/
https://www.ncbi.nlm.nih.gov/pubmed/36981323
http://dx.doi.org/10.3390/e25030434
work_keys_str_mv AT abbottmichaelc farfromasymptopiaunbiasedhighdimensionalinferencecannotassumeunlimiteddata
AT machtabenjaminb farfromasymptopiaunbiasedhighdimensionalinferencecannotassumeunlimiteddata