Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation
Variational inference is a powerful framework, used to approximate intractable posteriors through variational distributions. The de facto standard is to rely on Gaussian variational families, which come with numerous advantages: they are easy to sample from, simple to parametrize, and many expectati...
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8393997/ https://www.ncbi.nlm.nih.gov/pubmed/34441130 http://dx.doi.org/10.3390/e23080990 |
_version_ | 1783743851125538816 |
---|---|
author | Galy-Fajou, Théo Perrone, Valerio Opper, Manfred |
author_facet | Galy-Fajou, Théo Perrone, Valerio Opper, Manfred |
author_sort | Galy-Fajou, Théo |
collection | PubMed |
description | Variational inference is a powerful framework, used to approximate intractable posteriors through variational distributions. The de facto standard is to rely on Gaussian variational families, which come with numerous advantages: they are easy to sample from, simple to parametrize, and many expectations are known in closed-form or readily computed by quadrature. In this paper, we view the Gaussian variational approximation problem through the lens of gradient flows. We introduce a flexible and efficient algorithm based on a linear flow leading to a particle-based approximation. We prove that, with a sufficient number of particles, our algorithm converges linearly to the exact solution for Gaussian targets, and a low-rank approximation otherwise. In addition to the theoretical analysis, we show, on a set of synthetic and real-world high-dimensional problems, that our algorithm outperforms existing methods with Gaussian targets while performing on a par with non-Gaussian targets. |
format | Online Article Text |
id | pubmed-8393997 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-83939972021-08-28 Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation Galy-Fajou, Théo Perrone, Valerio Opper, Manfred Entropy (Basel) Article Variational inference is a powerful framework, used to approximate intractable posteriors through variational distributions. The de facto standard is to rely on Gaussian variational families, which come with numerous advantages: they are easy to sample from, simple to parametrize, and many expectations are known in closed-form or readily computed by quadrature. In this paper, we view the Gaussian variational approximation problem through the lens of gradient flows. We introduce a flexible and efficient algorithm based on a linear flow leading to a particle-based approximation. We prove that, with a sufficient number of particles, our algorithm converges linearly to the exact solution for Gaussian targets, and a low-rank approximation otherwise. In addition to the theoretical analysis, we show, on a set of synthetic and real-world high-dimensional problems, that our algorithm outperforms existing methods with Gaussian targets while performing on a par with non-Gaussian targets. MDPI 2021-07-30 /pmc/articles/PMC8393997/ /pubmed/34441130 http://dx.doi.org/10.3390/e23080990 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Galy-Fajou, Théo Perrone, Valerio Opper, Manfred Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation |
title | Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation |
title_full | Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation |
title_fullStr | Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation |
title_full_unstemmed | Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation |
title_short | Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation |
title_sort | flexible and efficient inference with particles for the variational gaussian approximation |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8393997/ https://www.ncbi.nlm.nih.gov/pubmed/34441130 http://dx.doi.org/10.3390/e23080990 |
work_keys_str_mv | AT galyfajoutheo flexibleandefficientinferencewithparticlesforthevariationalgaussianapproximation AT perronevalerio flexibleandefficientinferencewithparticlesforthevariationalgaussianapproximation AT oppermanfred flexibleandefficientinferencewithparticlesforthevariationalgaussianapproximation |