Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation

Variational inference is a powerful framework, used to approximate intractable posteriors through variational distributions. The de facto standard is to rely on Gaussian variational families, which come with numerous advantages: they are easy to sample from, simple to parametrize, and many expectati...

Descripción completa

Detalles Bibliográficos
Autores principales: Galy-Fajou, Théo, Perrone, Valerio, Opper, Manfred
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8393997/
https://www.ncbi.nlm.nih.gov/pubmed/34441130
http://dx.doi.org/10.3390/e23080990
_version_ 1783743851125538816
author Galy-Fajou, Théo
Perrone, Valerio
Opper, Manfred
author_facet Galy-Fajou, Théo
Perrone, Valerio
Opper, Manfred
author_sort Galy-Fajou, Théo
collection PubMed
description Variational inference is a powerful framework, used to approximate intractable posteriors through variational distributions. The de facto standard is to rely on Gaussian variational families, which come with numerous advantages: they are easy to sample from, simple to parametrize, and many expectations are known in closed-form or readily computed by quadrature. In this paper, we view the Gaussian variational approximation problem through the lens of gradient flows. We introduce a flexible and efficient algorithm based on a linear flow leading to a particle-based approximation. We prove that, with a sufficient number of particles, our algorithm converges linearly to the exact solution for Gaussian targets, and a low-rank approximation otherwise. In addition to the theoretical analysis, we show, on a set of synthetic and real-world high-dimensional problems, that our algorithm outperforms existing methods with Gaussian targets while performing on a par with non-Gaussian targets.
format Online
Article
Text
id pubmed-8393997
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-83939972021-08-28 Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation Galy-Fajou, Théo Perrone, Valerio Opper, Manfred Entropy (Basel) Article Variational inference is a powerful framework, used to approximate intractable posteriors through variational distributions. The de facto standard is to rely on Gaussian variational families, which come with numerous advantages: they are easy to sample from, simple to parametrize, and many expectations are known in closed-form or readily computed by quadrature. In this paper, we view the Gaussian variational approximation problem through the lens of gradient flows. We introduce a flexible and efficient algorithm based on a linear flow leading to a particle-based approximation. We prove that, with a sufficient number of particles, our algorithm converges linearly to the exact solution for Gaussian targets, and a low-rank approximation otherwise. In addition to the theoretical analysis, we show, on a set of synthetic and real-world high-dimensional problems, that our algorithm outperforms existing methods with Gaussian targets while performing on a par with non-Gaussian targets. MDPI 2021-07-30 /pmc/articles/PMC8393997/ /pubmed/34441130 http://dx.doi.org/10.3390/e23080990 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Galy-Fajou, Théo
Perrone, Valerio
Opper, Manfred
Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation
title Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation
title_full Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation
title_fullStr Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation
title_full_unstemmed Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation
title_short Flexible and Efficient Inference with Particles for the Variational Gaussian Approximation
title_sort flexible and efficient inference with particles for the variational gaussian approximation
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8393997/
https://www.ncbi.nlm.nih.gov/pubmed/34441130
http://dx.doi.org/10.3390/e23080990
work_keys_str_mv AT galyfajoutheo flexibleandefficientinferencewithparticlesforthevariationalgaussianapproximation
AT perronevalerio flexibleandefficientinferencewithparticlesforthevariationalgaussianapproximation
AT oppermanfred flexibleandefficientinferencewithparticlesforthevariationalgaussianapproximation