Cargando…
Variational Information Bottleneck for Semi-Supervised Classification
In this paper, we consider an information bottleneck (IB) framework for semi-supervised classification with several families of priors on latent space representation. We apply a variational decomposition of mutual information terms of IB. Using this decomposition we perform an analysis of several re...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597214/ https://www.ncbi.nlm.nih.gov/pubmed/33286710 http://dx.doi.org/10.3390/e22090943 |
_version_ | 1783602292720664576 |
---|---|
author | Voloshynovskiy, Slava Taran, Olga Kondah, Mouad Holotyak, Taras Rezende, Danilo |
author_facet | Voloshynovskiy, Slava Taran, Olga Kondah, Mouad Holotyak, Taras Rezende, Danilo |
author_sort | Voloshynovskiy, Slava |
collection | PubMed |
description | In this paper, we consider an information bottleneck (IB) framework for semi-supervised classification with several families of priors on latent space representation. We apply a variational decomposition of mutual information terms of IB. Using this decomposition we perform an analysis of several regularizers and practically demonstrate an impact of different components of variational model on the classification accuracy. We propose a new formulation of semi-supervised IB with hand crafted and learnable priors and link it to the previous methods such as semi-supervised versions of VAE (M1 + M2), AAE, CatGAN, etc. We show that the resulting model allows better understand the role of various previously proposed regularizers in semi-supervised classification task in the light of IB framework. The proposed IB semi-supervised model with hand-crafted and learnable priors is experimentally validated on MNIST under different amount of labeled data. |
format | Online Article Text |
id | pubmed-7597214 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-75972142020-11-09 Variational Information Bottleneck for Semi-Supervised Classification Voloshynovskiy, Slava Taran, Olga Kondah, Mouad Holotyak, Taras Rezende, Danilo Entropy (Basel) Article In this paper, we consider an information bottleneck (IB) framework for semi-supervised classification with several families of priors on latent space representation. We apply a variational decomposition of mutual information terms of IB. Using this decomposition we perform an analysis of several regularizers and practically demonstrate an impact of different components of variational model on the classification accuracy. We propose a new formulation of semi-supervised IB with hand crafted and learnable priors and link it to the previous methods such as semi-supervised versions of VAE (M1 + M2), AAE, CatGAN, etc. We show that the resulting model allows better understand the role of various previously proposed regularizers in semi-supervised classification task in the light of IB framework. The proposed IB semi-supervised model with hand-crafted and learnable priors is experimentally validated on MNIST under different amount of labeled data. MDPI 2020-08-27 /pmc/articles/PMC7597214/ /pubmed/33286710 http://dx.doi.org/10.3390/e22090943 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Voloshynovskiy, Slava Taran, Olga Kondah, Mouad Holotyak, Taras Rezende, Danilo Variational Information Bottleneck for Semi-Supervised Classification |
title | Variational Information Bottleneck for Semi-Supervised Classification |
title_full | Variational Information Bottleneck for Semi-Supervised Classification |
title_fullStr | Variational Information Bottleneck for Semi-Supervised Classification |
title_full_unstemmed | Variational Information Bottleneck for Semi-Supervised Classification |
title_short | Variational Information Bottleneck for Semi-Supervised Classification |
title_sort | variational information bottleneck for semi-supervised classification |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7597214/ https://www.ncbi.nlm.nih.gov/pubmed/33286710 http://dx.doi.org/10.3390/e22090943 |
work_keys_str_mv | AT voloshynovskiyslava variationalinformationbottleneckforsemisupervisedclassification AT taranolga variationalinformationbottleneckforsemisupervisedclassification AT kondahmouad variationalinformationbottleneckforsemisupervisedclassification AT holotyaktaras variationalinformationbottleneckforsemisupervisedclassification AT rezendedanilo variationalinformationbottleneckforsemisupervisedclassification |