Cargando…

CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction

This work proposes a new computational framework for learning a structured generative model for real-world datasets. In particular, we propose to learn a Closed-loop Transcriptionbetween a multi-class, multi-dimensional data distribution and a Linear discriminative representation (CTRL) in the featu...

Descripción completa

Detalles Bibliográficos
Autores principales: Dai, Xili, Tong, Shengbang, Li, Mingyang, Wu, Ziyang, Psenka, Michael, Chan, Kwan Ho Ryan, Zhai, Pengyuan, Yu, Yaodong, Yuan, Xiaojun, Shum, Heung-Yeung, Ma, Yi
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9031319/
https://www.ncbi.nlm.nih.gov/pubmed/35455120
http://dx.doi.org/10.3390/e24040456
_version_ 1784692361608036352
author Dai, Xili
Tong, Shengbang
Li, Mingyang
Wu, Ziyang
Psenka, Michael
Chan, Kwan Ho Ryan
Zhai, Pengyuan
Yu, Yaodong
Yuan, Xiaojun
Shum, Heung-Yeung
Ma, Yi
author_facet Dai, Xili
Tong, Shengbang
Li, Mingyang
Wu, Ziyang
Psenka, Michael
Chan, Kwan Ho Ryan
Zhai, Pengyuan
Yu, Yaodong
Yuan, Xiaojun
Shum, Heung-Yeung
Ma, Yi
author_sort Dai, Xili
collection PubMed
description This work proposes a new computational framework for learning a structured generative model for real-world datasets. In particular, we propose to learn a Closed-loop Transcriptionbetween a multi-class, multi-dimensional data distribution and a Linear discriminative representation (CTRL) in the feature space that consists of multiple independent multi-dimensional linear subspaces. In particular, we argue that the optimal encoding and decoding mappings sought can be formulated as a two-player minimax game between the encoder and decoderfor the learned representation. A natural utility function for this game is the so-called rate reduction, a simple information-theoretic measure for distances between mixtures of subspace-like Gaussians in the feature space. Our formulation draws inspiration from closed-loop error feedback from control systems and avoids expensive evaluating and minimizing of approximated distances between arbitrary distributions in either the data space or the feature space. To a large extent, this new formulation unifies the concepts and benefits of Auto-Encoding and GAN and naturally extends them to the settings of learning a both discriminative and generative representation for multi-class and multi-dimensional real-world data. Our extensive experiments on many benchmark imagery datasets demonstrate tremendous potential of this new closed-loop formulation: under fair comparison, visual quality of the learned decoder and classification performance of the encoder is competitive and arguably better than existing methods based on GAN, VAE, or a combination of both. Unlike existing generative models, the so-learned features of the multiple classes are structured instead of hidden: different classes are explicitly mapped onto corresponding independent principal subspaces in the feature space, and diverse visual attributes within each class are modeled by the independent principal components within each subspace.
format Online
Article
Text
id pubmed-9031319
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-90313192022-04-23 CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction Dai, Xili Tong, Shengbang Li, Mingyang Wu, Ziyang Psenka, Michael Chan, Kwan Ho Ryan Zhai, Pengyuan Yu, Yaodong Yuan, Xiaojun Shum, Heung-Yeung Ma, Yi Entropy (Basel) Article This work proposes a new computational framework for learning a structured generative model for real-world datasets. In particular, we propose to learn a Closed-loop Transcriptionbetween a multi-class, multi-dimensional data distribution and a Linear discriminative representation (CTRL) in the feature space that consists of multiple independent multi-dimensional linear subspaces. In particular, we argue that the optimal encoding and decoding mappings sought can be formulated as a two-player minimax game between the encoder and decoderfor the learned representation. A natural utility function for this game is the so-called rate reduction, a simple information-theoretic measure for distances between mixtures of subspace-like Gaussians in the feature space. Our formulation draws inspiration from closed-loop error feedback from control systems and avoids expensive evaluating and minimizing of approximated distances between arbitrary distributions in either the data space or the feature space. To a large extent, this new formulation unifies the concepts and benefits of Auto-Encoding and GAN and naturally extends them to the settings of learning a both discriminative and generative representation for multi-class and multi-dimensional real-world data. Our extensive experiments on many benchmark imagery datasets demonstrate tremendous potential of this new closed-loop formulation: under fair comparison, visual quality of the learned decoder and classification performance of the encoder is competitive and arguably better than existing methods based on GAN, VAE, or a combination of both. Unlike existing generative models, the so-learned features of the multiple classes are structured instead of hidden: different classes are explicitly mapped onto corresponding independent principal subspaces in the feature space, and diverse visual attributes within each class are modeled by the independent principal components within each subspace. MDPI 2022-03-25 /pmc/articles/PMC9031319/ /pubmed/35455120 http://dx.doi.org/10.3390/e24040456 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Dai, Xili
Tong, Shengbang
Li, Mingyang
Wu, Ziyang
Psenka, Michael
Chan, Kwan Ho Ryan
Zhai, Pengyuan
Yu, Yaodong
Yuan, Xiaojun
Shum, Heung-Yeung
Ma, Yi
CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction
title CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction
title_full CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction
title_fullStr CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction
title_full_unstemmed CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction
title_short CTRL: Closed-Loop Transcription to an LDR via Minimaxing Rate Reduction
title_sort ctrl: closed-loop transcription to an ldr via minimaxing rate reduction
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9031319/
https://www.ncbi.nlm.nih.gov/pubmed/35455120
http://dx.doi.org/10.3390/e24040456
work_keys_str_mv AT daixili ctrlclosedlooptranscriptiontoanldrviaminimaxingratereduction
AT tongshengbang ctrlclosedlooptranscriptiontoanldrviaminimaxingratereduction
AT limingyang ctrlclosedlooptranscriptiontoanldrviaminimaxingratereduction
AT wuziyang ctrlclosedlooptranscriptiontoanldrviaminimaxingratereduction
AT psenkamichael ctrlclosedlooptranscriptiontoanldrviaminimaxingratereduction
AT chankwanhoryan ctrlclosedlooptranscriptiontoanldrviaminimaxingratereduction
AT zhaipengyuan ctrlclosedlooptranscriptiontoanldrviaminimaxingratereduction
AT yuyaodong ctrlclosedlooptranscriptiontoanldrviaminimaxingratereduction
AT yuanxiaojun ctrlclosedlooptranscriptiontoanldrviaminimaxingratereduction
AT shumheungyeung ctrlclosedlooptranscriptiontoanldrviaminimaxingratereduction
AT mayi ctrlclosedlooptranscriptiontoanldrviaminimaxingratereduction