Cargando…

Common Information Components Analysis

Wyner’s common information is a measure that quantifies and assesses the commonality between two random variables. Based on this, we introduce a novel two-step procedure to construct features from data, referred to as Common Information Components Analysis (CICA). The first step can be interpreted a...

Descripción completa

Detalles Bibliográficos
Autores principales: Sula, Erixhen, Gastpar, Michael C.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7912312/
https://www.ncbi.nlm.nih.gov/pubmed/33530532
http://dx.doi.org/10.3390/e23020151
_version_ 1783656548145299456
author Sula, Erixhen
Gastpar, Michael C.
author_facet Sula, Erixhen
Gastpar, Michael C.
author_sort Sula, Erixhen
collection PubMed
description Wyner’s common information is a measure that quantifies and assesses the commonality between two random variables. Based on this, we introduce a novel two-step procedure to construct features from data, referred to as Common Information Components Analysis (CICA). The first step can be interpreted as an extraction of Wyner’s common information. The second step is a form of back-projection of the common information onto the original variables, leading to the extracted features. A free parameter [Formula: see text] controls the complexity of the extracted features. We establish that, in the case of Gaussian statistics, CICA precisely reduces to Canonical Correlation Analysis (CCA), where the parameter [Formula: see text] determines the number of CCA components that are extracted. In this sense, we establish a novel rigorous connection between information measures and CCA, and CICA is a strict generalization of the latter. It is shown that CICA has several desirable features, including a natural extension to beyond just two data sets.
format Online
Article
Text
id pubmed-7912312
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-79123122021-02-28 Common Information Components Analysis Sula, Erixhen Gastpar, Michael C. Entropy (Basel) Article Wyner’s common information is a measure that quantifies and assesses the commonality between two random variables. Based on this, we introduce a novel two-step procedure to construct features from data, referred to as Common Information Components Analysis (CICA). The first step can be interpreted as an extraction of Wyner’s common information. The second step is a form of back-projection of the common information onto the original variables, leading to the extracted features. A free parameter [Formula: see text] controls the complexity of the extracted features. We establish that, in the case of Gaussian statistics, CICA precisely reduces to Canonical Correlation Analysis (CCA), where the parameter [Formula: see text] determines the number of CCA components that are extracted. In this sense, we establish a novel rigorous connection between information measures and CCA, and CICA is a strict generalization of the latter. It is shown that CICA has several desirable features, including a natural extension to beyond just two data sets. MDPI 2021-01-26 /pmc/articles/PMC7912312/ /pubmed/33530532 http://dx.doi.org/10.3390/e23020151 Text en © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Sula, Erixhen
Gastpar, Michael C.
Common Information Components Analysis
title Common Information Components Analysis
title_full Common Information Components Analysis
title_fullStr Common Information Components Analysis
title_full_unstemmed Common Information Components Analysis
title_short Common Information Components Analysis
title_sort common information components analysis
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7912312/
https://www.ncbi.nlm.nih.gov/pubmed/33530532
http://dx.doi.org/10.3390/e23020151
work_keys_str_mv AT sulaerixhen commoninformationcomponentsanalysis
AT gastparmichaelc commoninformationcomponentsanalysis