Cargando…
Common Information Components Analysis
Wyner’s common information is a measure that quantifies and assesses the commonality between two random variables. Based on this, we introduce a novel two-step procedure to construct features from data, referred to as Common Information Components Analysis (CICA). The first step can be interpreted a...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7912312/ https://www.ncbi.nlm.nih.gov/pubmed/33530532 http://dx.doi.org/10.3390/e23020151 |
_version_ | 1783656548145299456 |
---|---|
author | Sula, Erixhen Gastpar, Michael C. |
author_facet | Sula, Erixhen Gastpar, Michael C. |
author_sort | Sula, Erixhen |
collection | PubMed |
description | Wyner’s common information is a measure that quantifies and assesses the commonality between two random variables. Based on this, we introduce a novel two-step procedure to construct features from data, referred to as Common Information Components Analysis (CICA). The first step can be interpreted as an extraction of Wyner’s common information. The second step is a form of back-projection of the common information onto the original variables, leading to the extracted features. A free parameter [Formula: see text] controls the complexity of the extracted features. We establish that, in the case of Gaussian statistics, CICA precisely reduces to Canonical Correlation Analysis (CCA), where the parameter [Formula: see text] determines the number of CCA components that are extracted. In this sense, we establish a novel rigorous connection between information measures and CCA, and CICA is a strict generalization of the latter. It is shown that CICA has several desirable features, including a natural extension to beyond just two data sets. |
format | Online Article Text |
id | pubmed-7912312 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-79123122021-02-28 Common Information Components Analysis Sula, Erixhen Gastpar, Michael C. Entropy (Basel) Article Wyner’s common information is a measure that quantifies and assesses the commonality between two random variables. Based on this, we introduce a novel two-step procedure to construct features from data, referred to as Common Information Components Analysis (CICA). The first step can be interpreted as an extraction of Wyner’s common information. The second step is a form of back-projection of the common information onto the original variables, leading to the extracted features. A free parameter [Formula: see text] controls the complexity of the extracted features. We establish that, in the case of Gaussian statistics, CICA precisely reduces to Canonical Correlation Analysis (CCA), where the parameter [Formula: see text] determines the number of CCA components that are extracted. In this sense, we establish a novel rigorous connection between information measures and CCA, and CICA is a strict generalization of the latter. It is shown that CICA has several desirable features, including a natural extension to beyond just two data sets. MDPI 2021-01-26 /pmc/articles/PMC7912312/ /pubmed/33530532 http://dx.doi.org/10.3390/e23020151 Text en © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Sula, Erixhen Gastpar, Michael C. Common Information Components Analysis |
title | Common Information Components Analysis |
title_full | Common Information Components Analysis |
title_fullStr | Common Information Components Analysis |
title_full_unstemmed | Common Information Components Analysis |
title_short | Common Information Components Analysis |
title_sort | common information components analysis |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7912312/ https://www.ncbi.nlm.nih.gov/pubmed/33530532 http://dx.doi.org/10.3390/e23020151 |
work_keys_str_mv | AT sulaerixhen commoninformationcomponentsanalysis AT gastparmichaelc commoninformationcomponentsanalysis |