Cargando…
The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy
Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables....
Autores principales: | , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2018
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7512685/ https://www.ncbi.nlm.nih.gov/pubmed/33265260 http://dx.doi.org/10.3390/e20030169 |
_version_ | 1783586215150223360 |
---|---|
author | Chicharro, Daniel Pica, Giuseppe Panzeri, Stefano |
author_facet | Chicharro, Daniel Pica, Giuseppe Panzeri, Stefano |
author_sort | Chicharro, Daniel |
collection | PubMed |
description | Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (2010) proposed a partial information decomposition (PID) that separates the mutual information that a set of sources contains about a set of targets into nonnegative terms interpretable as these pieces. Quantifying redundancy requires assigning an identity to different information pieces, to assess when information is common across sources. Harder et al. (2013) proposed an identity axiom that imposes necessary conditions to quantify qualitatively common information. However, Bertschinger et al. (2012) showed that, in a counterexample with deterministic target-source dependencies, the identity axiom is incompatible with ensuring PID nonnegativity. Here, we study systematically the consequences of information identity criteria that assign identity based on associations between target and source variables resulting from deterministic dependencies. We show how these criteria are related to the identity axiom and to previously proposed redundancy measures, and we characterize how they lead to negative PID terms. This constitutes a further step to more explicitly address the role of information identity in the quantification of redundancy. The implications for studying neural coding are discussed. |
format | Online Article Text |
id | pubmed-7512685 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2018 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-75126852020-11-09 The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy Chicharro, Daniel Pica, Giuseppe Panzeri, Stefano Entropy (Basel) Article Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (2010) proposed a partial information decomposition (PID) that separates the mutual information that a set of sources contains about a set of targets into nonnegative terms interpretable as these pieces. Quantifying redundancy requires assigning an identity to different information pieces, to assess when information is common across sources. Harder et al. (2013) proposed an identity axiom that imposes necessary conditions to quantify qualitatively common information. However, Bertschinger et al. (2012) showed that, in a counterexample with deterministic target-source dependencies, the identity axiom is incompatible with ensuring PID nonnegativity. Here, we study systematically the consequences of information identity criteria that assign identity based on associations between target and source variables resulting from deterministic dependencies. We show how these criteria are related to the identity axiom and to previously proposed redundancy measures, and we characterize how they lead to negative PID terms. This constitutes a further step to more explicitly address the role of information identity in the quantification of redundancy. The implications for studying neural coding are discussed. MDPI 2018-03-05 /pmc/articles/PMC7512685/ /pubmed/33265260 http://dx.doi.org/10.3390/e20030169 Text en © 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Chicharro, Daniel Pica, Giuseppe Panzeri, Stefano The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy |
title | The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy |
title_full | The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy |
title_fullStr | The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy |
title_full_unstemmed | The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy |
title_short | The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy |
title_sort | identity of information: how deterministic dependencies constrain information synergy and redundancy |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7512685/ https://www.ncbi.nlm.nih.gov/pubmed/33265260 http://dx.doi.org/10.3390/e20030169 |
work_keys_str_mv | AT chicharrodaniel theidentityofinformationhowdeterministicdependenciesconstraininformationsynergyandredundancy AT picagiuseppe theidentityofinformationhowdeterministicdependenciesconstraininformationsynergyandredundancy AT panzeristefano theidentityofinformationhowdeterministicdependenciesconstraininformationsynergyandredundancy AT chicharrodaniel identityofinformationhowdeterministicdependenciesconstraininformationsynergyandredundancy AT picagiuseppe identityofinformationhowdeterministicdependenciesconstraininformationsynergyandredundancy AT panzeristefano identityofinformationhowdeterministicdependenciesconstraininformationsynergyandredundancy |