Cargando…

The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy

Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables....

Descripción completa

Detalles Bibliográficos
Autores principales: Chicharro, Daniel, Pica, Giuseppe, Panzeri, Stefano
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2018
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7512685/
https://www.ncbi.nlm.nih.gov/pubmed/33265260
http://dx.doi.org/10.3390/e20030169
_version_ 1783586215150223360
author Chicharro, Daniel
Pica, Giuseppe
Panzeri, Stefano
author_facet Chicharro, Daniel
Pica, Giuseppe
Panzeri, Stefano
author_sort Chicharro, Daniel
collection PubMed
description Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (2010) proposed a partial information decomposition (PID) that separates the mutual information that a set of sources contains about a set of targets into nonnegative terms interpretable as these pieces. Quantifying redundancy requires assigning an identity to different information pieces, to assess when information is common across sources. Harder et al. (2013) proposed an identity axiom that imposes necessary conditions to quantify qualitatively common information. However, Bertschinger et al. (2012) showed that, in a counterexample with deterministic target-source dependencies, the identity axiom is incompatible with ensuring PID nonnegativity. Here, we study systematically the consequences of information identity criteria that assign identity based on associations between target and source variables resulting from deterministic dependencies. We show how these criteria are related to the identity axiom and to previously proposed redundancy measures, and we characterize how they lead to negative PID terms. This constitutes a further step to more explicitly address the role of information identity in the quantification of redundancy. The implications for studying neural coding are discussed.
format Online
Article
Text
id pubmed-7512685
institution National Center for Biotechnology Information
language English
publishDate 2018
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-75126852020-11-09 The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy Chicharro, Daniel Pica, Giuseppe Panzeri, Stefano Entropy (Basel) Article Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (2010) proposed a partial information decomposition (PID) that separates the mutual information that a set of sources contains about a set of targets into nonnegative terms interpretable as these pieces. Quantifying redundancy requires assigning an identity to different information pieces, to assess when information is common across sources. Harder et al. (2013) proposed an identity axiom that imposes necessary conditions to quantify qualitatively common information. However, Bertschinger et al. (2012) showed that, in a counterexample with deterministic target-source dependencies, the identity axiom is incompatible with ensuring PID nonnegativity. Here, we study systematically the consequences of information identity criteria that assign identity based on associations between target and source variables resulting from deterministic dependencies. We show how these criteria are related to the identity axiom and to previously proposed redundancy measures, and we characterize how they lead to negative PID terms. This constitutes a further step to more explicitly address the role of information identity in the quantification of redundancy. The implications for studying neural coding are discussed. MDPI 2018-03-05 /pmc/articles/PMC7512685/ /pubmed/33265260 http://dx.doi.org/10.3390/e20030169 Text en © 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Chicharro, Daniel
Pica, Giuseppe
Panzeri, Stefano
The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy
title The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy
title_full The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy
title_fullStr The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy
title_full_unstemmed The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy
title_short The Identity of Information: How Deterministic Dependencies Constrain Information Synergy and Redundancy
title_sort identity of information: how deterministic dependencies constrain information synergy and redundancy
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7512685/
https://www.ncbi.nlm.nih.gov/pubmed/33265260
http://dx.doi.org/10.3390/e20030169
work_keys_str_mv AT chicharrodaniel theidentityofinformationhowdeterministicdependenciesconstraininformationsynergyandredundancy
AT picagiuseppe theidentityofinformationhowdeterministicdependenciesconstraininformationsynergyandredundancy
AT panzeristefano theidentityofinformationhowdeterministicdependenciesconstraininformationsynergyandredundancy
AT chicharrodaniel identityofinformationhowdeterministicdependenciesconstraininformationsynergyandredundancy
AT picagiuseppe identityofinformationhowdeterministicdependenciesconstraininformationsynergyandredundancy
AT panzeristefano identityofinformationhowdeterministicdependenciesconstraininformationsynergyandredundancy