Cargando…

Probability Mass Exclusions and the Directed Components of Mutual Information

Information is often described as a reduction of uncertainty associated with a restriction of possible choices. Despite appearing in Hartley’s foundational work on information theory, there is a surprising lack of a formal treatment of this interpretation in terms of exclusions. This paper addresses...

Descripción completa

Detalles Bibliográficos
Autores principales: Finn, Conor, Lizier, Joseph T.
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2018
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7512388/
https://www.ncbi.nlm.nih.gov/pubmed/33266550
http://dx.doi.org/10.3390/e20110826
_version_ 1783586146344763392
author Finn, Conor
Lizier, Joseph T.
author_facet Finn, Conor
Lizier, Joseph T.
author_sort Finn, Conor
collection PubMed
description Information is often described as a reduction of uncertainty associated with a restriction of possible choices. Despite appearing in Hartley’s foundational work on information theory, there is a surprising lack of a formal treatment of this interpretation in terms of exclusions. This paper addresses the gap by providing an explicit characterisation of information in terms of probability mass exclusions. It then demonstrates that different exclusions can yield the same amount of information and discusses the insight this provides about how information is shared amongst random variables—lack of progress in this area is a key barrier preventing us from understanding how information is distributed in complex systems. The paper closes by deriving a decomposition of the mutual information which can distinguish between differing exclusions; this provides surprising insight into the nature of directed information.
format Online
Article
Text
id pubmed-7512388
institution National Center for Biotechnology Information
language English
publishDate 2018
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-75123882020-11-09 Probability Mass Exclusions and the Directed Components of Mutual Information Finn, Conor Lizier, Joseph T. Entropy (Basel) Article Information is often described as a reduction of uncertainty associated with a restriction of possible choices. Despite appearing in Hartley’s foundational work on information theory, there is a surprising lack of a formal treatment of this interpretation in terms of exclusions. This paper addresses the gap by providing an explicit characterisation of information in terms of probability mass exclusions. It then demonstrates that different exclusions can yield the same amount of information and discusses the insight this provides about how information is shared amongst random variables—lack of progress in this area is a key barrier preventing us from understanding how information is distributed in complex systems. The paper closes by deriving a decomposition of the mutual information which can distinguish between differing exclusions; this provides surprising insight into the nature of directed information. MDPI 2018-10-28 /pmc/articles/PMC7512388/ /pubmed/33266550 http://dx.doi.org/10.3390/e20110826 Text en © 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Finn, Conor
Lizier, Joseph T.
Probability Mass Exclusions and the Directed Components of Mutual Information
title Probability Mass Exclusions and the Directed Components of Mutual Information
title_full Probability Mass Exclusions and the Directed Components of Mutual Information
title_fullStr Probability Mass Exclusions and the Directed Components of Mutual Information
title_full_unstemmed Probability Mass Exclusions and the Directed Components of Mutual Information
title_short Probability Mass Exclusions and the Directed Components of Mutual Information
title_sort probability mass exclusions and the directed components of mutual information
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7512388/
https://www.ncbi.nlm.nih.gov/pubmed/33266550
http://dx.doi.org/10.3390/e20110826
work_keys_str_mv AT finnconor probabilitymassexclusionsandthedirectedcomponentsofmutualinformation
AT lizierjosepht probabilitymassexclusionsandthedirectedcomponentsofmutualinformation