Empirical Estimation of Information Measures: A Literature Guide

We give a brief survey of the literature on the empirical estimation of entropy, differential entropy, relative entropy, mutual information and related information measures. While those quantities are of central importance in information theory, universal algorithms for their estimation are increasi...

Descripción completa

Detalles Bibliográficos
Autor principal: Verdú, Sergio
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2019
Materias:
Article
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7515235/
https://www.ncbi.nlm.nih.gov/pubmed/33267434
http://dx.doi.org/10.3390/e21080720
_version_ 1783586772902477824
author Verdú, Sergio
author_facet Verdú, Sergio
author_sort Verdú, Sergio
collection PubMed
description We give a brief survey of the literature on the empirical estimation of entropy, differential entropy, relative entropy, mutual information and related information measures. While those quantities are of central importance in information theory, universal algorithms for their estimation are increasingly important in data science, machine learning, biology, neuroscience, economics, language, and other experimental sciences.
format Online
Article
Text
id pubmed-7515235
institution National Center for Biotechnology Information
language English
publishDate 2019
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-75152352020-11-09 Empirical Estimation of Information Measures: A Literature Guide Verdú, Sergio Entropy (Basel) Article We give a brief survey of the literature on the empirical estimation of entropy, differential entropy, relative entropy, mutual information and related information measures. While those quantities are of central importance in information theory, universal algorithms for their estimation are increasingly important in data science, machine learning, biology, neuroscience, economics, language, and other experimental sciences. MDPI 2019-07-24 /pmc/articles/PMC7515235/ /pubmed/33267434 http://dx.doi.org/10.3390/e21080720 Text en © 2019 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Verdú, Sergio
Empirical Estimation of Information Measures: A Literature Guide
title Empirical Estimation of Information Measures: A Literature Guide
title_full Empirical Estimation of Information Measures: A Literature Guide
title_fullStr Empirical Estimation of Information Measures: A Literature Guide
title_full_unstemmed Empirical Estimation of Information Measures: A Literature Guide
title_short Empirical Estimation of Information Measures: A Literature Guide
title_sort empirical estimation of information measures: a literature guide
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7515235/
https://www.ncbi.nlm.nih.gov/pubmed/33267434
http://dx.doi.org/10.3390/e21080720
work_keys_str_mv AT verdusergio empiricalestimationofinformationmeasuresaliteratureguide

Ejemplares similares