Cargando…
Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression
This paper provides tight bounds on the Rényi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one to one. To that end, a tight lower bound on the Rényi entropy of a discrete random variable with a finite support is der...
Autor principal: | |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2018
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7512480/ https://www.ncbi.nlm.nih.gov/pubmed/33266620 http://dx.doi.org/10.3390/e20120896 |
_version_ | 1783586167962206208 |
---|---|
author | Sason, Igal |
author_facet | Sason, Igal |
author_sort | Sason, Igal |
collection | PubMed |
description | This paper provides tight bounds on the Rényi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one to one. To that end, a tight lower bound on the Rényi entropy of a discrete random variable with a finite support is derived as a function of the size of the support, and the ratio of the maximal to minimal probability masses. This work was inspired by the recently published paper by Cicalese et al., which is focused on the Shannon entropy, and it strengthens and generalizes the results of that paper to Rényi entropies of arbitrary positive orders. In view of these generalized bounds and the works by Arikan and Campbell, non-asymptotic bounds are derived for guessing moments and lossless data compression of discrete memoryless sources. |
format | Online Article Text |
id | pubmed-7512480 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2018 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-75124802020-11-09 Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression Sason, Igal Entropy (Basel) Article This paper provides tight bounds on the Rényi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one to one. To that end, a tight lower bound on the Rényi entropy of a discrete random variable with a finite support is derived as a function of the size of the support, and the ratio of the maximal to minimal probability masses. This work was inspired by the recently published paper by Cicalese et al., which is focused on the Shannon entropy, and it strengthens and generalizes the results of that paper to Rényi entropies of arbitrary positive orders. In view of these generalized bounds and the works by Arikan and Campbell, non-asymptotic bounds are derived for guessing moments and lossless data compression of discrete memoryless sources. MDPI 2018-11-22 /pmc/articles/PMC7512480/ /pubmed/33266620 http://dx.doi.org/10.3390/e20120896 Text en © 2018 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Sason, Igal Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression |
title | Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression |
title_full | Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression |
title_fullStr | Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression |
title_full_unstemmed | Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression |
title_short | Tight Bounds on the Rényi Entropy via Majorization with Applications to Guessing and Compression |
title_sort | tight bounds on the rényi entropy via majorization with applications to guessing and compression |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7512480/ https://www.ncbi.nlm.nih.gov/pubmed/33266620 http://dx.doi.org/10.3390/e20120896 |
work_keys_str_mv | AT sasonigal tightboundsontherenyientropyviamajorizationwithapplicationstoguessingandcompression |