Cargando…

Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory

Two Rényi-type generalizations of the Shannon cross-entropy, the Rényi cross-entropy and the Natural Rényi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial networks. In this work, we derive the Rényi and Natural Rényi differential cr...

Descripción completa

Detalles Bibliográficos
Autores principales: Thierrin, Ferenc Cole, Alajaji, Fady, Linder, Tamás
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9601846/
https://www.ncbi.nlm.nih.gov/pubmed/37420437
http://dx.doi.org/10.3390/e24101417
_version_ 1784817165867679744
author Thierrin, Ferenc Cole
Alajaji, Fady
Linder, Tamás
author_facet Thierrin, Ferenc Cole
Alajaji, Fady
Linder, Tamás
author_sort Thierrin, Ferenc Cole
collection PubMed
description Two Rényi-type generalizations of the Shannon cross-entropy, the Rényi cross-entropy and the Natural Rényi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial networks. In this work, we derive the Rényi and Natural Rényi differential cross-entropy measures in closed form for a wide class of common continuous distributions belonging to the exponential family, and we tabulate the results for ease of reference. We also summarise the Rényi-type cross-entropy rates between stationary Gaussian processes and between finite-alphabet time-invariant Markov sources.
format Online
Article
Text
id pubmed-9601846
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-96018462022-10-27 Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory Thierrin, Ferenc Cole Alajaji, Fady Linder, Tamás Entropy (Basel) Article Two Rényi-type generalizations of the Shannon cross-entropy, the Rényi cross-entropy and the Natural Rényi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial networks. In this work, we derive the Rényi and Natural Rényi differential cross-entropy measures in closed form for a wide class of common continuous distributions belonging to the exponential family, and we tabulate the results for ease of reference. We also summarise the Rényi-type cross-entropy rates between stationary Gaussian processes and between finite-alphabet time-invariant Markov sources. MDPI 2022-10-04 /pmc/articles/PMC9601846/ /pubmed/37420437 http://dx.doi.org/10.3390/e24101417 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Thierrin, Ferenc Cole
Alajaji, Fady
Linder, Tamás
Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory
title Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory
title_full Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory
title_fullStr Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory
title_full_unstemmed Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory
title_short Rényi Cross-Entropy Measures for Common Distributions and Processes with Memory
title_sort rényi cross-entropy measures for common distributions and processes with memory
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9601846/
https://www.ncbi.nlm.nih.gov/pubmed/37420437
http://dx.doi.org/10.3390/e24101417
work_keys_str_mv AT thierrinferenccole renyicrossentropymeasuresforcommondistributionsandprocesseswithmemory
AT alajajifady renyicrossentropymeasuresforcommondistributionsandprocesseswithmemory
AT lindertamas renyicrossentropymeasuresforcommondistributionsandprocesseswithmemory