Mutual Information Based Learning Rate Decay for Stochastic Gradient Descent Training of Deep Neural Networks

This paper demonstrates a novel approach to training deep neural networks using a Mutual Information (MI)-driven, decaying Learning Rate (LR), Stochastic Gradient Descent (SGD) algorithm. MI between the output of the neural network and true outcomes is used to adaptively set the LR for the network,...

Descripción completa

Detalles Bibliográficos
Autor principal: Vasudevan, Shrihari
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2020
Materias:
Article
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7517082/
https://www.ncbi.nlm.nih.gov/pubmed/33286332
http://dx.doi.org/10.3390/e22050560

Internet

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7517082/
https://www.ncbi.nlm.nih.gov/pubmed/33286332
http://dx.doi.org/10.3390/e22050560

Ejemplares similares