A Geometric Interpretation of Stochastic Gradient Descent Using Diffusion Metrics

This paper is a step towards developing a geometric understanding of a popular algorithm for training deep neural networks named stochastic gradient descent (SGD). We built upon a recent result which observed that the noise in SGD while training typical networks is highly non-isotropic. That motivat...

Descripción completa

Detalles Bibliográficos
Autores principales: Fioresi, Rita, Chaudhari, Pratik, Soatto, Stefano
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2020
Materias:
Article
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7516401/
https://www.ncbi.nlm.nih.gov/pubmed/33285876
http://dx.doi.org/10.3390/e22010101

Internet

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7516401/
https://www.ncbi.nlm.nih.gov/pubmed/33285876
http://dx.doi.org/10.3390/e22010101

Ejemplares similares