Cargando…
A Geometric Interpretation of Stochastic Gradient Descent Using Diffusion Metrics
This paper is a step towards developing a geometric understanding of a popular algorithm for training deep neural networks named stochastic gradient descent (SGD). We built upon a recent result which observed that the noise in SGD while training typical networks is highly non-isotropic. That motivat...
| Autores principales: | , , |
|---|---|
| Formato: | Online Artículo Texto |
| Lenguaje: | English |
| Publicado: |
MDPI
2020
|
| Materias: | |
| Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7516401/ https://www.ncbi.nlm.nih.gov/pubmed/33285876 http://dx.doi.org/10.3390/e22010101 |