Cargando…

Role of Layers and Neurons in Deep Learning With the Rectified Linear Unit

Deep learning is used to classify data into several groups based on nonlinear curved surfaces. In this paper, we focus on the theoretical analysis of deep learning using the rectified linear unit (ReLU) activation function. Because layers approximate a nonlinear curved surface, increasing the number...

Descripción completa

Detalles Bibliográficos
Autores principales: Takekawa, Akira, Kajiura, Masayuki, Fukuda, Hiroya
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Cureus 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8601259/
https://www.ncbi.nlm.nih.gov/pubmed/34820210
http://dx.doi.org/10.7759/cureus.18866
Descripción
Sumario:Deep learning is used to classify data into several groups based on nonlinear curved surfaces. In this paper, we focus on the theoretical analysis of deep learning using the rectified linear unit (ReLU) activation function. Because layers approximate a nonlinear curved surface, increasing the number of layers improves the approximation accuracy of the curved surface. While neurons perform a layer-by-layer approximation of the most appropriate hyperplanes, increasing their number cannot improve the results obtained via canonical correlation analysis (CCA). These results illustrate the functions of layers and neurons in deep learning with ReLU.