PCA Based Kernel Initialization for Convolutional Neural Networks

The initialization of Convolutional Neural Networks (CNNs) is about providing reasonable initial values for the convolution kernels and the fully connected layers. In this paper, we proposed a convolution kernel initialization method based on the two-dimensional principal component analysis (2DPCA),...

Descripción completa

Detalles Bibliográficos
Autores principales: Wang, Yifeng, Rong, Yuxi, Pan, Hongyue, Liu, Ke, Hu, Yang, Wu, Fangmin, Peng, Wei, Xue, Xingsi, Chen, Junfeng
Formato: Online Artículo Texto
Lenguaje:English
Publicado: 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7351687/
http://dx.doi.org/10.1007/978-981-15-7205-0_7
_version_ 1783557488166043648
author Wang, Yifeng
Rong, Yuxi
Pan, Hongyue
Liu, Ke
Hu, Yang
Wu, Fangmin
Peng, Wei
Xue, Xingsi
Chen, Junfeng
author_facet Wang, Yifeng
Rong, Yuxi
Pan, Hongyue
Liu, Ke
Hu, Yang
Wu, Fangmin
Peng, Wei
Xue, Xingsi
Chen, Junfeng
author_sort Wang, Yifeng
collection PubMed
description The initialization of Convolutional Neural Networks (CNNs) is about providing reasonable initial values for the convolution kernels and the fully connected layers. In this paper, we proposed a convolution kernel initialization method based on the two-dimensional principal component analysis (2DPCA), in which a parametric equalization normalization method is used to adjust the scale between each neuron weight. After that the weight initial value can be adaptively adjusted according to different data samples. This method enables each neuron to fully back-propagate errors and accelerate network model training. Finally, a network model was built and experiments were performed using Tanh and ReLU activation functions. The experimental results verify the effectiveness of the proposed method through the distribution of histograms and the curve comparison diagrams of model training.
format Online
Article
Text
id pubmed-7351687
institution National Center for Biotechnology Information
language English
publishDate 2020
record_format MEDLINE/PubMed
spelling pubmed-73516872020-07-13 PCA Based Kernel Initialization for Convolutional Neural Networks Wang, Yifeng Rong, Yuxi Pan, Hongyue Liu, Ke Hu, Yang Wu, Fangmin Peng, Wei Xue, Xingsi Chen, Junfeng Data Mining and Big Data Article The initialization of Convolutional Neural Networks (CNNs) is about providing reasonable initial values for the convolution kernels and the fully connected layers. In this paper, we proposed a convolution kernel initialization method based on the two-dimensional principal component analysis (2DPCA), in which a parametric equalization normalization method is used to adjust the scale between each neuron weight. After that the weight initial value can be adaptively adjusted according to different data samples. This method enables each neuron to fully back-propagate errors and accelerate network model training. Finally, a network model was built and experiments were performed using Tanh and ReLU activation functions. The experimental results verify the effectiveness of the proposed method through the distribution of histograms and the curve comparison diagrams of model training. 2020-07-11 /pmc/articles/PMC7351687/ http://dx.doi.org/10.1007/978-981-15-7205-0_7 Text en © Springer Nature Singapore Pte Ltd. 2020 This article is made available via the PMC Open Access Subset for unrestricted research re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic.
spellingShingle Article
Wang, Yifeng
Rong, Yuxi
Pan, Hongyue
Liu, Ke
Hu, Yang
Wu, Fangmin
Peng, Wei
Xue, Xingsi
Chen, Junfeng
PCA Based Kernel Initialization for Convolutional Neural Networks
title PCA Based Kernel Initialization for Convolutional Neural Networks
title_full PCA Based Kernel Initialization for Convolutional Neural Networks
title_fullStr PCA Based Kernel Initialization for Convolutional Neural Networks
title_full_unstemmed PCA Based Kernel Initialization for Convolutional Neural Networks
title_short PCA Based Kernel Initialization for Convolutional Neural Networks
title_sort pca based kernel initialization for convolutional neural networks
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7351687/
http://dx.doi.org/10.1007/978-981-15-7205-0_7
work_keys_str_mv AT wangyifeng pcabasedkernelinitializationforconvolutionalneuralnetworks
AT rongyuxi pcabasedkernelinitializationforconvolutionalneuralnetworks
AT panhongyue pcabasedkernelinitializationforconvolutionalneuralnetworks
AT liuke pcabasedkernelinitializationforconvolutionalneuralnetworks
AT huyang pcabasedkernelinitializationforconvolutionalneuralnetworks
AT wufangmin pcabasedkernelinitializationforconvolutionalneuralnetworks
AT pengwei pcabasedkernelinitializationforconvolutionalneuralnetworks
AT xuexingsi pcabasedkernelinitializationforconvolutionalneuralnetworks
AT chenjunfeng pcabasedkernelinitializationforconvolutionalneuralnetworks