PCA Based Kernel Initialization for Convolutional Neural Networks
The initialization of Convolutional Neural Networks (CNNs) is about providing reasonable initial values for the convolution kernels and the fully connected layers. In this paper, we proposed a convolution kernel initialization method based on the two-dimensional principal component analysis (2DPCA),...
Autores principales: | , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7351687/ http://dx.doi.org/10.1007/978-981-15-7205-0_7 |
_version_ | 1783557488166043648 |
---|---|
author | Wang, Yifeng Rong, Yuxi Pan, Hongyue Liu, Ke Hu, Yang Wu, Fangmin Peng, Wei Xue, Xingsi Chen, Junfeng |
author_facet | Wang, Yifeng Rong, Yuxi Pan, Hongyue Liu, Ke Hu, Yang Wu, Fangmin Peng, Wei Xue, Xingsi Chen, Junfeng |
author_sort | Wang, Yifeng |
collection | PubMed |
description | The initialization of Convolutional Neural Networks (CNNs) is about providing reasonable initial values for the convolution kernels and the fully connected layers. In this paper, we proposed a convolution kernel initialization method based on the two-dimensional principal component analysis (2DPCA), in which a parametric equalization normalization method is used to adjust the scale between each neuron weight. After that the weight initial value can be adaptively adjusted according to different data samples. This method enables each neuron to fully back-propagate errors and accelerate network model training. Finally, a network model was built and experiments were performed using Tanh and ReLU activation functions. The experimental results verify the effectiveness of the proposed method through the distribution of histograms and the curve comparison diagrams of model training. |
format | Online Article Text |
id | pubmed-7351687 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
record_format | MEDLINE/PubMed |
spelling | pubmed-73516872020-07-13 PCA Based Kernel Initialization for Convolutional Neural Networks Wang, Yifeng Rong, Yuxi Pan, Hongyue Liu, Ke Hu, Yang Wu, Fangmin Peng, Wei Xue, Xingsi Chen, Junfeng Data Mining and Big Data Article The initialization of Convolutional Neural Networks (CNNs) is about providing reasonable initial values for the convolution kernels and the fully connected layers. In this paper, we proposed a convolution kernel initialization method based on the two-dimensional principal component analysis (2DPCA), in which a parametric equalization normalization method is used to adjust the scale between each neuron weight. After that the weight initial value can be adaptively adjusted according to different data samples. This method enables each neuron to fully back-propagate errors and accelerate network model training. Finally, a network model was built and experiments were performed using Tanh and ReLU activation functions. The experimental results verify the effectiveness of the proposed method through the distribution of histograms and the curve comparison diagrams of model training. 2020-07-11 /pmc/articles/PMC7351687/ http://dx.doi.org/10.1007/978-981-15-7205-0_7 Text en © Springer Nature Singapore Pte Ltd. 2020 This article is made available via the PMC Open Access Subset for unrestricted research re-use and secondary analysis in any form or by any means with acknowledgement of the original source. These permissions are granted for the duration of the World Health Organization (WHO) declaration of COVID-19 as a global pandemic. |
spellingShingle | Article Wang, Yifeng Rong, Yuxi Pan, Hongyue Liu, Ke Hu, Yang Wu, Fangmin Peng, Wei Xue, Xingsi Chen, Junfeng PCA Based Kernel Initialization for Convolutional Neural Networks |
title | PCA Based Kernel Initialization for Convolutional Neural Networks |
title_full | PCA Based Kernel Initialization for Convolutional Neural Networks |
title_fullStr | PCA Based Kernel Initialization for Convolutional Neural Networks |
title_full_unstemmed | PCA Based Kernel Initialization for Convolutional Neural Networks |
title_short | PCA Based Kernel Initialization for Convolutional Neural Networks |
title_sort | pca based kernel initialization for convolutional neural networks |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7351687/ http://dx.doi.org/10.1007/978-981-15-7205-0_7 |
work_keys_str_mv | AT wangyifeng pcabasedkernelinitializationforconvolutionalneuralnetworks AT rongyuxi pcabasedkernelinitializationforconvolutionalneuralnetworks AT panhongyue pcabasedkernelinitializationforconvolutionalneuralnetworks AT liuke pcabasedkernelinitializationforconvolutionalneuralnetworks AT huyang pcabasedkernelinitializationforconvolutionalneuralnetworks AT wufangmin pcabasedkernelinitializationforconvolutionalneuralnetworks AT pengwei pcabasedkernelinitializationforconvolutionalneuralnetworks AT xuexingsi pcabasedkernelinitializationforconvolutionalneuralnetworks AT chenjunfeng pcabasedkernelinitializationforconvolutionalneuralnetworks |