Cargando…

Optimization of neural networks via finite-value quantum fluctuations

We numerically test an optimization method for deep neural networks (DNNs) using quantum fluctuations inspired by quantum annealing. For efficient optimization, our method utilizes the quantum tunneling effect beyond the potential barriers. The path integral formulation of the DNN optimization gener...

Descripción completa

Detalles Bibliográficos
Autores principales: Ohzeki, Masayuki, Okada, Shuntaro, Terabe, Masayoshi, Taguchi, Shinichiro
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Nature Publishing Group UK 2018
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6028692/
https://www.ncbi.nlm.nih.gov/pubmed/29967442
http://dx.doi.org/10.1038/s41598-018-28212-4
_version_ 1783336820059144192
author Ohzeki, Masayuki
Okada, Shuntaro
Terabe, Masayoshi
Taguchi, Shinichiro
author_facet Ohzeki, Masayuki
Okada, Shuntaro
Terabe, Masayoshi
Taguchi, Shinichiro
author_sort Ohzeki, Masayuki
collection PubMed
description We numerically test an optimization method for deep neural networks (DNNs) using quantum fluctuations inspired by quantum annealing. For efficient optimization, our method utilizes the quantum tunneling effect beyond the potential barriers. The path integral formulation of the DNN optimization generates an attracting force to simulate the quantum tunneling effect. In the standard quantum annealing method, the quantum fluctuations will vanish at the last stage of optimization. In this study, we propose a learning protocol that utilizes a finite value for quantum fluctuations strength to obtain higher generalization performance, which is a type of robustness. We demonstrate the performance of our method using two well-known open datasets: the MNIST dataset and the Olivetti face dataset. Although computational costs prevent us from testing our method on large datasets with high-dimensional data, results show that our method can enhance generalization performance by induction of the finite value for quantum fluctuations.
format Online
Article
Text
id pubmed-6028692
institution National Center for Biotechnology Information
language English
publishDate 2018
publisher Nature Publishing Group UK
record_format MEDLINE/PubMed
spelling pubmed-60286922018-07-09 Optimization of neural networks via finite-value quantum fluctuations Ohzeki, Masayuki Okada, Shuntaro Terabe, Masayoshi Taguchi, Shinichiro Sci Rep Article We numerically test an optimization method for deep neural networks (DNNs) using quantum fluctuations inspired by quantum annealing. For efficient optimization, our method utilizes the quantum tunneling effect beyond the potential barriers. The path integral formulation of the DNN optimization generates an attracting force to simulate the quantum tunneling effect. In the standard quantum annealing method, the quantum fluctuations will vanish at the last stage of optimization. In this study, we propose a learning protocol that utilizes a finite value for quantum fluctuations strength to obtain higher generalization performance, which is a type of robustness. We demonstrate the performance of our method using two well-known open datasets: the MNIST dataset and the Olivetti face dataset. Although computational costs prevent us from testing our method on large datasets with high-dimensional data, results show that our method can enhance generalization performance by induction of the finite value for quantum fluctuations. Nature Publishing Group UK 2018-07-02 /pmc/articles/PMC6028692/ /pubmed/29967442 http://dx.doi.org/10.1038/s41598-018-28212-4 Text en © The Author(s) 2018 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
spellingShingle Article
Ohzeki, Masayuki
Okada, Shuntaro
Terabe, Masayoshi
Taguchi, Shinichiro
Optimization of neural networks via finite-value quantum fluctuations
title Optimization of neural networks via finite-value quantum fluctuations
title_full Optimization of neural networks via finite-value quantum fluctuations
title_fullStr Optimization of neural networks via finite-value quantum fluctuations
title_full_unstemmed Optimization of neural networks via finite-value quantum fluctuations
title_short Optimization of neural networks via finite-value quantum fluctuations
title_sort optimization of neural networks via finite-value quantum fluctuations
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6028692/
https://www.ncbi.nlm.nih.gov/pubmed/29967442
http://dx.doi.org/10.1038/s41598-018-28212-4
work_keys_str_mv AT ohzekimasayuki optimizationofneuralnetworksviafinitevaluequantumfluctuations
AT okadashuntaro optimizationofneuralnetworksviafinitevaluequantumfluctuations
AT terabemasayoshi optimizationofneuralnetworksviafinitevaluequantumfluctuations
AT taguchishinichiro optimizationofneuralnetworksviafinitevaluequantumfluctuations