Cargando…
Nonideality‐Aware Training for Accurate and Robust Low‐Power Memristive Neural Networks
Recent years have seen a rapid rise of artificial neural networks being employed in a number of cognitive tasks. The ever‐increasing computing requirements of these structures have contributed to a desire for novel technologies and paradigms, including memristor‐based hardware accelerators. Solution...
Autores principales: | , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
John Wiley and Sons Inc.
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9189678/ https://www.ncbi.nlm.nih.gov/pubmed/35508766 http://dx.doi.org/10.1002/advs.202105784 |
_version_ | 1784725640207925248 |
---|---|
author | Joksas, Dovydas Wang, Erwei Barmpatsalos, Nikolaos Ng, Wing H. Kenyon, Anthony J. Constantinides, George A. Mehonic, Adnan |
author_facet | Joksas, Dovydas Wang, Erwei Barmpatsalos, Nikolaos Ng, Wing H. Kenyon, Anthony J. Constantinides, George A. Mehonic, Adnan |
author_sort | Joksas, Dovydas |
collection | PubMed |
description | Recent years have seen a rapid rise of artificial neural networks being employed in a number of cognitive tasks. The ever‐increasing computing requirements of these structures have contributed to a desire for novel technologies and paradigms, including memristor‐based hardware accelerators. Solutions based on memristive crossbars and analog data processing promise to improve the overall energy efficiency. However, memristor nonidealities can lead to the degradation of neural network accuracy, while the attempts to mitigate these negative effects often introduce design trade‐offs, such as those between power and reliability. In this work, authors design nonideality‐aware training of memristor‐based neural networks capable of dealing with the most common device nonidealities. The feasibility of using high‐resistance devices that exhibit high I‐V nonlinearity is demonstrated—by analyzing experimental data and employing nonideality‐aware training, it is estimated that the energy efficiency of memristive vector‐matrix multipliers is improved by almost three orders of magnitude (0.715 TOPs(−1)W(−1) to 381 TOPs(−1)W(−1)) while maintaining similar accuracy. It is shown that associating the parameters of neural networks with individual memristors allows to bias these devices toward less conductive states through regularization of the corresponding optimization problem, while modifying the validation procedure leads to more reliable estimates of performance. The authors demonstrate the universality and robustness of this approach when dealing with a wide range of nonidealities. |
format | Online Article Text |
id | pubmed-9189678 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | John Wiley and Sons Inc. |
record_format | MEDLINE/PubMed |
spelling | pubmed-91896782022-06-16 Nonideality‐Aware Training for Accurate and Robust Low‐Power Memristive Neural Networks Joksas, Dovydas Wang, Erwei Barmpatsalos, Nikolaos Ng, Wing H. Kenyon, Anthony J. Constantinides, George A. Mehonic, Adnan Adv Sci (Weinh) Research Articles Recent years have seen a rapid rise of artificial neural networks being employed in a number of cognitive tasks. The ever‐increasing computing requirements of these structures have contributed to a desire for novel technologies and paradigms, including memristor‐based hardware accelerators. Solutions based on memristive crossbars and analog data processing promise to improve the overall energy efficiency. However, memristor nonidealities can lead to the degradation of neural network accuracy, while the attempts to mitigate these negative effects often introduce design trade‐offs, such as those between power and reliability. In this work, authors design nonideality‐aware training of memristor‐based neural networks capable of dealing with the most common device nonidealities. The feasibility of using high‐resistance devices that exhibit high I‐V nonlinearity is demonstrated—by analyzing experimental data and employing nonideality‐aware training, it is estimated that the energy efficiency of memristive vector‐matrix multipliers is improved by almost three orders of magnitude (0.715 TOPs(−1)W(−1) to 381 TOPs(−1)W(−1)) while maintaining similar accuracy. It is shown that associating the parameters of neural networks with individual memristors allows to bias these devices toward less conductive states through regularization of the corresponding optimization problem, while modifying the validation procedure leads to more reliable estimates of performance. The authors demonstrate the universality and robustness of this approach when dealing with a wide range of nonidealities. John Wiley and Sons Inc. 2022-05-04 /pmc/articles/PMC9189678/ /pubmed/35508766 http://dx.doi.org/10.1002/advs.202105784 Text en © 2022 The Authors. Advanced Science published by Wiley‐VCH GmbH https://creativecommons.org/licenses/by/4.0/This is an open access article under the terms of the http://creativecommons.org/licenses/by/4.0/ (https://creativecommons.org/licenses/by/4.0/) License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. |
spellingShingle | Research Articles Joksas, Dovydas Wang, Erwei Barmpatsalos, Nikolaos Ng, Wing H. Kenyon, Anthony J. Constantinides, George A. Mehonic, Adnan Nonideality‐Aware Training for Accurate and Robust Low‐Power Memristive Neural Networks |
title | Nonideality‐Aware Training for Accurate and Robust Low‐Power Memristive Neural Networks |
title_full | Nonideality‐Aware Training for Accurate and Robust Low‐Power Memristive Neural Networks |
title_fullStr | Nonideality‐Aware Training for Accurate and Robust Low‐Power Memristive Neural Networks |
title_full_unstemmed | Nonideality‐Aware Training for Accurate and Robust Low‐Power Memristive Neural Networks |
title_short | Nonideality‐Aware Training for Accurate and Robust Low‐Power Memristive Neural Networks |
title_sort | nonideality‐aware training for accurate and robust low‐power memristive neural networks |
topic | Research Articles |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9189678/ https://www.ncbi.nlm.nih.gov/pubmed/35508766 http://dx.doi.org/10.1002/advs.202105784 |
work_keys_str_mv | AT joksasdovydas nonidealityawaretrainingforaccurateandrobustlowpowermemristiveneuralnetworks AT wangerwei nonidealityawaretrainingforaccurateandrobustlowpowermemristiveneuralnetworks AT barmpatsalosnikolaos nonidealityawaretrainingforaccurateandrobustlowpowermemristiveneuralnetworks AT ngwingh nonidealityawaretrainingforaccurateandrobustlowpowermemristiveneuralnetworks AT kenyonanthonyj nonidealityawaretrainingforaccurateandrobustlowpowermemristiveneuralnetworks AT constantinidesgeorgea nonidealityawaretrainingforaccurateandrobustlowpowermemristiveneuralnetworks AT mehonicadnan nonidealityawaretrainingforaccurateandrobustlowpowermemristiveneuralnetworks |