Cargando…
Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations
Although double-precision floating-point arithmetic currently dominates high-performance computing, there is increasing interest in smaller and simpler arithmetic types. The main reasons are potential improvements in energy efficiency and memory footprint and bandwidth. However, simply switching to...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
The Royal Society Publishing
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7015297/ https://www.ncbi.nlm.nih.gov/pubmed/31955687 http://dx.doi.org/10.1098/rsta.2019.0052 |
_version_ | 1783496781551632384 |
---|---|
author | Hopkins, Michael Mikaitis, Mantas Lester, Dave R. Furber, Steve |
author_facet | Hopkins, Michael Mikaitis, Mantas Lester, Dave R. Furber, Steve |
author_sort | Hopkins, Michael |
collection | PubMed |
description | Although double-precision floating-point arithmetic currently dominates high-performance computing, there is increasing interest in smaller and simpler arithmetic types. The main reasons are potential improvements in energy efficiency and memory footprint and bandwidth. However, simply switching to lower-precision types typically results in increased numerical errors. We investigate approaches to improving the accuracy of reduced-precision fixed-point arithmetic types, using examples in an important domain for numerical computation in neuroscience: the solution of ordinary differential equations (ODEs). The Izhikevich neuron model is used to demonstrate that rounding has an important role in producing accurate spike timings from explicit ODE solution algorithms. In particular, fixed-point arithmetic with stochastic rounding consistently results in smaller errors compared to single-precision floating-point and fixed-point arithmetic with round-to-nearest across a range of neuron behaviours and ODE solvers. A computationally much cheaper alternative is also investigated, inspired by the concept of dither that is a widely understood mechanism for providing resolution below the least significant bit in digital signal processing. These results will have implications for the solution of ODEs in other subject areas, and should also be directly relevant to the huge range of practical problems that are represented by partial differential equations. This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’. |
format | Online Article Text |
id | pubmed-7015297 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | The Royal Society Publishing |
record_format | MEDLINE/PubMed |
spelling | pubmed-70152972020-02-18 Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations Hopkins, Michael Mikaitis, Mantas Lester, Dave R. Furber, Steve Philos Trans A Math Phys Eng Sci Articles Although double-precision floating-point arithmetic currently dominates high-performance computing, there is increasing interest in smaller and simpler arithmetic types. The main reasons are potential improvements in energy efficiency and memory footprint and bandwidth. However, simply switching to lower-precision types typically results in increased numerical errors. We investigate approaches to improving the accuracy of reduced-precision fixed-point arithmetic types, using examples in an important domain for numerical computation in neuroscience: the solution of ordinary differential equations (ODEs). The Izhikevich neuron model is used to demonstrate that rounding has an important role in producing accurate spike timings from explicit ODE solution algorithms. In particular, fixed-point arithmetic with stochastic rounding consistently results in smaller errors compared to single-precision floating-point and fixed-point arithmetic with round-to-nearest across a range of neuron behaviours and ODE solvers. A computationally much cheaper alternative is also investigated, inspired by the concept of dither that is a widely understood mechanism for providing resolution below the least significant bit in digital signal processing. These results will have implications for the solution of ODEs in other subject areas, and should also be directly relevant to the huge range of practical problems that are represented by partial differential equations. This article is part of a discussion meeting issue ‘Numerical algorithms for high-performance computational science’. The Royal Society Publishing 2020-03-06 2020-01-20 /pmc/articles/PMC7015297/ /pubmed/31955687 http://dx.doi.org/10.1098/rsta.2019.0052 Text en © 2020 The Authors. http://creativecommons.org/licenses/by/4.0/ Published by the Royal Society under the terms of the Creative Commons Attribution License http://creativecommons.org/licenses/by/4.0/, which permits unrestricted use, provided the original author and source are credited. |
spellingShingle | Articles Hopkins, Michael Mikaitis, Mantas Lester, Dave R. Furber, Steve Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations |
title | Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations |
title_full | Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations |
title_fullStr | Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations |
title_full_unstemmed | Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations |
title_short | Stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations |
title_sort | stochastic rounding and reduced-precision fixed-point arithmetic for solving neural ordinary differential equations |
topic | Articles |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7015297/ https://www.ncbi.nlm.nih.gov/pubmed/31955687 http://dx.doi.org/10.1098/rsta.2019.0052 |
work_keys_str_mv | AT hopkinsmichael stochasticroundingandreducedprecisionfixedpointarithmeticforsolvingneuralordinarydifferentialequations AT mikaitismantas stochasticroundingandreducedprecisionfixedpointarithmeticforsolvingneuralordinarydifferentialequations AT lesterdaver stochasticroundingandreducedprecisionfixedpointarithmeticforsolvingneuralordinarydifferentialequations AT furbersteve stochasticroundingandreducedprecisionfixedpointarithmeticforsolvingneuralordinarydifferentialequations |