Cargando…

Integrating geometries of ReLU feedforward neural networks

This paper investigates the integration of multiple geometries present within a ReLU-based neural network. A ReLU neural network determines a piecewise affine linear continuous map, M, from an input space ℝ(m) to an output space ℝ(n). The piecewise behavior corresponds to a polyhedral decomposition...

Descripción completa

Detalles Bibliográficos
Autores principales: Liu, Yajing, Caglar, Turgay, Peterson, Christopher, Kirby, Michael
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10682363/
https://www.ncbi.nlm.nih.gov/pubmed/38033354
http://dx.doi.org/10.3389/fdata.2023.1274831
_version_ 1785150958405156864
author Liu, Yajing
Caglar, Turgay
Peterson, Christopher
Kirby, Michael
author_facet Liu, Yajing
Caglar, Turgay
Peterson, Christopher
Kirby, Michael
author_sort Liu, Yajing
collection PubMed
description This paper investigates the integration of multiple geometries present within a ReLU-based neural network. A ReLU neural network determines a piecewise affine linear continuous map, M, from an input space ℝ(m) to an output space ℝ(n). The piecewise behavior corresponds to a polyhedral decomposition of ℝ(m). Each polyhedron in the decomposition can be labeled with a binary vector (whose length equals the number of ReLU nodes in the network) and with an affine linear function (which agrees with M when restricted to points in the polyhedron). We develop a toolbox that calculates the binary vector for a polyhedra containing a given data point with respect to a given ReLU FFNN. We utilize this binary vector to derive bounding facets for the corresponding polyhedron, extraction of “active” bits within the binary vector, enumeration of neighboring binary vectors, and visualization of the polyhedral decomposition (Python code is available at https://github.com/cglrtrgy/GoL_Toolbox). Polyhedra in the polyhedral decomposition of ℝ(m) are neighbors if they share a facet. Binary vectors for neighboring polyhedra differ in exactly 1 bit. Using the toolbox, we analyze the Hamming distance between the binary vectors for polyhedra containing points from adversarial/nonadversarial datasets revealing distinct geometric properties. A bisection method is employed to identify sample points with a Hamming distance of 1 along the shortest Euclidean distance path, facilitating the analysis of local geometric interplay between Euclidean geometry and the polyhedral decomposition along the path. Additionally, we study the distribution of Chebyshev centers and related radii across different polyhedra, shedding light on the polyhedral shape, size, clustering, and aiding in the understanding of decision boundaries.
format Online
Article
Text
id pubmed-10682363
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-106823632023-11-30 Integrating geometries of ReLU feedforward neural networks Liu, Yajing Caglar, Turgay Peterson, Christopher Kirby, Michael Front Big Data Big Data This paper investigates the integration of multiple geometries present within a ReLU-based neural network. A ReLU neural network determines a piecewise affine linear continuous map, M, from an input space ℝ(m) to an output space ℝ(n). The piecewise behavior corresponds to a polyhedral decomposition of ℝ(m). Each polyhedron in the decomposition can be labeled with a binary vector (whose length equals the number of ReLU nodes in the network) and with an affine linear function (which agrees with M when restricted to points in the polyhedron). We develop a toolbox that calculates the binary vector for a polyhedra containing a given data point with respect to a given ReLU FFNN. We utilize this binary vector to derive bounding facets for the corresponding polyhedron, extraction of “active” bits within the binary vector, enumeration of neighboring binary vectors, and visualization of the polyhedral decomposition (Python code is available at https://github.com/cglrtrgy/GoL_Toolbox). Polyhedra in the polyhedral decomposition of ℝ(m) are neighbors if they share a facet. Binary vectors for neighboring polyhedra differ in exactly 1 bit. Using the toolbox, we analyze the Hamming distance between the binary vectors for polyhedra containing points from adversarial/nonadversarial datasets revealing distinct geometric properties. A bisection method is employed to identify sample points with a Hamming distance of 1 along the shortest Euclidean distance path, facilitating the analysis of local geometric interplay between Euclidean geometry and the polyhedral decomposition along the path. Additionally, we study the distribution of Chebyshev centers and related radii across different polyhedra, shedding light on the polyhedral shape, size, clustering, and aiding in the understanding of decision boundaries. Frontiers Media S.A. 2023-11-14 /pmc/articles/PMC10682363/ /pubmed/38033354 http://dx.doi.org/10.3389/fdata.2023.1274831 Text en Copyright © 2023 Liu, Caglar, Peterson and Kirby. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Big Data
Liu, Yajing
Caglar, Turgay
Peterson, Christopher
Kirby, Michael
Integrating geometries of ReLU feedforward neural networks
title Integrating geometries of ReLU feedforward neural networks
title_full Integrating geometries of ReLU feedforward neural networks
title_fullStr Integrating geometries of ReLU feedforward neural networks
title_full_unstemmed Integrating geometries of ReLU feedforward neural networks
title_short Integrating geometries of ReLU feedforward neural networks
title_sort integrating geometries of relu feedforward neural networks
topic Big Data
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10682363/
https://www.ncbi.nlm.nih.gov/pubmed/38033354
http://dx.doi.org/10.3389/fdata.2023.1274831
work_keys_str_mv AT liuyajing integratinggeometriesofrelufeedforwardneuralnetworks
AT caglarturgay integratinggeometriesofrelufeedforwardneuralnetworks
AT petersonchristopher integratinggeometriesofrelufeedforwardneuralnetworks
AT kirbymichael integratinggeometriesofrelufeedforwardneuralnetworks