C++ Code Generation for Fast Inference of Deep Learning Models in ROOT/TMVA

Mostrar otras versiones (2)

We report the latest development in ROOT/TMVA, a new tool that takes trained ONNX deep learning models and emits C++ code that can be easily included and invoked for fast inference of the model, with minimal dependency. An introduction to SOFIE (System for Optimized Fast Inference code Emit) is pres...

Descripción completa

Detalles Bibliográficos
Autores principales: An, Sitong, Moneta, Lorenzo, Sengupta, Sanjiban, Hamdan, Ahmat, Sossai, Federico, Saxena, Aaradhya
Lenguaje:eng
Publicado: 2023
Acceso en línea:https://dx.doi.org/10.1088/1742-6596/2438/1/012013
http://cds.cern.ch/record/2862109
_version_ 1780977856954761216
author An, Sitong
Moneta, Lorenzo
Sengupta, Sanjiban
Hamdan, Ahmat
Sossai, Federico
Saxena, Aaradhya
author_facet An, Sitong
Moneta, Lorenzo
Sengupta, Sanjiban
Hamdan, Ahmat
Sossai, Federico
Saxena, Aaradhya
author_sort An, Sitong
collection CERN
description We report the latest development in ROOT/TMVA, a new tool that takes trained ONNX deep learning models and emits C++ code that can be easily included and invoked for fast inference of the model, with minimal dependency. An introduction to SOFIE (System for Optimized Fast Inference code Emit) is presented, with examples of interface and generated code. We discuss the latest expanded support of a variety of neural network operators, including convolutional and recurrent layers, as well as the integration with RDataFrame. We demonstrate the latest performance of this framework with a set of benchmarks.
id cern-2862109
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2023
record_format invenio
spelling cern-28621092023-06-16T19:24:23Zdoi:10.1088/1742-6596/2438/1/012013http://cds.cern.ch/record/2862109engAn, SitongMoneta, LorenzoSengupta, SanjibanHamdan, AhmatSossai, FedericoSaxena, AaradhyaC++ Code Generation for Fast Inference of Deep Learning Models in ROOT/TMVAWe report the latest development in ROOT/TMVA, a new tool that takes trained ONNX deep learning models and emits C++ code that can be easily included and invoked for fast inference of the model, with minimal dependency. An introduction to SOFIE (System for Optimized Fast Inference code Emit) is presented, with examples of interface and generated code. We discuss the latest expanded support of a variety of neural network operators, including convolutional and recurrent layers, as well as the integration with RDataFrame. We demonstrate the latest performance of this framework with a set of benchmarks.oai:cds.cern.ch:28621092023
spellingShingle An, Sitong
Moneta, Lorenzo
Sengupta, Sanjiban
Hamdan, Ahmat
Sossai, Federico
Saxena, Aaradhya
C++ Code Generation for Fast Inference of Deep Learning Models in ROOT/TMVA
title C++ Code Generation for Fast Inference of Deep Learning Models in ROOT/TMVA
title_full C++ Code Generation for Fast Inference of Deep Learning Models in ROOT/TMVA
title_fullStr C++ Code Generation for Fast Inference of Deep Learning Models in ROOT/TMVA
title_full_unstemmed C++ Code Generation for Fast Inference of Deep Learning Models in ROOT/TMVA
title_short C++ Code Generation for Fast Inference of Deep Learning Models in ROOT/TMVA
title_sort c++ code generation for fast inference of deep learning models in root/tmva
url https://dx.doi.org/10.1088/1742-6596/2438/1/012013
http://cds.cern.ch/record/2862109
work_keys_str_mv AT ansitong ccodegenerationforfastinferenceofdeeplearningmodelsinroottmva
AT monetalorenzo ccodegenerationforfastinferenceofdeeplearningmodelsinroottmva
AT senguptasanjiban ccodegenerationforfastinferenceofdeeplearningmodelsinroottmva
AT hamdanahmat ccodegenerationforfastinferenceofdeeplearningmodelsinroottmva
AT sossaifederico ccodegenerationforfastinferenceofdeeplearningmodelsinroottmva
AT saxenaaaradhya ccodegenerationforfastinferenceofdeeplearningmodelsinroottmva

Ejemplares similares