Scalable FPGA-based controller of a thousand-ASIC system in the CERN CMS HGCAL detector

The upcoming of the European Organization for Nuclear Research (CERN) High-Luminosity LargeHadron Collider (LHC) (HL-LHC) phase motivates the replacement of the endcap calorimeters of theCompact Muon Solenoid (CMS) detector with the High-Granularity Calorimeter (HGCAL). To read outits 6 million channels, the HGCAL uses a complex electronic readout chain that comprises a front-endand a back-end. The front-end is located in the experimental cavern comprising about 150 000 radiationtolerant Application Specific Integrated Circuits (ASICs). The back-end is shielded from radiation andconsists of about 100 Field Programmable Gate Arrays (FPGAs). Each FPGA is connected to 108 optical links, each providing a 10.24 Gbit/s transmission rate, and is responsible for configuring up to 3500ASICs.This dissertation reports on the work contributed to the control system implemented in the back-endof HGCAL to configure the front-end electronics, known as asynchronous (slow) control. By using development boards to emulate the HGCAL hardware still under development, it was possible to prototypethe slow control FPGA hardware and validate the communication with the target front-end ASICs whileensuring complete portability between the prototype and the final detector systems. Furthermore, theconfiguration time of all the HGCAL electronics was roughly estimated at around 1 minute.Another contribution of this work to HGCAL is the design of an accumulator system to accelerate thecomputation of the mean and standard deviation of several metrics in the testing of the HGCAL ReadoutChip (ROC) (HGCROC) ASIC, allowing up to four times faster analysis in testing.Hadron Collider (LHC) high-luminosity phase is a detector withmore than 6 million channels that will provide precise sensingand measurement of position, timing, and energy of the particlesproduced in the collisions of the beams. The HGCAL electronicsare a large and complex set of processing systems split into frontend and back-end. The front-end, located in the experimentalcavern, consists of ???150 thousand radiation tolerant ASICs.The high-density FPGA-based back-end is housed away fromthe radiation area in a set of Advanced TelecommunicationsComputing Architecture (ATCA) boards and crates hosting ???100FPGAs. Each ATCA back-end board will comprise one (or two)FPGAs, managing up to ???120 optical links, each providinga transmission rate of 10.24 Gb/s between the back-end andthe front-end electronics. Each back-end FPGA is responsiblefor configuring and monitoring up to ???3500 front-end ASICsand will be controlled by software running on a back-endMPSoC that provides the entry point for the whole controlprocedure. This paper presents the design and implementation ofthe prototyping infrastructure deployed to test and validate theslow-control block of the HGCAL back-end electronics, togetherwith the related interfaces with the controller MPSoC and thefront-end transceiver ASICs. The required functionalities havebeen validated with a ZCU102 Xilinx Ultrascale+ developmentboard, which emulated the back-end elements that are still underdevelopment and not yet available for this comprehensive test.This development board was connected to other custom ASICdevelopment boards via optical links, emulating the front-end sideof the system, also still under development. Besides providingreliable testing and validation of the operation of the wholeinfrastructure, the prototyping platform also allowed to attainthe required software/hardware portability that ensures easyintegration/replacement of all the (still) emulated componentswith their final implementations.Index Terms???Fast Prototyping, System Emulation, Testingand Validation, Field-Programmable Gate-Array