Scaling Security Operations Center message processing: from Scala to Go

<!--HTML-->The Security Operations Center of CERN is responsible for ingesting, storing and aggregating an immense amount of data each day (around 2.5TB). The main message processing pipeline used to be based on Spark streaming jobs deployed across a cluster of machines, but was still struggling with the daily load. Nowadays, everything is performed by a single Go binary deployed on 4 machines, with enough resources to spare. This talk focuses on why we switched languages, why it works better, as well as some key implementation takeaways.