Scaling Security Operations Center message processing: from Scala to Go

<!--HTML-->The Security Operations Center of CERN is responsible for ingesting, storing and aggregating an immense amount of data each day (around 2.5TB). The main message processing pipeline used to be based on Spark streaming jobs deployed across a cluster of machines, but was still struggli...

Descripción completa

Detalles Bibliográficos
Autor principal: Schuszter, Cristian
Lenguaje:eng
Publicado: 2019
Materias:
IT Lightning Talks (ITLT)
Acceso en línea:http://cds.cern.ch/record/2673979
_version_ 1780962553771327488
author Schuszter, Cristian
author_facet Schuszter, Cristian
author_sort Schuszter, Cristian
collection CERN
description <!--HTML-->The Security Operations Center of CERN is responsible for ingesting, storing and aggregating an immense amount of data each day (around 2.5TB). The main message processing pipeline used to be based on Spark streaming jobs deployed across a cluster of machines, but was still struggling with the daily load. Nowadays, everything is performed by a single Go binary deployed on 4 machines, with enough resources to spare. This talk focuses on why we switched languages, why it works better, as well as some key implementation takeaways.
id cern-2673979
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2019
record_format invenio
spelling cern-26739792022-11-02T22:13:24Zhttp://cds.cern.ch/record/2673979engSchuszter, CristianScaling Security Operations Center message processing: from Scala to GoIT Lightning Talks: session #18IT Lightning Talks (ITLT)<!--HTML-->The Security Operations Center of CERN is responsible for ingesting, storing and aggregating an immense amount of data each day (around 2.5TB). The main message processing pipeline used to be based on Spark streaming jobs deployed across a cluster of machines, but was still struggling with the daily load. Nowadays, everything is performed by a single Go binary deployed on 4 machines, with enough resources to spare. This talk focuses on why we switched languages, why it works better, as well as some key implementation takeaways.oai:cds.cern.ch:26739792019
spellingShingle IT Lightning Talks (ITLT)
Schuszter, Cristian
Scaling Security Operations Center message processing: from Scala to Go
title Scaling Security Operations Center message processing: from Scala to Go
title_full Scaling Security Operations Center message processing: from Scala to Go
title_fullStr Scaling Security Operations Center message processing: from Scala to Go
title_full_unstemmed Scaling Security Operations Center message processing: from Scala to Go
title_short Scaling Security Operations Center message processing: from Scala to Go
title_sort scaling security operations center message processing: from scala to go
topic IT Lightning Talks (ITLT)
url http://cds.cern.ch/record/2673979
work_keys_str_mv AT schusztercristian scalingsecurityoperationscentermessageprocessingfromscalatogo
AT schusztercristian itlightningtalkssession18

Ejemplares similares