Top-level DB design for Big Data in ATLAS Experiment at CERN

This presentation describes a system that accumulates a set of key quantities for a very large number of particle collision events recorded by the ATLAS experiment at the LHC (Large Hadron Collider) at CERN. The main project requirements are the handling of tens of billions of rows per year with minimal DB resources, and providing outstanding performance for the fundamental use cases. Various challenges were faced in the process of project development, such as large data volume, large transactions (tens to hundreds of million of rows per transaction) requiring significant amount of undo, row duplication checks, adequate table statistics gathering, and SQL execution plan stability. Currently the system hosts about 120 billion rows as the data ingestion rate has gone beyond the initially foreseen 30 billion rows per year. The crucial DB schema design decisions and the Oracle DB features and techniques will be shared with the audience. By attending this session you will learn how big physics data can be organized in a very efficient way in order to become small-sized physics data.