The one-million table partitions challenge in an ATLAS experiment DB application

<!--HTML-->How can a new database system which manages billions of rows with relatively short lifetime (weeks to months) be designed ? The system should resemble a Whiteboard: - data, grouped by given properties form data collections, are added when requested - data collections are consumed by the requestor - when not needed any more, data collections are removed by a "Whiteboard sponge" process. I will show you what options I explored and which one is potentially the best in terms of efficiency.