Object store characterisation for ATLAS distributed computing

In this work we developed a platform for running functional and stress tests against object storage instances. The approach taken was to closely mimic the use case seen in ATLAS distributed computing in terms of client libraries, object sizes, connection counts, and request rates. This was achieved by using Kubernetes clusters to scale the number of concurrent client processes at a level expected to be seen on Grid resources. Initial results show some interesting differences and bottleneck on both the storage side and client side. The decision to use Kubernetes as a platform has highlighted the need to understand the details of such a system compared to traditional running on local platforms. Differences in site behaviour are observed and feedback to site operators has proven useful for their deployments. Much larger scale and more distributed tests are planned and we expect these to provide a better understanding of object store performance and limits.