Ceph at the Flatiron Institute

<!--HTML-->The Flatiron Institute, a division of the Simons Foundation, is a privately funded non-profit organization in Manhattan with a mission to advance scientific knowledge via computational methods. Operating in a variety of disciplines, from astrophysics to biology, quantum physics and mathematics, the breadth of computational problems our researchers tackle present unique challenges to our infrastructure. We are early adopters of Ceph and CephFS from the Hammer days, and now run close to 30PB of Ceph storage that serves our HPC environment. The open source development model of Ceph enabled us to make customizations and apply patches both for early fixes as well as for custom enhancements specific to our environment. This talk will give an overview of our over four year journey with Ceph, highlighting choices we made for our setup, the unique issues we face, some of the tools/patches we are working on for our environment and disasters that Ceph successfully saved us from over the years.