Introducing Object Storage in Hadoop Ecosystem

CERN deals with one of the highest rates of data every day. The current Hadoop Ecosystem uses Hadoop Data File System (HDFS) as a storage option. While HDFS does not generate any problems at the moment, it has some inefficiencies. First, storage capacity overhead cost. Second, HDFS is no longer a top Big Data standard in the industry. The community is moving towards solutions based on object storage. Object storage potentially offers a better cost-effective solution. This project aims to evaluate Apache Ozone as a possible object storage solution for the current Hadoop architecture. It would look into the implementation of Ozone and how it connects and operates with some current services of the Hadoop architecture like Yarn and Apache Spark.