CMS Data Provenance: CERN open data portal data curation automation

The CERN Open Data portal is the access point to a growing range of data produced through the research performed at CERN. It disseminates the preserved output from various research activities and includes accompanying software and documentation needed to understand and analyze the data. The portal adheres to established global standards in data preservation and Open Science: the products are shared under open licenses; they are issued with a Digital Object Identifier (DOI) to make them citable objects.[1] The portal has a wide range of datasets coming from CERN main experiments ALICE, ATLAS, CMS, LHCb and OPERA. The datasets are very huge in size -Gigabytes to Terabytes-, and only metadata information is generated as JSON records that go directly to the portal by the data curation scripts. data curation scripts contain a collection of data ingestion and curation tools used to prepare the datasets’ metadata, software, and any accompanying material for public open data releases on the CERN Open Data portal.[2]