Cargando…

Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs

We have developed remote data access for large volumes of data over the Wide Area Network based on the Lustre filesystem and Kerberos authentication for security. In this paper we explore a prototype for two-step data access from worker nodes at Florida Tier3 centers, located behind a firewall and u...

Descripción completa

Detalles Bibliográficos
Autor principal: Rodriguez, Jorge Luis
Lenguaje:eng
Publicado: 2013
Materias:
Acceso en línea:http://cds.cern.ch/record/1627823
_version_ 1780933912206245888
author Rodriguez, Jorge Luis
author_facet Rodriguez, Jorge Luis
author_sort Rodriguez, Jorge Luis
collection CERN
description We have developed remote data access for large volumes of data over the Wide Area Network based on the Lustre filesystem and Kerberos authentication for security. In this paper we explore a prototype for two-step data access from worker nodes at Florida Tier3 centers, located behind a firewall and using a private network, to data hosted on the Lustre filesystem at the University of Florida CMS Tier2 center. The Tier2-Tier3 links are 10 Gigabit per second, and the typical round trip times are 10-15 msec. For each Tier3 center we use a client which mounts securely the Lustre filesystem and hosts an XrootD server. The worker nodes access the data from the Tier3 client using POSIX compliant tools via the XrootD-fs filesystem. We perform scalability tests with up to 200 jobs running in parallel on the Tier3 worker nodes.
id cern-1627823
institution Organización Europea para la Investigación Nuclear
language eng
publishDate 2013
record_format invenio
spelling cern-16278232019-09-30T06:29:59Zhttp://cds.cern.ch/record/1627823engRodriguez, Jorge LuisDistributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fsDetectors and Experimental TechniquesWe have developed remote data access for large volumes of data over the Wide Area Network based on the Lustre filesystem and Kerberos authentication for security. In this paper we explore a prototype for two-step data access from worker nodes at Florida Tier3 centers, located behind a firewall and using a private network, to data hosted on the Lustre filesystem at the University of Florida CMS Tier2 center. The Tier2-Tier3 links are 10 Gigabit per second, and the typical round trip times are 10-15 msec. For each Tier3 center we use a client which mounts securely the Lustre filesystem and hosts an XrootD server. The worker nodes access the data from the Tier3 client using POSIX compliant tools via the XrootD-fs filesystem. We perform scalability tests with up to 200 jobs running in parallel on the Tier3 worker nodes.CMS-CR-2013-392oai:cds.cern.ch:16278232013-10-31
spellingShingle Detectors and Experimental Techniques
Rodriguez, Jorge Luis
Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs
title Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs
title_full Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs
title_fullStr Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs
title_full_unstemmed Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs
title_short Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs
title_sort distributing cms data between the florida t2 and t3 centers using lustre and xrootd-fs
topic Detectors and Experimental Techniques
url http://cds.cern.ch/record/1627823
work_keys_str_mv AT rodriguezjorgeluis distributingcmsdatabetweenthefloridat2andt3centersusinglustreandxrootdfs