Cargando…
Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs
We have developed remote data access for large volumes of data over the Wide Area Network based on the Lustre filesystem and Kerberos authentication for security. In this paper we explore a prototype for two-step data access from worker nodes at Florida Tier3 centers, located behind a firewall and u...
Autor principal: | |
---|---|
Lenguaje: | eng |
Publicado: |
2013
|
Materias: | |
Acceso en línea: | http://cds.cern.ch/record/1627823 |
_version_ | 1780933912206245888 |
---|---|
author | Rodriguez, Jorge Luis |
author_facet | Rodriguez, Jorge Luis |
author_sort | Rodriguez, Jorge Luis |
collection | CERN |
description | We have developed remote data access for large volumes of data over the Wide Area Network based on the Lustre filesystem and Kerberos authentication for security. In this paper we explore a prototype for two-step data access from worker nodes at Florida Tier3 centers, located behind a firewall and using a private network, to data hosted on the Lustre filesystem at the University of Florida CMS Tier2 center. The Tier2-Tier3 links are 10 Gigabit per second, and the typical round trip times are 10-15 msec. For each Tier3 center we use a client which mounts securely the Lustre filesystem and hosts an XrootD server. The worker nodes access the data from the Tier3 client using POSIX compliant tools via the XrootD-fs filesystem. We perform scalability tests with up to 200 jobs running in parallel on the Tier3 worker nodes. |
id | cern-1627823 |
institution | Organización Europea para la Investigación Nuclear |
language | eng |
publishDate | 2013 |
record_format | invenio |
spelling | cern-16278232019-09-30T06:29:59Zhttp://cds.cern.ch/record/1627823engRodriguez, Jorge LuisDistributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fsDetectors and Experimental TechniquesWe have developed remote data access for large volumes of data over the Wide Area Network based on the Lustre filesystem and Kerberos authentication for security. In this paper we explore a prototype for two-step data access from worker nodes at Florida Tier3 centers, located behind a firewall and using a private network, to data hosted on the Lustre filesystem at the University of Florida CMS Tier2 center. The Tier2-Tier3 links are 10 Gigabit per second, and the typical round trip times are 10-15 msec. For each Tier3 center we use a client which mounts securely the Lustre filesystem and hosts an XrootD server. The worker nodes access the data from the Tier3 client using POSIX compliant tools via the XrootD-fs filesystem. We perform scalability tests with up to 200 jobs running in parallel on the Tier3 worker nodes.CMS-CR-2013-392oai:cds.cern.ch:16278232013-10-31 |
spellingShingle | Detectors and Experimental Techniques Rodriguez, Jorge Luis Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs |
title | Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs |
title_full | Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs |
title_fullStr | Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs |
title_full_unstemmed | Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs |
title_short | Distributing CMS Data between the Florida T2 and T3 Centers using Lustre and Xrootd-fs |
title_sort | distributing cms data between the florida t2 and t3 centers using lustre and xrootd-fs |
topic | Detectors and Experimental Techniques |
url | http://cds.cern.ch/record/1627823 |
work_keys_str_mv | AT rodriguezjorgeluis distributingcmsdatabetweenthefloridat2andt3centersusinglustreandxrootdfs |