Change search
ReferencesLink to record
Permanent link

Direct link
Adaptive data management in the ARC Grid middleware
University of Oslo, Nordic Data Grid Facility, Kastrup, Danmark. (Department of Physics)
KTH, School of Computer Science and Communication (CSC), Centres, Centre for High Performance Computing, PDC.
University of Oslo. (Department of Physics)
University of Oslo, Vilnius University Institute of Applied Research. (Department of Physics)
2011 (English)In: Journal of Physics, Conference Series, ISSN 1742-6588, E-ISSN 1742-6596, Vol. 331, no 062006Article in journal (Refereed) Published
Abstract [en]

The Advanced Resource Connector (ARC) Grid middleware was designed almost 10 years ago, and has proven to be an attractive distributed computing solution and successful in adapting to new data management and storage technologies. However, with an ever-increasing user base and scale of resources to manage, along with the introduction of more advanced data transfer protocols, some limitations in the current architecture have become apparent. The simple first-in first-out approach to data transfer leads to bottlenecks in the system, as does the built-in assumption that all data is immediately available from remote data storage. We present an entirely new data management architecture for ARC which aims to alleviate these problems, by introducing a three-layer structure. The top layer accepts incoming requests for data transfer and directs them to the middle layer, which schedules individual transfers and negotiates with various intermediate catalog and storage systems until the physical file is ready to be transferred. The lower layer performs all operations which use large amounts of bandwidth, i.e. the physical data transfer. Using such a layered structure allows more efficient use of the available bandwidth as well as enabling late-binding of jobs to data transfer slots based on a priority system. Here we describe in full detail the design and implementation of the new system.

Place, publisher, year, edition, pages
2011. Vol. 331, no 062006
National Category
Computer and Information Science
URN: urn:nbn:se:kth:diva-66434DOI: 10.1088/1742-6596/331/6/062006ISI: 000301120500006ScopusID: 2-s2.0-84858142842OAI: diva2:484026
QC 20120201Available from: 2012-01-26 Created: 2012-01-26 Last updated: 2012-02-01Bibliographically approved

Open Access in DiVA

No full text

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Gholami, Ali
By organisation
Centre for High Performance Computing, PDC
In the same journal
Journal of Physics, Conference Series
Computer and Information Science

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 44 hits
ReferencesLink to record
Permanent link

Direct link