kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Distributed Hierarchical File Systems strike back in the Cloud
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Software and Computer systems, SCS. Log Clocks AB, Stockholm, Sweden..ORCID iD: 0000-0002-6578-3902
Log Clocks AB, Stockholm, Sweden..
Oracle AB, Santa Clara, CA USA..
Oracle AB, Santa Clara, CA USA..
Show others and affiliations
2020 (English)In: 2020 IEEE 40th international conference on distributed computing systems (ICDCS), Institute of Electrical and Electronics Engineers (IEEE) , 2020, p. 820-830Conference paper, Published paper (Refereed)
Abstract [en]

Cloud service providers have aligned on availability zones as an important unit of failure and replication for storage systems. An availability zone (AZ) has independent power, networking, and cooling systems and consists of one or more data centers. Multiple AZs in close geographic proximity form a region that can support replicated low latency storage services that can survive the failure of one or more AZs. Recent reductions in inter-AZ latency have made synchronous replication protocols increasingly viable, instead of traditional quorum-based replication protocols. We introduce HopsFS-CL, a distributed hierarchical file system with support for high-availability (HA) across AZs, backed by AZ-aware synchronously replicated metadata and AZ-aware block replication. HopsFS-CL is a redesign of HopsFS, a version of HDFS with distributed metadata, and its design involved making replication protocols and block placement protocols AZ-aware at all layers of its stack: the metadata serving, the metadata storage, and block storage layers. In experiments on a real-world workload from Spotify, we show that HopsFS-CL, deployed in HA mode over 3 AZs, reaches 1.66 million ops/s, and has similar performance to HopsFS when deployed in a single AZ, while preserving the same semantics.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE) , 2020. p. 820-830
Series
IEEE International Conference on Distributed Computing Systems, ISSN 1063-6927
National Category
Computer Systems
Identifiers
URN: urn:nbn:se:kth:diva-299114DOI: 10.1109/ICDCS47774.2020.00108ISI: 000667971400075Scopus ID: 2-s2.0-85101968318OAI: oai:DiVA.org:kth-299114DiVA, id: diva2:1582683
Conference
40th IEEE International Conference on Distributed Computing Systems (ICDCS), NOV 29-DEC 01, 2020, ELECTR NETWORK
Note

QC 20210803

Not duplicate with DiVA 1467134

Available from: 2021-08-03 Created: 2021-08-03 Last updated: 2022-06-25Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Ismail, MahmoudHaridi, SeifDowling, Jim

Search in DiVA

By author/editor
Ismail, MahmoudHaridi, SeifDowling, Jim
By organisation
Software and Computer systems, SCS
Computer Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 58 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf