kth.sePublikationer KTH
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Locality-aware workflow orchestration for big data
Visa övriga samt affilieringar
2021 (Engelska)Ingår i: ACM International Conference Proceeding Series, Association for Computing Machinery , 2021, s. 62-70Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

The development of the Edge computing paradigm shifts data processing from centralised infrastructures to heterogeneous and geographically distributed infrastructure. Such a paradigm requires data processing solutions that consider data locality in order to reduce the performance penalties from data transfers between remote (in network terms) data centres. However, existing Big Data processing solutions have limited support for handling data locality and are inefficient in processing small and frequent events specific to Edge environments. This paper proposes a novel architecture and a proof-of-concept implementation for software container-centric Big Data workflow orchestration that puts data locality at the forefront. Our solution considers any available data locality information by default, leverages long-lived containers to execute workflow steps, and handles the interaction with different data sources through containers. We compare our system with Argo workflow and show significant performance improvements in terms of speed of execution for processing units of data using our data locality aware Big Data workflow approach. 

Ort, förlag, år, upplaga, sidor
Association for Computing Machinery , 2021. s. 62-70
Nyckelord [en]
Big data workflows, Data locality, Software containers, Big data, Data handling, Data transfer, Big data workflow, Centralised, Computing paradigm, Edge computing, Locality aware, Paradigm shifts, Processing solutions, Software container, Work-flows, Containers
Nationell ämneskategori
Datavetenskap (datalogi) Datorsystem
Identifikatorer
URN: urn:nbn:se:kth:diva-313850DOI: 10.1145/3444757.3485106ISI: 001429314700010Scopus ID: 2-s2.0-85120811728OAI: oai:DiVA.org:kth-313850DiVA, id: diva2:1668288
Konferens
13th International Conference on Management of Digital EcoSystems, MEDES 2021, Online/Hammamet, Tunisia, 1-3 November 2021
Anmärkning

Part of proceedings: ISBN 978-1-4503-8314-1

QC 20220613

Tillgänglig från: 2022-06-13 Skapad: 2022-06-13 Senast uppdaterad: 2025-12-05Bibliografiskt granskad

Open Access i DiVA

Fulltext saknas i DiVA

Övriga länkar

Förlagets fulltextScopushttps://medes.sigappfr.org/21/

Person

Matskin, MihhailPayberah, Amir H.

Sök vidare i DiVA

Av författaren/redaktören
Matskin, MihhailPayberah, Amir H.
Av organisationen
Programvaruteknik och datorsystem, SCS
Datavetenskap (datalogi)Datorsystem

Sök vidare utanför DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetricpoäng

doi
urn-nbn
Totalt: 109 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf