Endre søk
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Big Data Workflows: Locality-Aware Orchestration Using Software Containers
Univ Oslo, Dept Informat, N-0373 Oslo, Norway..
SINTEF AS, Software & Serv Innovat, N-0373 Oslo, Norway..
Norwegian Univ Sci & Technol, Dept Comp Sci, N-2815 Gjovik, Norway..
OsloMet Oslo Metropolitan Univ, Dept Comp Sci, N-0166 Oslo, Norway..ORCID-id: 0000-0001-6034-4137
Vise andre og tillknytning
2021 (engelsk)Inngår i: Sensors, E-ISSN 1424-8220, Vol. 21, nr 24, artikkel-id 8212Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

The emergence of the edge computing paradigm has shifted data processing from centralised infrastructures to heterogeneous and geographically distributed infrastructures. Therefore, data processing solutions must consider data locality to reduce the performance penalties from data transfers among remote data centres. Existing big data processing solutions provide limited support for handling data locality and are inefficient in processing small and frequent events specific to the edge environments. This article proposes a novel architecture and a proof-of-concept implementation for software container-centric big data workflow orchestration that puts data locality at the forefront. The proposed solution considers the available data locality information, leverages long-lived containers to execute workflow steps, and handles the interaction with different data sources through containers. We compare the proposed solution with Argo workflows and demonstrate a significant performance improvement in the execution speed for processing the same data units. Finally, we carry out experiments with the proposed solution under different configurations and analyze individual aspects affecting the performance of the overall solution.

sted, utgiver, år, opplag, sider
MDPI AG , 2021. Vol. 21, nr 24, artikkel-id 8212
Emneord [en]
big data workflows, orchestration, data locality, software containers
HSV kategori
Identifikatorer
URN: urn:nbn:se:kth:diva-311310DOI: 10.3390/s21248212ISI: 000778247100006PubMedID: 34960302Scopus ID: 2-s2.0-85120809412OAI: oai:DiVA.org:kth-311310DiVA, id: diva2:1655968
Merknad

QC 20220504

Tilgjengelig fra: 2022-05-04 Laget: 2022-05-04 Sist oppdatert: 2022-06-25bibliografisk kontrollert

Open Access i DiVA

Fulltekst mangler i DiVA

Andre lenker

Forlagets fulltekstPubMedScopus

Person

Matskin, MihhailPayberah, Amir H.

Søk i DiVA

Av forfatter/redaktør
Soylu, AhmetMatskin, MihhailPayberah, Amir H.
Av organisasjonen
I samme tidsskrift
Sensors

Søk utenfor DiVA

GoogleGoogle Scholar

doi
pubmed
urn-nbn

Altmetric

doi
pubmed
urn-nbn
Totalt: 107 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf