kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Conceptualization and scalable execution of big data workflows using domain-specific languages and software containers
SINTEF AS, Forskningsveien 1, N-0373 Oslo, Norway..
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Software and Computer systems, SCS.
Norwegian Univ Sci & Technol NTNU, Teknol Vegen 22, N-2815 Gjovik, Norway..
OsloMet Oslo Metropolitan Univ, Pilestredet 46, N-0167 Oslo, Norway..
Show others and affiliations
2021 (English)In: INTERNET OF THINGS, ISSN 2543-1536, Vol. 16, p. 100440-, article id 100440Article in journal (Refereed) Published
Abstract [en]

Big Data processing, especially with the increasing proliferation of Internet of Things (IoT) technologies and convergence of IoT, edge and cloud computing technologies, involves handling massive and complex data sets on heterogeneous resources and incorporating different tools, frameworks, and processes to help organizations make sense of their data collected from various sources. This set of operations, referred to as Big Data workflows, requires taking advantage of Cloud infrastructures' elasticity for scalability. In this article, we present the design and prototype implementation of a Big Data workflow approach based on the use of software container technologies, message-oriented middleware (MOM), and a domain-specific language (DSL) to enable highly scalable workflow execution and abstract workflow definition. We demonstrate our system in a use case and a set of experiments that show the practical applicability of the proposed approach for the specification and scalable execution of Big Data workflows. Furthermore, we compare our proposed approach's scalability with that of Argo Workflows - one of the most prominent tools in the area of Big Data workflows - and provide a qualitative evaluation of the proposed DSL and overall approach with respect to the existing literature.

Place, publisher, year, edition, pages
Elsevier BV , 2021. Vol. 16, p. 100440-, article id 100440
Keywords [en]
Big data workflows, Internet of Things, Domain-specific languages, Software containers
National Category
Computer Systems
Identifiers
URN: urn:nbn:se:kth:diva-306770DOI: 10.1016/j.iot.2021.100440ISI: 000723420400012Scopus ID: 2-s2.0-85119205685OAI: oai:DiVA.org:kth-306770DiVA, id: diva2:1622747
Note

QC 20211223

Available from: 2021-12-23 Created: 2021-12-23 Last updated: 2022-11-28Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Dessalk, Yared DejeneMatskin, MihhailPayberah, Amir H.

Search in DiVA

By author/editor
Dessalk, Yared DejeneMatskin, MihhailPayberah, Amir H.
By organisation
Software and Computer systems, SCS
Computer Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 80 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf