Change search
ReferencesLink to record
Permanent link

Direct link
SAASFEE: Scalable scientific workflow execution engine
KTH.
Show others and affiliations
2015 (English)In: Proceedings of the VLDB Endowment, ISSN 2150-8097, E-ISSN 2150-8097, Vol. 8, no 12, 1892-1895 p.Article in journal (Refereed) PublishedText
Abstract [en]

Across many fields of science, primary data sets like sensor read-outs, time series, and genomic sequences are analyzed by complex chains of specialized tools and scripts exchanging intermediate results in domain-specific file formats. Scientific work ow management systems (SWfMSs) support the development and execution of these tool chains by providing work ow specification languages, graphical editors, fault-tolerant execution engines, etc. However, many SWfMSs are not prepared to handle large data sets because of inadequate support for distributed computing. On the other hand, most SWfMSs that do support distributed computing only allow static task execution orders. We present SAASFEE, a SWfMS which runs arbitrarily complex work ows on Hadoop YARN. Work ows are specified in Cuneiform, a functional work ow language focusing on parallelization and easy integration of existing software. Cuneiform work ows are executed on Hi-WAY, a higher-level scheduler for running work ows on YARN. Distinct features of SAASFEE are the ability to execute iterative work ows, an adaptive task scheduler, re-executable provenance traces, and compatibility to selected other work ow systems. In the demonstration, we present all components of SAASFEE using real-life work ows from the field of genomics.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM), 2015. Vol. 8, no 12, 1892-1895 p.
Keyword [en]
Chains, Computational linguistics, Engines, Information management, Specification languages, Wool, Yarn, Execution engine, Genomic sequence, Graphical editors, Intermediate results, Management systems, Parallelizations, Scientific workflows, Specialized tools, Distributed computer systems
National Category
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-181254ScopusID: 2-s2.0-84953879839OAI: oai:DiVA.org:kth-181254DiVA: diva2:900934
Conference
11 September 2006 through 11 September 2006, Seoul
Note

QC 20160205

Available from: 2016-02-05 Created: 2016-01-29 Last updated: 2016-02-05Bibliographically approved

Open Access in DiVA

No full text

Scopus

Search in DiVA

By author/editor
Dowling, Jim
By organisation
KTHSoftware and Computer systems, SCS
In the same journal
Proceedings of the VLDB Endowment
Computer Science

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 19 hits
ReferencesLink to record
Permanent link

Direct link