Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Combining analytics framework and Cloud schedulers in order to optimise resource utilisation in a distributed Cloud
KTH, School of Information and Communication Technology (ICT).
2015 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

Analytics frameworks were initially created to run on bare-metal hardware so they contain scheduling mechanisms to optimise the distribution of the cpu load and data allocation. Generally, the scheduler is part of the analytics framework resource manager. There are different resources managers used in the market and the open-source community that can serve for different analytics frameworks. For example, Spark is initially built with Mesos. Hadoop is now using YARN. Spark is also available as a YARN application. On the other hand, cloud environments (Like OpenStack) contain theirs own mechanisms of distributing resources between users and services.

While analytics applications are increasingly being migrated to the cloud, the scheduling decisions for running an analytic job is still done in isolation between the different scheduler layers (Cloud/Infrastructure vs analytics resource manager). This can seriously impact performance of analytics or other services running jointly in the same infrastructure as well as limit load-balancing, and autoscaling capabilities. This master thesis identifies what are the scheduling decisions that should be taken at the different layers (Infrastructure, Platform and Software) as well as the required metrics from the environment when mul-tiple schedulers are used in order to get the best performance and maximise the resource utilisation.

Place, publisher, year, edition, pages
2015. , 51 p.
Series
TRITA-ICT-EX, 2015:154
National Category
Computer and Information Science
Identifiers
URN: urn:nbn:se:kth:diva-177582OAI: oai:DiVA.org:kth-177582DiVA: diva2:873440
Examiners
Available from: 2015-11-25 Created: 2015-11-24 Last updated: 2017-08-04Bibliographically approved

Open Access in DiVA

fulltext(2011 kB)27 downloads
File information
File name FULLTEXT01.pdfFile size 2011 kBChecksum SHA-512
90c9b57cf98376b4ad5c389ad2ef97512dc1cfb2d304c8cbba2a89020f924334c25f56b8168b81e80ee8bae66927b89817083c12d99ba6a4e351feb9b442d4d4
Type fulltextMimetype application/pdf

By organisation
School of Information and Communication Technology (ICT)
Computer and Information Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 27 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 109 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf