Change search
ReferencesLink to record
Permanent link

Direct link
Combining analytics framework and Cloud schedulers in order to optimise resource utilisation in a distributed Cloud
KTH, School of Information and Communication Technology (ICT).
2015 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

Analytics frameworks were initially created to run on bare-metal hardware so they contain scheduling mechanisms to optimise the distribution of the cpu load and data allocation. Generally, the scheduler is part of the analytics framework resource manager. There are different resources managers used in the market and the open-source community that can serve for different analytics frameworks. For example, Spark is initially built with Mesos. Hadoop is now using YARN. Spark is also available as a YARN application. On the other hand, cloud environments (Like OpenStack) contain theirs own mechanisms of distributing resources between users and services.

While analytics applications are increasingly being migrated to the cloud, the scheduling decisions for running an analytic job is still done in isolation between the different scheduler layers (Cloud/Infrastructure vs analytics resource manager). This can seriously impact performance of analytics or other services running jointly in the same infrastructure as well as limit load-balancing, and autoscaling capabilities. This master thesis identifies what are the scheduling decisions that should be taken at the different layers (Infrastructure, Platform and Software) as well as the required metrics from the environment when mul-tiple schedulers are used in order to get the best performance and maximise the resource utilisation.

Place, publisher, year, edition, pages
2015. , 51 p.
TRITA-ICT-EX, 2015:154
National Category
Computer and Information Science
URN: urn:nbn:se:kth:diva-177582OAI: diva2:873440
Available from: 2015-11-25 Created: 2015-11-24 Last updated: 2015-11-25Bibliographically approved

Open Access in DiVA

No full text

By organisation
School of Information and Communication Technology (ICT)
Computer and Information Science

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 29 hits
ReferencesLink to record
Permanent link

Direct link