Change search
ReferencesLink to record
Permanent link

Direct link
Evaluation and benchmarking of Tachyon as a memory-centric distributed storage system for Apache Hadoop
KTH, School of Information and Communication Technology (ICT).
2016 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

Hadoop was developed as an open-source software framework that leveraged initially the MapReduce programming model and therefore was able to efficiently analyse and process large datasets. At the core of Hadoop is the Hadoop distributed file system or HDFS, which is used as the default storage across the cluster. Hadoop can also be used with other types of storage, with or without HDFS, such as Amazon S3, Windows Azure Storage Blobs, GlusterFS, Tachyon etc. This thesis focuses on Tachyon, a distributed file system that claims to enable reliable data sharing at memory speed across cluster computing frameworks. We benchmark and evaluate HDFS with and without Tachyon in regards to performance. To do so we used TestDFSIO as a benchmark to simulate different MapReduce workloads and an in-production Spark job from Spotify. Tachyon's different writetypes were also put to the test and evaluated. To see how cloud solutions compare, we perform the same evaluations of Tachyon over Google Cloud Storage.

Place, publisher, year, edition, pages
2016. , 38 p.
TRITA-ICT-EX, 2016:12
National Category
Information Systems, Social aspects
URN: urn:nbn:se:kth:diva-189571OAI: diva2:947054
Subject / course
Information and Communication Technology
Educational program
Master of Science - Software Engineering of Distributed Systems
Available from: 2016-07-06 Created: 2016-07-06 Last updated: 2016-07-06Bibliographically approved

Open Access in DiVA

fulltext(27302 kB)21 downloads
File information
File name FULLTEXT01.pdfFile size 27302 kBChecksum SHA-512
Type fulltextMimetype application/pdf

By organisation
School of Information and Communication Technology (ICT)
Information Systems, Social aspects

Search outside of DiVA

GoogleGoogle Scholar
Total: 21 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 32 hits
ReferencesLink to record
Permanent link

Direct link