Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Architectural Impact on Performance of In-memoryData Analytics: Apache Spark Case Study
KTH, School of Information and Communication Technology (ICT), Software and Computer systems, SCS.ORCID iD: 0000-0002-7510-6286
KTH, School of Information and Communication Technology (ICT), Software and Computer systems, SCS.ORCID iD: 0000-0002-9637-2065
KTH, School of Information and Communication Technology (ICT), Software and Computer systems, SCS.
Barcelona Super Computing Center and Technical University of Catalunya.
(English)Manuscript (preprint) (Other academic)
Abstract [en]

While cluster computing frameworks are contin-uously evolving to provide real-time data analysis capabilities,Apache Spark has managed to be at the forefront of big data an-alytics for being a unified framework for both, batch and streamdata processing. However, recent studies on micro-architecturalcharacterization of in-memory data analytics are limited to onlybatch processing workloads. We compare micro-architectural per-formance of batch processing and stream processing workloadsin Apache Spark using hardware performance counters on a dualsocket server. In our evaluation experiments, we have found thatbatch processing are stream processing workloads have similarmicro-architectural characteristics are bounded by the latency offrequent data access to DRAM. For data accesses we have foundthat simultaneous multi-threading is effective in hiding the datalatencies. We have also observed that (i) data locality on NUMAnodes can improve the performance by 10% on average and(ii)disabling next-line L1-D prefetchers can reduce the executiontime by up-to 14% and (iii) multiple small executors can provideup-to 36% speedup over single large executor

Keyword [en]
Performance Characterization, Apache Spark, Micro-architecture
National Category
Computer Systems
Research subject
Information and Communication Technology
Identifiers
URN: urn:nbn:se:kth:diva-185580OAI: oai:DiVA.org:kth-185580DiVA: diva2:922527
Note

QC 20160425

Available from: 2016-04-22 Created: 2016-04-22 Last updated: 2016-04-25Bibliographically approved
In thesis
1. Performance Characterization of In-Memory Data Analytics on a Scale-up Server
Open this publication in new window or tab >>Performance Characterization of In-Memory Data Analytics on a Scale-up Server
2016 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

The sheer increase in volume of data over the last decade has triggered research in cluster computing frameworks that enable web enterprises to extract big insights from big data. While Apache Spark defines the state of the art in big data analytics platforms for (i) exploiting data-flow and in-memory computing and (ii) for exhibiting superior scale-out performance on the commodity machines, little effort has been devoted at understanding the performance of in-memory data analytics with Spark on modern scale-up servers. This thesis characterizes the performance of in-memory data analytics with Spark on scale-up servers.

Through empirical evaluation of representative benchmark workloads on a dual socket server, we have found that in-memory data analytics with Spark exhibit poor multi-core scalability beyond 12 cores due to thread level load imbalance and work-time inflation. We have also found that workloads are bound by the latency of frequent data accesses to DRAM. By enlarging input data size, application performance degrades significantly due to substantial increase in wait time during I/O operations and garbage collection, despite 10% better instruction retirement rate (due to lower L1 cache misses and higher core utilization).

For data accesses we have found that simultaneous multi-threading is effective in hiding the data latencies. We have also observed that (i) data locality on NUMA nodes can improve the performance by 10% on average, (ii) disabling next-line L1-D prefetchers can reduce the execution time by up-to 14%. For GC impact, we match memory behaviour with the garbage collector to improve performance of applications between 1.6x to 3x. and recommend to use multiple small executors that can provide up-to 36% speedup over single large executor.

Place, publisher, year, edition, pages
KTH Royal Institute of Technology, 2016. 111 p.
Series
TRITA-ICT, 2016:07
National Category
Computer Systems
Research subject
Information and Communication Technology
Identifiers
urn:nbn:se:kth:diva-185581 (URN)978-91-7595-926-9 (ISBN)
Presentation
2016-05-23, Ka-210, Electrum 229, Kista, Stockholm, 09:15 (English)
Opponent
Supervisors
Note

QC 20160425

Available from: 2016-04-25 Created: 2016-04-22 Last updated: 2017-03-02Bibliographically approved

Open Access in DiVA

paper(1120 kB)180 downloads
File information
File name FULLTEXT01.pdfFile size 1120 kBChecksum SHA-512
775aec701dd91da748209287d162eb148bc07529cd6082a1446652f930bff8a5a546ed11c8d92f6300425f3dd6fdae83a71954de5fc04d7ca48ff46628a551ad
Type fulltextMimetype application/pdf

Authority records BETA

Awan, Ahsan JavedBrorsson, Mats

Search in DiVA

By author/editor
Awan, Ahsan JavedBrorsson, MatsVlassov, Vladimir
By organisation
Software and Computer systems, SCS
Computer Systems

Search outside of DiVA

GoogleGoogle Scholar
Total: 180 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 133 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf