Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
MPI Trace Compression Using Event Flow Graphs
KTH, School of Computer Science and Communication (CSC), High Performance Computing and Visualization (HPCViz).ORCID iD: 0000-0001-9693-6265
Ludwig-Maximilians-Universitat (LMU). (Computer Science Department, MNM Team)
KTH, School of Computer Science and Communication (CSC), High Performance Computing and Visualization (HPCViz).ORCID iD: 0000-0002-9901-9857
2014 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Understanding how parallel applications behave is crucial for using high-performance computing (HPC) resources efficiently. However, the task of performance analysis is becoming increasingly difficult due to the growing complexity of scientific codes and the size of machines. Even though many tools have been developed over the past years to help in this task, current approaches either only offer an overview of the application discarding temporal information, or they generate huge trace files that are often difficult to handle.

In this paper we propose the use of event flow graphs for monitoring MPI applications, a new and different approach that balances the low overhead of profiling tools with the abundance of information available from tracers. Event flow graphs are captured with very low overhead, require orders of magnitude less storage than standard trace files, and can still recover the full sequence of events in the application. We test this new approach with the NERSC-8/Trinity Benchmark suite and achieve compression ratios up to 119x.

Place, publisher, year, edition, pages
2014. 1-12 p.
Series
Lecture Notes in Computer Science, ISSN 0302-9743 ; 8632
Keyword [en]
MPI event flow graphs, trace compression, trace reconstruction, performance monitoring
National Category
Computer Systems
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-165042Scopus ID: 2-s2.0-84958532986OAI: oai:DiVA.org:kth-165042DiVA: diva2:806807
Conference
Euro-Par 2014 Parallel Processing
Note

QC 20150423. QC 20160314

Available from: 2015-04-21 Created: 2015-04-21 Last updated: 2017-04-28Bibliographically approved
In thesis
1. Towards Scalable Performance Analysis of MPI Parallel Applications
Open this publication in new window or tab >>Towards Scalable Performance Analysis of MPI Parallel Applications
2015 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

  A considerably fraction of science discovery is nowadays relying on computer simulations. High Performance Computing  (HPC) provides scientists with the means to simulate processes ranging from climate modeling to protein folding. However, achieving good application performance and making an optimal use of HPC resources is a heroic task due to the complexity of parallel software. Therefore, performance tools  and runtime systems that help users to execute  applications in the most optimal way are of utmost importance in the landscape of HPC.  In this thesis, we explore different techniques to tackle the challenges of collecting, storing, and using  fine-grained performance data. First, we investigate the automatic use of real-time performance data in order to run applications in an optimal way. To that end, we present a prototype of an adaptive task-based runtime system that uses real-time performance data for task scheduling. This runtime system has a performance monitoring component that provides real-time access to the performance behavior of anapplication while it runs. The implementation of this monitoring component is presented and evaluated within this thesis. Secondly, we explore lossless compression approaches  for MPI monitoring. One of the main problems that  performance tools face is the huge amount of fine-grained data that can be generated from an instrumented application. Collecting fine-grained data from a program is the best method to uncover the root causes of performance bottlenecks, however, it is unfeasible with extremely parallel applications  or applications with long execution times. On the other hand, collecting coarse-grained data is scalable but  sometimes not enough to discern the root cause of a performance problem. Thus, we propose a new method for performance monitoring of MPI programs using event flow graphs. Event flow graphs  provide very low overhead in terms of execution time and  storage size, and can be used to reconstruct fine-grained trace files of application events ordered in time.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2015. viii, 39 p.
Series
TRITA-CSC-A, ISSN 1653-5723 ; 2015:05
Keyword
parallel computing, performance monitoring, performance tools, event flow graphs
National Category
Computer Systems
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-165043 (URN)978-91-7595-518-6 (ISBN)
Presentation
2015-05-20, The Visualization Studio, room 4451, Lindstedtsvägen 5, KTH, Stockholm, 10:00 (English)
Opponent
Supervisors
Note

QC 20150508

Available from: 2015-05-08 Created: 2015-04-21 Last updated: 2015-05-08Bibliographically approved

Open Access in DiVA

No full text

Scopus

Authority records BETA

Aguilar, XavierLaure, Erwin

Search in DiVA

By author/editor
Aguilar, XavierLaure, Erwin
By organisation
High Performance Computing and Visualization (HPCViz)
Computer Systems

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 153 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf