kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Dynamic Resource Management for In-Situ Techniques Using MPI-Sessions
Max Planck Comp & Data Facil, D-85748 Garching, Germany..
Tech Univ Munich, D-85748 Garching, Germany..
KTH, School of Engineering Sciences (SCI), Engineering Mechanics, Fluid Mechanics.ORCID iD: 0000-0001-5204-8549
Max Planck Inst Plasma Phys, D-85748 Garching, Germany..
Show others and affiliations
2025 (English)In: Recent advances in the message passing interface, EUROMPI 2024 / [ed] Blaas-Schenner, C Niethammer, C Haas, T, Springer Nature , 2025, p. 105-120Conference paper, Published paper (Refereed)
Abstract [en]

The computational power of High-Performance Computing (HPC) systems increases continuously and rapidly. Data-intensive applications are designed to leverage the high computational capacity of HPC resources and typically generate a large amount of data for traditional post-processing data analytics. However, the HPC systems' in-/output (IO) subsystem develops relatively slowly, and the storage capacity is limited. This could lead to limited actual performance and scientific discovery. In-situ techniques are a partial remedy to these problems by reducing or avoiding the data flow through the IO subsystem to/from the storage. However, in current practice, asynchronous in-situ techniques with static resource management often allocate separate computing resources for executing in-situ task(s), which remain idle if no in-situ work is at hand. In the present work, we target improving the efficiency of computing resource usage by launching and releasing necessary additional computing resources for in-situ task(s). Our approach is based on extensions for MPI Sessions that enable the required dynamic resource management. In this paper, we propose a basic and an advanced in-situ techniques with dynamic resource management enabled by MPI Sessions, their implementations on two real-world use cases, and a critical analysis of the experimental results.

Place, publisher, year, edition, pages
Springer Nature , 2025. p. 105-120
Series
Lecture Notes in Computer Science, ISSN 0302-9743 ; 15267
Keywords [en]
In-situ, HPC, Dynamic resource management, MPI Session
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:kth:diva-357272DOI: 10.1007/978-3-031-73370-3_7ISI: 001329986700007Scopus ID: 2-s2.0-85206070581OAI: oai:DiVA.org:kth-357272DiVA, id: diva2:1918931
Conference
31st European MPI Users' Group Meeting (EuroMPI), September 25-27, 2024, Pawsey Supercomput Res Centre, Perth, Australia
Note

Part of ISBN 978-3-031-73369-7, 978-3-031-73370-3

QC 20241206

Available from: 2024-12-06 Created: 2024-12-06 Last updated: 2024-12-06Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Perez Martinez, AdalbertoMarkidis, StefanoSchlatter, Philipp

Search in DiVA

By author/editor
Perez Martinez, AdalbertoMarkidis, StefanoSchlatter, Philipp
By organisation
Fluid MechanicsComputational Science and Technology (CST)Engineering Mechanics
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 89 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf