kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Experience and Analysis of Scalable High-Fidelity Computational Fluid Dynamics on Modular Supercomputing Architectures
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Computational Science and Technology (CST).ORCID iD: 0000-0003-3374-8093
Forschungszentrum Julich GmbH, Juelich Supercomputing Centre; Rheinische Friedrich-Wilhelms-Universität Bonn, Institut für Informatik.ORCID iD: 0000-0003-0748-7264
Forschungszentrum Julich GmbH, Juelich Supercomputing Centre.
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Computational Science and Technology (CST).ORCID iD: 0000-0002-6384-2630
Show others and affiliations
(Swedish)Manuscript (preprint) (Other academic)
Abstract [en]

The never-ending computational demand from simulations of turbulence makes computational fluid dynamics (CFD) a prime application use case for current and future exascale systems. High-order finite element methods, such as the spectral element method, have been gaining traction as they offer high performance on both multicore CPUs and modern GPU-based accelerators. In this work, we assess how high-fidelity CFD using the spectral element method can exploit the modular supercomputing architecture at scale through domain partitioning, where the computational domain is split between GPUs and CPUs. We investigate several different flow cases and computer systems based on the MSA. We observe that for our simulations, the communication overhead and load balancing issues incurred by incorporating different computing architectures are seldom worthwhile, especially when I/O is also considered, but when the simulation at hand requires more than the combined global memory on the GPUs, utilizing additional CPUs to increase the available memory can be fruitful. We support our results with a simple performance model to assess when running across modules might be beneficial. For a smaller supercomputer where the computation takes significant amounts of time on the CPU module, it can be beneficial to also use a GPU module to decrease the execution time significantly.

National Category
Computer Systems Fluid Mechanics Computational Mathematics
Identifiers
URN: urn:nbn:se:kth:diva-345656OAI: oai:DiVA.org:kth-345656DiVA, id: diva2:1852092
Funder
EU, Horizon 2020, 955606Swedish Research Council, 2019-04723Swedish e‐Science Research Center
Note

QC 20240417

Available from: 2024-04-16 Created: 2024-04-16 Last updated: 2025-02-05Bibliographically approved
In thesis
1. Direct Numerical Simulation of Turbulence on Heterogenous Computer Systems: Architectures, Algorithms, and Applications
Open this publication in new window or tab >>Direct Numerical Simulation of Turbulence on Heterogenous Computer Systems: Architectures, Algorithms, and Applications
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Direct numerical simulations (DNS) of turbulence have a virtually unbounded need for computing power. To carry out these simulations, software, computer architectures, and algorithms must operate as efficiently as possible to amortize the large computational cost. However, in a computing landscape increasingly incorporating heterogeneous computer systems, changes are necessary. In this thesis, we consider how DNS can be carried out efficiently on upcoming heterogeneous computer systems. This work relates to developing algorithms for upcoming heterogeneous computer architectures, overcoming software challenges associated with large-scale DNS on these platforms, and applying these developments to new flow cases that were previously too costly to carry out. We consider in particular the spectral element method for DNS and evaluate how this method maps to field-programmable gate arrays, graphics processing units, as well as conventional processors. We also consider the issue of trading arithmetic operations for less communication, reducing the cost of solving the linear systems that arise in the spectral element method. Our developments are incorporated into the spectral element framework Neko, enabling Neko to strong-scale efficiently on the largest supercomputers in the world. Finally, we have carried out several DNS such as the simulation of a Flettner rotor in a turbulent boundary layer and simulating Rayleigh-Bénard convection at very high Rayleigh numbers. The developments in this thesis enable the high-fidelity simulation of turbulence on emerging computer systems with high parallel efficiency and performance.

Abstract [sv]

Direct numerisk simulering (DNS) av turbulens kräver enorma mängder datorkraft. För att utföra simuleringar som DNS krävs det att mjukvara, datorarkitekturer och algoritmer samverkar så effektivt som möjligt tillsammans. Idag förändras superdatorer snabbt och inkoporerar nya heterogena datorarkitekturer. Detta innebär att nya tillvägagångssätt är nödvändiga för att tillgodogöra sig all beräkningskraft. I den här avhandlingen fokuserar vi på DNS på heterogena, storskaliga, datorsystem för att möjligöra nya simuleringar av turbulenta flöden. För att nå detta mål undersöker vi nya datorarkitekturer, analyserar och förbättrar de numeriska metoderna och algoritmerna vi använder och applicerar slutligen våra utvecklingar på nya simuleringar av turbulens. Vi fokuserar speciellt på den spektrala element metoden (SEM) för DNS och undersöker hur den beter sig på eng. field-programmable gate arrays, grafikkort och konventionella processorer. Vi bidrar även med analys av hur vi löser det linjära systemet som utgör kärnan i SEM för att bättre utnyttja den tillgängliga datorkraften och minska mängden data som behöver överföras. Våra förbättringar inkorporeras i SEM lösaren Neko och möjligör att Neko kan skala effektivt på de största superdatorerna i världen. Vi använder sedan detta ramverk för att genomföra flera storskaliga simuleringar. Vi genomför den första simuleringen av en Flettner rotor och dess interaktion med turbulent skjuvströmning samt simulering av Rayleigh-Bénard konvektion i en cylindrisk domän vid mycket höga Rayleigh tal. Avhandlingen möjligör detaljerad numerisk simulering av turbulens med hög skalbarhet och prestanda i dagens föränderliga datorlandskap. 

Place, publisher, year, edition, pages
Stockholm, Sweden: KTH Royal Institute of Technology, 2024. p. 54
Series
TRITA-EECS-AVL ; 2024:36
Keywords
High Performance Computing, Turbulence, Computational Fluid Dynamics, Heterogenous Computer Architectures, Högprestandaberäkningar, Turbulens, Numerisk Strömingsmekanik, Heterogena Datorarkitekturer
National Category
Computer Sciences Fluid Mechanics
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-345851 (URN)978-91-8040-910-0 (ISBN)
Public defence
2024-05-24, https://kth-se.zoom.us/s/61541415709, Kollegiesalen, Brinellvägen 6, Stockholm, 09:15 (English)
Opponent
Supervisors
Funder
Swedish e‐Science Research Center, SESSI
Note

QC 20240423

Available from: 2024-04-23 Created: 2024-04-22 Last updated: 2025-02-05Bibliographically approved

Open Access in DiVA

No full text in DiVA

Authority records

Karp, MartinAndersson, MånsSchlatter, PhilippMarkidis, StefanoJansson, Niclas

Search in DiVA

By author/editor
Karp, MartinSuarez, EstelaAndersson, MånsSchlatter, PhilippMarkidis, StefanoJansson, Niclas
By organisation
Computational Science and Technology (CST)Turbulent simulations laboratoryCentre for High Performance Computing, PDC
Computer SystemsFluid MechanicsComputational Mathematics

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 243 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf