kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Reducing Communication in the Conjugate Gradient Method: A Case Study on High-Order Finite Elements
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Computational Science and Technology (CST).ORCID iD: 0000-0003-3374-8093
KTH, School of Electrical Engineering and Computer Science (EECS), Centres, Centre for High Performance Computing, PDC.ORCID iD: 0000-0002-5020-1631
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Computational Science and Technology (CST).ORCID iD: 0000-0001-5452-6794
KTH, School of Engineering Sciences (SCI), Engineering Mechanics, Fluid Mechanics and Engineering Acoustics.ORCID iD: 0000-0001-9627-5903
Show others and affiliations
2022 (English)In: Proceedings of the Platform for Advanced Scientific Computing Conference, PASC 2022, Association for Computing Machinery (ACM) , 2022, article id 2Conference paper, Published paper (Refereed)
Abstract [en]

Currently, a major bottleneck for several scientific computations is communication, both communication between different processors, so-called horizontal communication, and vertical communication between different levels of the memory hierarchy. With this bottleneck in mind, we target a notoriously communication-bound solver at the core of many high-performance applications, namely the conjugate gradient method (CG). To reduce the communication we present lower bounds on the vertical data movement in CG and go on to make a CG solver with reduced data movement. Using our theoretical analysis we apply our CG solver on a high-performance discretization used in practice, the spectral element method (SEM). Guided by our analysis, we show that for the Poisson equation on modern GPUs we can improve the performance by 30% by both rematerializing the discrete system and by reformulating the system to work on unique degrees of freedom. In order to investigate how horizontal communication can be reduced, we compare CG to two communication-reducing techniques, namely communication-avoiding and pipelined CG. We strong scale up to 4096 CPU cores and showcase performance improvements of upwards of 70% for pipelined CG compared to standard CG when applied on SEM at scale. We show that in addition to improving the scaling capabilities of the solver, initial measurements indicate that the convergence of SEM is largely unaffected by pipelined CG.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM) , 2022. article id 2
National Category
Information Systems
Identifiers
URN: urn:nbn:se:kth:diva-317542DOI: 10.1145/3539781.3539785Scopus ID: 2-s2.0-85134847143OAI: oai:DiVA.org:kth-317542DiVA, id: diva2:1695263
Conference
2022 Platform for Advanced Scientific Computing Conference, PASC 2022, 27 June 2022 through 29 June 2022, Basel, Switzerland
Note

QC 20220913

Part of proceedings: ISBN 978-145039410-9

Available from: 2022-09-13 Created: 2022-09-13 Last updated: 2024-04-22Bibliographically approved
In thesis
1. Direct Numerical Simulation of Turbulence on Heterogenous Computer Systems: Architectures, Algorithms, and Applications
Open this publication in new window or tab >>Direct Numerical Simulation of Turbulence on Heterogenous Computer Systems: Architectures, Algorithms, and Applications
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Direct numerical simulations (DNS) of turbulence have a virtually unbounded need for computing power. To carry out these simulations, software, computer architectures, and algorithms must operate as efficiently as possible to amortize the large computational cost. However, in a computing landscape increasingly incorporating heterogeneous computer systems, changes are necessary. In this thesis, we consider how DNS can be carried out efficiently on upcoming heterogeneous computer systems. This work relates to developing algorithms for upcoming heterogeneous computer architectures, overcoming software challenges associated with large-scale DNS on these platforms, and applying these developments to new flow cases that were previously too costly to carry out. We consider in particular the spectral element method for DNS and evaluate how this method maps to field-programmable gate arrays, graphics processing units, as well as conventional processors. We also consider the issue of trading arithmetic operations for less communication, reducing the cost of solving the linear systems that arise in the spectral element method. Our developments are incorporated into the spectral element framework Neko, enabling Neko to strong-scale efficiently on the largest supercomputers in the world. Finally, we have carried out several DNS such as the simulation of a Flettner rotor in a turbulent boundary layer and simulating Rayleigh-Bénard convection at very high Rayleigh numbers. The developments in this thesis enable the high-fidelity simulation of turbulence on emerging computer systems with high parallel efficiency and performance.

Abstract [sv]

Direct numerisk simulering (DNS) av turbulens kräver enorma mängder datorkraft. För att utföra simuleringar som DNS krävs det att mjukvara, datorarkitekturer och algoritmer samverkar så effektivt som möjligt tillsammans. Idag förändras superdatorer snabbt och inkoporerar nya heterogena datorarkitekturer. Detta innebär att nya tillvägagångssätt är nödvändiga för att tillgodogöra sig all beräkningskraft. I den här avhandlingen fokuserar vi på DNS på heterogena, storskaliga, datorsystem för att möjligöra nya simuleringar av turbulenta flöden. För att nå detta mål undersöker vi nya datorarkitekturer, analyserar och förbättrar de numeriska metoderna och algoritmerna vi använder och applicerar slutligen våra utvecklingar på nya simuleringar av turbulens. Vi fokuserar speciellt på den spektrala element metoden (SEM) för DNS och undersöker hur den beter sig på eng. field-programmable gate arrays, grafikkort och konventionella processorer. Vi bidrar även med analys av hur vi löser det linjära systemet som utgör kärnan i SEM för att bättre utnyttja den tillgängliga datorkraften och minska mängden data som behöver överföras. Våra förbättringar inkorporeras i SEM lösaren Neko och möjligör att Neko kan skala effektivt på de största superdatorerna i världen. Vi använder sedan detta ramverk för att genomföra flera storskaliga simuleringar. Vi genomför den första simuleringen av en Flettner rotor och dess interaktion med turbulent skjuvströmning samt simulering av Rayleigh-Bénard konvektion i en cylindrisk domän vid mycket höga Rayleigh tal. Avhandlingen möjligör detaljerad numerisk simulering av turbulens med hög skalbarhet och prestanda i dagens föränderliga datorlandskap. 

Place, publisher, year, edition, pages
Stockholm, Sweden: KTH Royal Institute of Technology, 2024. p. 54
Series
TRITA-EECS-AVL ; 2024:36
Keywords
High Performance Computing, Turbulence, Computational Fluid Dynamics, Heterogenous Computer Architectures, Högprestandaberäkningar, Turbulens, Numerisk Strömingsmekanik, Heterogena Datorarkitekturer
National Category
Computer Sciences Fluid Mechanics
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-345851 (URN)978-91-8040-910-0 (ISBN)
Public defence
2024-05-24, https://kth-se.zoom.us/s/61541415709, Kollegiesalen, Brinellvägen 6, Stockholm, 09:15 (English)
Opponent
Supervisors
Funder
Swedish e‐Science Research Center, SESSI
Note

QC 20240423

Available from: 2024-04-23 Created: 2024-04-22 Last updated: 2025-02-05Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Karp, MartinJansson, NiclasPodobas, ArturSchlatter, PhilippMarkidis, Stefano

Search in DiVA

By author/editor
Karp, MartinJansson, NiclasPodobas, ArturSchlatter, PhilippMarkidis, Stefano
By organisation
Computational Science and Technology (CST)Centre for High Performance Computing, PDCFluid Mechanics and Engineering Acoustics
Information Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 109 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf