kth.sePublications KTH
System disruptions
There are recurring service disruptions on some of the search portals due to high traffic. We are working to resolve the issue, you may temporarily encounter an error message.
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Effect of hyper-parameters on streaming and parallel proper orthogonal decomposition
KTH, School of Engineering Sciences (SCI), Engineering Mechanics, Fluid Mechanics.ORCID iD: 0000-0001-5204-8549
Department of Fluids and Environment, University of Manchester, Manchester, United Kingdom.ORCID iD: 0000-0002-9610-9910
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Computational Science and Technology (CST).ORCID iD: 0000-0003-0639-0639
KTH, School of Engineering Sciences (SCI), Engineering Mechanics, Fluid Mechanics. Institute of Fluid Mechanics (LSTM), Friedrich–Alexander Universität Erlangen–Nürnberg (FAU), DE-910 58 Erlangen, Germany.ORCID iD: 0000-0001-9627-5903
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Proper orthogonal decomposition (POD) is a quintessential tool in the analysis of flow fields in experimental and computational fluid dynamics as a whole and turbulence in particular. It is a data driven technique that can be used to extract coherent structures in the flow and to create reduced-order dynamical systems that represent the most energetic flow structures. While extremely useful, its main drawback is that, at least traditionally, the POD algorithm is executed in serial using all the available data at once, while the original full-order system (simulation or experiment) is typically conducted and concluded before, either using a parallel simulation code or an (optical field-based) experiment. This can potentially limit its applications, in particular for large-scale data sets, e.g., from three-dimensional turbulent flows. In this article, we collect and review some of the more flexible methods for calculation of the POD (in particular, based on singular value decomposition, SVD) that have appeared over the yearsi n order to tackle these limitations. We summarize an implementation that combines the characteristics of the (distributed-memory) parallel and streaming methods and conduct detailed uncertainty quantification studies to address the effect of the hyper-parameters that typically appear with the introduction of streaming and parallel computations. We find that in general, the number of parallel processes used to compute the POD has no significant effect. On the other hand, the accuracy of the results depends on the number of updated modes and the number of snapshots treated simultaneously, with 60% and 30% of global sensitivity, respectively, in the error in the obtained POD modes. Given these findings, we formulate the recommendation that it is beneficial that more POD modes than those desired for later analysis are continuously updated. We find that the errors introduced by using streaming and parallel algorithms are however minimal in all cases. Therefore, the use of these algorithms for either computation on smaller memory systems, or online (in-situ) computation of POD modes as simulations advance is a reliable and a robust way forward to increase the rate of scientific discovery in large-scale fluid-mechanics problems.

National Category
Engineering and Technology
Identifiers
URN: urn:nbn:se:kth:diva-371427OAI: oai:DiVA.org:kth-371427DiVA, id: diva2:2005579
Note

QC 20251016

Available from: 2025-10-10 Created: 2025-10-10 Last updated: 2025-10-16Bibliographically approved
In thesis
1. Data analysis and data reduction for large-scale turbulence simulations
Open this publication in new window or tab >>Data analysis and data reduction for large-scale turbulence simulations
2025 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Computational fluid dynamics (CFD) and direct numerical simulations (DNS),when applied to the study of turbulence, have traditionally been treated as acompute bound discipline, where the size of problems of interest is limited bythe capacity of supercomputers. While this remains true, the relatively recentadoption of specialized hardware such as graphics processing units (GPUs) hasallowed researchers to start studying problems that were not thought possiblebefore. This trend brings benefits for scientific discovery, however, it alsoaccentuates the importance of robust methodologies to manage and processthe increasing amount of data that is being produced by the simulations. Thepresent thesis explores techniques to process large scale data sets produced,mainly, by the spectral element method (SEM).This study explores the possibility to exploit the computational resourcesused by the simulations to perform data analysis and transformations in whatis termed, in-situ data processing. It is shown that it is viable to apply amultitude of processing tasks, such as data compression and image visualizationefficiently, as long as the hardware being used is taken into consideration, whichis relevant for modern heterogeneous systems. Furthermore it is shown that datacompression is an efficient technique to reduce storage requirements while keepingaccuracy, even for turbulence research. On this note, this thesis introduces amethod that incorporates uncertainty quantification (UQ) techniques for datacompression to facilitate the data quality evaluation.Data compression is a large focus in the present work, however, methodsto facilitate data analysis are also studied. Streaming and parallel modaldecompositions, in particular proper orthogonal decomposition (POD), aredeveloped and made available to the turbulence community with the additionof uncertainty quantification studies to ease its adoption. It is found thatthis sort of technique is excellent at increasing the interpretability of the data,while being able to exploit computational resources with in-situ execution.Additionally, parallel high-order interpolation techniques are introduced, whichbecome essential to reduce the memory footprint of large data sets whenperforming post-processing tasks, while aiding to simplify the data distributionof traditional SEM meshes.

Abstract [sv]

Beräkningsvätskedynamik (CFD) och direkta numeriska simuleringar (DNS)har, när de tillämpas på studien av turbulens, traditionellt behandlats som enberäkningsbegränsad disciplin där storleken på intressanta problem begränsas avsuperdatorers kapacitet. Även om detta fortfarande stämmer har den relativt nyaanvändningen av specialiserad hårdvara, såsom grafikprocessorer (GPU:er), gjortdet möjligt för forskare att börja studera problem som tidigare betraktades somoåtkomliga. Denna utveckling gynnar den vetenskapliga kunskapsutvecklingen,men den betonar också behovet av robusta metoder för att hantera och bear-beta den växande datamängd som genereras av simuleringarna. Föreliggandeavhandling undersöker tekniker för att bearbeta storskaliga datamängder somhuvudsakligen produceras med spektralelementmetoden (SEM).Denna studie utforskar möjligheten att utnyttja de beräkningsresurser somanvänds av simuleringarna för att utföra dataanalys och transformationer inomså kallad in situ-databehandling. Den belyser att det är möjligt att effektivtgenomföra en rad bearbetningsuppgifter, såsom datakomprimering och visualise-ring, under förutsättning att den använda hårdvaran beaktas, vilket är särskiltrelevant för moderna heterogena system. Vidare åskådliggörs att datakomprime-ring är en effektiv metod för att minska lagringsbehovet samtidigt som noggrann-heten bibehåll, även inom turbulensforskning. I detta sammanhang introduceraravhandlingen en metod som integrerar tekniker för osäkerhetskvantifiering (UQ)i datakomprimering för att underlätta bedömningen av datakvalitet.Datakomprimering är ett centralt fokus i arbetet, men även metodersom underlättar dataanalys studeras. Strömmande och parallella modalned-brytningar, i synnerhet Proper Orthogonal Decomposition (POD), utveck-las och görs tillgängliga för turbulensfältet, kompletterade med studier avosäkerhetskvantifiering för att underlätta införandet. Det konstateras att dennatyp av teknik i hög grad ökar datans tolkbarhet samtidigt som den kan utnyttjaberäkningsresurser genom in situ. Dessutom introduceras parallella interpola-tionsmetoder av hög ordning, vilka är avgörande för att minska minnesavtryckethos stora datamängder vid efterbehandling och som samtidigt bidrar till attförenkla datadistributionen i traditionella SEM-nät.

Place, publisher, year, edition, pages
Stockholm, Sweden: KTH Royal Institute of Technology, 2025. p. 56
Series
TRITA-SCI-FOU ; 2025:39
Keywords
Turbulence, Data compression, Data processing, Modal decompositions, Interpolation, Turbulens, Datakompression, Databehandling, Modaldekomposition, Interpolation
National Category
Fluid Mechanics
Research subject
Engineering Mechanics
Identifiers
urn:nbn:se:kth:diva-371434 (URN)978-91-8106-376-9 (ISBN)
Public defence
2025-11-14, F3, Lindstedtvägen 26, https://kth-se.zoom.us/j/69836085996, Stockholm, 10:15 (English)
Opponent
Supervisors
Note

QC 251013

Available from: 2025-10-13 Created: 2025-10-10 Last updated: 2025-12-02Bibliographically approved

Open Access in DiVA

No full text in DiVA

Authority records

Perez Martinez, AdalbertoRezaeiravesh, SalehMarkidis, StefanoSchlatter, Philipp

Search in DiVA

By author/editor
Perez Martinez, AdalbertoRezaeiravesh, SalehMarkidis, StefanoSchlatter, Philipp
By organisation
Fluid MechanicsComputational Science and Technology (CST)
Engineering and Technology

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 85 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf