Recent trends and advancements in including more diverse and heterogeneous hardware in High-Performance Computing (HPC) are challenging scientific software developers in their pursuit of efficient numerical methods with sustained performance across a diverse set of platforms. As a result, researchers are today forced to re-factor their codes to leverage these powerful new heterogeneous systems. We present our design considerations of Neko—a portable framework for high-fidelity spectral element flow simulations. Unlike prior works, Neko adopts a modern object-oriented Fortran 2008 approach, allowing multi-tier abstractions of the solver stack and facilitating various hardware backends ranging from general-purpose processors, accelerators down to exotic vector processors and Field-Programmable Gate Arrays (FPGAs). Focusing on the performance and portability of Neko, we describe the framework's device abstraction layer managing device memory, data transfer and kernel launches from Fortran, allowing for a solver written in a hardware-neutral yet performant way. Accelerator-specific optimizations are also discussed, with auto-tuning of key kernels and various communication strategies using device-aware MPI. Finally, we present performance measurements on a wide range of computing platforms, including the EuroHPC pre-exascale system LUMI, where Neko achieves excellent parallel efficiency for a large direct numerical simulation (DNS) of turbulent fluid flow using up to 80% of the entire LUMI supercomputer.

Rotors play a major role in various applications including ventilation and propulsion systems such as in helicopters, drones, gas turbines and wind turbines. This visualization of instantaneous vortical structures (identified by the  criterion) shows complex flow structures emanating from a twisted drone rotor that is impulsively starting to rotate at 1600 rpm. Initially, a starting vortex is formed as a result of lift generation and shed as a connected vortex tube from the entire surface of the blade, which has a strong connection to the blade tip via the so-called tip vortex. Leading edge separation occurs at span positions of high twist, followed by wave-induced breakdown to turbulence along the whole wing span. This turbulence then sheds as small-scale vortices into the wake and dissipates. Understanding the behaviour of these vortices from such complex blades and how they interact with the other blade is critical to design more efficient and potentially more silent propellers. 

The computational power of High-Performance Computing (HPC) systems increases continuously and rapidly. Data-intensive applications are designed to leverage the high computational capacity of HPC resources and typically generate a large amount of data for traditional post-processing data analytics. However, the HPC systems' in-/output (IO) subsystem develops relatively slowly, and the storage capacity is limited. This could lead to limited actual performance and scientific discovery. In-situ techniques are a partial remedy to these problems by reducing or avoiding the data flow through the IO subsystem to/from the storage. However, in current practice, asynchronous in-situ techniques with static resource management often allocate separate computing resources for executing in-situ task(s), which remain idle if no in-situ work is at hand. In the present work, we target improving the efficiency of computing resource usage by launching and releasing necessary additional computing resources for in-situ task(s). Our approach is based on extensions for MPI Sessions that enable the required dynamic resource management. In this paper, we propose a basic and an advanced in-situ techniques with dynamic resource management enabled by MPI Sessions, their implementations on two real-world use cases, and a critical analysis of the experimental results.

An experimental characterization of the turbulent boundary layers developing around a NACA 4412 wing profile is carried out in the Minimum Turbulence Level (MTL) wind tunnel located at KTH Royal Institute of Technology. The campaign included collecting wall-pressure data via built-in pressure taps, capturing velocity signals in the turbulent boundary layers (TBLs) using hot-wire anemometry (HWA), and conducting direct skin-friction measurements with oil-film interferometry (OFI). The research spanned two chord-based Reynolds numbers (Rec=4×105 and 106) and four angles of attack (5°, 8°, 11° and 14°), encompassing a broad spectrum of flow conditions, from mild to strong adverse-pressure gradients (APGs), including scenarios where the TBL detaches from the wing surface. This dataset offers crucial insights into TBL behavior under varied flow conditions, particularly in the context of APGs. Key features include the quasi-independence of the pressure coefficient distributions from Reynolds number, which aids in distinguishing Reynolds-number effects from those due to APG strengths. The study also reveals changes in TBL dynamics as separation approaches, with energy shifting from the inner to the outer region and the eventual transition to a free-shear flow state post-separation. Additionally, the diagnostic scaling in the outer region under spatial-resolution effects is considered, showing further evidence for its applicability for small L+, however with inconsistent results for larger L+. The findings and database resulting from this campaign may be of special relevance for the development and validation of turbulence models, especially in the context of aeronautical applications.

Fully-convolutional neural networks (FCN) were proven to be effective for predicting the instantaneous state of a fully-developed turbulent flow at different wall-normal locations using quantities measured at the wall. In Guastoni et al. (J Fluid Mech 928:A27, 2021. https://doi.org/10.1017/jfm.2021.812), we focused on wall-shear-stress distributions as input, which are difficult to measure in experiments. In order to overcome this limitation, we introduce a model that can take as input the heat-flux field at the wall from a passive scalar. Four different Prandtl numbers Pr=ν/α=(1,2,4,6) are considered (where ν is the kinematic viscosity and α is the thermal diffusivity of the scalar quantity). A turbulent boundary layer is simulated since accurate heat-flux measurements can be performed in experimental settings: first we train the network on aptly-modified DNS data and then we fine-tune it on the experimental data. Finally, we test our network on experimental data sampled in a water tunnel. These predictions represent the first application of transfer learning on experimental data of neural networks trained on simulations. This paves the way for the implementation of a non-intrusive sensing approach for the flow in practical applications.

The statistics obtained from turbulent flow simulations are generally uncertain due to finite time averaging. Most techniques available in the literature to accurately estimate these uncertainties typically only work in an offline mode, that is, they require access to all available samples of a time series at once. In addition to the impossibility of online monitoring of uncertainties during the course of simulations, such an offline approach can lead to input/output (I/O) deficiencies and large storage/memory requirements, which can be problematic for large-scale simulations of turbulent flows. Here, we designed, implemented and tested a framework for estimating time-averaging uncertainties in turbulence statistics in an in-situ (online/streaming/updating) manner. The proposed algorithm relies on a novel low-memory update formula for computing the sample-estimated autocorrelation functions (ACFs). Based on this, smooth modeled ACFs of turbulence quantities can be generated to accurately estimate the time-averaging uncertainties in the corresponding sample mean estimators. The resulting uncertainty estimates are highly robust, accurate, and quantitatively the same as those obtained by standard offline estimators. Moreover, the computational overhead added by the in-situ algorithm is found to be negligible allowing for online estimation of uncertainties for multiple points and quantities. The framework is general and can be used with any flow solver and also integrated into the simulations over conformal and complex meshes created by adopting adaptive mesh refinement techniques. The results of the study are encouraging for the further development of the in-situ framework for other uncertainty quantification and data-driven analyses relevant not only to large-scale turbulent flow simulations, but also to the simulation of other dynamical systems leading to time-varying quantities with autocorrelated samples.

Thermal anemometry sensors for time-resolved velocity measurements average the measured signal over the length of their sensor, thereby attenuating fluctuations stemming from scales smaller than the wire length. Several compensation methods have emerged for wall turbulence, the most prominent ones relying on the small-scale universality in canonical flows or on the reconstruction based on two attenuated variance profiles obtained with sensors of different length. To extend these methods to non-canonical flows, the present work considers various adverse-pressure gradient (APG) turbulent boundary layer (TBL) flows in order to explore how the small-scale energy is affected in the inner and outer layer and how the two prominent correction methods perform as function of wall-distance, wire length and flow condition. Our findings show that the increased levels of small-scale energy in the inner, but also outer layer associated with APG TBLs reduces the applicability of empirical methods based on the universality of small-scale energy. On the other hand, a correction based on the relationship between the spanwise Taylor microscale and the two-point streamwise velocity correlation function, is able to correct the attenuated profiles of non-canonical cases. Combining the strength of both methods, a composite profile for the spanwise Taylor microscale is suggested, which then is used for the correction of probe-length attenuation effects across a multitude of flow conditions.

It is well understood that spanwise arrays of roughness elements can be used to generate steady streaks in boundary layers. This modulation of the boundary layer has the potential to attenuate the growth of Tollmien–Schlichting (TS) waves which can lead to the transition to turbulence in low turbulence intensity environments, such as those experienced by an aircraft's fuselage in atmospheric flight. This article applies density based topology optimization in order to design roughness elements capable of exploiting the aforementioned stabilizing effect as a means of passive flow control. The geometry of the roughness elements are represented using a Brinkman penalization when conducting Direct Numerical Simulations (DNS) to simulate the streaky boundary layer flow. Similarly, the unsteady linearized Navier–Stokes equations are evolved to assess the spatial growth of the TS waves across the flat plate. The optimization procedure aims to minimize the TS wave amplitude at a given downstream position while a novel constraint is used promoting a stable baseflow. The optimization problem is solved with gradient descent algorithms where the adjoint-variable method is used to compute gradients. This method has been applied to three initial material distributions yielding three distinct and novel designs capable of damping the downstream growth of the TS wave significantly more than a reference Minature Vortex Generator (MVG) of comparable size. The optimized designs and streaky baseflows they induce are then studied using an energy budget analysis and local stability analysis.

The continuous Galerkin spectral-element method (SEM) has long been used for high-fidelity simulations of turbulent flows, particularly for direct numerical simulations. To circumvent the usual limitations on the maximum affordable Reynolds number, wall-modeled large-eddy simulation (WMLES) using SEM has been proposed in the literature. However, existing works are confined to channel flow, and several methodological questions remain, such as numerical stability and subgrid scale modeling, when grid quality is that typical of applications. Here, WMLES is conducted using the SEM solver Nek5000, considering three flows: turbulent channel flow, a flat-plate turbulent boundary layer, and flow over the A-airfoil at near-stall conditions. Thus, we expand the assessment of WMLES-based SEM to a larger class of flow conditions, with the principal conclusion that state-of-the-art predictive accuracy can be achieved. However, further methodological improvements are needed to obtain smooth solutions on coarser grids, which are typical of WMLES. We show that established algebraic subgrid scale models (Vreman, Sigma) introduce sufficient diffusion to stabilize simulations and outperform modal filtering, yet mean turbulent viscosity profiles still exhibit spurious peaks due to velocity derivative jumps across elements. We also investigate the impact of time-averaging the velocity input to the wall model and find it to be negligible. Finally, our simulations of the A-airfoil highlight the critical need for accurately resolving the laminar region near the leading edge and developing wall models capable of detecting and handling transition.

A framework is presented for the spectral-element code Nek5000, which has been, and still is, widely used in the computational fluid dynamics (CFD) community to perform high-fidelity numerical simulations of transitional and high Reynolds number flows. Despite the widespread usage, there is a deficiency in having a comprehensive set of tools specifically designed for conducting simulations using Nek5000. To address this issue, we have created a unique framework that allows, inter alia, to perform stability analysis and compute statistics of a turbulent flow. The framework encapsulates modules that provide tools, run-time parameters and memory structures, defining interfaces and performing different tasks. First, the framework architecture is described, showing its non-intrusive approach. Then, the modules are presented, explaining the main tools that have been implemented and describing some of the test cases. The code is open-source and available online, with proper documentation, to-run instructions and related examples.