The simulation of turbulent flows requires high spatial resolution in potentially a priori unknown, solution -dependent locations. To achieve adaptive refinement of the mesh, we rely on error indicators. We assess the differences between an error measure relying on the local convergence properties of the numerical solution and a goal-oriented error measure based on the computation of an adjoint problem. The latter method aims at optimizing the mesh for the calculation of a predefined integral quantity, or functional of interest. This work follows on from a previous study conducted on steady flows in Offermans et al. (2020) and we extend the use of the so-called adjoint error estimator to three-dimensional, turbulent flows. They both represent a way to achieve error control and automatic mesh refinement (AMR) for the numerical approximation of the Navier-Stokes equations, with a spectral element method discretization and non-conforming h-refinement.The current study consists of running the same physical flow case on gradually finer meshes, starting from a coarse initial grid, and to compare the results and mesh refinement patterns when using both error measures. As a flow case, we consider the turbulent flow in a constricted, periodic channel, also known as the periodic hill flow, at four different Reynolds numbers: Re = 700, Re = 1400, Re = 2800 and Re = 5600. Our results show that both error measures allow for effective control of the error, but they adjust the mesh differently. Well-resolved simulations are achieved by automatically focusing refinement on the most critical regions of the domain, while significant saving in the overall number of elements is attained, compared to statically generated meshes. At all Reynolds numbers, we show that relevant physical quantities, such as mean velocity profiles and reattachment/separation points, converge well to reference literature data. At the highest Reynolds number achieved (Re = 5600), relevant quantities, i.e. reattachment and separation locations, are estimated with the same level of accuracy as the reference data while only using one-third of the degrees of freedom of the reference. Moreover, we observe distinct mesh refinement patterns for both error measures. With the spectral error indicator, the mesh resolution is more uniform and turbulent structures are more resolved within the whole domain. On the other hand, the adjoint error estimator tends to focus the refinement within a localized zone in the domain, dependent on the functional of interest, leaving large parts of the domain marginally resolved.

Adaptive mesh refinement (AMR) is an important component of modern numerical solvers, as it allows to control the computational error during the simulation, increasing the reliability of the numerical modelling and giving the possibility to study a broad range of different phenomena even without knowing the physics a priori. In this work we present selected aspects of the implementation and parallel performance of a new h-type AMR framework developed for the high-order CFD solver Nek5000; the development was done within the ExaFLOW EU project. We utilise in this case the natural domain decomposition inherent to the spectral element method (SEM), which constitutes the main source of parallelism and provides meshing flexibility that can be exploited in AMR. We use standard libraries for parallel mesh management (p4est) and partitioning (ParMetis) and focus on developing efficient preconditioners for the pressure problem solved on non-conforming meshes. Two different approaches are considered: an additive overlapping Schwarz and a hybrid Schwarz-multigrid method. The strong scaling is shown on the example of the simulation of the turbulent flow around a NACA4412 wing section at Re = 200, 000. 

Adaptive mesh refinement is performed in the framework of the spectral element method augmented by approaches to error estimation and control. The h-refinement technique is used for adapting the mesh, where selected grid elements are split by a quadtree (2D) or octree (3D) structure. Continuity between parent-child elements is enforced by high-order interpolation of the solution across the common faces. Parallel mesh partitioning and grid management respectively, are taken care of by the external libraries ParMETIS and p4est. Two methods are considered for estimating and controlling the error of the solution. The first error estimate is local and based on the spectral properties of the solution on each element. This method gives a local measure of the L-2-norm of the solution over the entire computational domain. The second error estimate uses the dual-weighted residuals method - it is based on and takes into account both the local properties of the solution and the global dependence of the error in the solution via an adjoint problem. The objective of this second approach is to optimize the computation of a given functional of physical interest. The simulations are performed by using the code Nek5000 and three steady-state test cases are studied: a two-dimensional lid-driven cavity at Re = 7, 500, a two-dimensional flow past a cylinder at Re = 40, and a three-dimensional lid-driven cavity at Re = 2,000 with a moving lid tilted by an angle of 30 degrees. The efficiency of both error estimators is compared in terms of refinement patterns and accuracy on the functional of interest. In the case of the adjoint error estimators, the trend on the error of the functional is shown to be correctly represented up to a multiplicative constant.

The implementation of adaptive mesh refinement (AMR) in the spectral-element method code Nek5000 is used for the first time on the well-resolved large-eddy  simulation (LES) of the turbulent flow over wings. In particular, the flow over a NACA4412 profile with a 5° angle of attack at chord-based Reynolds number Re_c=200,000 is analysed in the present work. The mesh, starting from a coarse resolution, is progressively refined by means of AMR, which allows for high resolution near the wall and wake whereas significantly larger elements are used in the far-field. The resulting mesh is of higher resolution than those in previous conformal cases, and it allows for the use of larger computational domains, avoiding the use of precursor RANS simulations to determine the boundary conditions. All of this with, approximately, 3 times lower total number of grid points if the same spanwise length is used. Turbulence statistics obtained in the AMR simulation show good agreement with the ones obtained with the conformal mesh. Finally, using AMR on wings will enable simulations at Re_c beyond 1 million, thus allowing the study of pressure-gradient effects at high Reynolds numbers relevant for practical applications.

Simulations have matured to be an important tool in the design and analysis of many modern industrial devices, in particular in the process, vehicle and aeronautical industries. Most of the considered flows are turbulent, for which it is impossible to perform straightforward grid-convergence studies without considering long-term statistics. Therefore, when performing simulations in computational fluid dynamics (CFD), the use of adaptive mesh refinement (AMR) is an effective strategy to increase the reliability of the solution at a reduced computational cost. Such tools allow for error control, reduced simulation time, easier mesh generation, better mesh quality and better resolution of the a priori unknown physics. In the present contribution, the design of an optimal mesh, using AMR, is investigated in Nek5000 [3], a highly scalable code based on the spectral element method (SEM) [9] and aimed at the direct numerical simulation (DNS) of the incompressible Navier–Stokes equations (further discussed in section “Numerical Method”)

Adaptive mesh refinement (AMR) is an important component of modern numerical solvers, as it allows to control the computational error during the simulation, increasing the reliability of the numerical modelling and giving the possibility to study a broad range of different phenomena even without knowing the physics a priori. In this work we present selected aspects of the implementation and parallel performance of a new h−type AMR framework developed for the high-order CFD solver Nek5000; the development was done within the ExaFLOW EU project. We utilise in this case the natural domain decomposition inherent to the spectral element method (SEM), which constitutes the main source of parallelism and provides meshing flexibility that can be exploited in AMR. We use standard libraries for parallel mesh management (p4est) and partitioning (ParMetis) and focus on developing efficient preconditioners for the pressure problem solved on non-conforming meshes. Two different approaches are considered: an additive overlapping Schwarz and a hybrid Schwarz-multigrid method.The strong scaling is shown on the example of the simulation of the turbulent flow around a NACA4412 wing section at Rec = 200, 000.

BoomerAMG, the algebraic multigrid solver from the hypre library, is used to solve a coarse grid problem which is part of the preconditioning strategy for thepressure equation arising from the numerical resolution of the Navier–Stokes equations. A set of optimal parameters for the setup phase is determined and used for selected strong scaling tests on two different supercomputers, namely Mira and Hazel Hen, on up to 131, 072 compute cores. The results are compared to an existing algebraic multigrid solver, designed specifically for the coarse gridproblem at hand. It is shown that the BoomerAMG solver is fast and scalable, and that performance depends on the computer architecture. The test cases considered are the turbulent flow past a NACA4412 airfoil and the turbulent flow inside wire-tapped pin bundles.

Hypre, a library for linear algebra, is used to replace a Matlab code for performing the setup step of an Algebraic Multigrid Method (AMG). The AMG method is used to compute part of the preconditioner in Nek5000, a code for Computational Fluid Dynamics based on the spectral element method. However, the solution of the AMG problem is not performed via Hypre but by Nek5000’s internal solver. The new AMG setup is shown to be faster by at least one order of magnitude, while it does not significantly impact the efficiency of the AMG solver, as is shown from its application to relevant test cases.

Unsteady adjoint error estimators based on the dual-weighted residuals method are implemented for the spectral element method in Nek5000. The time-integration of the adjoint problem is performed based on the nonlinear direction solution recomputed via the revolve algorithm, which uses an optimal check-pointing strategy. Adaptive mesh refinement is performed on the flow inside a constricted periodic channel, the so-called periodic hill case, at four different Reynolds numbers, Re = 700, 1400, 2800 and 5600. This case is fully turbulent at all regimes, with significant flow separation, requires curved meshes, but yet has a number of accurate reference solutions in the literature. The chosen method to adapt the mesh is h-refinement, where selected elements are split by an oct-tree structure in three dimensions. The objective function for the adjoint estimators is the integral of the friction forces along the flat bottom wall between the hills, for which the location of the reattachment becomes crucial. The refinement process is compared between the adjoint error estimators and classical straightforward a posteriori spectral error indicators based on the local approximation properties of the solution.The turbulent simulations using mesh adaptation are stable, free of spurious numerical noise and accurate, as shown by comparing the statistical profiles of relevant flow quantities with reference data. The comparison between the error estimators shows that the adjoint error estimators tend to refine the mesh only around localized regions in the computational domain while leaving other areas under-resolved. However, only the locally refined regions are shown to have a significant impact on the value of the objective function and thus on the location of the reattachment point. Conversely, the spectral error indicatorstend to homogenize the error on the solution over the whole domain but have a lesser direct influence on the location of the reattachment point.

The implementation of adaptive mesh refinement (AMR) in Nek5000 is used for the first time on the simulation of the flow over wings. This is done by simulating the flow over a NACA4412 profile with 5 degrees angle of attack at chord-based Reynolds number 200,000. The mesh is progressively refined by means of AMR which allows for high resolution near the wall whereas significantly larger elements are used in the far-field. The resultant mesh shows higher resolution than previous conformal meshes, and it allows for larger computational domains,which avoid the use of RANS to determine the boundary condition, all of this with, approximately, 3 times lower total number of grid points. The results ofthe turbulence statistics show a good agreement with the ones obtained with the conformal mesh. Finally, using AMR on wings leads to simulations at higher Reynolds numbers (i.e. Rec = 850, 000) in order to analyse the effect of adverse pressure gradients at high Reynolds numbers.