Author: Lena Bühler (High Performance Computing Center Stuttgart, Germany)

After three years of working on key algorithmic challenges in CFD (Computational Fluid Dynamics), the ExaFLOW project reminisces about what has been achieved during the project runtime. With three flagship runs, ExaFLOW has managed to work on different specific CFD use cases which highlight the importance of their outcomes for both industry and academia.

CFD is a prime contender for reaching exascale performance: In fluid dynamics and in particular turbulence there is virtually no limit to the size of a relevant system to be studied via numerical simulations, which can be exploited for extreme parallel scaling. Moreover, turbulent flows are an essential element of many industrial and academic problems: A crude estimate shows that 10% of the energy use in the world is spent overcoming turbulent friction.

In order to enable the use of accurate simulation models in exascale environments, ExaFLOW conducted three Flagship runs with high industrial relevance, addressing key innovation areas such as mesh adaptivity, resilience, and strong scaling at exascale. Each of the flagship runs addresses specific CFD use cases, demonstrating the improvements made to the utilized open-source community codes: Nektar++, Nek5000, and OpenSBLI.

Flagship run No. 1: Mesh refinement of a turbulent flow with Nek5000

For incompressible flow around an asymmetric wing profile - the NACA 4412 airfoil - adaptive mesh refinement was coupled to the highly accurate spectral-element open source code Nek5000. As a result, the mesh is evolving dynamically depending on the estimated computational error at any given point in space and time: the simulation determines the required mesh on its own, leading to less grid points and making the design of an initial mesh much simpler. Therefore, the through-put time for a computational engineer is much faster, keeping specific error measures of the solution fulfilled. The researchers anticipate that solution-aware adaptive simulation techniques will play a major role in future CFD.

Flagshiprun1: Turbulent (incompressible) flow around a NACA-4412 profile 

Flagship run No. 2: Error indicators in a compressible flow with OpenSBLI

For the compressible airfoil flagship runs, error indicators were implemented into OpenSBLI. This compressible open source code can automatically generate code for GPU and CPU architectures based on high-level problem specifications. The error indicators enabled rapid validation of grids containing up to 5 billion points for simulation that were subsequently run on Europe’s fastest current supercomputers on more than 60,000 cores. Such simulations will contribute to the design of next-generation aircraft.

Flagship run No. 3: Turbulent flow around the Imperial Front Wing with Nektar++

The Imperial front wing combines the challenge of modelling both highly loaded wing profiles in ground effect and the interaction of transient bluff body wakes with vortical structures on a complex geometry where the challenges of high order meshing and solver robustness are evident. We were able to successfully run this geometry at full experimental Reynolds and varying polynomial order in Nektar++ and compare it with state of the art finite volume LES and time resolved experimental data. This demonstrates significant progress in leveraging Nektar++ for the goal of moving high-order CFD from an academic to industrial tool, and some of the benefits of high-order methods can be presented to the industrial community. 

All partners will continue their efforts improving the open-source community codes with the aim to enable fast and fault tolerant CFD simulations for both academia and industry on the exascale. The ExaFLOW improvements can be found on the websites which link to the respective repositories:

https://nek5000.mcs.anl.gov/

https://www.nektar.info/

https://opensbli.github.io/