As heterogeneous supercomputing architectures leveraging GPUs become increasingly central to high-performance computing (HPC), it is crucial for computational fluid dynamics (CFD) simulations, a de-facto HPC workload, to efficiently utilize such hardware. One of the key challenges of HPC codes is performance portability, i.e. the ability to maintain near-optimal performance across different accelerators. In the context of the REFMAP project, which targets scalable, GPU-enabled multi-fidelity CFD for urban airflow prediction, this paper analyzes the performance portability of SOD2D, a state-of-the-art Spectral Elements simulation framework across AMD and NVIDIA GPU architectures. We first discuss the physical and numerical models underlying SOD2D, highlighting its computational hotspots. Then, we examine its performance and scalability in a multi-level manner, i.e. defining and characterizing an extensive full-stack design space spanning across application, software and hardware infrastructure related parameters. Single-GPU performance characterization across server-grade NVIDIA and AMD GPU architectures and vendor-specific compiler stacks, show the potential as well as the diverse effect of memory access optimizations, i.e. 0.69× - 3.91× deviations in acceleration speedup. Performance variability of SOD2D at scale is further examined on the LUMI multi-GPU cluster, where profiling reveals similar throughput variations, highlighting the limits of performance projections and the need for multi-level, informed tuning.

As heterogeneous supercomputing architectures leveraging GPUs become increasingly central to high-performance computing (HPC), it is crucial for computational fluid dynamics (CFD) simulations to maintain performance portability. In this paper, we examine the performance and scalability of CFD framework SOD2D in a crosslayer manner, i.e. across application, software and hardware infrastructure related parameters. Single-GPU performance characterization across server-grade NVIDIA and AMD GPU architectures and vendor-specific compiler stacks, show the potential as well as the diverse effect of memory access optimizations, i.e. 0.69x - 3.96x deviations in acceleration speedup. Performance variability of SOD2D at scale is then further examined on the LUMI multi-GPU cluster, showcasing analogous diverse effects on throughput, demonstrating the ineffectiveness of adopting performance projections, thus underscoring the importance and necessity of cross-layer informed performance analysis and tuning for multi-GPU configurations.

We present new results on the strong parallel scaling for the OpenACC-accelerated implementation of the high-order spectral element fluid dynamics solver Nek5000. The test case considered consists of a direct numerical simulation of fully-developed turbulent flow in a straight pipe, at two different Reynolds numbers Reτ = 360 and Reτ = 550, based on friction velocity and pipe radius. The strong scaling is tested on several GPU-enabled HPC systems, including the Swiss Piz Daint system, TACC's Longhorn, Jülich's JUWELS Booster, and Berzelius in Sweden. The performance results show that speed-up between 3-5 can be achieved using the GPU accelerated version compared with the CPU version on these different systems. The run-time for 20 timesteps reduces from 43.5 to 13.2 seconds with increasing the number of GPUs from 64 to 512 for Reτ = 550 case on JUWELS Booster system. This illustrates the GPU accelerated version the potential for high throughput. At the same time, the strong scaling limit is significantly larger for GPUs, at about 2000 - 5000 elements per rank; compared to about 50 - 100 for a CPU-rank.

The Swedish academic computing landscape is organized under the auspices of SNIC, the Swedish National Infrastructure for Computing. SNIC coordinates investments in computing and storage infrastructure at its six national centers and manages the national process for allocating research time on its computing resources. Since its formation in 2003, SNIC has significantly increased the computational capacity available to Swedish researchers and firmly put Sweden on the international computational science map. When the Partnership for Advanced Computing in Europe (PRACE) started in 2010, SNIC joined this European HPC effort and worked with the Swedish Research 142Council to allocate additional funds for a national high-end system that would also be made available to European researchers via PRACE. These efforts resulted in the installation of a CRAY XE6 supercomputer, named Lindgren, at the PDC Center for High-Performance Computing at the KTH Royal Institute of Technology in Stockholm. 

Photosynthetic reaction centers convert the energy content of light into a transmembrane potential difference and so provide the major pathway for energy input into the biosphere. We applied time-resolved Laue diffraction to study light-induced conformational changes in the photosynthetic reaction center complex of Blastochloris viridis. The side chain of TyrL162, which lies adjacent to the special pair of bacteriochlorophyll molecules that are photooxidized in the primary light conversion event of photosynthesis, was observed to move 1.3 angstroms closer to the special pair after photoactivation. Free energy calculations suggest that this movement results from the deprotonation of this conserved tyrosine residue and provides a mechanism for stabilizing the primary charge separation reactions of photosynthesis.