Moving to C++20

[ax3l]

Hi,

@WeiqunZhang and I would like to start the discussion when people are ready to move to require C++20 for AMReX.

We have some optional recent features, like SIMD support, that already need C++20 when used. Generally, C++20 adds new features and math functions that we would like to exploit, as usual.

From looking at other projects reporting regressions: The usual candidate that holds us back is the CUDA Toolkit deployed on HPC machines that we run production runs on. From what I saw, I think we need to require at least CUDA/NVCC 12.4.1, even though NVCC added C++20 support already in CUDA 12.0+.

Community: What are your thoughts? Any important machines holding you back?

[BenWibking]

We have migrated Quokka to C++20 already. The issues we've seen are related to missing <concepts> and <ranges> headers on SLES 15 since the system libstdc++ is so old. No AMReX or GPU-specific issues so far.

The one other exception to this is when using <ranges> with HIP and libc++ (only): ROCm/rocm-systems#2141. This is probably very uncommon in practice.

[WeiqunZhang]

@BenWibking How do you work around the missing header issue?

[BenWibking]

@BenWibking How do you work around the missing header issue?

@WeiqunZhang On Frontier, it doesn't seem to be an issue with these modules:

module load PrgEnv-cray
module load cce/20.0.0
module load rocm/6.4.2
module load cmake/4.1.0
export CC=$(which hipcc)
export CXX=$(which hipcc)

I haven't tested other combinations.

In our SLES container (used for CI), I added libstdc++6-devel-gcc13 with the system package manager, and that seems to just work without changing anything else:
https://github.com/quokka-astro/quokka/blob/1dcebb5ac78db19d127a2205a5ac8e63951eba8e/.devcontainer/rocm-container/Dockerfile#L79

[mirenradia]

One of the systems I use (Tursa) currently only provides CUDA 12.3.0 in their CUDA modules (and Open MPI/UCX built with this) which I have been using to build AMReX. Thankfully, their CUDA drivers are new enough (CUDA 12.6) such that I could install a newer CUDA version/Open MPI/etc myself using Spack if required.