Code_Saturne solves the Navier-Stokes equations for 2D, 2D-axisymmetric and 3D flows, steady or unsteady, laminar or turbulent, incompressible or weakly dilatable, isothermal or not, with scalar transport if required. Several turbulence models are available, from Reynolds-averaged models to large-eddy simulation (LES) models. In addition, a number of specific physical models are also available as "modules": gas, coal and heavy-fuel oil combustion, semi-transparent radiative transfer, particle-tracking with Lagrangian modeling, Joule effect, electrics arcs, weakly compressible flows, atmospheric flows, rotor/stator interaction for hydraulic machines.
Using Code_Saturne on ARCHER2
Code_Saturne is released under the GNU General Public Licence v2 and so is freely available to all users on ARCHER2.
You can load the default GCC build of Code_Saturne for use by running the following command:
module load code_saturne
This will load the default
code_saturne/7.0.1-gcc11 module. A build using the CCE compilers,
code_saturne/7.0.1-cce12, has also been made optionally available to users on the full ARCHER2 system as testing indicates that this may provide improved performance over the GCC build.
Running parallel Code_Saturne jobs
After setting up a case it should be initialized by running the following command from the case directory, where setup.xml is the input file:
code_saturne run --initialize --param setup.xml
This will create a directory named for the current date and time (e.g. 20201019-1636) inside the RESU directory. Inside the new directory will be a script named run_solver. You may alter this to resemble the script below, or you may wish to simply create a new one with the contents shown.
If you wish to alter the existing run_solver script you will need to
add all the
#SBATCH options shown to set the job name, size and so on.
You should also add the two
module commands, and
srun --distribution=block:block --hint=nomultithread as well as the
--mpi option to the line executing
./cs_solver to ensure parallel execution on the compute nodes. The
export LD_LIBRARY_PATH=... and
cd commands are redundant and may be
retained or removed.
This script will run an MPI-only Code_Saturne job using the default GCC build and UCX over 4 nodes (128 x 4 = 512 cores) for a maximum of 20 minutes.
#!/bin/bash #SBATCH --export=none #SBATCH --job-name=CSExample #SBATCH --time=0:20:0 #SBATCH --nodes=4 #SBATCH --tasks-per-node=128 #SBATCH --cpus-per-task=1 # Replace [budget code] below with your project code (e.g. t01) #SBATCH --account=[budget code] #SBATCH --partition=standard #SBATCH --qos=standard # Load the GCC build of Code_Saturne 7.0.1 module load cpe/21.09 module load PrgEnv-gnu module load code_saturne # Switch to mpich-ucx implementation (see info note below) module swap craype-network-ofi craype-network-ucx module swap cray-mpich cray-mpich-ucx # Prevent threading. export OMP_NUM_THREADS=1 # Run solver. srun --distribution=block:block --hint=nomultithread ./cs_solver --mpi $@
The script can then be submitted to the batch system with
There is a known issue with the default MPI collectives which is
causing performance issues on Code_Saturne. The suggested workaround is to
switch to the mpich-ucx implementation. For this to link correctly on the full system, the extra
PrgEnv-gnu modules also have to be explicitly loaded.
The latest instructions for building Code_Saturne on ARCHER2 may be found in the GitHub repository of build instructions: