Flat node

Among the Nurion KNL nodes, those that support flat mode, which allows MCDRAM (16GB) to be used alongside DDR4, are available for general users. To use these nodes, you must specify the flat queue when submitting a job.

To use flat mode, you must use the numactl command, which specifies the preferred or default memory mode. For example, if you want to run an executable file my_app.x in flat mode, you can specify the corresponding NUMA node using the -m option with the numactl command, as shown below. However, if my_app.x requires more than 16GB of memory, the program will terminate due to insufficient memory.

Therefore, it is recommended to use the -p option, which prioritizes the use of MCDRAM, so that the task does not terminate even if the executable file requires more than 16GB of memory.

Example of writing a flat mode job script

#!/bin/sh
#PBS -N flat_job
#PBS -V
#PBS -q flat
#PBS -A {PBS option name} # refer to the table of PBS option name per application 
#PBS -l select=1:ncpus=68:mpiprocs=32:ompthreads=1
#PBS -l walltime=12:00:00

cd $PBS_O_WORKDIR

mpirun numactl -m 1 my_app.x
Alternatively,
mpirun numactl -p 1 my_app.x

※ Flat must be selected for the queue being submitted with the PBS option (i.e., -q flat)

Last updated on November 08, 2024.