Using SCARF’s AMD nodes¶
Overview¶
With the release of SCARF20, SCARF now has both Intel and AMD processors. This page documents the steps you’ll need to take to run and compile jobs on SCARF’s AMD nodes.
Introduction¶
As compilers can optimise code for specific processors we’ve decide to recompile the software we provide on the AMD nodes to ensure that it works. In particular binaries compiled on an Intel host with the Intel Compiler using the -xHost flag will not execute on AMD systems and must be recompiled. Binaries produced by other compilers may work, but you may wish to recompile on the AMD nodes anyway for better performance
Getting Access to AMD compatible software and tools¶
AMD compiled software and tools are in a separate area of our applications repository, to get access to them, run
module load AMDmodules
A number of toolchains and applications have been made available, please get in touch if you would like additional ones provided.
Running jobs on AMD nodes¶
Due to the potential incompatibility between Intel compiled binaries and the AMD nodes, we have added a default constraint of “intel” for all jobs that do not specify a constraint. To use AMD nodes, add the parameter below to your submission parameters:
-C amd
If you have verified that your binaries work on Intel nodes and AMD nodes and have no preference to what you run on, then you can specify
-C "[intel|amd]"
If you wish to run on the latest AMD nodes or the latest Intel nodes specify
-C "[scarf18|amd]"
Alternatively you can use the constraint scarf20 in place of amd in the previous examples, this will limit you to running on the current generation of AMD nodes, but would exclude any potential future AMD nodes that SCARF deploys. The combinations noted above are examples, others combinations will also work, please see the available features here
The default constraint of intel will be kept under review and may be changed as the processor mix of SCARF changes or as more software is available for AMD nodes.
Compiler Flag recommendations¶
For versions of gcc greater than 9.3 use the flag
-march=znver2
to compile for the Epyc2 processors in SCARF 20, For the Intel compiler use the flag
-march=core-avx2
Please see the AMD Compiler Options Quick Reference Guide for AMD’s recomended settings for a number of compilers (note that not all the mentioned compilers and libraries in the linked document are available on SCARF)