Features & Constraints

In a heterogeneous environment, often hardware resources are grouped together into one big partition. On M3, we group various nodes into one partition comp (default) and this is particularly useful when your software or code is hardware agnostic. In the case where your job needs a specific hardware resource, you can flag the constraint option in your job script to ask for a specific feature. For example, you can specify skylake in your job script if your program can utilise the AVX-512 instruction set. The CPU type of a particular node can be viewed by running this command

scontrol show node <nodename>

and then looking for the Feature field.

An example:

NodeName=m3i000 Arch=x86_64 CoresPerSocket=18
CPUAlloc=36 CPUTot=36 CPULoad=0.01
AvailableFeatures=skylake
ActiveFeatures=skylake

and add the constraint flag in your job script.

#SBATCH --constraint=<feature_name>

An example of using constraint in the job script

#!/bin/bash
#SBATCH --job-name=MyJob
#SBATCH --ntasks=4
#SBATCH --ntasks-per-node=2
#SBATCH --constraint=skylake
#SBATCH --output=output.%j

module load openmpi

mpirun <program>

This feature should only be used if you must have a particular processor. Your job will have access to more resources if you do not use it.

To request multiple features, you can use operator function in the constraint flag in your job script:

Operator Function	Operator Symbol	Example
AND	&	#SBATCH --constraint=“K80&P100”
OR	| (pipe)	#SBATCH --constraint=“haswell|broadwell”

Features Available

Here is a list of the features available on M3. You can request the compute nodes with a variety of characteristics such as machines with a particular CPU model or GPU architecture. All nodes have hyper-threading turned off.

Feature	Description	Node
haswell	Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz	m3a,m3c,m3d
broadwell	Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz	m3h,m3e,dgx
skylake	Intel(R) Xeon(R) Gold 6150 CPU @ 2.70GHz	m3i,m3j,m3g,m3m
K80	NVIDIA Tesla K80	m3c,m3e
P100	NVIDIA Tesla P100	m3h
V100-16G	NVIDIA Tesla V100 16G variant	m3g[000-019],dgx000
V100-32G	NVIDIA Tesla V100 32G variant	m3g[020-023],dgx[001-005]