Features & Constraints

In a heterogeneous environment, often hardware resources are grouped together into one big partition. On M3, we group various nodes into one partition comp (default) and this is particularly useful when your software or code is hardware agnostic. In the case where your job needs a specific hardware resource, you can flag the constraint option in your job script to ask for a specific feature. For example, you can specify skylake in your job script if your program can utilise the AVX-512 instruction set. The CPU type of a particular node can be viewed by running this command

scontrol show node <nodename>

and then looking for the Feature field.

An example:

NodeName=m3i000 Arch=x86_64 CoresPerSocket=18
CPUAlloc=36 CPUTot=36 CPULoad=0.01
AvailableFeatures=skylake
ActiveFeatures=skylake

and add the constraint flag in your job script.

#SBATCH --constraint=<feature_name>

An example of using constraint in the job script

#!/bin/bash
#SBATCH --job-name=MyJob
#SBATCH --ntasks=4
#SBATCH --ntasks-per-node=2
#SBATCH --constraint=skylake
#SBATCH --output=output.%j

module load openmpi

mpirun <program>

This feature should only be used if you must have a particular processor. Your job will have access to more resources if you do not use it.

To request multiple features, you can use operator function in the constraint flag in your job script:

Operator Function

Operator Symbol

Example

AND

&

#SBATCH --constraint=“K80&P100”

OR

| (pipe)

#SBATCH --constraint=“haswell|broadwell”

Features Available

Here is a list of the features available on M3. You can request the compute nodes with a variety of characteristics such as machines with a particular CPU model or GPU architecture. All nodes have hyper-threading turned off.

Feature

Description

Node

haswell

Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz

m3a,m3c,m3d

broadwell

Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz

m3h,m3e,dgx

skylake

Intel(R) Xeon(R) Gold 6150 CPU @ 2.70GHz

m3i,m3j,m3g,m3m

K80

NVIDIA Tesla K80

m3c,m3e

P100

NVIDIA Tesla P100

m3h

V100-16G

NVIDIA Tesla V100 16G variant

m3g[000-019],dgx000

V100-32G

NVIDIA Tesla V100 32G variant

m3g[020-023],dgx[001-005]