Attention

This documentation is under active development, meaning that it can change over time as we refine it. Please email help@massive.org.au if you require assistance, or have suggestions to improve this documentation.

GPU Look-Up Tables#

These look-up tables provide an overview of key information about our GPUs, to assist you when choosing a GPU on M3 for your research. For more detailed discussion of GPU selection, see Starter Guide: GPUs on M3, or for more detailed hardware information, see About M3.

Look-Up Tables#

We have compute GPUs accessible via the queue or interactively, and GPUs specifically reserved for desktops. The tables are split accordingly.

Compute GPUs#

GPU

Should I use this?

More details

QoS/Partition

P100 (Pascal)

  • These have 16GB of RAM and are more than powerful enough for most jobs. These are older than the V100s.

  • The queue for these may be long as there are 7 servers, but it is typically shorter than the V100 queue.

  • 7 servers (nodes)

  • 2 P100 GPUs per server

  • 28 CPU cores per server

  • 16GB of RAM per GPU

  • 240GB of RAM per server

  • #SBATCH --partition=m3h

V100 16GB (Volta)

  • In single GPU jobs these match the DGX GPUs on performance but are available for everyone to submit to.

  • There are long queue times for these.

  • In general, the wait time for these is justified by the performance.

  • 20 servers (nodes)

  • 3 V100 GPUs per server

  • 36 CPU cores per server

  • 16GB of RAM per GPU

  • 340GB of RAM per server

  • #SBATCH --partition=m3g

V100 32GB (Volta)

  • In single GPU jobs these match the DGX GPUs on performance but are available for everyone to submit to.

  • There are only 4 servers with 32GB memory V100s. Queue time is therefore longer, so if you dont need 32GB of memory, consider using the 16GB variety instead - they’re just as fast.

  • 4 servers (nodes)

  • 3 V100 GPUs per server

  • 36 CPU cores per server

  • 32GB of RAM per GPU

  • 340GB of RAM per server

  • #SBATCH --partition=m3g

  • To specify you need a 32GB V100, you also need to add: #SBATCH --constraint=V100-32G

T4 (Turing)

  • Successor to the P4 GPUs with higher clock-speeds and 16GB of GDDR6 RAM.

  • Also available as desktops via Strudel2 (see Desktop GPUs below.)

  • 2 servers (nodes)

  • 8 T4 GPUs per server

  • 52 CPUs per server

  • 16GB of RAM per GPU

  • 900GB of RAM per server

  • #SBATCH --partition=gpu

A40 (Ampere)

  • These GPUs are the newest and come with the most GPU RAM of the GPUs on M3.

  • Also available as desktops via Strudel2 (see Desktop GPUs below.)

  • 4 servers (nodes)

  • 4 A40 GPUs per server

  • 52 CPUs per server

  • 48GB of RAM per GPU

  • 1TB of RAM per server

  • #SBATCH --partition=gpu

DGX (Volta)

  • These contain 8 GPUS per server, and are purpose built for deep learning.

  • Use these when you require multiple GPUs on one server, or leverage the NVLink capabilities.

  • You must apply for access to the DGX.

  • Jobs submitted to the DGX must use a minimum of 4 GPUs.

  • They are a limited resource and thus have a lengthy queue time - these should be reserved for jobs that demonstrate their scalability.

  • 11 servers (nodes)

  • 8 DGX GPUs per server

  • 40 CPU cores per server

  • 32GB of RAM per GPU

  • 512GB of RAM per server

Desktop GPUs#

We have some GPUs available through the desktop. As these are accessed via desktops, there is no partition column in this table; select the GPU when setting up your desktop session as described in the Strudel documentation.

GPU

Should I use this?

More details

K1 (Kepler)

  • These shouldn’t be used for computation - they’re designed to enable visualisation with applications on the desktops.

  • 32 desktops available

  • 1 K1 GPU per desktop

  • 3 CPU cores per desktop

  • 11GB of RAM per desktop

P4 (Pascal)

  • This is the option most likely to meet your desktop needs.

  • Currently one of M3’s least powerful GPUs, but still sufficient for many activities, including testing your work before submitting to a more powerful GPU. Some visualisation software may require a P4.

  • Typically minimal wait time to start a P4 desktop.

  • 60 desktops available

  • 1 P4 GPU per desktop

  • 6 CPU cores per desktop

  • 55GB of RAM per desktop

K80 (Kepler)

  • These are the oldest GPUs on M3.

  • There are comparatively fewer K80s on M3, so queue time is longer for these than a K1 or P4.

  • Use a K80 when your job requires more memory than a P4 desktop can provide.

  • 28 K80 desktops available

  • 2 K80 GPUs per desktop

  • 12 CPU cores per desktop

  • 117GB of RAM per desktop

T4 (Turing)

  • Successor to the P4 GPUs with higher clock-speeds and 16GB of GDDR6 RAM.

  • Available in either 1 or 2 GPUs per desktop configurations.

  • The dual T4 GPU desktop can be useful for testing multi-GPU workflows before queueing for more powerful GPUs.

  • 32 T4 GPUs available in total

  • 1 or 2 T4 GPUs per desktop

  • 6 or 13 CPU cores per desktop

  • 100GB or 225GB of RAM per desktop

A40 (Ampere)

  • Highest GPU RAM of the GPUs available on M3 (48GB).

  • 32 A40 desktops available

  • 1 A40 GPU per desktop

  • 13 CPU cores per desktop

  • 250GB of RAM per desktop