Attention

This documentation is under active development, meaning that it can change over time as we refine it. Please email help@massive.org.au if you require assistance, or have suggestions to improve this documentation.

GPU Look-Up Tables

These look-up tables provide an overview of key information about our GPUs, to assist you when choosing a GPU on M3 for your research. For more detailed discussion of GPU selection, see Starter Guide: GPUs on M3, or for more detailed hardware information, see About M3.

Look-Up Tables

We have compute GPUs accessible via the queue or interactively, and GPUs specifically reserved for desktops. The tables are split accordingly.

Compute GPUs

GPU

Should I use this?

More details

QoS/Partition

P100 (Pascal)

  • These have 16GB of RAM and are more than powerful enough for most jobs. These are older than the V100s.

  • The queue for these may be long as there are 10 servers, but it is typically shorter than the V100 queue.

  • 10 servers (nodes)

  • 2 P100 GPUs per server

  • 28 CPU cores per server

  • 16GB of RAM per GPU

  • 240GB of RAM per server

  • #SBATCH --partition=m3h

V100 16GB (Volta)

  • In single GPU jobs these match the DGX GPUs on performance but are available for everyone to submit to.

  • There are long queue times for these.

  • In general, the wait time for these is justified by the performance.

  • 20 servers (nodes)

  • 3 V100 GPUs per server

  • 36 CPU cores per server

  • 16GB of RAM per GPU

  • 340GB of RAM per server

  • #SBATCH --partition=m3g

V100 32GB (Volta)

  • In single GPU jobs these match the DGX GPUs on performance but are available for everyone to submit to.

  • There are 4 servers with the 32GB memory V100s. Queue time will be longer because of this, so if you dont need 32GB of memory, consider using the 16GB variety instead - they’re just as fast.

  • 4 servers (nodes)

  • 3 V100 GPUs per server

  • 36 CPU cores per server

  • 32GB of RAM per GPU

  • 340GB of RAM per server

  • #SBATCH --partition=m3g

  • To specify you need a 32GB V100, you also need to add: #SBATCH --constraint=V100-32GB

DGX (Volta)

  • These are our most advanced GPUs with 8 GPUS per server, and are purpose built for deep learning.

  • Use these when you require multiple GPUs on one server, or leverage the NVLink capabilities.

  • You must apply for access to the DGX.

  • Jobs submitted to the DGX must use a minimum of 4 GPUs.

  • They are a limited resource and thus have a lengthy queue time - these should be reserved for jobs that demonstrate their scalability.

  • 11 servers (nodes)

  • 8 DGX GPUs per server

  • 40 CPU cores per server

  • 32GB of RAM per GPU

  • 512GB of RAM per server

  • #SBATCH --qos=dgx

  • #SBATCH --partition=dgx

  • You need to apply to use these here.

Desktop GPUs

We have some GPUs available through the desktop. As these are accessed via desktops, there is no partition column in this table; select the GPU when setting up your desktop session as described here.

GPU

Should I use this?

More details

K1 (Kepler)

  • These shouldn’t be used for computation - they’re designed to enable visualisation with applications on the desktops.

  • 32 desktops available

  • 1 K1 GPU per desktop

  • 3 CPU cores per desktop

  • 11GB of RAM per desktop

P4 (Pascal)

  • This is the option most likely to meet your desktop needs.

  • Less powerful than a V100 or P100, but still sufficient for a variety of activities, including testing your work before submitting to a more powerful GPU. Some visualisation software may also require a P4.

  • Usually no wait time to start a P4 desktop.

  • 66 desktops available

  • 1 P4 GPU per desktop

  • 6 CPU cores per desktop

  • 55GB of RAM per desktop

K80 (Kepler)

  • These are the oldest GPUs on M3.

  • There is a more limited number of K80s, so queue time is longer for these than a K1 or P4.

  • Use a K80 when your job requires more memory than a P4 desktop can provide.

  • 28 K80 desktops available

  • 2 K80 GPUs per desktop

  • 12 CPU cores per desktop

  • 117GB of RAM per desktop