Attention

This documentation is under active development, meaning that it can change over time as we refine it. Please email help@massive.org.au if you require assistance.

About M3

M3 is the third stage of MASSIVE.

MASSIVE is a High Performance Computing (HPC) facility designed specifically to process complex data. Since 2010, MASSIVE has played a key role in driving discoveries across many disciplines including biomedical sciences, materials research, engineering and geosciences.

MASSIVE is pioneering and building high performance computing upon Monash’s specialist Research Cloud fabric. M3 has been supplied by Dell with a Mellanox low latency network and NVIDIA GPUs.

System configuration

  • Total number of cores: 4,188
  • Total number of GPU cores: 741,888 CUDA cores
  • Total memory: 32 TByte
  • Storage capacity: 2.9 PByte Lustre parallel file system
  • Interconnect: 100 Gb/s Ethernet Mellanox Spectrum network

M3 utilises SLURM scheduler to manage the resources with 5 partitions available to users, with different configurations to suit a variety of computational requirements. Details about the partitions can be found here

M3 consists of 165 nodes with the following configurations:

  • “Compute” nodes in 4 configurations
    • Standard Memory:
      • Number of nodes: 22
      • Number of cores per node: 24
      • Processor model: 2 x Intel Xeon CPU E5-2680 v3
      • Processor frequency: 2.50GHz, with max Turbo frequency 3.30GHz
      • Memory per node: 128GB RAM
      • Partition: m3a
    • Medium Memory:
      • Number of nodes: 13
      • Number of cores per node: 24
      • Processor model: 2 x Intel Xeon CPU E5-2680 v3
      • Processor frequency: 2.50GHz, with max Turbo frequency 3.30GHz
      • Memory per node: 256GB RAM
      • Partition: m3d
    • High-Density CPUs:
      • Number of nodes: 48
      • Number of cores per node: 36
      • Processor model: 2 x Intel Xeon Gold 6150
      • Processor frequency: 2.70 GHz, with max Turbo frequency 3.70GHz
      • Memory per node: 192GB RAM
      • Partition: m3i
    • High-Density CPUs with High Memory:
      • Number of nodes: 1
      • Number of cores per node: 36
      • Processor model: 2 x Intel Xeon Gold 6150
      • Processor frequency: 2.70 GHz, with max Turbo frequency 3.70GHz
      • Memory per node: 1TB RAM
      • Partition: m3m
  • “GPU” nodes in 6 configurations
    • Desktops:
      • Number of desktop sessions: 32
      • Number of cores per desktop session: 3
      • Processor model: 2 x Intel Xeon CPU E5-2680 v3
      • Processor frequency: 2.50GHz, with max Turbo frequency 3.30GHz
      • GPU model: nVidia Grid K1
      • Number of GPU per desktop session: 1
      • GPU cores : 192 CUDA cores
      • Memory per desktop session: 16GB RAM
      • Partition name: m3f
    • K80:
      • Number of desktop sessions: 26
      • Number of cores per desktop session: 12
      • Processor model: 2 x Intel Xeon CPU E5-2680 v3
      • Processor frequency: 2.50GHz, with max Turbo frequency 3.30GHz
      • GPU model: nVidia Tesla K80
      • Number of GPUs per desktop session: 2
      • GPU cores per card: 4,992 CUDA cores
      • Total GPU cores per desktop session: 9,984 CUDA cores
      • Memory per desktop session: 128GB RAM
      • Partition name: m3c
    • K80 with High-Density GPUs:
      • Number of nodes: 1
      • Number of cores per node: 24
      • Processor model: 2 x Intel Xeon CPU E5-2680 v3
      • Processor frequency: 2.50GHz, with max Turbo frequency 3.30GHz
      • GPU model: nVidia Tesla K80
      • Number of GPUs per node: 8
      • GPU cores per card: 4,992 CUDA cores
      • Total GPU cores per node: 39,936 CUDA cores
      • Memory per node: 256GB RAM
      • Partition name: m3e
    • P100:
      • Number of nodes: 14
      • Number of cores per node: 28
      • Processor model: 2 x Intel Xeon CPU E5-2680 v4
      • Processor frequency: 2.40GHz, with max Turbo frequency 3.30GHz
      • GPU model: nVidia Tesla P100
      • Number of GPUs per node: 2
      • GPU cores per card: 3,584 CUDA cores
      • Total GPU cores per node: 7,168 CUDA cores
      • Memory per node: 256GB RAM
      • Partition name: m3h
    • V100:
      • Number of nodes: 20
      • Number of cores per node: 36
      • Processor model: 2 x Intel Xeon Gold 6150
      • Processor frequency: 2.70 GHz, with max Turbo frequency 3.70GHz
      • GPU model: nVidia Tesla V100
      • Number of GPUs per node: 3
      • GPU cores per card: 5,120 CUDA cores
      • Total GPU cores per node: 15,360 CUDA cores
      • Memory per node: 384GB RAM
      • Partition name: m3g
    • DGX1:
      • Number of nodes: 1
      • Number of cores per node: 40
      • Processor model: 2 x Intel Xeon CPU E5-2698 v4
      • Processor frequency: 2.20GHz, with max Turbo frequency 3.60GHz
      • GPU model: nVidia Tesla GP100
      • Number of GPUs per node: 8
      • GPU cores per card: 3,584 CUDA cores
      • Total GPU cores per node: 28,672 CUDA cores
      • Memory per node: 512GB RAM
      • Partition name: TBU