Welcome to the M3 user guide!

Important

Scheduled Outages

We have scheduled quarterly outages for M3. The purpose of this is to ensure that we have communicated outages to our HPC users in advance, and at the same time to ensure that the system is up-to-date as well as apply fixes to bugs and security vulnerabilities.

• Dates are:

  • 20/03/19 (COMPLETED)

  • 26/06/19 (COMPLETED)

  • 18/09/19

  • 11/12/19

This site contains the documentation for the MASSIVE HPC systems. M3 is the newest addition to the facility.

Help and Support

• Help and Support
• Scheduled Maintenance
• MASSIVE Terms of Use

Using M3

• About M3
  • System configuration
  • Compute
  • GPU Compute
  • Remote desktops through the CVL
• Requesting an account
  • Overview of the process
    • 1. Log in to the HPC ID system
    • 2. Create an M3 account
    • 3. Create or join an M3 project
    • 4. Set your M3 account password
    • 5. Log in to M3
    • Request an M3 account via email
• Requesting help on M3
  • Software issues
  • System issues
  • Contacting us
    • Help Desk
    • General Enquiries
    • Postal Address
• Connecting to M3
  • Connecting to M3 via ssh
    • Linux and OS X Users
    • Windows Users
  • Connecting to M3 via the MASSIVE desktop
    • MASSIVE Desktop (Strudel)
    • MASSIVE Desktop in browser (Strudel Web)
• File Systems on M3
  • What to put on each file system?
    • Home directory (~2GB)
    • Project directory (Shared with everyone on your project)
    • Scratch directory (Shared with everyone on your project)
  • Disk Quotas
  • Default Quotas for New Projects
  • Scratch Usage Policies
  • System Backups and File Recovery
    • File Recovery Procedure
  • Information for Desktop Users
    • Thumbnails Generating Too Much Data
    • Remember to empty your trash folder
    • Already over quota?
  • Storage outside of M3
    • Instructions to access Market share
    • Instructions to access Vault
• Copying files to and from M3
  • GUI Tool - Windows, Mac OS, and Linux Users
    • FileZilla
  • Command Line Interface - Linux and OS X Users
    • rsync
• Software on M3
  • Modules
  • Installed software modules
  • Requesting an install
  • Docker based work flows
  • Running QIIME on M3
• Running jobs on M3
  • Partitions Available
  • Other partitions
  • Checking the status of M3
    • The STATUS field explained
  • Slurm Accounts
    • Default accounts
    • Setting the account for a job
    • Questions about slurm accounts
  • Quick Start
  • Running Simple Batch Jobs
    • An example Slurm job script
    • Cancelling jobs
  • Running MPI Jobs
    • An example Slurm MPI job script
  • Running Multi-threading Jobs
    • An example Slurm Multi-threading job script
  • Running Interactive Jobs
    • Submitting an Interactive Job
    • How long do interactive jobs last for?
    • Reconnecting to/Disconnecting from an Active Interactive Job
  • Running GPU Jobs
    • Running GPU Batch Jobs
    • Compiling your own CUDA or OpenCL codes for use on M3
  • Running Array Jobs
    • An example of Slurm Array job script
  • QoS (Quality of Service)
  • How to run jobs with QoS
    • Explanation
  • Checking job status
    • Method 1: show_job
    • Method 2: Slurm commands
  • Project Allocation
    • Project Space
    • Questions about allocation
• Lustre File System Quickstart Guide
  • About Lustre

Communities

• MX2 Eiger
  • The Big Picture
  • Authorising MASSIVE to download your data (CAP Leaders only)
  • Creating research projects (research lab leaders only)
  • Requesting access to M3
  • Getting started on M3
  • Connecting to M3
  • Accessing your MX data
  • Reprocessing your MX data
  • Recommended Settings
  • Known Issues
    • Cannot open or read filename.tmp error
  • Need help?
• Deep Learning
  • Software
    • Caffe
    • TensorFlow
  • Reference datasets
    • ImageNet
  • Quick guide for checkpointing
    • Why checkpointing?
• Neuroimaging
  • Human Connectome Project Data Set
    • Human Connectome Project Steps for Access
    • MASSIVE Steps for Access
  • Using SLURM to submit a simple FSL job
    • Background
    • Data and scripts
• Cryo EM
  • Relion
    • The Graphical Interface
    • Using the Queue
    • Motion Correction
    • 2d/3d Classification & Refinement
    • Particle Polishing
  • Cryo EM Pre-Processing Tool
  • Cryo EM Benchmarking and Optimisation
    • Cryo EM Benchmarking and Optimisation
• Bioinformatics
  • Requesting an account on M3
  • Getting started
    • Importing the bioinformatics module environment
    • Installing additional software with Bioconda
  • Running the RNAsik RNA-seq pipeline on M3
  • FAQ

FAQs

• Frequently asked questions
  • Accounts
  • Jobs
  • M3, desktops and Strudel
• About Field of Research (FOR) and Socio-Economic Objective (SEO) codes
• Miscellaneous
• CloudStor Plus
  • Getting Started
  • CloudStor Plus on the CVL Desktop
  • Deactivating your CloudStor Plus Account on the CVL Desktop