Hoffman2:Batch Mode: Difference between revisions

From Center for Cognitive Neuroscience
Jump to navigation Jump to search
Line 44: Line 44:


  #$ -j y
  #$ -j y
:Merge error log with joblog.$JOB_ID
:Merge error log with standard out put (in file joblog.$JOB_ID)
 
So you'll be able to find error output together with standard output of your job script in this joblog.$JOB_ID log file.


  #$ -pe shared 2
  #$ -pe shared 2
Line 58: Line 56:


  #$ -M $USER@mail
  #$ -M $USER@mail
Send notification email to your user email address
:Send notification email to your user email address


   #$ -m bea
   #$ -m bea
Specify the timing of the notification email to be sent out:
:Specify the timing of the notification email to be sent out:
:* b - when the job begins
:* b - when the job begins
:* e - when the job ends
:* e - when the job ends

Revision as of 20:53, 19 December 2019

Back to all things Hoffman2

Here we show how you can submit your script job to the working node with batch mode.

To use a batch job, you need to create a batch file with bash or tcsh. This find should have three parts:

  • Part 1: List all the resource you want to reserve for your job
  • Part 2: Load your modules, export the Unix environment that needed for your script to run
  • Part 3: Call your job script

Once you have your batch file, you can submit it by qsub command, for example (if your batch file is named as myjob.sh)

qsub myjob.sh

Here are some batch file templates you can start with

Job Submission Templates

Please use these job submission template scripts to simplify job submission.

Part 1: Request Computing Resource

This example uses the code from the following template Submit job

The first part of the batch file should let the Hoffman job scheduler know how much resource you want to reserve for your job

#!/bin/bash
#$ -cwd
# error = Merged with joblog
#$ -o joblog.$JOB_ID
#$ -j y
#$ -pe shared 2
#$ -l h_rt=8:00:00,h_data=4G
# Email address to notify
#$ -M $USER@mail
# Notify when
#$ -m bea

Here's the meaning of each line:

#$ -cwd
Use the current directory for the job
#$ -o joblog.$JOB_ID
Write standard output to file joblog.$JOB_ID. $JOB_ID will be replaced by your job ID which is assigned once you submit your job.
#$ -j y
Merge error log with standard out put (in file joblog.$JOB_ID)
#$ -pe shared 2
Request 2 processor cores
#$ -l h_rt=8:00:00,h_data=4G
Use -l option to specify job running time length and reserve memory
h_rt=8:00:00 : reserve 8 hours for your job running time
h_data=4G: reserve 4G per-core (since -pe 2 is used above, it will reserve 2 core x 4G memory = 8G total memory)
#$ -M $USER@mail
Send notification email to your user email address
 #$ -m bea
Specify the timing of the notification email to be sent out:
  • b - when the job begins
  • e - when the job ends
  • a - when the job is aborted (ends in an error state)

Part 2: Setup the Environment

In the second part, you should setup your Unix environment for your code to run, which includes loading the needed modules and export paths for libraries.

To use any module provided by Hoffman and CCN, you'll need the following two lines

# load the job environment:
. /u/local/Modules/default/init/modules.sh
module use /u/project/CCN/apps/modulefiles

For example, load FSL for your job to run

# Load the FSL module
module load fsl

Another example, you want to export a variable for FSL so it won't run parallel

# This is optional
# More info here: https://www.ccn.ucla.edu/wiki/index.php/Hoffman2:FSL 
export NO_FSL_JOBS=true

Or export an additional PATH for a custom installed library in your .local space

export PATH=$HOME/.local/bin:$PATH

Part 3: Call your job script

This is where you call your job script.

For example:

/bin/bash mycode.sh


Make sure your job script has executive privileges by command

chmod ug+x mycode.sh


Other methods


Job Array

Job Array is a type of batch mode. It can be used to run the same processing job towards many subjects on separate nodes in parallel. More about Job Array