Hoffman2:Batch Mode

From Center for Cognitive Neuroscience
Jump to navigation Jump to search

Back to all things Hoffman2

Here we show how you can submit your job with batch mode.

To use a batch job, you need to create a batch file with bash or tcsh. This file should have three parts:

  • Part 1: List all the resources you want to reserve for your job
  • Part 2: Load your modules, export the Linux environment that is needed for your script to run
  • Part 3: Call your job script

Once you have your batch file, you can submit it using the qsub command. For example (if your batch file is named as myjob.sh) 

qsub myjob.sh


Job Submission Templates

Here are some batch file templates you can start with:

Job Submission Template Scripts

Part 1: Request Computing Resource

This example is based on code from the Submit Job template.

The first part of the batch script file should let the Hoffman job scheduler know what resources you want to reserve for your job: 

#!/bin/bash
#$ -cwd
#$ -o joblog.$JOB_ID
#$ -j y
#$ -pe shared 2
#$ -l h_rt=8:00:00,h_data=4G
#$ -M $USER@mail
#$ -m bea


Here's the meaning of each line: 

#$ -cwd
Use the current directory for the job


#$ -o joblog.$JOB_ID
Write standard output to file joblog.$JOB_ID. $JOB_ID will be replaced by your job ID which is assigned once you submit your job.


#$ -j y
Merge error log with standard output (in file joblog.$JOB_ID)


#$ -pe shared 2
Request 2 processor cores


#$ -l h_rt=8:00:00,h_data=4G
Use -l option to specify job running time length and reserve memory
h_rt=8:00:00 : reserve 8 hours for your job running time
h_data=4G: reserve 4G per-core (since -pe 2 is used above, it will reserve 2 core x 4G memory = 8G total memory)


#$ -M $USER@mail
Send notification to your user email address


 #$ -m bea
Specify the timing of the notification email to be sent out:
  • b - when the job begins
  • e - when the job ends
  • a - when the job is aborted (ends in an error state)

Part 2: Setup the Environment

In the second part of the batch script, you should setup your Unix environment for your code to run, which includes loading modules and export paths for libraries.

To use any module provided by Hoffman and CCN, you'll need the following two lines 

# load the job environment:
. /u/local/Modules/default/init/modules.sh
module use /u/project/CCN/apps/modulefiles

For example, load FSL 

# Load the FSL module
module load fsl

Another example, export a FSL variable 

# This is optional
# More info here: https://www.ccn.ucla.edu/wiki/index.php/Hoffman2:FSL 
export NO_FSL_JOBS=true

Or export an additional PATH for a custom installed library under your .local/ directory 

export PATH=$HOME/.local/bin:$PATH

Part 3: Call your job script

The third part of the batch script should call commands or other scrip for analysis.

For example, if you run feat 

 feat /my/path/to/design.fsf


Or if you have a script named as mycode.sh containing all the commands for your analysis,

Make sure your job script has executive privileges by using chmod command 

chmod ug+x mycode.sh

call your script at the last part of your batch script

For example: 

/bin/bash mycode.sh

Once the batch script is ready, you can submit it with qsub 

 qsub myjob.sh


To confirm the status of the submitted job, use command "myjob" 

 myjob

This will show the status of your jobs

Other methods

Job Array

Job Array is a type of batch mode. It is used to run the same processing job towards a large number of subjects on separate nodes in parallel. More about Job Array

Retrieved from "https://www.ccn.ucla.edu/wiki/index.php?title=Hoffman2:Batch_Mode&oldid=4965"