Hoffman2:Batch Mode: Difference between revisions

From Center for Cognitive Neuroscience
Jump to navigation Jump to search
No edit summary
No edit summary
Line 3: Line 3:
To use a batch job, you need to create a batch file with bash or tcsh. This find should have three parts:
To use a batch job, you need to create a batch file with bash or tcsh. This find should have three parts:


*:Part 1: List all the resource you want to reserve for your job
:*Part 1: List all the resource you want to reserve for your job
*:Part 2: Load your modules, export the Unix environment that needed for your script to run
:*Part 2: Load your modules, export the Unix environment that needed for your script to run
*:Part 3: Call your job script
:*Part 3: Call your job script


Here are some batch file templates you can start with
Here are some batch file templates you can start with

Revision as of 19:32, 18 December 2019

Back to all things Hoffman2

To use a batch job, you need to create a batch file with bash or tcsh. This find should have three parts:

  • Part 1: List all the resource you want to reserve for your job
  • Part 2: Load your modules, export the Unix environment that needed for your script to run
  • Part 3: Call your job script

Here are some batch file templates you can start with

Job Submission Templates

Please use these job submission template scripts to simplify job submission.

What's in the Part 1

The first part of the batch file should let the job scheduler know how much resource you want to reserve for your job

#!/bin/bash
#$ -cwd
# error = Merged with joblog
#$ -o joblog.$JOB_ID
#$ -j y
#$ -pe shared 2
#$ -l h_rt=8:00:00,h_data=4G
# Email address to notify
#$ -M $USER@mail
# Notify when
#$ -m bea

Here are what they mean:

#$ -cwd
Using the current directory for the job
#$ -o joblog.$JOB_ID

Write standard output to file joblog.$JOB_ID. $JOB_ID will be replaced by your job ID which is assigned once you submit your job.

#$ -j y

Merge error log with joblog.$JOB_ID

So you'll be able to find error output together with standard output of your job script in this joblog.$JOB_ID log file.

#$ -pe shared 2

Request 2 processor cores

#$ -l h_rt=8:00:00,h_data=4G

Use -l option to specify job running time length and reserve memory

h_rt=8:00:00 : reserve 8 hours for your job running time
h_data=4G: reserve 4G per-core (since -pe 2 is used above, it will reserve 2 core x 4G memory = 8G total memory)
#$ -M $USER@mail

Send notification email to your user email address

 #$ -m bea

Specify the timing of the notification email to be sent out:

  • b - when the job begins
  • e - when the job ends
  • a - when the job is aborted (ends in an error state)


What goes in Part 2

# load the job environment:
. /u/local/Modules/default/init/modules.sh
module use /u/project/CCN/apps/modulefiles
# Load the FSL module
module load fsl
# This is optional
# More info here: https://www.ccn.ucla.edu/wiki/index.php/Hoffman2:FSL 
export NO_FSL_JOBS=true