Hoffman2:Batch Mode: Difference between revisions
No edit summary |
|||
(38 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
[[Hoffman2|Back to all things Hoffman2]] | [[Hoffman2|Back to all things Hoffman2]] | ||
Here we show how you can submit your | Here we show how you can submit your job with batch mode. | ||
To use a batch job, you need to create a batch file with bash or tcsh. This | To use a batch job, you need to create a batch file with bash or tcsh. This file should have three parts: | ||
:*Part 1: List all the | :*Part 1: List all the resources you want to reserve for your job | ||
:*Part 2: Load your modules, export the | :*Part 2: Load your modules, export the Linux environment that is needed for your script to run | ||
:*Part 3: Call your job script | :*Part 3: Call your job script | ||
Once you have your batch file, you can submit it | Once you have your batch file, you can submit it using the <code>qsub</code> command. For example (if your batch file is named as myjob.sh) | ||
qsub myjob.sh | qsub myjob.sh | ||
= Job Submission Templates = | = Job Submission Templates = | ||
Here are some batch file templates you can start with: | |||
:[[Hoffman2:Job_Submission_Templates|Job Submission Template Scripts]] | |||
= Part 1: Request Computing Resource = | = Part 1: Request Computing Resource = | ||
This example | This example is based on code from the | ||
'''[[Hoffman2:Submit_job|Submit | '''[[Hoffman2:Submit_job|Submit Job]]''' template. | ||
The first part of the batch file should let the job scheduler know | The first part of the batch script file should let the Hoffman job scheduler know what resources you want to reserve for your job: | ||
#!/bin/bash | #!/bin/bash | ||
#$ -cwd | #$ -cwd | ||
#$ -o joblog.$JOB_ID | #$ -o joblog.$JOB_ID | ||
#$ -j y | #$ -j y | ||
#$ -pe shared 2 | #$ -pe shared 2 | ||
#$ -l h_rt=8:00:00,h_data=4G | #$ -l h_rt=8:00:00,h_data=4G | ||
#$ -M $USER@mail | #$ -M $USER@mail | ||
#$ -m bea | #$ -m bea | ||
Here | |||
Here's the meaning of each line: | |||
#$ -cwd | #$ -cwd | ||
:Use the current directory for the job | |||
#$ -o joblog.$JOB_ID | #$ -o joblog.$JOB_ID | ||
Write standard output to file joblog.$JOB_ID. $JOB_ID will be replaced by your job ID which is assigned once you submit your job. | :Write standard output to file joblog.$JOB_ID. $JOB_ID will be replaced by your job ID which is assigned once you submit your job. | ||
#$ -j y | #$ -j y | ||
Merge error log with joblog.$JOB_ID | :Merge error log with standard output (in file joblog.$JOB_ID) | ||
#$ -pe shared 2 | #$ -pe shared 2 | ||
Request 2 processor cores | :Request 2 processor cores | ||
#$ -l h_rt=8:00:00,h_data=4G | #$ -l h_rt=8:00:00,h_data=4G | ||
Use -l option to specify job running time length and reserve memory | :Use -l option to specify job running time length and reserve memory | ||
:<code>h_rt=8:00:00</code> : reserve 8 hours for your job running time | :<code>h_rt=8:00:00</code> : reserve 8 hours for your job running time | ||
:<code>h_data=4G</code>: reserve 4G per-core (since -pe 2 is used above, it will reserve 2 core x 4G memory = 8G total memory) | :<code>h_data=4G</code>: reserve 4G per-core (since -pe 2 is used above, it will reserve 2 core x 4G memory = 8G total memory) | ||
#$ -M $USER@mail | #$ -M $USER@mail | ||
Send notification | :Send notification to your user email address | ||
#$ -m bea | #$ -m bea | ||
Specify the timing of the notification email to be sent out: | :Specify the timing of the notification email to be sent out: | ||
:* b - when the job begins | :* b - when the job begins | ||
:* e - when the job ends | :* e - when the job ends | ||
Line 67: | Line 69: | ||
= Part 2: Setup the Environment = | = Part 2: Setup the Environment = | ||
In the second part, you should setup your Unix environment for your code to run, which includes loading | In the second part of the batch script, you should setup your Unix environment for your code to run, which includes loading modules and export paths for libraries. | ||
To use any module provided by Hoffman and CCN, you'll need the following two lines | To use any module provided by Hoffman and CCN, you'll need the following two lines | ||
Line 74: | Line 76: | ||
module use /u/project/CCN/apps/modulefiles | module use /u/project/CCN/apps/modulefiles | ||
For example, load FSL | For example, load FSL | ||
# Load the FSL module | # Load the FSL module | ||
module load fsl | module load fsl | ||
Another example, | Another example, export a FSL variable | ||
# This is optional | # This is optional | ||
# More info here: https://www.ccn.ucla.edu/wiki/index.php/Hoffman2:FSL | # More info here: https://www.ccn.ucla.edu/wiki/index.php/Hoffman2:FSL | ||
export NO_FSL_JOBS=true | export NO_FSL_JOBS=true | ||
Or export an additional PATH for a custom installed library | Or export an additional PATH for a custom installed library under your .local/ directory | ||
export PATH=$HOME/.local/bin:$PATH | export PATH=$HOME/.local/bin:$PATH | ||
= Part 3: Call your job script = | = Part 3: Call your job script = | ||
The third part of the batch script should call commands or other scrip for analysis. | |||
For example, if you run feat | |||
feat /my/path/to/design.fsf | |||
Or if you have a script named as mycode.sh containing all the commands for your analysis, | |||
Make sure your job script has executive privileges by using chmod command | |||
chmod ug+x mycode.sh | |||
call your script at the last part of your batch script | |||
For example: | For example: | ||
/bin/bash mycode.sh | /bin/bash mycode.sh | ||
Once the batch script is ready, you can submit it with qsub | |||
qsub myjob.sh | |||
To confirm the status of the submitted job, use command "myjob" | |||
myjob | |||
This will show the status of your jobs | |||
= Other methods = | = Other methods = | ||
Line 105: | Line 125: | ||
* Use qsub in one line command: [[Hoffman2:Submitting_Jobs#qsub|examples]] | * Use qsub in one line command: [[Hoffman2:Submitting_Jobs#qsub|examples]] | ||
= Job Array = | = Job Array = | ||
Job Array is a type of batch mode. It | Job Array is a type of batch mode. It is used to run the same processing job towards a large number of subjects on separate nodes in parallel. | ||
[[Hoffman2: | [[Hoffman2:Job Array|More about Job Array]] |
Latest revision as of 17:29, 30 September 2022
Here we show how you can submit your job with batch mode.
To use a batch job, you need to create a batch file with bash or tcsh. This file should have three parts:
- Part 1: List all the resources you want to reserve for your job
- Part 2: Load your modules, export the Linux environment that is needed for your script to run
- Part 3: Call your job script
Once you have your batch file, you can submit it using the qsub
command. For example (if your batch file is named as myjob.sh)
qsub myjob.sh
Job Submission Templates
Here are some batch file templates you can start with:
Part 1: Request Computing Resource
This example is based on code from the Submit Job template.
The first part of the batch script file should let the Hoffman job scheduler know what resources you want to reserve for your job:
#!/bin/bash #$ -cwd #$ -o joblog.$JOB_ID #$ -j y #$ -pe shared 2 #$ -l h_rt=8:00:00,h_data=4G #$ -M $USER@mail #$ -m bea
Here's the meaning of each line:
#$ -cwd
- Use the current directory for the job
#$ -o joblog.$JOB_ID
- Write standard output to file joblog.$JOB_ID. $JOB_ID will be replaced by your job ID which is assigned once you submit your job.
#$ -j y
- Merge error log with standard output (in file joblog.$JOB_ID)
#$ -pe shared 2
- Request 2 processor cores
#$ -l h_rt=8:00:00,h_data=4G
- Use -l option to specify job running time length and reserve memory
h_rt=8:00:00
: reserve 8 hours for your job running timeh_data=4G
: reserve 4G per-core (since -pe 2 is used above, it will reserve 2 core x 4G memory = 8G total memory)
#$ -M $USER@mail
- Send notification to your user email address
#$ -m bea
- Specify the timing of the notification email to be sent out:
- b - when the job begins
- e - when the job ends
- a - when the job is aborted (ends in an error state)
Part 2: Setup the Environment
In the second part of the batch script, you should setup your Unix environment for your code to run, which includes loading modules and export paths for libraries.
To use any module provided by Hoffman and CCN, you'll need the following two lines
# load the job environment: . /u/local/Modules/default/init/modules.sh module use /u/project/CCN/apps/modulefiles
For example, load FSL
# Load the FSL module module load fsl
Another example, export a FSL variable
# This is optional # More info here: https://www.ccn.ucla.edu/wiki/index.php/Hoffman2:FSL export NO_FSL_JOBS=true
Or export an additional PATH for a custom installed library under your .local/ directory
export PATH=$HOME/.local/bin:$PATH
Part 3: Call your job script
The third part of the batch script should call commands or other scrip for analysis.
For example, if you run feat
feat /my/path/to/design.fsf
Or if you have a script named as mycode.sh containing all the commands for your analysis,
Make sure your job script has executive privileges by using chmod command
chmod ug+x mycode.sh
call your script at the last part of your batch script
For example:
/bin/bash mycode.sh
Once the batch script is ready, you can submit it with qsub
qsub myjob.sh
To confirm the status of the submitted job, use command "myjob"
myjob
This will show the status of your jobs
Other methods
- Use an interactive way to create your batch job file in Hoffman, read more about job.q
- Use qsub in one line command: examples
Job Array
Job Array is a type of batch mode. It is used to run the same processing job towards a large number of subjects on separate nodes in parallel. More about Job Array