How to have a script wait for jobs before proceeding

If you want to have some dependency between different jobs, you can use the "-hold_jid" flag in qsub command or cmd file. For example, if you want c.sh runs when a.sh and b.sh both finish or quit, the commands can be as the following:

qsub -N job1 ./a.sh
qsub -N job2 ./b.sh
qsub -hold_jid job1,job2 -N job3 ./c.sh

[The -N names your job for simplicity]

Old Example Create a script named jobhold.sh that contains the following:

#!/bin/bash

# jobhold.sh
# JB 8/2010
# usage: 
#       jobhold.sh qsub <my_command>
#       jobhold.sh q.sh <my_command>
# For commands that are submitted to cluster, hold until jobs have completed

shopt -s expand_aliases

sleep_time=30 # seconds; don't make this too short! don't want to tax system with excessive qstat calls

me=`whoami`
alias myqstat='qstat | grep $me'
stdout=`$@` # call the command and capture the stdout
id=`echo $stdout | awk -F' ' '{print $3}'` # get the jobid
status=`myqstat | grep $id` # check to see if job is running
while [ -n "$status" ] # while $status is not empty
	do
		sleep $sleep_time
		status=`myqstat | grep $id`
	done

Then, in the script, just put the jobhold.sh command in front of the step for which the script should be held until it has finished running on the cluster:

> job_hold.sh <your_command>

eg

> job_hold.sh q.sh bet -in structural.nii.gz -out structural_brain.nii.gz