Difference between revisions of "Submitting Matlab scripts to the SGE cluster"

From Center for Cognitive Neuroscience
Jump to: navigation, search
(Submitting Multiple Jobs without Using Multiple Licenses)
(Submitting Multiple Jobs without Using Multiple Licenses)
Line 62: Line 62:
 
:Note that the -a option can make compilation take a long time
 
:Note that the -a option can make compilation take a long time
 
::Consider using "-I folder1 -I folder2"
 
::Consider using "-I folder1 -I folder2"
 +
::Note that folder1 and folder2 must contain ALL of the dependencies of the compiled function, excluding the default base that MATLAB gives you.
  
 
Without using -a:
 
Without using -a:
Line 79: Line 80:
 
  $@
 
  $@
 
  CMD
 
  CMD
 +
 +
If you have more questions, ask Wesley and/or Hong-jing.

Revision as of 00:22, 16 August 2011

Submitting matlab jobs to the SGE cluster is a breeze, all that needs to be done is copying the script below and substituting runme6 with your particular matlab .m file.

#!/bin/sh
#$ -cwd
#$ -j y
#$ -S /bin/sh
#$ -V
. /etc/profile
/Volumes/local/bin/matlab6 -nojvm -nosplash -nodisplay -r runme6

Explanation of script line by line:

#!/bin/sh
this is a sh script
#$ -cwd
before executing the script, change into the current directory
#$ -j y
the output and errors will be put in the same file (in your current directory)
#$ -S /bin/sh
SGE will use the /bin/sh shell
#$ -V
export the current environment to SGE
. /etc/profile
make sure all our paths are right
/Volumes/local/bin/matlab6 -nojvm -nosplash -nodisplay -r runme6
Use the matlab6 binary (change this to the version you desire/need), no gui, no splash screen, and no display used, run the runme6.m script found in the current directory (change this to your particular script.

Wrappers for general script submission

A set of scripts has been created to ease the use of matlab & clustering. The available scripts are as follows:

matsub6
submits jobs using the matlab 6.5 version
matsub7
submits jobs using the matlab 7.0 version
matsub75
submits jobs using the matlab 7.5 version

Usage

$ matsub6 myAwesomeMatlabScript.m

Output will be placed in the current directory in a file called mat_log

Licensing and SGE Matlab Jobs

Please note that each job you submit to SGE will use up one of our available license seats. Submitting multiple jobs at once could use up all the seats so no one but you will have access to matlab.

Please be courteous to other matlab users and only submit one job at a time.

Submitting Multiple Jobs without Using Multiple Licenses

The key to this approach is compiling your code using the MATLAB Compiler (MCC) then running it using the queue. Note that the MATLAB Compiler has some limitations to it.

- Each job must have a single associated .m file. - You CANNOT use the path() function inside the .m file.

Commands to do it:

qrsh
module load matlab
mcc -m my_m_file.m -a top_path
q.sh my_m_file
mcc -m my_m_file.m -a top_path
This will create an executable called my_m_file
top_path adds all files inside top_path and inside it's subfolders
Note that the -a option can make compilation take a long time
Consider using "-I folder1 -I folder2"
Note that folder1 and folder2 must contain ALL of the dependencies of the compiled function, excluding the default base that MATLAB gives you.

Without using -a:

qrsh
module load matlab
mcc -m my_m_file.m -I folder1 -I folder2
q.sh my_m_file
q.sh is a simple wrapper script by Jesse and the contents are
qsub <<CMD
#!bin/bash
#$ -cwd
#$ -j y
#$ -S /bin/bash
#$ -V
#$ -l highp,h_data=1024M,h_rt=72:00:00
$@
CMD

If you have more questions, ask Wesley and/or Hong-jing.