Difference between revisions of "Hoffman2:Scripts:setup subject 3"

From Center for Cognitive Neuroscience
Jump to: navigation, search
(Setup_subject script usage)
m (Setup_subject script usage: add link to setup_subject_2)
 
(62 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
==Setup_subject script usage==
 
==Setup_subject script usage==
Setup_subject_3 is built on top of setup_subject_2. The major goal is to apply BIDs format on the data structure. Besides of using BIDs structure, it added other useful features as well.
+
Setup_subject_3 (also named as setup_subject_bids) is built on top of [https://www.ccn.ucla.edu/wiki/index.php/Hoffman2:Scripts:setup_subject setup_subject_2]. The major goal is to apply BIDs format on the data structure. Besides of using BIDs structure, it added other useful features as well.
  
 
Setup_subject_3 has all the functions which exist in setup_subject_2 and works the same way with the same options as setup_subject_2.
 
Setup_subject_3 has all the functions which exist in setup_subject_2 and works the same way with the same options as setup_subject_2.
Line 10: Line 10:
 
basedir is where you want the processed data to be stored in your Hoffman account,
 
basedir is where you want the processed data to be stored in your Hoffman account,
  
subjectID is the subject folder name which has your subject Dicom data
+
subjectID is the subject folder name under /Dicom/data/YOURGROUP/ in Dicom server
  
 
===Options:===
 
===Options:===
  --download : This option will download data from Dicom server to you hoffman account
+
 
 +
====Options for both BIDs mode and non-BIDs mode====
 +
  --download hostname,userid,path
 +
download data from Dicom server to you hoffman account
 
It requires 3 argument, separated by comma(,)
 
It requires 3 argument, separated by comma(,)
 
(If you have downloaded the data already, this option is not needed)
 
(If you have downloaded the data already, this option is not needed)
  
   1. Hostname - (ccn for ccn Dicom server, bmap for BMC Dicom server).  
+
   1. hostname - (ccn for ccn Dicom server, bmap for BMC Dicom server).  
   2. Your Dicom server user ID
+
   2. userid: Your Dicom server user ID
   3. Path to your data in dicom server
+
   3. path: path to the subject data in dicom server (/Dicom/data/YOURGROUP)
  
 +
 +
-d (or --do) : use this option to specify the processing you want to add
 
After NIFTI convert, these options can be used to run FSL mcflirt, bet and melodic also diag, and mprage.
 
After NIFTI convert, these options can be used to run FSL mcflirt, bet and melodic also diag, and mprage.
-d (or --do) : this option to specify the processing you want to add
 
 
   mcflirt : run mcflirt  
 
   mcflirt : run mcflirt  
 
   bet : run bet
 
   bet : run bet
Line 32: Line 36:
 
(Using option diags requires mcflirt, bet and melodic to be processed first)
 
(Using option diags requires mcflirt, bet and melodic to be processed first)
  
  -s (or --skip) this option to specify the processing you want to skip
+
 
 +
  -s (or --skip)  
 +
this option to specify the processing you want to skip
 
   mcflirt : run mcflirt  
 
   mcflirt : run mcflirt  
 
   bet : run bet
 
   bet : run bet
Line 38: Line 44:
 
   diags : run diagnostics
 
   diags : run diagnostics
 
   mprage : process  anatomical scans
 
   mprage : process  anatomical scans
   sge: not using batch mode, which means you need to start interactive mode first before running setup_subject_* command
+
   sge: not using batch mode, which means you need to start interactive mode first before running setup_subject_3 script
 
    
 
    
Set SPM compatability
+
 
 
  --spm
 
  --spm
 +
Set SPM compatibility
 +
 +
====Options for BIDs mode only====
 +
--bids
 +
BIDs format: specify using BIDs structure to group result files. However this option often being used with --config option to choose configuration file.
 +
Without configuration file, the name conversion might not be correct under BIDs format rule.
 +
 +
--config FILEPATH/FILENAME
 +
Specifying the BIDs naming configuration file path and name. Users should generate this file based on their data and BIDSs formatting rule. Here's more about the [[BIDs configuration file]].
 +
 +
--sessdir ses-NN
 +
Specify session directory name. When the subject has multiple sessions, by default the processed results will all go under sub-xxxx/[anat | func | dwi | fmap] without separation based on sessions. If you want to have all the NifTi converted files to be located in one specific session directory, you can use this option and all files will go under sub-xxxx/ses-NN/[anat | func | dwi | fmap ]  (NN represents session number, e.g. 02)
 +
 +
--notar
 +
No compression of dicom/ folder. Normally raw dicom data will be compressed into dicom*.tar.gz at the end of NiFTI convert. With this option, it will not compress.
 +
This option is good when you know you want to run the setup_subject_3 script again and want to save time on compression and decompression of the data.
 +
You should run without this option on the last run so it will save the disk usage.
 +
 +
--setgrp GROUPNAME
 +
Force the data's group to be set as the GROUPNAME specified. The GROUPNAME need to be an existing valid group name in Hoffman.
 +
 +
--pipeline
 +
Specify pipeline name used for analysis ( fsl is the default value)
 +
 +
===Data Structure===
 +
 +
=====Non-BIDs mode=====
 +
folder subjectID will be created under basedir the user specified.
 +
Then Dicom Raw data will be under dicom/ folder during NiFTI convert then compressed into dicom.tar.gz
 +
Result of NiFTI convert will be under raw/ folder
 +
 +
ls basedir/subjectID
 +
analysis  behav  dicom  notes  raw
 +
 +
=====BIDs mode=====
 +
basedir/
 +
            |_
 +
                sub-[subjectID]/
 +
                    |_
 +
                        [ses-SESSIONID]/
 +
                              |_
 +
                                    anat/  dwi/  fmap/  func/
 +
            |_
 +
                sourcedata/
 +
                      |_
 +
                          sub-[subjectID]/
 +
                                |_
 +
                                    [ses-SESSIONID]/
 +
                                            |_
 +
                                                dicom/ setup_log.xxxxxxx
 +
            |_
 +
                derivatives/
 +
                      |_ 
 +
                        PipelineName/
 +
                                |_
 +
                                    sub-[subjectID]/
 +
                                          |_
 +
                                              [ses-SESSIONID]/
 +
                                                    |_
 +
                                                        anat/  dwi/  fmap/  func/
  
 
===Examples: ===
 
===Examples: ===
Line 47: Line 113:
 
Load the script module first:
 
Load the script module first:
 
  module load ccnscripts
 
  module load ccnscripts
 +
 +
====NON-BIDs Mode====
  
 
Download data and run NIFTI convert
 
Download data and run NIFTI convert
  setup_subject_3 --download ccn,dicomuserid,/dicom/data/YOURGROUP/ basedir subjectID
+
  setup_subject_3 --download ccn,dicomuserid,/dicom/data/YOURGROUP/ -s mprage basedir subjectID
( /dicom/data/YOURGROUP/subjectID should be where the raw data is located and accessable by your Dicom server user account)
+
( /dicom/data/YOURGROUP/subjectID is the path to the raw data in Dicom server. We skip mprage here since it will be run by default)
  
 
Download data and run NIFTI convert and other FSL processing:
 
Download data and run NIFTI convert and other FSL processing:
  setup_subject_2 --download ccn,userid,/dicom/data/YOURGROUP/ -d mcflirt,bet,melodic,diags,mprage --spm basedir subjectID
+
  setup_subject_3 --download ccn,userid,/dicom/data/YOURGROUP/ -d mcflirt,bet,melodic,diags,mprage --spm basedir subjectID
  
 
Process data only without download
 
Process data only without download
  setup_subject_2 -d mcflirt,bet,melodic,diags basedir subjectID
+
  setup_subject_3 -d mcflirt,bet,melodic,diags basedir subjectID
  setup_subject_2 -d mcflirt,bet,melodic,diags,mprage basedir subjectID
+
  setup_subject_3 -d mcflirt,bet,melodic,diags,mprage basedir subjectID
  setup_subject_2 -d mprage basedir subjectID
+
  setup_subject_3 -d mprage basedir subjectID
  
 
Skip mcflirt,bet,melodic,diags
 
Skip mcflirt,bet,melodic,diags
   setup_subject_2 -s mcflirt,bet,melodic,diags basedir subjectID
+
   setup_subject_3 -s mcflirt,bet,melodic,diags basedir subjectID
 +
 
 +
====BIDs mode====
 +
 
 +
Download data and run debabel with Bids format, skip mprage.
 +
 
 +
setup_subject_3 --download ccn,userid,/dicom/data/YOURGROUP/ --bids --config ~/bidsname-sub123.conf -s mprage basedir subjectID
  
 
===Notes===
 
===Notes===
Log File: provides useful details about what commands were run, their output and the script's status. It's located at
+
Log File: provides useful details about what commands were run, their output and the script's status. It's located at the following location for bids mode and non-bids mode
 +
 
 +
BIDs mode
 +
 
 +
basedir/sourcedata/sub-subjectID/setup_log.date_numbers
 +
 
 +
Non-BIDs mode
  
 
  basedir/subjectID/notes/setup_log.date_numbers
 
  basedir/subjectID/notes/setup_log.date_numbers

Latest revision as of 22:39, 5 June 2020

Setup_subject script usage

Setup_subject_3 (also named as setup_subject_bids) is built on top of setup_subject_2. The major goal is to apply BIDs format on the data structure. Besides of using BIDs structure, it added other useful features as well.

Setup_subject_3 has all the functions which exist in setup_subject_2 and works the same way with the same options as setup_subject_2.


Basic Usage:

setup_subject_3 [option] basedir subjectID

basedir is where you want the processed data to be stored in your Hoffman account,

subjectID is the subject folder name under /Dicom/data/YOURGROUP/ in Dicom server

Options:

Options for both BIDs mode and non-BIDs mode

--download hostname,userid,path

download data from Dicom server to you hoffman account It requires 3 argument, separated by comma(,) (If you have downloaded the data already, this option is not needed)

 1. hostname - (ccn for ccn Dicom server, bmap for BMC Dicom server). 
 2. userid: Your Dicom server user ID
 3. path: path to the subject data in dicom server (/Dicom/data/YOURGROUP)


-d (or --do) : use this option to specify the processing you want to add

After NIFTI convert, these options can be used to run FSL mcflirt, bet and melodic also diag, and mprage.

  mcflirt : run mcflirt 
  bet : run bet
  melodic : run melodic
  diags : run diagnostics
  mprage : process  anatomical scans
  sge: submit the job in batch mode

(Using option diags requires mcflirt, bet and melodic to be processed first)


-s (or --skip) 

this option to specify the processing you want to skip

  mcflirt : run mcflirt 
  bet : run bet
  melodic : run melodic
  diags : run diagnostics
  mprage : process  anatomical scans
  sge: not using batch mode, which means you need to start interactive mode first before running setup_subject_3 script
 
--spm

Set SPM compatibility

Options for BIDs mode only

--bids

BIDs format: specify using BIDs structure to group result files. However this option often being used with --config option to choose configuration file. Without configuration file, the name conversion might not be correct under BIDs format rule.

--config FILEPATH/FILENAME

Specifying the BIDs naming configuration file path and name. Users should generate this file based on their data and BIDSs formatting rule. Here's more about the BIDs configuration file.

--sessdir ses-NN

Specify session directory name. When the subject has multiple sessions, by default the processed results will all go under sub-xxxx/[anat | func | dwi | fmap] without separation based on sessions. If you want to have all the NifTi converted files to be located in one specific session directory, you can use this option and all files will go under sub-xxxx/ses-NN/[anat | func | dwi | fmap ] (NN represents session number, e.g. 02)

--notar 

No compression of dicom/ folder. Normally raw dicom data will be compressed into dicom*.tar.gz at the end of NiFTI convert. With this option, it will not compress. This option is good when you know you want to run the setup_subject_3 script again and want to save time on compression and decompression of the data. You should run without this option on the last run so it will save the disk usage.

--setgrp GROUPNAME

Force the data's group to be set as the GROUPNAME specified. The GROUPNAME need to be an existing valid group name in Hoffman.

--pipeline

Specify pipeline name used for analysis ( fsl is the default value)

Data Structure

Non-BIDs mode

folder subjectID will be created under basedir the user specified. Then Dicom Raw data will be under dicom/ folder during NiFTI convert then compressed into dicom.tar.gz Result of NiFTI convert will be under raw/ folder

ls basedir/subjectID
analysis  behav  dicom  notes  raw
BIDs mode
basedir/
           |_
               sub-[subjectID]/
                   |_
                       [ses-SESSIONID]/
                              |_
                                   anat/  dwi/  fmap/  func/
            |_
                sourcedata/
                     |_
                         sub-[subjectID]/
                                |_
                                    [ses-SESSIONID]/
                                           |_ 
                                               dicom/ setup_log.xxxxxxx
            |_
                derivatives/
                     |_  
                        PipelineName/
                               |_
                                   sub-[subjectID]/
                                          |_
                                             [ses-SESSIONID]/
                                                   |_ 
                                                       anat/  dwi/  fmap/  func/

Examples:

Load the script module first:

module load ccnscripts

NON-BIDs Mode

Download data and run NIFTI convert

setup_subject_3 --download ccn,dicomuserid,/dicom/data/YOURGROUP/ -s mprage basedir subjectID

( /dicom/data/YOURGROUP/subjectID is the path to the raw data in Dicom server. We skip mprage here since it will be run by default)

Download data and run NIFTI convert and other FSL processing:

setup_subject_3 --download ccn,userid,/dicom/data/YOURGROUP/ -d mcflirt,bet,melodic,diags,mprage --spm basedir subjectID

Process data only without download

setup_subject_3 -d mcflirt,bet,melodic,diags basedir subjectID
setup_subject_3 -d mcflirt,bet,melodic,diags,mprage basedir subjectID
setup_subject_3 -d mprage basedir subjectID

Skip mcflirt,bet,melodic,diags

 setup_subject_3 -s mcflirt,bet,melodic,diags basedir subjectID

BIDs mode

Download data and run debabel with Bids format, skip mprage.

setup_subject_3 --download ccn,userid,/dicom/data/YOURGROUP/ --bids --config ~/bidsname-sub123.conf -s mprage basedir subjectID

Notes

Log File: provides useful details about what commands were run, their output and the script's status. It's located at the following location for bids mode and non-bids mode

BIDs mode

basedir/sourcedata/sub-subjectID/setup_log.date_numbers

Non-BIDs mode

basedir/subjectID/notes/setup_log.date_numbers