Hoffman2:Software Tools: NDATools: Difference between revisions

From Center for Cognitive Neuroscience
Jump to navigation Jump to search
No edit summary
No edit summary
Line 1: Line 1:
[https://github.com/NDAR/nda-tools | More information of NDATools]
[https://github.com/NDAR/nda-tools More information of NDATools]


NDATools is for downloadig or uploading data to NDA. Here is how to use it in Hoffman
NDATools is for downloadig or uploading data to NDA. Here is how to use it in Hoffman

Revision as of 23:52, 8 March 2021

More information of NDATools

NDATools is for downloadig or uploading data to NDA. Here is how to use it in Hoffman

Load modules

module load python/3.6.1_shared
module load ndatools


Generate Temporary Tokens

To access NDA data in AWS S3, you need temporary token generated by your NDA user credential

generate_token.sh ‘USERID’ ‘PASSWORD’ 'https://nda.nih.gov/DataManager/dataManager'

Then you'll get something like the following

Beginning token request...
Access Key:    AAAAAAAAAAAAAAA
Secret Key:    SSSSSSSSSSSSSSSSSSSSSSSS
Session Token: 
ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCD
Expiration:    2021-03-04T12:55:00Z

Download Data

Use command downloadcmd to download data

To check the usage of command downloadcmd

downloadcmd -h

To start downloading the package, you'll need the packageID. The packageID can be found in NDA's website after you login and submit your request for data access.

downloadcmd <packageID> -dp -d /u/project/MYGROUP/MYNDADATA_FOLDER

If the download got interrupted and you want to resume the download

downloadcmd <packageID> -dp -d /u/project/MYGROUP/MYNDADATA_FOLDER -r /u/project/MYGROUP/MYNDADATA_FOLDER