Hoffman2:Software Tools: NDATools: Difference between revisions
Jump to navigation
Jump to search
| Line 10: | Line 10: | ||
==Generate Temporary Tokens== | ==Generate Temporary Tokens== | ||
To access NDA data in AWS S3, you need temporary token generated by your NDA user credential | To access NDA data in AWS S3, you need temporary token generated by your NDA user credential. | ||
generate_token.sh ‘USERID’ ‘PASSWORD’ 'https://nda.nih.gov/DataManager/dataManager' | generate_token.sh ‘USERID’ ‘PASSWORD’ 'https://nda.nih.gov/DataManager/dataManager' | ||
Then you'll get something like the following | Replace USERID and PASSWORD with your NDA login ID and password. | ||
Then you'll get something like the following. These keys and token will be used at the next step. | |||
Beginning token request... | Beginning token request... | ||
Access Key: AAAAAAAAAAAAAAA | Access Key: AAAAAAAAAAAAAAA | ||
Revision as of 23:53, 8 March 2021
NDATools is for downloadig or uploading data to NDA. Here is how to use it in Hoffman
Load modules
module load python/3.6.1_shared module load ndatools
Generate Temporary Tokens
To access NDA data in AWS S3, you need temporary token generated by your NDA user credential.
generate_token.sh ‘USERID’ ‘PASSWORD’ 'https://nda.nih.gov/DataManager/dataManager'
Replace USERID and PASSWORD with your NDA login ID and password.
Then you'll get something like the following. These keys and token will be used at the next step.
Beginning token request... Access Key: AAAAAAAAAAAAAAA Secret Key: SSSSSSSSSSSSSSSSSSSSSSSS Session Token: ABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCDEFGHIJKLMNOPQRSTUVWXYZABCD Expiration: 2021-03-04T12:55:00Z
Download Data
Use command downloadcmd to download data
To check the usage of command downloadcmd
downloadcmd -h
To start downloading the package, you'll need the packageID. The packageID can be found in NDA's website after you login and submit your request for data access.
downloadcmd <packageID> -dp -d /u/project/MYGROUP/MYNDADATA_FOLDER
If the download got interrupted and you want to resume the download
downloadcmd <packageID> -dp -d /u/project/MYGROUP/MYNDADATA_FOLDER -r /u/project/MYGROUP/MYNDADATA_FOLDER