Hoffman2:Singularity: Difference between revisions
Jump to navigation
Jump to search
| (10 intermediate revisions by the same user not shown) | |||
| Line 4: | Line 4: | ||
Singularity is a type of container technology. It is provided in Hoffman2 currently. | Singularity is a type of container technology. It is provided in Hoffman2 currently. | ||
===How to use Singularity=== | * Note: Singularity's newer versions are under its new name <b>"Apptainer"</b>. It's recommended to use Apptainer instead of Singularity. | ||
Just need to replace all the "singularity" with "apptainer" in the following command and it should work the same way. | |||
===How to use Singularity / Apptainer=== | |||
The example code here is from IDRE team's | |||
[https://gitlab.idre.ucla.edu/cpeterson/singularity_ws/-/tree/master | gitlab]. | |||
Please check it up for more information. | |||
====Interactive mode ==== | ====Interactive mode ==== | ||
In a CentOs 7 Node, load Singularity module as | In a CentOs 7 Node, load Singularity / Apptainer module as | ||
module load singularity | module load singularity | ||
singularity shell --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif | singularity shell --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif | ||
or | |||
module load apptainer | |||
apptainer shell --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif | |||
Then run your command inside of the container | Then run your command inside of the container | ||
| Line 19: | Line 31: | ||
Add the following line into your batch job script | Add the following line into your batch job script | ||
module load singularity | module load singularity | ||
Use "singularity exec" to run command in Singularity container | Or | ||
module load apptainer | |||
Use "singularity exec" or "apptainer exec" to run command in Singularity/apptainer container | |||
Example | Example | ||
| Line 27: | Line 42: | ||
singularity exec --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif python3 tf-example.py > tf-example-batch.out | singularity exec --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif python3 tf-example.py > tf-example-batch.out | ||
or | |||
apptainer exec --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif python3 tf-example.py > tf-example-batch.out | |||
===Convert Docker image for Singularity=== | |||
Docker image can be converted for Singularity | |||
export SINGULARITY_CACHEDIR=$SCRATCH/singularity_cache | |||
singularity pull tensorflow-1.15.5-gpu-jupyter.sif docker://tensorflow/tensorflow:1.15.5-gpu-jupyter | |||
or | |||
export APPTAINER_CACHEDIR=$SCRATCH/apptainer_cache | |||
apptainer pull tensorflow-1.15.5-gpu-jupyter.sif docker://tensorflow/tensorflow:1.15.5-gpu-jupyter | |||
Latest revision as of 22:15, 22 June 2026
Singularity
Singularity is a type of container technology. It is provided in Hoffman2 currently.
* Note: Singularity's newer versions are under its new name "Apptainer". It's recommended to use Apptainer instead of Singularity. Just need to replace all the "singularity" with "apptainer" in the following command and it should work the same way.
How to use Singularity / Apptainer
The example code here is from IDRE team's | gitlab. Please check it up for more information.
Interactive mode
In a CentOs 7 Node, load Singularity / Apptainer module as
module load singularity singularity shell --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif
or
module load apptainer apptainer shell --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif
Then run your command inside of the container
python3 tf-example.py > tf-example-batch.out
Batch mode
Add the following line into your batch job script
module load singularity
Or
module load apptainer
Use "singularity exec" or "apptainer exec" to run command in Singularity/apptainer container
Example
singularity exec --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif python3 tf-example.py > tf-example-batch.out
or
apptainer exec --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif python3 tf-example.py > tf-example-batch.out
Convert Docker image for Singularity
Docker image can be converted for Singularity
export SINGULARITY_CACHEDIR=$SCRATCH/singularity_cache singularity pull tensorflow-1.15.5-gpu-jupyter.sif docker://tensorflow/tensorflow:1.15.5-gpu-jupyter
or
export APPTAINER_CACHEDIR=$SCRATCH/apptainer_cache apptainer pull tensorflow-1.15.5-gpu-jupyter.sif docker://tensorflow/tensorflow:1.15.5-gpu-jupyter