Hoffman2:Singularity

Singularity

Singularity is a type of container technology. It is provided in Hoffman2 currently.

* Note: Singularity's newer versions are under its new name "Apptainer". It's recommended to use Apptainer instead of Singularity.
  Just need to replace all the "singularity" with "apptainer" in the following command and it should work the same way.

How to use Singularity / Apptainer

The example code here is from IDRE team's | gitlab. Please check it up for more information.

Interactive mode

In a CentOs 7 Node, load Singularity / Apptainer module as

module load singularity
singularity shell --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif

or

module load apptainer
apptainer shell --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif

Then run your command inside of the container

python3 tf-example.py > tf-example-batch.out

Batch mode

Add the following line into your batch job script

module load singularity

Or

module load apptainer

Use "singularity exec" or "apptainer exec" to run command in Singularity/apptainer container

Example

singularity exec --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif  python3 tf-example.py > tf-example-batch.out

or

apptainer exec --userns $H2_CONTAINER_LOC/tensorflow-2.4.1-gpu-jupyter.sif  python3 tf-example.py > tf-example-batch.out

Convert Docker image for Singularity

Docker image can be converted for Singularity

export SINGULARITY_CACHEDIR=$SCRATCH/singularity_cache
singularity pull tensorflow-1.15.5-gpu-jupyter.sif docker://tensorflow/tensorflow:1.15.5-gpu-jupyter

or

export APPTAINER_CACHEDIR=$SCRATCH/apptainer_cache
apptainer pull tensorflow-1.15.5-gpu-jupyter.sif docker://tensorflow/tensorflow:1.15.5-gpu-jupyter