Sam’s Notebook - Singularity - RStudio Server Container on Mox

Aidan recently needed to use R on a machine with more memory. Additionally, it would be ideal if he could use RStudio. So, I managed to figure out how to set up a Singularity container running rocker/rstudio.

For those of you not interested in the process, just jump over to our handbook guide on how to use the container to run RStudio Server on Mox (or, build your own).

Now, for the lengthy, and probably unneccessary, parts… Sometimes it’s useful to document the pain.

The UW high-perfomance computing (HPC) cluster, Hyak, used to have an RStudio Server module installed, as well as instructions on how to get it going. However, that module no longer exists, so wasn’t usable. I hit up UW IT and they directed me to this Rocker SLURM job example.. After a few different attempts to get this work, I finally found someone else who’s had the same issue!

I created a functional SLURM script modified from this GitHub Issue and it totally worked! Except, due to the fact that a SLURM script is run on a execute/compute node on Mox, there’s no internet access; which means no installing new packages via RStudio Server when it’s running on Mox. Although I haven’t documented it yet, I was unable to get this to work on build node on Mox (which does have internet access) because parts of the process require the use of sudo which isn’t available on Mox.

So, I turned to installing Singularity on my own computer and building/updating the container locally. Singularity installation didn’t go as smoothly as I would’ve hoped, but I eventually found this Singularity installation guide (GitHub Issue) which took care of any issues I had encountered.

To retrieve the container from Rocker and make it accessible, I needed to build it as a sandbox. The command below also specifies to pull Rstudio with R v4.0.2.

singularity build --sandbox rstudio-4.0.2.sandbox.simg docker://rocker/rstudio:4.0.2

From here, I learned how to get into the container, which would potentially allow me to update/install system and R packages, but you need root access inside the container. To get that, I ran:

sudo singularity shell rstudio-4.0.2.sandbox.simg/

screenshot showing differences in singularity with/without sudo

I also quickly learned that the container needs to be writable:

After making some feable attempt to install some R packages, I encountered a number of errors indicating missing system packages. Those were installed via the shell in the container:

Install libbz2:

apt install libbz2-dev

Install liblzma:

apt install liblzma-dev

Install libxml2

apt install libxml2

Install libz-dev

apt install libz-dev

Install libxtst6 to allow R Markdown image rendering:

apt install libxtst6

After resolving those issues, I could start R and install stuff:

BioConductor:

if (!requireNamespace("BiocManager", quietly = TRUE))
    install.packages("BiocManager")
BiocManager::install(version = "3.12")

DESeq2:

BiocManager::install("DESeq2")

Errors:

MatrixGenerics:

BiocManager::install("MatrixGenerics")

Error:

matrixStats:

install.packages("https://cran.rstudio.com/src/contrib/matrixStats_0.58.0.tar.gz", repos=NULL, type="source")

Methylkit:

BiocManager::install("methylKit")

WGCNA:

BiocManager::install("WGCNA")

tidyverse:

install.packages("tidyverse")

After getting all the desired packages installed, I needed to actually build the container image. The image is a single file, whereas the sandbox representation is actually a way to interact with all the system directories.

Build container (from StackOverFlow):

sudo singularity build rstudio-4.0.2.sjw-01 rstudio-4.0.2.sandbox.simg/

This container image was then rsync’d to the following Mox location for everyone to access:

/gscratch/srlab/programs/singularity_containers

This will suffice for now to allow people to play around/test/use it for some things. However, there are other things I’d like to work on:

Ideally, get this set up to be able to build containers on a Mox build node.
Get Singularity installed on one of the birds (e.g. Emu, Roadrunner) so that people can build their own containers. Singularity is only avialbe for Linux, thus most other lab members won’t be able to use their own computers to build containers; they’ll need one of the birds which run Ubuntu.