Which tmpsf partition is used in singularity and how can I increase it - singularity-container

Question

I'm wondering how can I increase tmpfs size in singularity sif image and how it relates to tmpfs on host system. According to a post by Pawsey Centre:
Singularity has a flag for rendering containers from SIF image files ephemerally writable. --writable-tmpfs will allocate a small amount of RAM for this purpose (configured by the sys admins, by default just a bunch of MB)
On my host system I have the following tmpfs:
$ df -h | grep tmpfs
tmpfs 13G 2,9M 13G 1% /run
tmpfs 63G 84M 63G 1% /dev/shm
tmpfs 5,0M 0 5,0M 0% /run/lock
tmpfs 63G 0 63G 0% /sys/fs/cgroup
While inside container I have:
overlay 16M 12K 16M 1% /
...
tmpfs 63G 84M 63G 1% /dev/shm
tmpfs 16M 12K 16M 1% /.singularity.d/libs
tmpfs 13G 2.9M 13G 1% /run/nvidia-persistenced/socket
I can only write small files in my container (couple of KB, otherwise "no space error" is thrown) Which tmpfs does singularity use and why? How can I increase it?

Answer 1

The size of the tmpfs partition is set by the admin using the sessiondir max size config entry. It defaults to 16MB, but you can check it via sudo singularity config global --get "sessiondir max size" or check the config file directly if you don't have sufficient permissions: grep sessiondir /usr/local/etc/singularity/singularity.conf.
You can change the config value (or ask your admins to do it) to increase it to the desired size. If that's not an option, you'll need to make sure the host filesystem is mounted at locations where the data is being written. This is likely a good idea anyway if you'll be writing a lot of data. Docker also does this behind the scenes when using volumes.