When I'm building a Singularity container I'd like to read environment variables from the host system in the %post section. I've been looking online for a way to achieve this, but to no avail. I'm starting to question if this is even possible at the moment, but I can't find any mentions of it being possible/impossible.
Example:
Singularity definition file: recipe
BootStrap: docker
From: continuumio/anaconda3
%runscript
%post
echo $TEST_ENV_VARIABLE
On the host system / OS
export TEST_ENV_VARIABLE='foo'
sudo singularity build test.sif recipe
prints only a blank line when echoing TEST_ENV_VARIABLE.
If there is no way of reading host system's environment variables in the %post section, are there any other ways of passing arguments into the recipe that could be used build-time?
That is not currently possible, though there is an open issue for that functionality. I'm not personally a fan of dynamic build options as it makes it harder to guarantee reproducibility.
If you do want something more dynamic, you could use a template to create different definition files. A very simplistic example:
$ cat gen_def.py
#!/usr/bin/env python3
import sys
my_def = """BootStrap: docker
From: continuumio/anaconda3
%post
echo This is {0}
echo This is {1}"""
print(my_def.format(*sys.argv[1:]))
$ ./gen_def.py one two > Singularity.custom
$ sudo singularity build test.sif Singularity.custom
Related
I am trying to use a shell script as a custom shell in Github Actions like this:
- name: Test bash-wrapper
shell: bash-wrapper {0}
run: |
echo Hello world
However, when I try to run it, I get Permission denied.
Background: I have set up a chroot jail, which I use with QEMU user mode emulation in order to build for non-IA64 architectures with toolchains that lack cross-compilation support.
The script is intended to provide a bash shell on the target architecture and looks like this:
#!/bin/bash
sudo chroot --userspec=`whoami`:`whoami` $CROSS_ROOT qemu-arm-static /bin/bash -c "$*"
It resides in /bin/bash-wrapper and it thus on $PATH.
Digging a bit deeper, I found:
Running bash-wrapper "echo Hello world" in a GHA step with the default shell works as expected.
Running bash-wrapper 'echo Running as $(whoami)' from the default shell correctly reports we are running as user runner.
Removing --userspec from the chroot command in bash-wrapper (thus running the command as root) does not make a difference – the custom shell gives the same error.
GHA converts each step into a script file and passes it to the shell.
File ownership on these files is runner:docker, runner being the user that runs the job by default.
Interestingly, the script files generated by GHA are not executable. I suspect that is what is ultimately causing the permission error.
Indeed, if I modify bash-wrapper to set the executable bit on the script before running it, everything works as expected.
I imagine non-executable script files would cause all sorts of troubles with various shells, thus I would expect GHA would have a way of dealing with that – in fact I am a bit surprised these on-the-fly scripts are not executable by default.
Is there a less hacky way of fixing this, such as telling GHA to set the executable bit on temporary scripts? (How does Github expect this to be solved?)
When calling your script try running it like this:
- name: Test bash-wrapper
shell: bash-wrapper {0}
run: |
bash <your_script>.sh
Alternatively, try running this command locally and the commit and push the repository:
git update-index --chmod=+x <your_script>.sh
I wrote a small test project for Singularity Compose, consisting of a small server application, with the following YAML file:
version: "1.0"
instances:
server:
build:
context: ./server
recipe: server.recipe
ports:
- 9999:9999
When I call singularity-compose build, it successfully builds server.sif. Calling singularity-compose up also seemingly works without error, and calling singularity-compose ps results in something that looks just fine:
+ singularity-compose ps
INSTANCES NAME PID IMAGE
1 server 4176911 server.sif
However, the server application does not work, calling my test client results in it saying that there is no answer from the server.
But if I run server.sif directly without compose, everything works just fine.
Also, I tripple checked, my test application listens to port 9999, thus should be reachable from the outside.
What did I do wrong?
Edit:
I also checked whether there actually is any process listening at port 9999 by calling sudo lsof -i -P -n | grep LISTEN, this is not the case. Only when I manually start server.sif without compose it shows me the process listening.
Edit:
I went into the Singularity Compose shell and tried to start the Server application directly in there, just as a test, and it resulted in Permission denied. Not sure if that means anything.
Edit:
I now gave the application execution rights within the shell and called in there, this works. Am now trying to add execution rights in the recipe. If that works, it would be kind of strange, as the executable was build right there, and thus should already have execution rights.
Edit:
I added chmod +x in my recipe both after building Server and before executing it. Doesn't work either.
Also checked whether any bridges exist using brctl show, this is not the case.
Edit: My recipe, adjusted by the input of tsnowlan in his answer below:
Bootstrap: docker
From: ubuntu:20.04
%files
connection.cpp
connection.h
main.cpp
server.cpp
server.h
server.pro
%post
# get some basics
apt update
apt-get install -y wget
apt-get install -y software-properties-common
# get C++ compiler
apt-get install -y g++
apt-get install -y build-essential
apt-get install -y build-essential cmake
# get Qt
apt-get install -y qt5-default
# compile
qmake
make
ls
%runscript
/Server
%startscript
/Server
Again, note that the application works just fine both when compiled and startet normally and when started within a Singularity image (but without Singularity Compose).
The ls at the end of the %post block is used to verify that the Server application was build successfully.
Please share the server.recipe, as it is difficult to identify should be/is happening without it.
Without having that, my guess is that you have a %runscript in your definition file, but no %startscript. When the image is executed directly or via singularity run image.sif, the contents of %runscript determine what happens. To emulate the docker-compose style, the singularity images are started as persistent instances. In this case, the %startscript block determines what runs. If it is empty, it will just start up and sit there doing nothing. This would explain why when run by hand it works but not when using compose.
I have a singularity container that has been made for me (to run tensorflow on comet GPU nodes) but I need to modify the keras install for my purposes.
I understand that .simg files are not editable (and that the writable .img format is deprecated), so the process of converting to an .img file, editing, and then converting back to .simg is discouraged:
sudo singularity build --writable development.img production.simg
## make changes
sudo singularity build production2.img development.simg
It seems to me the best way might be to extract the contents (say into a sandbox), edit them, and then rebuild the sandbox into an .simg image.
I know how to do the second conversion (singularity build new-sif sandbox), but how can I do the first?
I have tried the following, but the command never finishes:
sudo singularity build tf_gpu tensorflow-gpu.simg
WARNING: Authentication token file not found : Only pulls of public images will succeed
Build target already exists. Do you want to overwrite? [N/y] y
2018/10/12 08:39:54 bufio.Scanner: token too long
INFO: Starting build...
You can easily convert between a sandbox and a production build using the following:
sudo singularity build lolcow.sif docker://godlovedc/lolcow # pulls and builds an example container
sudo singularity build --sandbox lolcow_sandbox/ lolcow.sif # converts from container to a writable sandbox
sudo singularity build lolcow2 lolcow_sandbox/ # converts from sandbox to container
So, you can edit the sandbox and then rebuild accordingly.
I am not sure if this is possible without creating my own base image, but I use environment variables in /etc/environment on our servers and typically make them accessible to apache by doing the following:
$ printf "HTTP_VAR1=var1-value\n\
HTTP_VAR2=var2-value"\
>> /etc/environment
$ mkdir /usr/lib/systemd/system/httpd.service.d
$ printf "[Service]\n\
EnvironmentFile=/etc/environment"\
> /usr/lib/systemd/system/httpd.service.d/environment.conf
$ systemctl daemon-reload
$ systemctl restart httpd
$ reboot
The variables are then available in any PHP calls to getenv('HTTP_VAR1'); and etc. However, in running this from a docker file I get dbus errors on the systemctl commands. Without the systemctl commands it seems the variables are not available to apache as it seems the new EnvironmentFile directive doesn't take effect. My docker file snippet:
FROM centos/httpd:latest
RUN printf "HTTP_VAR1=var1-value\n\
HTTP_VAR2=var2-value"\
>> /etc/environment
RUN mkdir /usr/lib/systemd/system/httpd.service.d &&\
printf "[Service]\n\
EnvironmentFile=/etc/environment"\
> /usr/lib/systemd/system/httpd.service.d/environment.conf
RUN systemctl daemon-reload &&\
systemctl restart httpd
COPY entrypoint.sh /entrypoint.sh
So I happened upon the answer to the issue today. It seems that systemd drops backslashes inside single quotes, but it may effect double-quotes too from what I saw in testing. I found the systemd development mailing list thread from April 2014 where patching the issue was being discussed. It seems as though the fix never made it in. So we have to work around it.
In attempting to work around it I noticed some issues with actually reading the variables at all. It seemed as though either Apache or php-cli would get the correct variables, and sometimes not at all, it took a bit of sleuthing to figure out what was going on. Then I started reading into the systemd's EnvironmentFile directive to see if there was more to gain from the docs. It turns out it does not evaluate bash so export won't work. It expects a text file with variable assignments and herein lies one of the main issues that might keep this from being resolved.
I then devised a workable solution. Utilizing systemd's ExecStartPre directive I am able to run a script on startup of the httpd service. I then read in the environment file and write a new plain text one that will then be used by httpd's systemd unit. Here is the code:
Firstly, I moved my variables to /etc/profile.d/ directory rather than /etc/environment file.
file: /etc/profile.d/environment.sh
This is where we store all our environment variables, this gets easily sourced on all interactive shell logins. In the rarer cases where we need to have these variables available non-interactively we can either provide --login flag to /bin/bash or source it manually.
export HTTP_VAR1=var1-value-with-a-back\slash
export HTTP_VAR2=var2-value
file: /usr/lib/systemd/system/httpd.service.d/environment.conf
Our drop-in unit file to extend how the httpd service works. I add in a script that runs before httpd starts up. This gets ran on all httpd restarts and starts. The script that runs generates a plain text file at /etc/profile.d/environment.env which we subsequently tell systemd to load as an EnvironmentFile.
[Service]
ExecStartPre=/usr/bin/bash -c "/usr/local/bin/generate-plain-environment-file"
EnvironmentFile=/etc/profile.d/environment.env
file: /usr/local/bin/generate-plain-environment-file
Here is the script I am using, I whipped this together really fast, I really don't think it is that robust and it could be better. It just removes the export from the beginning of the lines and then escapes any backslashes since systemd drops single backslashes. A more proper solution might be to use bash to evaluate each line and obtain the variable value in case of usage of variables or other bash in the actual bash variables, then output them as plain text name=value assignments, however this is not part of my use-case so I didn't bother.
#!/bin/bash
cd /etc/profile.d/
rm -rf "./environment.env"
while IFS='' read -r line || [[ -n "$line" ]]; do
echo $(echo "${line}" | sed 's/^export //' | sed 's/\\/\\\\/g') >> "./environment.env";
done < "./environment.sh"
file: /etc/profile.d/environment.env
This is the resulting file generated by the described script.
HTTP_VAR1=var1-value-with-a-back\\slash
HTTP_VAR2=var2-value
Conclusion is that the I now have two files with the same thing in them but I only need to maintain the one, the other is generated each time we restart httpd. Also, we fix the backslash issue in the process. Hurray!
I have a package that is using the autotools to build and install.
Part of the package is a website that can be run on the local machine.
So in the package there is a .conf file that is meant to be either
copied or linked to the /etc/apache2/conf.d directory. What's the
standard way that packages would do this? If possible, I'd like for
the user not to have an extra step to make the website work. I'd like
to have them install the package and then be able to browse to
http://localhost/newpackage to get up and running.
Also, is there a way that autoconf knows about the apache install or a
standard way through then environment some how? If someone could
point me in the right direction that would be great.
Steve
The first thing you should do is to locate the apache extension tool apxs or apxs2 (depends on apache version and/or platform you are building for). After you know where your tool is located you can run queries to get certain apache config params. For example to get system config dir you can run:
apxs2 -q SYSCONFDIR
Here is a snippet of how you can locate apache extension tool: (be careful it may contain syntax errors)
dnl Note: AC_DEFUN goes here plus other stuff
AC_MSG_CHECKING(for apache APXS)
AC_ARG_WITH(apxs,
[AS_HELP_STRING([[--with-apxs[=FILE]]],
[path to the apxs, defaults to "apxs".])],
[
if test "$withval" = "yes"; then
APXS=apxs
else
APXS="$withval"
fi
])
if test -z "$APXS"; then
for i in /usr/sbin /usr/local/apache/bin /usr/bin ; do
if test -f "$i/apxs2"; then
APXS="$i/apxs2"
break
fi
if test -f "$i/apxs"; then
APXS="$i/apxs"
break
fi
done
fi
AC_SUBST(APXS)
The way to use APXS in your automake Makefile.am would look something like this:
## Find apache sys config dir
APACHE2_SYSCONFDIR = `#APXS# -q SYSCONFDIR`
## Misc automake stuff goes here
install: install-am
cp my.conf $(DESTDIR)${APACHE2_SYSCONFDIR}/conf.d/my.conf
I assume you are familiar with automake and autoconf tools.