I am setting up a foreign chroot environment to build for architectures other than amd64 from a GitLab CI image. Steps were mostly taken from https://www.hellion.org.uk/blog/posts/foreign-chroots-with-schroot-and-qemu/, except that I am skipping the schroot/sbuild part.
- export CROSS_ARCH=armhf
- export CROSS_ROOT=/opt/chroot/$CROSS_ARCH
- export DISTRO=stretch
- export CROSS_MIRROR=http://deb.debian.org/debian/
- apt-get update
- apt-get -y install debootstrap qemu-user-static binfmt-support
- mkdir -p $CROSS_ROOT
- debootstrap --variant=buildd --include=fakeroot,build-essential --arch=$CROSS_ARCH --foreign $DISTRO $CROSS_ROOT $CROSS_MIRROR
- mkdir -p $CROSS_ROOT/usr/bin
- cp /usr/bin/qemu-arm-static $CROSS_ROOT/usr/bin/
- chroot $CROSS_ROOT ./debootstrap/debootstrap --second-stage
When I now try to run a command in the target environment like this:
chroot $CROSS_ROOT qemu-arm-static uname -a
the command exits with an error (nonzero exit status), but no error message is printed. It works, however, if I specify the path:
chroot $CROSS_ROOT qemu-arm-static /bin/uname -a
And it gives me the following output, which indicates I am running inside the armhf environment:
Linux runner--azerasq-project-40807358-concurrent-0 5.4.109+ #1 SMP Wed Jun 16 20:00:10 PDT 2021 armv7l GNU/Linux
Oddly, the following works:
chroot $CROSS_ROOT qemu-arm-static /bin/bash -c "uname -a"
i.e. full path to bash, but no path for the command after -c.
Suspecting that there could be something wrong with $PATH, I ran:
chroot $CROSS_ROOT qemu-arm-static /bin/bash -c set
I get all of the GitLab-specific variables, as well as a bunch of others, including the following ones:
MACHTYPE=arm-unknown-linux-gnueabihf
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
When I run
chroot $CROSS_ROOT qemu-arm-static /usr/bin/env
some variables (such as $MACHTYPE) are missing but $PATH is the same. So $PATH seems to be set correctly, and a diff of the outputs (after sorting) does not indicate anything that looks related – the extra variables for bash -c set look mostly bash-specific.
Why won’t qemu-arm-static accept binaries without a full path if they are on $PATH? Where else should I look to debug?
Why won’t qemu-arm-static accept binaries without a full path if they
are on $PATH?
Because qemu-user is not a shell, it doesn't have code that would search the PATH. This is the piece of qemu-user code that opens executable image when it's started as in the examples that you give, and as you can see here exec_path comes directly from the command line.
On the other hand you can install qemu-user as a binfmt-misc handler, in which case the shell will do the PATH search and the kernel will invoke qemu-user with an open file descriptor of the executable file in the AT_EXECFD entry in the aux vectors.
Related
I am trying to use a shell script as a custom shell in Github Actions like this:
- name: Test bash-wrapper
shell: bash-wrapper {0}
run: |
echo Hello world
However, when I try to run it, I get Permission denied.
Background: I have set up a chroot jail, which I use with QEMU user mode emulation in order to build for non-IA64 architectures with toolchains that lack cross-compilation support.
The script is intended to provide a bash shell on the target architecture and looks like this:
#!/bin/bash
sudo chroot --userspec=`whoami`:`whoami` $CROSS_ROOT qemu-arm-static /bin/bash -c "$*"
It resides in /bin/bash-wrapper and it thus on $PATH.
Digging a bit deeper, I found:
Running bash-wrapper "echo Hello world" in a GHA step with the default shell works as expected.
Running bash-wrapper 'echo Running as $(whoami)' from the default shell correctly reports we are running as user runner.
Removing --userspec from the chroot command in bash-wrapper (thus running the command as root) does not make a difference – the custom shell gives the same error.
GHA converts each step into a script file and passes it to the shell.
File ownership on these files is runner:docker, runner being the user that runs the job by default.
Interestingly, the script files generated by GHA are not executable. I suspect that is what is ultimately causing the permission error.
Indeed, if I modify bash-wrapper to set the executable bit on the script before running it, everything works as expected.
I imagine non-executable script files would cause all sorts of troubles with various shells, thus I would expect GHA would have a way of dealing with that – in fact I am a bit surprised these on-the-fly scripts are not executable by default.
Is there a less hacky way of fixing this, such as telling GHA to set the executable bit on temporary scripts? (How does Github expect this to be solved?)
When calling your script try running it like this:
- name: Test bash-wrapper
shell: bash-wrapper {0}
run: |
bash <your_script>.sh
Alternatively, try running this command locally and the commit and push the repository:
git update-index --chmod=+x <your_script>.sh
I wrote a small test project for Singularity Compose, consisting of a small server application, with the following YAML file:
version: "1.0"
instances:
server:
build:
context: ./server
recipe: server.recipe
ports:
- 9999:9999
When I call singularity-compose build, it successfully builds server.sif. Calling singularity-compose up also seemingly works without error, and calling singularity-compose ps results in something that looks just fine:
+ singularity-compose ps
INSTANCES NAME PID IMAGE
1 server 4176911 server.sif
However, the server application does not work, calling my test client results in it saying that there is no answer from the server.
But if I run server.sif directly without compose, everything works just fine.
Also, I tripple checked, my test application listens to port 9999, thus should be reachable from the outside.
What did I do wrong?
Edit:
I also checked whether there actually is any process listening at port 9999 by calling sudo lsof -i -P -n | grep LISTEN, this is not the case. Only when I manually start server.sif without compose it shows me the process listening.
Edit:
I went into the Singularity Compose shell and tried to start the Server application directly in there, just as a test, and it resulted in Permission denied. Not sure if that means anything.
Edit:
I now gave the application execution rights within the shell and called in there, this works. Am now trying to add execution rights in the recipe. If that works, it would be kind of strange, as the executable was build right there, and thus should already have execution rights.
Edit:
I added chmod +x in my recipe both after building Server and before executing it. Doesn't work either.
Also checked whether any bridges exist using brctl show, this is not the case.
Edit: My recipe, adjusted by the input of tsnowlan in his answer below:
Bootstrap: docker
From: ubuntu:20.04
%files
connection.cpp
connection.h
main.cpp
server.cpp
server.h
server.pro
%post
# get some basics
apt update
apt-get install -y wget
apt-get install -y software-properties-common
# get C++ compiler
apt-get install -y g++
apt-get install -y build-essential
apt-get install -y build-essential cmake
# get Qt
apt-get install -y qt5-default
# compile
qmake
make
ls
%runscript
/Server
%startscript
/Server
Again, note that the application works just fine both when compiled and startet normally and when started within a Singularity image (but without Singularity Compose).
The ls at the end of the %post block is used to verify that the Server application was build successfully.
Please share the server.recipe, as it is difficult to identify should be/is happening without it.
Without having that, my guess is that you have a %runscript in your definition file, but no %startscript. When the image is executed directly or via singularity run image.sif, the contents of %runscript determine what happens. To emulate the docker-compose style, the singularity images are started as persistent instances. In this case, the %startscript block determines what runs. If it is empty, it will just start up and sit there doing nothing. This would explain why when run by hand it works but not when using compose.
So currently I have Busybox installed on an embedded kernel in its /system/bin/ folder and can call manually to the VI editor by typing busybox vi and vi will be executed. HOWEVER, I want to create a symbolic link to busybox vi by just typing vi file.txt instead of busybox vi file.txt so I won't have to type busybox every time. How to do this? I already tried this:
Installing Busybox
If the Busybox executable is renamed to one of the commands it supports, it will act as that command automatically:
ln -s busybox pwd
./pwdfrom
...from Busybox's website but still doesn't work, all it says is on my terminal for which command is:
127|root#nitrogen6x:/system/bin # ln -s busbox which
root#nitrogen6x:/system/bin # which ls
/system/bin/sh: which: not found
127|root#nitrogen6x:/system/bin # ls -la which lrwxrwxrwx root root 1970-01-03 18:15 which -> busbox
any ideas what I'm doing wrong? My $PATH is: /sbin:/vendor/bin:/system/sbin:/system/bin:/system/xbin
I figured out how to get this to work
HERE'S HOW:
So I went to root directory:
cd /
Then I remounted the /system/ directory:
mount -o rw,remount /system
Then I went into the binary folder where busybox was located:
cd /system/bin/
Then I used the link command for the busybox binary I wanted:
ln -s busybox lsusb (remember you must be in /system/bin directory already)
For Already Linked Files:
For already linked files like ls, remove the linked file and replace with Busybox binary instead (I know it sounds crazy but you can always go back to system's binary utilities):
sudo rm /system/bin/ls
ln -s busybox ls (remember you must be in /system/bin directory already)
You should get something like this when you do ls -l ls:
lrwxrwxrwx 1 0 0 7 Jan 4 21:53 ls -> busybox
One point to consider is that you have to be on the same file system.
For example if you are trying to create a symbolic link from one mounted file system to a file on another file system then that's an issue.
If your / and /usr are not on the same mounted file system as there might be the case for embedded systems, then you cannot create a symbolic link /usr/bin/which to point to /bin/busybox.
One possible solution is to put a copy of busybox binary in /urs/bin and create link to that.
I have Arch Linux on /dev/sdb1 and NetBSD-7.0 on /dev/sdb2.
On Arch Linux when I run sudo grub-mkconfig -o /boot/grub/grub.cfg I get a message like Found unknown Linux distribution on /dev/sdb2 but when I reboot, there is no grub option for that unknown Linux distribution which I know it is NetBSD-7.0.
How can I add NetBSD-7.0 to my grub menu option when rebooting.
There is a similar post, currently looking into it.
UPDATE: I mounted NetBSD partition with sudo mount -t ufs -o ro,ufstype=ufs2 /dev/sdb2 /mnt/ (ufstype=44bsd did not work) and then ran grub-mkconfig -o /boot/grub/grub.cfg but yet the issue persists.
UPDATE: Rebooted and pressed c to get the grub command line. Following commands booted the NetBSD-7.0:
ls
Ran ls to see the correct name of disks and partitions, /dev/sdb2 on Linux was (hd0,gpt2) on Grub. Then ran the following:
insmod ufs2
set root=(hd0,gpt2)
knetbsd /netbsd
boot
And NetBSD-7.0 booted.
To add NetBSD option to Grub menu, modified file /etc/grub/40_custom on Arch Linux like below:
menuentry "NetBSD-7.0"{
insmod ufs2
set root=(hd0,gpt2)
knetbsd /netbsd
}
However, after modifying 40_custom like above, NetBSD option does not appear on Grub menu. I don't know why.
Unless you have a typo, it looks like the 40_custom file is in the wrong directory. it should be located at /etc/grub.d/40_custom, notice the .d.
If your /boot is located on a separate partition, make sure that it is mounted with mount /boot before generating the grub.cfg. Otherwise your new grub.cfg won't be used.
Check which partition grub is loading the configuration from by running echo ${prefix} within the grub command line. It's possible that grub is loading the configuration from a partition that you don't expect.
Verify that netbsd was added to the config with grep -i netbsd /boot/grub/grub.cfg before rebooting to avoid some frustration after generating grub.cfg
I am setting up a Debian Etch server to host ruby and php applications with nginx. I have successfully configured inittab to start the php-cgi process on boot with the respawn action. After serving 1000 requests, the php-cgi worker processes die and are respawned by init. The inittab record looks like this:
50:23:respawn:/usr/local/bin/spawn-fcgi -n -a 127.0.0.1 -p 8000 -C 3 -u someuser -- /usr/bin/php-cgi
I initially wrote the process entry (everything after the 3rd colon) in a separate script (simply because it was long) and put that script name in the inittab record, but because the script would run its single line and die, the syslog was filled with errors like this:
May 7 20:20:50 sb init: Id "50" respawning too fast: disabled for 5 minutes
Thus, I got rid of the script file and just put the whole line in the inittab. Henceforth, no errors show up in the syslog.
Now I'm attempting the same with thin to serve a rails application. I can successfully start the thin server by running this command:
sudo thin -a 127.0.0.1 -e production -l /var/log/thin/thin.log -P /var/run/thin/thin.pid -c /path/to/rails/app -p 8010 -u someuser -g somegroup -s 2 -d start
It works apparently exactly the same whether I use the -d (daemonize) flag or not. Command line control comes immediately back (the processes have been daemonized) either way. If I put that whole command (minus the sudo and with absolute paths) into inittab, init complains (in syslog) that the process entry is too long, so I put the options into an exported environment variable in /etc/profile. Now I can successfully start the server with:
sudo thin $THIN_OPTIONS start
But when I put this in an inittab record with the respawn action
51:23:respawn:/usr/local/bin/thin $THIN_OPTIONS start
the logs clearly indicate that the environment variable is not visible to init; it's as though the command were simply "thin start."
How can I shorten the inittab process entry? Is there another file than /etc/profile where I could set the THIN_OPTIONS environment variable? My earlier experience with php-cgi tells me I can't just put the whole command in a separate script.
And why don't you call a wrapper who start thin whith your options?
start_thin.sh:
#!/bin/bash
/usr/local/bin/thin -a 127.0.0.1 -e production -l /var/log/thin/thin.log -P /var/run/thin/thin.pid -c /path/to/rails/app -p 8010 -u someuser -g somegroup -s 2 -d start
and then:
51:23:respawn:/usr/local/bin/start_thin
init.d script
Use a script in
/etc/rc.d/init.d
and set the runlevel
Here are some examples with thin, ruby, apache
http://articles.slicehost.com/2009/4/17/centos-apache-rails-and-thin
http://blog.fiveruns.com/2008/9/24/rails-automation-at-slicehost
http://elwoodicious.com/2008/07/15/nginx-haproxy-thin-fastcgi-php5-load-balanced-rails-with-php-support/
Which provide example initscripts to use.
edit:
Asker pointed out this will not allow respawning. I suggested forking in the init script and disowning the process so init doesn't hang (it might fork() the script itself, will check). And then creating an infinite loop that waits on the server process to die and restarts it.
edit2:
It seems init will fork the script. Just a loop should do it.