I have an established yocto build which I'm now trying to switch over to having a root file system (eg. EXTRA_IMAGE_FEATURES += "read-only-rootfs").
However I'm running into an issue with a recipe in the meta-mono layer: mozroot-certdata. I see the culprit is the pkg_postint script (http://git.yoctoproject.org/cgit/cgit.cgi/meta-mono/tree/recipes-mono/mozroot-certdata/mozroot-certdata_1.0.0.bb) which needs to modify the root file system on first boot which the build system is correctly flagging as impossible with a read only root file system:
ERROR: The following packages could not be configured offline and rootfs is read-only: ['mozroot-certdata']
My question is: is there a way to get these mozroot certs installed and configured with mono during the build process, such that the root file system does not need to be modified at boot/run time?
Well, I had a brief look at this late this summer, as I'm also using a read-only rootfs. The problem is that mozroot.exe is hardcoded to write into /usr/share/.mono/certs and does not respect your sysroot. You could probably hack mozroot.exe to actually write the imported files into the sysroot, though my time limit didn't allow me to try this (and neither have I ever looked into mono at all...).
My solution was instead to do the import at every boot. (It could also be done only once, but then the issue about updates come along). To achive this I made a bind mount on the directory where mozroot.exe wants to write the certdata.
Details of my solution
Add a file volatile-binds.bbappend with the following contents:
VOLATILE_BINDS += "\
/tmp/mono-certs /usr/share/.mono/certs \n\
"
That will make a bind mount from /tmp/mono-certs to /usr/share/.mono/certs, thus you'll be able to import the certs.
Then I added a service file and a mozroot-certdata_%.bbappend:
FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:"
DEPENDS += "mono-native"
SRC_URI += "file://mozroot-certdata.service \
"
inherit systemd
SYSTEMD_SERVICE_${PN} = "mozroot-certdata.service"
do_install_append() {
mkdir -p ${D}${datadir}/.mono/certs
mkdir -p ${D}${systemd_system_unitdir}
install -m 440 ${WORKDIR}/mozroot-certdata.service ${D}${systemd_system_unitdir}/mozroot-certdata.service
}
FILES_${PN} += "${datadir}"
# Empty the postinstallation script, as we can import the cert offline.
pkg_postinst_${PN} () {
# mono $D/usr/lib/mono/4.5/mozroots.exe --import --machine --ask remove --file $D/${sysconfdir}/ssl/certdata.txt
}
The service file mozroot-certdata.service:
[Unit]
Description=Import certficates to Mono
After=tmp-mono-certs.service
[Service]
Type=oneshot
ExecStart=/usr/bin/mono /usr/lib/mono/4.5/mozroots.exe --import --machine --ask-remove --file /etc/ssl/certdata.txt
[Install]
WantedBy=multi-user.target
is there a way to get these mozroot certs installed and configured with mono during the build process
Yes but it requires mosroots binary to be executable at rootfs creation time. See Post-Installation Scripts in documentation.
The 'else' branch in pkg_postinst is what gets executed at that time and if it succeeds, then the delayed postinst is not needed (and you shouldn't get a build error). mono-native recipe already exists so you should be able to depend on that and to fix the else branch in the pkg_postinst function so it finds native mono & mosroots.exe and writes to the correct place under $D.
As Anders mentioned this alone is not enough if you care about package-based upgrades.
Related
Imagine we have a statically linked Linux executable.
How should I name it in the imported tar.gz so the WSL 1 will run it by default, when created and started like:
# import an archive as a WSL distro
wsl --import static tmp-root-dir static.tar.gz
# boot distro to a default app??
wsl -d static
PS WSL uses own proprietary boot process and seems doesn't use traditional Unix /sbin/init.
Short answer:
The smallest bootable (without errors or warnings) WSL rootfs will consist of three files:
/main: Your statically-linked application. It can be named whatever you want, as long as the name matches what is in passwd.
/etc/passwd: Defines the application (i.e. shell) to load for the default user.
/etc/wsl.conf: To suppress normal WSL functionality and (optionally) define the non-root user.
More detail:
This probably isn't exactly what you are wanting, but it will hopefully meet your needs.
To start with, the entry point for WSL (the first time a Linux ELF binary is started inside the instance) seems to be its /init binary, which, in addition to some "normal" Linux init process tasks, sets up some of the Windows-interop functionality. To my knowledge, it cannot currently be changed. As far as I can tell, for WSL1, it is injected into the instance by the LXSS manager when starting a WSL instance.
Note: WSL2 might be slightly different in this regard, as it does seem to use a kernel-processed initrd to load /init. It is possible to override the kernel command-line, but that would impact all WSL2 instances, so it's probably not a practical solution.
It's not quite clear from your question whether you want the "default application" to:
Run as the default application/shell every time wsl -d static is run, even if it was already running.
Or just run once when starting the WSL1 instance for the first time.
I believe you are looking for the first option.
Run as the default application
In the first case, the standard WSL1 /init process might get you to where you need to be. As part of the startup, as you would expect, it reads /etc/passwd to determine the user shell to start. It also reads /etc/wsl.conf to determine the default user ID (but falls back to the registry if there is no default user set in wsl.conf).
So, to start a different application (let's call it main), you can:
Place the binary in the root directory of your image.
Set the application as the "shell" of the root user in a single-line /etc/passwd:
user:x:1000:1000:user:/:/main
Side-note that this also sets the home directory to / so we don't have to create another directory.
Define a etc/wsl.conf with the following contents:
[user]
default=user
[automount]
enabled=false
mountFsTab=false
[interop]
appendWindowsPath=false
This will prevent WSL from performing the following startup tasks, which would produce an error without additional image support:
Mounting Windows drives into the instance
Attempting to process /etc/fstab (since we have no mount command in the image).
Appending Windows paths (since our instance won't have access to the Windows drives)
It also sets the default user to the UID 1000 user we created in /etc/passwd. This isn't strictly necessary - There's likely no concern with running as root in this single-use instance, but I've included a non-root user as a "best practice".
That should be it. The smallest bootable WSL rootfs will consist of just those three files:
/etc/wsl.conf
/etc/passwd
/main
This will work on WSL1 as well as WSL2, although for WSL2, you should invoke with wsl ~ -d static to make sure that it doesn't try to start on a Windows drive that it can't access. Otherwise, you'll receive an init error, but your application will still be invoked.
Run once
If you are looking for something that will, for instance, start up a daemon when the instance is started for the first time, then there are a few alternatives that I document in this answer. If you are on Windows 11, then there's a built-in mechanism via /etc/wsl.conf. Otherwise, on Windows 10, you'll probably need to include some binary that can handle conditional logic. Something like execline would probably be perfect for this, but I've had issues with it under WSL2, at least, and I'm not sure that it would run under WSL1 (but it might).
Side-note for WSL1/musl
musl is a commonly used alternative libc implementation. For instance, Rust (AFAICT), can only generate truly statically-linked executables using musl. Note, however, that WSL1 cannot run musl-based statically linked binaries.
WSL2 can handle them just fine.
I managed to get it working. Initially I missed an executable bit on the app when created TAR archive.
Take standard 64-bit assembly:
.data
msg:
.ascii "Hello, world!\n"
.set len, . - msg
.text
.globl _start
_start:
# write
mov $1, %rax
mov $1, %rdi
mov $msg, %rsi
mov $len, %rdx
syscall
# exit
mov $60, %rax
xor %rdi, %rdi
syscall
and create a minimal WSL system:
wsl as -64 -o minimal.o minimal.s
wsl ld -melf_x86_64 -o minimal minimal.o
tar czf minimal.tar.gz \
--mode=a=rx \
--xform='s#^minimal#/\0#' minimal
wsl --import minimal rootfs-minimal minimal.tar.gz --version 1
wsl --list
wsl -d minimal -e /minimal
To make executable default (shorten wsl -d minimal -e /minimal to wsl -d minimal) we need an extra file /etc/passwd:
root:x:0:0:root:/root:/minimal
First line of this file determine a default user and so path to the executable (entry point) unless you override the user with /etc/wsl.conf:
[user]
default=user
Basically WSL 1 treats only 2 files as magical (in addition to ignoring /sbin/init):
/etc/wsl.conf
/etc/passwd
In the Docker best practices guide it states:
You are strongly encouraged to use VOLUME for any mutable and/or user-serviceable parts of your image.
And by looking at the source code for e.g. the cpuguy83/nagios image this can clearly be seen done, as everything from nagios to apache config directories are made available as volumes.
However, looking at the same image the apache service (and cgi-scripts for nagios) are run as the nagios user by default. So now I'm in a pickle, as I can't seem to figure how to add my own config files in order to e.g. define more hosts for nagios monitoring. I've tried:
FROM cpuguy83/nagios
ADD my_custom_config.cfg /opt/nagios/etc/conf.d/
RUN chown nagios: /opt/nagios/etc/conf.d/my_custom_config.cfg
CMD ["/opt/local/bin/start_nagios"]
I build as normal, and try to run it with docker run -d -p 8000:80 <image_hash>, however I get the following error:
Error: Cannot open config file '/opt/nagios/etc/conf.d/my_custom_config.cfg' for reading: Permission denied
And sure enough, the permissions in the folder looks like (whist the apache process runs as nagios):
# ls -l /opt/nagios/etc/conf.d/
-rw-rw---- 1 root root 861 Jan 5 13:43 my_custom_config.cfg
Now, this has been answered before (why doesn't chown work in Dockerfile), but no proper solution other than "change the original Dockerfile" has been proposed.
To be honest, I think there's some core concept here I haven't grasped (as I can't see the point of declaring config directories as VOLUME nor running services as anything other than root) - so provided a Dockerfile as above (which follows Docker best practices by adding multiple volumes) is the solution/problem:
To change NAGIOS_USER/APACHE_RUN_USER to 'root' and run everything as root?
To remove the VOLUME declarations in the Dockerfile for nagios?
Other approaches?
How would you extend the nagios dockerfile above with your own config file?
Since you are adding your own my_custom_config.cfg file directly into the container at build time just change the permissions of the my_custom_config.cfg file on your host machine and then build your image using docker build. The host machine permissions are copied into the container image.
I'd really appreciate any help in tracking down and diagnosing an umask issue on Ubuntu:
I'm running php5-fpm with Apache via proxy_fcgi. The process is running with a umask of 0022 (confirmed by having PHP send the results of umask() into a file [the result is '18' == 0022]). I'd like to change this to 0002, but can't track down where the umask is coming from.
Apache is set with umask 0002, and as a test, if I disable proxy_fcgi and run my test above, I get a file with u+g having rw access (and the file contents confirm the umask as '2' == 0002).
If I sudo -iu fpmuser and run umask the results are 0002.
System info:
PHP: 5.5.3-1ubuntu2.1
Apache: 2.4.6
Ubuntu: 13.10
PHP-PFM is listing using TCP ports (as Unix ports aren't yet working/support)
So far I've tried the following (each followed by a system restart and a retest):
adding umask 0002 to the start of /etc/init.d/php5-fpm
adding --umask 0002 into the start-stop-daemon calls in /etc/init.d/php5-fpm
adding umask 0002 to .profile in the home of the fpm user
Something is clearly adjusting the umask of the php-fpm process - so, how can I begin tracing what is forcing the umask 0022 onto the php-fpm process?
EDIT (1):
adjusting the system wide umask via /etc/login.defs (see How to set system wide umask?) affects the umask elsewhere (e.g. comannds via sudo now have a umask of 0002), but still php-fpm creates files with a umask of 0022. Note that I verified that session optional pam_umask.so was also present in /etc/pam.d/common-session-noninteractive and I tested umasks of 002 and 0002.
EDIT (2):
I have been able to replicate the issue using nginx and php5-fpm (using unix sockets set to listen mode '0666').
I would love to trace where the umask is coming from but I'd settle for some way to force it to what I want.
I should add that the first test was done on an Amazon Ubuntu 13.10 image. My tests in 'edit 2' where completed using a copy of the Ubuntu13.10 server ISO setup from scratch in a virtual machine. All installations were completed via apt-get rather than by downloading the source and building.
EDIT (3):
I have confirmed I can manipulate the umask manually by either of the following (verified by checking the permissions on the test file created):
a. In a shell, set a umask then run /usr/sbin/php-fpm from the shell
b. In a shell, run the following with whatever umask value I like:
start-stop-daemon --start --quiet --umask 0002 --pidfile /var/run/php5-fpm.pid --exec /usr/sbin/php5-fpm -- --daemonize --fpm-config /etc/php5/fpm/php-fpm.conf
However this exact same command in the /etc/init.d/php5-fpm file fails to adjust the umask when running sudo service php5-fpm stop; sudo service php5-fpm start or at reboot.
Not a solution for generically tracing where umask settings are coming from on ubuntu (the only way I've found so far is the good old hard work approach of replicating the issue, attempting to isolate it to a script or a function, then stepping back through each script/function that is called recursively) but a solution to the php5-fpm umask issue. I've found a lot of hits on google, stackoverflow, and elsewhere for the problem, but so far no solution. Hopefully this is useful for people.
Edit /etc/init/php-fpm.conf to include the line umask 0002 (or whatever umask you wish). My version of the file now looks like this:
# php5-fpm - The PHP FastCGI Process Manager
description "The PHP FastCGI Process Manager"
author "Ondřej Surý <ondrej#debian.org>"
start on runlevel [2345]
stop on runlevel [016]
### my edit - change umask setting
umask 0002
pre-start exec /usr/lib/php5/php5-fpm-checkconf
respawn
exec /usr/sbin/php5-fpm --nodaemonize --fpm-config /etc/php5/fpm/php-fpm.conf
Explanation
Having traced through the service command which launches php5-fpm at startup, it runs some checks (line 118 on my copy) for /etc/init/${SERVICE}.conf, along with verifying initctl is present and can report it's version. If these tests are passed then upstart is used which in the case of php5-fpm uses the /etc/init/php-fpm.conf file.
The ubuntu upstart site gives pretty clear instructions. In particular you can check out the upstart cookbook for the specifics you need.
As best I can work out that means that therefore the 'service' command was never actually running the start-stop-daemon … commands found in /etc/init.d/php5-fpm which is why my previous edits were having no effect. Instead it passes off to upstart (actually initctl) when you use something like service php5-fpm start, etc.
If you use systemd, in the /etc/systemd/system directory, create a new directory called php7.2-fpm.service.d. The name of this directory will vary depending on your distro and PHP version. Run systemctl list-units --type=service | grep --ignore-case php to find out what to call it. Inside of this directory, place a file called umask.conf with the contents:
# /etc/systemd/system/php7.2-fpm.service.d/umask.conf
[Service]
UMask=0002
For the changes to take effect, run:
systemctl daemon-reload && systemctl restart php7.2-fpm
The benefit of this solution is that your customizations are not lost when packages get updated.
Explanation of how this works from the systemd manual:
Along with a unit file foo.service, a "drop-in" directory foo.service.d/ may exist. All files with the suffix ".conf" from this directory will be parsed after the file itself is parsed. This is useful to alter or add configuration settings for a unit, without having to modify unit files. Each drop-in file must have appropriate section headers. Note that for instantiated units, this logic will first look for the instance ".d/" subdirectory and read its ".conf" files, followed by the template ".d/" subdirectory and the ".conf" files there.
In addition to /etc/systemd/system, the drop-in ".d" directories for system services can be placed in /usr/lib/systemd/system or /run/systemd/system directories. Drop-in files in /etc take precedence over those in /run which in turn take precedence over those in /usr/lib. Drop-in files under any of these directories take precedence over unit files wherever located. Multiple drop-in files with different names are applied in lexicographic order, regardless of which of the directories they reside in.
better copy systemd script before editing php5-fpm.service or it will be overwritten on next update:
cp /lib/systemd/system/php5-fpm.service /etc/systemd/system/
vi /etc/systemd/system/php5-fpm.service
Add: UMask=0002 in [Service] section.
systemctl daemon-reload
systemctl restart php5-fpm
Source: https://ispire.me/running-php-fpm-with-different-user-group-using-umask/
okey, but this applies to all the pools.
Would be handy to be able to set it with something like
env[umask] = 0002
(no chance for this to work)
been googling, but doesn't seem to be a way to do this on a per host basis.
I have been given the unpleasant task of installing a Rails 3 app I have written on Windows Server 2008 (definitely not my choice - was promised a linux server but I.T. pulled the rug out at the last minute so please don't suggest a change in environment as a solution).
I followed the instructions on this blog post (with a few minor modifications) and now actually have my app up and running under Windows/IIS (proxying mongrel) after a great deal of frustration. The only thing remaining is to get mongrel running as a service.
Unfortunately the mongrel gem has not been kept up-to-date for Rails 3 and while I can get the app running under mongrel at the command line I am unable to use mongrel_service to get the app running as a service.
The solution to this appears to be to use the service_wrapper project on github which has been mentioned in this previous question. The project is not yet complete but apparently functional but comes without documentation/binaries. I have looked through the source-code and don't really understand what is it/how it works so was wondering if someone can point me in the right direction (or, even better, walk me through how) to get this installed.
So close, yet still so far.....
Alright I have this worked out (with a little help from luislavena himself - thanks).
Download service_wrapper-0.1.0-win32.zip from https://github.com/luislavena/service_wrapper/downloads and extract service_wrapper.exe from bin/. I extracted it to C:\service_wrapper.
Next set up a configuration file. I used the hello example and modified it for my app then placed it in the C:\service_wrapper directory.
; Service section, it will be the only section read by service_wrapper
[service]
; Provide full path to executable to avoid issues when executable path was not
; added to system PATH.
executable = C:\Ruby192\bin\ruby.exe
; Provide there the arguments you will pass to executable from the command line
arguments = C:\railsapp\script\rails s -e production
; Which directory will be used when invoking executable.
; Provide a full path to the directory (not to a file)
directory = C:\railsapp
; Optionally specify a logfile where both STDOUT and STDERR of executable will
; be redirected.
; Please note that full path is also required.
logfile = C:\railsapp\log\service_wrapper.log
Now just create the service with
sc create railsapp binPath= "C:\service_wrapper\service_wrapper.exe C:\service_wrapper\service_wrapper.conf" start= auto
(watch for the spaces after binPath= and start=. It won't work without them)
Then start it with
net start railsapp
And you're home and hosed!
I ought to contribute due to this article. For config of using bundle exec, use the following:
Note that I am setting up rubyCAS! it's a great OpenCAS authentication mechanism!!!
; Service section, it will be the only section read by service_wrapper
[service]
; Provide full path to executable to avoid issues when executable path was not
; added to system PATH.
executable = C:\Ruby\bin\ruby.exe
; Provide there the arguments you will pass to executable from the command line
arguments = D:\rubycas-server bundle exec rackup -s mongrel -p 11011
; Which directory will be used when invoking executable.
; Provide a full path to the directory (not to a file)
directory = D:\rubycas-server
; Optionally specify a logfile where both STDOUT and STDERR of executable will
; be redirected.
; Please note that full path is also required.
logfile = D:\rubycas-server\log\service_wrapper.log
I am trying to do distributed builds with Xcode, but I see this error while building from my build server (Build Sever is the host, dev machine is the client).
When I try to do this the other way, I am able to distribute builds (My Dev machine as the host and the Build Sever as the client)
Any thoughts?
[14:44:47]: Step 2/3 (6m:10s)
[14:44:57]: [Step 2/3] distcc[95606] (dcc_parse_multiplier) ERROR: bad multiplier "/0,lzo,cpp" in host specification
[14:44:57]: [Step 2/3] distcc[95606] (dcc_show_hosts) CRITICAL! Failed to get host list
[14:44:57]: [Step 2/3] /usr/bin/pump: error: pump mode requested, but distcc hosts list does not contain any hosts with ',cpp' option
Your milage may vary with this solution, but we've had to hack the distcc that comes with Xcode to force pump mode to be off to fix this problem.
Remove pump from /Developer/usr/bin and /usr/bin, just write out an empty file named pump in its place
Don't forget to chmod a+x your pump and distcc (in the next step)
In /Developer/usr/bin, rename distcc to distcc.bin and write out this distcc
#!/bin/bash
hosts=$DISTCC_HOSTS
hosts=${hosts//\,cpp/}
export DISTCC_HOSTS=$hosts
echo Modified DISTCC_HOSTS=\"$DISTCC_HOSTS\"
/Developer/usr/bin/distcc.bin $#
Apologies, this is a quick and dirty solution. There is probably a cleaner way to do this.
Please restart the build server and your own computer. That usually does the trick for me, also, update to the latest xcode 4