How to make /var/log symlink to a persistent storage in Yocto Rocko - embedded

I'm building a Yocto-based distribution with systemd and journald in its core.
And unfortunately I cannot get Yocto to store all logs in /var/log -> /data/log. I need journald logs as well as some other logs that are written there after multi-user.target to be persistent. /data is the persistent partition.
I have a very similar problem to this but unfortunately I couldn't modify it to work properly in my setup.
From my understanding there's two things I need to modify:
volatiles file in base-files which I hope is a config file for systemd-tmpfiles. It should tell it to create at runtime everything that journald needs. Here I modified one line:
L+ root root 0755 /var/log /data/log
fs-perms.txt
${localstatedir}/log link /data/log
I also tried to pull it off with VOLATILE_LOG_DIR set "no" (fs-perms-persistent-log.txtmodified but to no avail. And also adding some kind ofvar.confto/etc/tmpfiles.d` with a config similar to the one above. It also hasn't worked.
I launch a watch ls -l on the resulting rootfs/var and see that var/log is getting symlinked to `/data/log for a short while but later it's overridden somewhere to point to volatile/log once again.
I would greatly appreciate any advice because it seems like I'm overcomplicating this thing. It should be very easy. After all it's just making Yocto to make a symlink. But I guess this is a rather important directory to let me ln -sf /data/log /var/log.
I would also like to hear out implications of this approach.
Other than wearing out my eMMC. We can live with that because the log activity is very low compared to some other actions performed on the device. I'm mostly interested about mount order and stuff. If I remember correctly, journald will use a memory buffer until it has /var/log/journal created for it so I should be fine. But what should I do to ensure everything's in place before the logs are flushed? Do I need to modify systemd services to include RequireMountsFor or After=?
I want to be as defensive as possible so I'm looking forward to what you guys have to say on the topic.
EDIT:
Maybe I can just add a bind mount from /var/log to /data/log? If that is actually the solution I'd also like to know if there's no hidden hindrances down the road?

you can mount your persistent partition via tweaking the base-files recipe (base-files_%.bbappend)
do_install_append () {
cat >> ${D}${sysconfdir}/fstab <<EOF
# Data partition
/dev/mmcblk0p4 /data auto defaults,sync,noauto 0 2
EOF
}
dirs755 += "/data"
then you can tweak volatile-binds (volatile-binds.bbappend)
VOLATILE_BINDS = "\
/data/var/lib /var/lib\n\
/data/var/log /var/log\n\
/data/var/spool /var/spool\n\
/data/var/srv /srv\n\
"
This should hopefully help, I have not tested it fully here, but I hope this might provide you some starting point.

A persistent log data option has made it in Yocto 2.4: https://bugzilla.yoctoproject.org/show_bug.cgi?id=6132
Log data can be made persistent by defining the following in your distro config:
VOLATILE_LOG_DIR = "no"

Related

Redirecting output of bluez btmgmt to file from systemd service

I try to have separate ssp modes during connection using Bluetooth btmgmt utility. Basic idea is scan current device OUI and select ssp on/off modes. But I can't get any answer from neither btmgmt con or btmgmt info commands when I put them into .service files. My system is Arch Linux arm 32-bit and bluez stack version is 5.55-1. I tried
[Unit]
Description=check Bluetooth address
[Service]
Type=oneshot
ExecStart=/usr/bin/bash -c '/usr/bin/btmgmt info >> /usr/local/lib/mac 2>&1'
without any success: it just puts nothing in output file. Some tricks like add
User=root
Group=root
or substitute ExecStart with
ExecStart=/bin/bash -c 'echo -e "$(btmgmt con)" >> /usr/local/lib/mac 2>&1
did nothing. I tried changed thing by putting btmgmt stuff in different bash script instead of start them right from service file, i.e.
ExecStart=/usr/local/lib/test1
to no avail. I'm confused completely, because of:
It doesn't seem to be general btmgmt thing problem, because I can set ssp mode from service files
in very simple manner, just using
ExecStart=btmgmt off
or
ExecStart=btmgmt off
even without full path.
It doesn't seem to be redirecting command error as well, because if I add
ExecStartPre=/usr/bin/bash -c '/usr/bin/fdisk -l > /usr/local/lib/mac 2>&1'
it does work without any problem and I see fdisk info in file (I use fdisk because it requires elevated rights same as btmgmt one).
Moreover, btmgmt info works in the same way, i.e. shows nothing in out file. It makes me think something is wrong in output of btmgmt. I talk about output because input parameters work fine in btmgmt ssp on/of commands and journalctl and systemct don't show any errors in btmgmt con/info cases, so it seems to like output generating successfully but then sending somewhere to outer space, but I'm not sure completely.
Thanks for any help in advance
Well, I make a statement: btmgmt is quite intended to be used for output at any manner, but they tried make a zillion variants of output which directed to exact result it should be: bug is buried somewhere deep inside of bt_shell_printf and struct data. I'm not ready run through all that disgusting documented code (I mean total absence of comments or more-or-less satisfactory mans), so I ended up with a liitte bit hacked version of utility downgraded to simple fprintf in output, leaving only commands I need: con, ssp, power and simplified until only current setting printing info. Everything works fine, and I can use that kind of btmgmt not with systemd only, but even with udev rules (indirectly, of course)
I know this is an old question, but I had the same problem, and managed to fix it after hours and hours of trying.
I don't know why in the world this happens to be so hard with btmgmt but here's the fix:
[Unit]
After=bluetooth.service
Description=Bluetooth service
[Service]
ExecStart=<your-process>
Group=root
StandardInput=tty
TTYPath=/dev/tty2
TTYReset=yes
TTYVHangup=yes
Type=simple
User=root
Basically, by occupying a TTY, btmgmt will think it's running in an interactive terminal, and will output as usual.
Hope this saves anyone the hell I've been through!

Access pagemap in gem5 FS mode

I am trying to run an application which uses pagemap in gem5 FS mode.
But I am not able to use pagemap in gem5. It throws below error -
"assert(pagemap>=0) failed"
The line of code is:
int pagemap = open("/proc/self/pagemap", O_RDONLY);
assert(pagemap >= 0);
Also, If I try to run my application on gem5 terminal with sudo ,it throws error-
sudo command not found
How can I use sudo in gem5 ??
These problems are not gem5 specific, but rather image / Linux specific, and would likely happen on any simulator or real hardware. So I recommend that you remove gem5 from the equation completely, and ask a Linux or image specific question next time, saying exactly what image your are using, kernel configs, and provide a minimal C example that reproduces the problem: this will greatly improve the probability that you will get help.
I have just done open("/proc/self/pagemap", O_RDONLY) successfully with: this program and on this fs.py setup on aarch64, see also these comments.
If /proc/<pid>/pagemap is not present for any file, do the following:
ensure that procfs is mounted on /proc. This is normally done with an fstab entry of type:
proc /proc proc defaults 0 0
but your init script needs to use fstab as well.
Alternatively, you can mount proc manually with:
mount -t proc proc proc/
you will likely want to ensure that /sys and /dev are mounted as well.
grep the kernel to see if there is some config controlling the file creation.
These kinds of things are often easy to find without knowing anything about the kernel.
If I do:
git grep '"pagemap'
to find the pagemap string, which is likely the creation point, on v4.18 this leads me to fs/proc/base.c, which contains:
#ifdef CONFIG_PROC_PAGE_MONITOR
REG("pagemap", S_IRUSR, proc_pagemap_operations),
#endif
so make sure CONFIG_PROC_PAGE_MONITOR is set.
sudo: most embedded / simulator images don't have it, you just login as root directly and can do anything by default without it. This can be seen by the conventional # in the prompt instead of $.

Changing permissions of added file to a Docker volume

In the Docker best practices guide it states:
You are strongly encouraged to use VOLUME for any mutable and/or user-serviceable parts of your image.
And by looking at the source code for e.g. the cpuguy83/nagios image this can clearly be seen done, as everything from nagios to apache config directories are made available as volumes.
However, looking at the same image the apache service (and cgi-scripts for nagios) are run as the nagios user by default. So now I'm in a pickle, as I can't seem to figure how to add my own config files in order to e.g. define more hosts for nagios monitoring. I've tried:
FROM cpuguy83/nagios
ADD my_custom_config.cfg /opt/nagios/etc/conf.d/
RUN chown nagios: /opt/nagios/etc/conf.d/my_custom_config.cfg
CMD ["/opt/local/bin/start_nagios"]
I build as normal, and try to run it with docker run -d -p 8000:80 <image_hash>, however I get the following error:
Error: Cannot open config file '/opt/nagios/etc/conf.d/my_custom_config.cfg' for reading: Permission denied
And sure enough, the permissions in the folder looks like (whist the apache process runs as nagios):
# ls -l /opt/nagios/etc/conf.d/
-rw-rw---- 1 root root 861 Jan 5 13:43 my_custom_config.cfg
Now, this has been answered before (why doesn't chown work in Dockerfile), but no proper solution other than "change the original Dockerfile" has been proposed.
To be honest, I think there's some core concept here I haven't grasped (as I can't see the point of declaring config directories as VOLUME nor running services as anything other than root) - so provided a Dockerfile as above (which follows Docker best practices by adding multiple volumes) is the solution/problem:
To change NAGIOS_USER/APACHE_RUN_USER to 'root' and run everything as root?
To remove the VOLUME declarations in the Dockerfile for nagios?
Other approaches?
How would you extend the nagios dockerfile above with your own config file?
Since you are adding your own my_custom_config.cfg file directly into the container at build time just change the permissions of the my_custom_config.cfg file on your host machine and then build your image using docker build. The host machine permissions are copied into the container image.

Docker: How to live sync host folder with container folder?

I am working on a website powered by Node. So I have made a simple Dockerfile that adds my site's files to the container's FS, installs Node and runs the app when I run the container, exposing the private port 80.
But if I want to change a file for that app, I have rebuild the container image and re-run it. That takes some seconds.
Is there an easy way to have some sort of "live sync", NFS like, to have my host system's app files be in sync with the ones from the running container?
This way I only have to relaunch it to have changes apply, or even better, if I use something like supervisor, it will be done automatically.
You can use volumes in order to do this. You have two options:
Docker managed volumes:
docker run -v /src/path nodejsapp
docker run -i -t -volumes-from <container id> bash
The file you edit in the second container will update the first one.
Host directory volume:
docker run -v `pwd`/host/src/path:/container/src/path nodejsapp
The changes you make on the host will update the container.
If you are under OSX, those kind of volume shares can become very slow, especially with node-based apps ( a lot of files ). For this issue, http://docker-sync.io can help, by providing a volume-share like synchronisation, without using volume shares, this usually speeds up your container read/write speed of the code-directory from 50-80 times, depending on what docker-machine you use.
For performance see https://github.com/EugenMayer/docker-sync/wiki/4.-Performance and for easy examples how to use it, see the boilerplates https://github.com/EugenMayer/docker-sync-boilerplate for your case the unison example https://github.com/EugenMayer/docker-sync-boilerplate/tree/master/unison is the one you would need for NFS like sync
docker run -dit -v ~/my/local/path:/container/path/ myimageId
For /container/path/ you could use for instance /usr/src/app.
The flags:
-d = detached mode,
-it = interactive,
-v + paths = specifies the volume.
(If you just care about the volume, you can drop the -dit flag.)
Docker run reference
I use Scaffold's File Sync functionality for this. It gets the job done, and without needing overly complex configuration.
Setting up Scaffold in my project was as simple as installing Skaffold (through chocolatey, since I'm on Windows), running skaffold init --generate-manifests in my project folder, and answering a couple questions it asked.

iOS Processes Info (pid, uid, cpu, mem, ...)

I'm writing app like System Activity in Mac. I used modified darwin.h and darwin.c from this topic.
But it seems that it shows correct info only for certain fields (and some time ago it showed all correct info but only for current process - this app), for everything else it shows correct only pid, process name, group/owner. Everything else like cpu/mem/disk usage is nulled. I think it may be something with sandbox or security. Tried to move application to /Applications and setting root/wheel 755 but it's not work, still zeroed values.
Here's my output : click
Code changed in darwin.c : click, written after a lot of commented printf lines
Also there is a free apps approved in AppStore like SystemInfo (com.lymbilesoft.systeminfoliteforiphone) that shows detailed info about disk and processes, so how do it work? Tried nm-ing it's binary but did not find something useful.
Can someone please help me with it?
I'm not sure if this is the only problem, but it certainly is possible that your app needs to run as root for this to work correctly.
However, simply installing the app in /Applications, doing chown root.wheel, and chmod 755 will not cause it to run as root. It will still run as user mobile.
In order to get the app to run as root, see this answer
I've used this technique successfully, and if you have a jailbroken phone, you can navigate to /Applications/Cydia.app and see that this launch script is how Cydia does this, too.
I use ps aux from the Terminal/Command Line. That gives me all the stuff I need, regarding your concerns.