I have a development stack with multiple processes running: web server, auto-testing, compilation in background etc. All of these are basic command line commands such as node app.js or lein midje :autotest.
Is it possible with one script to run all these processes in "background" and merge their outputs into one stdout (that is: to show it on the screen in terminal)?
One of the problem with easy bash solution that I found (using &) is that on Ctrl+C the background processes are obviously kept alive, which is not possible desirable.
I have tried adding trap 'kill $(jobs -pr)' SIGINT SIGTERM EXIT but this doesn't seem to work reliably on OS X - surprisingly the node processes get killed, but the java ones are still living after the script exits (via Ctrl+C).
I can use any scripting language. I would prefer pure bash or JS, but Python or Ruby are OK too.
I would also like the ANSI escape colouring to be preserved in the merged output.
You might use multitail utility. It not only allows you to tail log files, but also output of arbitrary CLI programs (lein run, lein midje :autotest, ...).
Example:
$ multitail --mergeall -cT ANSI -l "lein midje :autotest" -cT ANSI -l "lein ring server-headless"
Ctrl-C than kills all processes which are being tailed.
If you are OSX user you can install multitail using brew install multitail (assuming that you already have homebrew installed - if not, see http://mxcl.github.io/homebrew/)
In order to get more info about multitail configuration you might read man multitail. There are also usage examples at http://www.vanheusden.com/multitail/index.php
Related
I have properly installed and setup WSL2. It works fine.
I also setup X11 forwarding and X server (VcXsrv). I can launch GUI apps such like konsole or gvim or even google-chrome from a bash shell.
Now I want to launch konsole by simply double clicking a short cut on the desktop without launching the bash command mode terminal. How should I do it?
I tried running this in cmd:
> wsl /usr/bin/konsole
and it reports:
qt.qpa.xcb: could not connect to display
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, xcb.
I'm guessing it is because some X11 forwarding configurations were not properly setup, so I created a k.sh as follows:
#!/usr/bin/bash
export DISPLAY=$(cat /etc/resolv.conf | grep nameserver | awk '{print $2; exit;}'):0.0
export LIBGL_ALWAYS_INDIRECT=1
/usr/bin/konsole &
The first two lines were the X11 settings in my .bashrc, the last line launches konsole.
It works fine under bash environment; but when I ran
wsl k.sh
from windows cmd environment, it silently quitted without launching the konsole.
I'm out of ideas. What should I do to directly launch konsole or other Linux GUI apps under windows without having to getting into bash?
Thanks in advance.
You are asking about two different command-lines, and while the failures in running them via the wsl command have the same root-cause, the underlying failures are likely slightly different.
In both cases, the wsl <command> invocation results in a non-login, non-interactive shell where the command simply "runs and exits".
Since the shell is non-login/non-interactive, your startup files (such as ~/.bashrc and ~/.bash_profile, among others) are not being processed.
When you run:
wsl /usr/bin/konsole
... the DISPLAY variable is not set, since, as you said, you normally set it in your ~/.bashrc.
Try using:
wsl -e bash -lic "/usr/bin/konsole"
That will force bash to run as a login (-l), interactive (-i) shell. The DISPLAY should be set correctly, and it should run konsole.
Note that the quotes probably aren't necessary in this case, but are useful for delineating the commands you are passing to bash. More complicated command-lines can be passed in via the quotes.
As for:
wsl k.sh
That's likely a similar problem. You are doing the right thing by setting DISPLAY in your script, but I notice that you aren't using a fully-qualified path it. This would normally work, of course, if your script is in a directory on the $PATH.
But I'm guessing that you might add that directory to the $PATH in your startup config, which means (again) that it isn't being set in this non-login, non-interactive shell.
As before, try:
wsl -e bash -lic "k.sh"`
You could also use a fully-qualified path, of course.
And, I'm fairly sure you are going to run into an issue with trying to put konsole in the background via the script. When WSL exits, and the bash shell process ends, the child konsole process will terminate as well.
You could get around this with a nohup in the script, but then you also need to redirect the stderr. It's probably easiest just to move the & from the script itself to the command-line. Change your k.sh to:
#!/usr/bin/bash
export DISPLAY=$(cat /etc/resolv.conf | grep nameserver | awk '{print $2; exit;}'):0.0
export LIBGL_ALWAYS_INDIRECT=1
/usr/bin/konsole
Then run it with:
wsl -e bash -lic "k.sh &"`
Finally, a side note that when and if you can upgrade to Windows 11, it will automatically create Windows Start Menu entries for any Linux GUI app you install that creates a .desktop file. You can manually create .desktop files to have WSL create Start menu items for most applications.
For reference, in Windows 11 it's easier. To run a GUI application without a terminal window popping up, you just need to call wslg.exe instead of wsl.exe.
So, for example:
target: C:\Windows\System32\wslg.exe konsole
start in: C:\WINDOWS\system32
shortcut key: None
comment: Konsole
This tutorial shows how to install VcXsrv and and edit .bashrc to ensure that the "DISPLAY env var is updated on every restart".
DISPLAY env var needs to be dynamic setting.
I've used it successfully with WSL2 on Windows10 Version 21H2 (OS build 19044.2130) to run Chrome, Edge, and thunar. I'm using the Ubuntu 20.04 Linux distro.
To edit .bashrc follow these instructions.
I try to have separate ssp modes during connection using Bluetooth btmgmt utility. Basic idea is scan current device OUI and select ssp on/off modes. But I can't get any answer from neither btmgmt con or btmgmt info commands when I put them into .service files. My system is Arch Linux arm 32-bit and bluez stack version is 5.55-1. I tried
[Unit]
Description=check Bluetooth address
[Service]
Type=oneshot
ExecStart=/usr/bin/bash -c '/usr/bin/btmgmt info >> /usr/local/lib/mac 2>&1'
without any success: it just puts nothing in output file. Some tricks like add
User=root
Group=root
or substitute ExecStart with
ExecStart=/bin/bash -c 'echo -e "$(btmgmt con)" >> /usr/local/lib/mac 2>&1
did nothing. I tried changed thing by putting btmgmt stuff in different bash script instead of start them right from service file, i.e.
ExecStart=/usr/local/lib/test1
to no avail. I'm confused completely, because of:
It doesn't seem to be general btmgmt thing problem, because I can set ssp mode from service files
in very simple manner, just using
ExecStart=btmgmt off
or
ExecStart=btmgmt off
even without full path.
It doesn't seem to be redirecting command error as well, because if I add
ExecStartPre=/usr/bin/bash -c '/usr/bin/fdisk -l > /usr/local/lib/mac 2>&1'
it does work without any problem and I see fdisk info in file (I use fdisk because it requires elevated rights same as btmgmt one).
Moreover, btmgmt info works in the same way, i.e. shows nothing in out file. It makes me think something is wrong in output of btmgmt. I talk about output because input parameters work fine in btmgmt ssp on/of commands and journalctl and systemct don't show any errors in btmgmt con/info cases, so it seems to like output generating successfully but then sending somewhere to outer space, but I'm not sure completely.
Thanks for any help in advance
Well, I make a statement: btmgmt is quite intended to be used for output at any manner, but they tried make a zillion variants of output which directed to exact result it should be: bug is buried somewhere deep inside of bt_shell_printf and struct data. I'm not ready run through all that disgusting documented code (I mean total absence of comments or more-or-less satisfactory mans), so I ended up with a liitte bit hacked version of utility downgraded to simple fprintf in output, leaving only commands I need: con, ssp, power and simplified until only current setting printing info. Everything works fine, and I can use that kind of btmgmt not with systemd only, but even with udev rules (indirectly, of course)
I know this is an old question, but I had the same problem, and managed to fix it after hours and hours of trying.
I don't know why in the world this happens to be so hard with btmgmt but here's the fix:
[Unit]
After=bluetooth.service
Description=Bluetooth service
[Service]
ExecStart=<your-process>
Group=root
StandardInput=tty
TTYPath=/dev/tty2
TTYReset=yes
TTYVHangup=yes
Type=simple
User=root
Basically, by occupying a TTY, btmgmt will think it's running in an interactive terminal, and will output as usual.
Hope this saves anyone the hell I've been through!
I am using Ubuntu via WSL 2.0 on Windows 10 and would like to run Texlive from the Windows command line. To do so I prepended the Texlive folder to the path in /etc/environment (I also tried a number of other locations eg. $HOME/.bashrc):
C:\Users\scott\Documents>wsl echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Windows/system32:...
C:\Users\scott\Documents>wsl
scott#SCOTT-PC:/mnt/c/Users/scott/Documents$ echo $PATH
/usr/local/texlive/2020/bin/x86_64-linux:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/mnt/c/Windows/system32:...
Why is there a difference between these two paths? Is it possible to change the first PATH variable?
To be honest, when I first looked at this question, I thought it would be an easy answer. Oh how wrong I was. There are a lot of nuances to how this works.
Let's start with the fairly "easy" part, though. The main difference between the first method and the second:
wsl by itself launches into a login (and interactive) shell
the shell launched with wsl echo $PATH is neither a login shell nor an interactive shell
So the first will source both login scripts (e.g. ~/.profile) and interactive startup scripts (e.g. ~/.bashrc). The second form does not get to source either of these.
You can see this a different way (and get to the solution) with the following commands:
wsl -e bash -c 'echo $PATH'
wsl -e bash -li -c 'echo $PATH'
The -li forces bash to run as a login and interactive shell, thus sourcing all of the applicable startup scripts. And, as #bovquier points out in the comments, a single quote is needed here to prevent PowerShell from interpolating the $ before it gets to Bash. That, or escape it.
You should be able to run TeX Live the same way, just replacing the "echo $PATH" with the startup command you need for TeX Live.
A second option would be to create a script that both adds the path and runs the command, and just launch that script through wsl /path/to/script.sh
That said, I honestly don't think that your current login/interactive PATH is coming from /etc/environment. In my testing, at least, /etc/environment has no use in WSL, and that's to be expected. /etc/environment is only sourced by PAM modules, and with no login check performed by WSL, there's no reason to invoke PAM in either the wsl nor the wsl echo $PATH commands.
I'd expect that you still have the PATH setting in ~/.bashrc or somewhere similar), and that's where the shell is picking it up from at the moment.
While this isn't necessarily critical to understanding the answer, you might also wonder, if /etc/environment isn't used for setting the default (non-login, non-interactive) path in WSL, what is? The answer seems to be that it is hard-coded into the init that starts up WSL. That init is also what appends the Windows path (assuming you don't have that feature disabled in /etc/wsl.conf).
I'm using singularity to run python in an environnement deprived of python. I'm also running a mysql instance as explained by the IOWA state university (running an instance of mysql, and closing it when done).
For clarity, I'm using a bash script to open mysql, then do what i have to do (a python script) and close mysql, and it works fine. But Python's only way to stop if an error occured is sys.exit([value]) and this not only stops the python script, but also the bash script that ran it. This makes it impossible for me to manage the errors and close the instance of mysql if the python script exits.
My question is : Is there a way for me to execute a 'singularity instance stop mysql' while being in the python sandbox. Something to tell singularity "hey, this command here must be used on the host !" ?
I keep searching but can't find anything.
I only tried to execute it with subprocess like any other command, but it returned an error message because I don't have this instance inside the python sandbox. I don't even have singularity in this sandbox.
For any clarifications, just ask me, I'm trying to be clear but I'm pretty sure it's not very clear.
Thanks a lot !
Generally speaking, it would be a big security issue if a process could be initiated from inside a container (docker or singularity) but run in the host OS's namespace.
If the bash script is exiting on the python failure, it sounds like you're using set -e or #!/bin/bash -e. This causes the script to abort if any command returns non-zero. It's commonly recommended for safer processing, but can cause problems like this at times. To bypass that for the python step you can modify your script:
# start mysql, do some stuff
set +x # disable abort on non-zero return
python my_script.py
set -x # re-enable abort on non-zero
# shut down mysql, do other stuff
I am currently setting up a web service powered by apache and running on CENTOS 6.4.
This service uses perl scripts (cgi-bin) launching in particular external homemade fortran compiled binaries.
Here is the issue: when I boot my server, everything goes well except that one of my binary crashes systematically (with a kernel segfault) when called by my perl scripts.
If I restart manually the httpd service (at the command line: service httpd restart), the issue is totally fixed.
I examined apache/system logs and nothing suspicious can be found.
It appears that the problem occurs only when httpd is launched by /etc/rc[0-6].d startup directives. I tried to change the launch order of http (S85httpd by default) to any other position without success.
To summarize, my web service is only functional (with no external binary crash) when httpd is launched at the command line once the server has fully booted up!
[EDIT] This issue is now resolved:
My fortran binary handles very large arrays and complex functions requiring an unlimited stack size.
Despite that the stack size limit was defined on a system-wide basis (in /etc/security/limits.conf), for any reason it appears that the "apache/perl/fortran binary" ensemble was not aware of that (causing my binary to crash each time it was called).
At the contrary, when I manually restarted apache at the shell prompt, the stacksize limit was correctly passed (.bashrc with 'ulimit -S -s unlimited').
As a workaround, I used BSD::Resource module (http://metacpan.org/pod/BSD::Resource) to define stacksize directly in my perl script by using e.g. setrlimit(RLIMIT_STACK, $softlimit, $hardlimit);
Thus, this new stack size limit is now directly passed from my perl script to my binary.
I've run into similar problems before. Maybe one way to solve this is to put the binary on a 'delayed start', so that it starts after everything else on your system is running. One way to do this is to put an at job in your /etc/rc.local script, to start the binary in X minutes.