Setting argv[0] for a valgrind child process? - valgrind

Is there any way to override the argv[0] specified by valgrind when it execve's the child process?
Why?
I'm using Valgrind with a tool that examines its argv[0] to determine the location of its executable in order to find related executables relative to itself. It exec()s a lot of children, most of which are not of any interest and should not be traced by Valgrind.
I can intercept invocations of the commands of interest by populating a directory with wrapper scripts that call the next executable of the same name on the PATH under the control of valgrind. But valgrind always sets argv[0] to the concrete name of the executable it invoked. I want it to pass the name of the wrapper executable instead, so the child command looks in my wrapper directory for related commands to run.
The usual workaround would be to create a symlink to the real executable from the wrapper dir, then invoke the real executable via the symlink. But that won't work here because that's where the wrapper scripts must exist.
Ugly workaround
So far the only solution I see is to re-exec my wrapper script under valgrind, detect that the wrapper script is running under valgrind, and exec the real target program without wrapping when the script detects it's already running under valgrind. That'll work, but it's ugly. It requires that valgrind --trace-children=yes in order to inspect the actual target, which for my use case is undesirable. And it's expensive to have those short-lived valgrind commands run each wrapper script a second time.
Things I tried
I've tried exec -a /path/to/wrapper/command valgrind /path/to/real/command (bash). But valgrind doesn't seem to notice or care that argv[0] isn't valgrind, and does not pass that on to the child process.
Sample wrapper script with hacky workaround
if [ "${RUNNING_UNDER_VALGRIND:-0}" -eq 0 ]; then
# Find the real executable that's next on the PATH. But don't run it
# yet; instead put its path in the environment so it's available
# when we re-exec ourselves under valgrind.
export NEXT_EXEC="$(type -pafP $mycmd | awk '{ if (NR == 2) { print; exit; } }')"
# Re-exec this wrapper under valgrind's control. Valgrind ignores
# argv[0] so there's no way to directly run NEXT_EXEC under valgrind
# and set its argv[0] to point to our $0.
#
RUNNING_UNDER_VALGRIND=1 exec valgrind --tool=memcheck --trace-children=yes "$0" "$#"
else
# We're under valgrind, so exec the real executable that's next on the
# PATH with an argv[0] that points to this wrapper, so it looks here for
# peer executables when it wants to exec them. We looked up NEXT_EXEC
# in our previous life and put it in the environment.
#
exec -a "$0" "${NEXT_EXEC}" "$#"
fi
Yes that's gross. It'd be possible to make a C executable that did the same thing a bit quicker, but the same issues apply with having to trace children, getting unwanted extra logs, etc.
Edit:
This works, so long as your target program(s) don't care about the executable name itself, only the directory.
NEXT_EXEC="$(type -pafP $mycmd | awk '{ if (NR == 2) { print; exit; } }')"
if ! [ "${0}.real" -ef "${NEXT_EXEC}" ]; then
rm -f "${0}.real"
ln "${NEXT_EXEC}" "${0}.real"
fi
exec valgrind --trace-children=no "${0}.real" "$#"
Edit 2
Beginnings of a valgrind patch to add support for a --with-argv0 argument. When passed, valgrind core will treat the first argument after the executable name as the argv[0] to supply in the target's command line. Normally it puts the executable name there, and treats the client argument list as starting at argv[1].

Related

From CMake setup 'make' to use '-j' option by default

I want my CMake project to be built by make -j N, whenever I call make from the terminal. I don't want to set -j option manually every time.
For that, I set CMAKE_MAKE_PROGRAM variable to the specific command line. I use the ProcessorCount() function, which gives the number of procesors to perform build in parallel.
When I do make, I do not see any speed up. However if I do make -j N, then it is built definitely faster.
Would you please help me on this issue? (I am developing this on Linux.)
Here is the snippet of the code that I use in CMakeList.txt:
include(ProcessorCount)
ProcessorCount(N)
message("number of processors: " ${N})
if(NOT N EQUAL 0)
set(CTEST_BUILD_FLAGS -j${N})
set(ctest_test_args ${ctest_test_args} PARALLEL_LEVEL ${N})
set(CMAKE_MAKE_PROGRAM "${CMAKE_MAKE_PROGRAM} -j ${N}")
endif()
message("cmake make program" ${CMAKE_MAKE_PROGRAM})
Thank you very much.
In case you want to speed up the build you can run multiple make processes in parallel but not cmake.
To perform every build with predefined number of parallel processes you can define this in MAKEFLAGS.
Set MAKEFLAGS in your environment script, e.g. ~/.bashrc as you want:
export MAKEFLAGS=-j8
On Linux the following sets MAKEFLAGS to the number of CPUs - 1: (Keep one CPU free for other tasks while build) and is useful in environments with dynamic ressources, e.g. VMware:
export MAKEFLAGS=-j$(($(grep -c "^processor" /proc/cpuinfo) - 1))
New from cmake v3.12 on:
The command line has a new option --parallel <JOBS>.
Example:
cmake --build build_arm --parallel 4 --target all
Example with number of CPUs- 1 using nproc:
cmake --build build_arm --parallel $(($(nproc) - 1)) --target all
Via setting the CMAKE_MAKE_PROGRAM variable you want to affect the build process. But:
This variable affects only the build via cmake --build, not on native tool (make) call:
The CMAKE_MAKE_PROGRAM variable is set for use by project code. The value is also used by the cmake(1) --build and ctest(1) --build-and-test tools to launch the native build process.
This variable should be a CACHEd one. It is used in such way by make-like generators:
These generators store CMAKE_MAKE_PROGRAM in the CMake cache so that it may be edited by the user.
That is, you need to set this variable with
set(CMAKE_MAKE_PROGRAM <program> CACHE PATH "Path to build tool" FORCE)
This variable should refer to the executable itself, not to a program with arguments:
The value may be the full path to an executable or just the tool name if it is expected to be in the PATH.
That is, value "make -j 2" cannot be used for that variable (splitting arguments as list
set(CMAKE_MAKE_PROGRAM make -j 2 CACHE PATH "Path to build tool" FORCE)
wouldn't help either).
In summary, you may redefine the behavior of cmake --build calls with setting the CMAKE_MAKE_PROGRAM variable to the script, which calls make with parallel options. But you may not affect the behavior of direct make calls.
You may set the env variable MAKEFLAGS using this command
export MAKEFLAGS=-j$(nproc)
My solution is to have a small script which will run make include all sorts of other features, not just the number of CPUs.
I call my script mk and I do a chmod 755 mk so I can run it with ./mk in the root of my project. I also have a few flags to be able to run various things with a simple command line. For example, while working on the code and I get many errors, I like to pipe the output to less. I can do that with ./mk -l without having to retype all the heavy duty Unix stuff...
As you can see, I have the -j4 in a couple of places where it makes sense. For the -l option, I don't want it because in this case it would eventually cause multiple errors to be printed at the same time (I tried that before!)
#!/bin/sh -e
#
# Execute make
case "$1" in
"-l")
make -C ../BUILD/Debug 2>&1 | less -R
;;
"-r")
make -j4 -C ../BUILD/Release
;;
"-d")
rm -rf ../BUILD/Debug/doc/lpp-doc-?.*.tar.gz \
../BUILD/Debug/doc/lpp-doc-?.*
make -C ../BUILD/Debug
;;
"-t")
make -C ../BUILD/Debug
../BUILD/Debug/src/lpp tests/suite/syntax-print.logo
g++ -std=c++14 -I rt l.cpp rt/*.cpp
;;
*)
make -j4 -C ../BUILD/Debug
;;
esac
# From the https://github.com/m2osw/lpp project
With CMake, it wouldn't work unless, as Tsyvarev mentioned, you create your own script. But I personally don't think it's sensible to call make from your make script. Plus it could break a build process which would not expect that strange script. Finally, my script, as I mentioned, allows me to vary the options depending on the situation.
I usually use alias in linux to set cm equal to cmake .. && make -j12. Or write a shell to specify make and clean progress ...
alias cm='cmake .. && make -j12'
Then use cm to make in a single command.

Adding home-brew to PATH

I just installed Home-brew and now I'm trying to insert the home-brew directory at the top of my path environment variable by typing in two commands inside my terminal. My questions are these:
What is a path environment variable?
Are the two codes provided me correct?
echo "export Path=/usr/local/bin:$PATH" >> ~/.bash_profile && source ~/.bash_profile
After this I am to type in brew doctor. Nothing is happening as far as I can see.
Can anyone offer me some advice or direction?
I installed brew in my new Mac M1 and ask me to put /opt/homebrew/bin in the path, so the right command for this case is:
echo "export PATH=/opt/homebrew/bin:$PATH" >> ~/.bash_profile && source ~/.bash_profile
TL;DR
echo "export PATH=/usr/local/bin:$PATH" >> ~/.bash_profile && source ~/.bash_profile
is what you want.
To answer your first question; in order to run (execute) a program (executable) the shell must know exactly where it is in your filesystem in order to run it. The PATH environment variable is a list of directories that the shell uses to search for executables. When you use a command that is not built into the shell you are using the shell will search through these directories in order and will execute the first matching executable it finds.
For example when you type: mv foo bar the shell is almost certainly actually using an executable located in the /bin directory. Thus fully the command is
/bin/mv foo bar
The PATH environment variable therefore saves you some extra typing. You can see what is in your PATH currently (as you can with all environment variables) by entering:
echo $<NAME OF VARIABLE>
So in this instance:
echo $PATH
As I mentioned earlier, ordering is important. Adding /usr/local/bin to the beginning of PATH means that the shell will search there first and so if you have an executable foo in that folder it will be used in preference to any other foo executables you may have in the folders in your path. This means that any executables you install with brew will be used in preference to the system defaults.
On to your second question. What the command you have provided is trying to do is add a line to your .bash_profile and then source it. The .bash_profile is a text file stored in your home directory that is sourced (read) every time bash (your shell) starts. The mistake in the line you've provided is that only the first letter of PATH is capitalised. To your shell Path and PATH are very different things.
To fix it you want:
echo "export PATH=/usr/local/bin:$PATH" >> ~/.bash_profile && source ~/.bash_profile
To explain
echo "export PATH=/usr/local/bin:$PATH"
simply prints or echoes what follows to stdout, which in the above instance is the terminal. (stdout, stderr and stdin are very important concepts on UNIX systems but rather off topic) Running this command produces the result:
export PATH=/usr/local/bin:/opt/local/sbin:/opt/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
on my system because using $PATH within double quotes means bash will substitute it with its value. >> is then used to redirect stdout to the end of the ~/.bash_profile file. ~ is shorthand for your home directory. (NB be very careful as > will redirect to the file and overwrite it rather than appending.)
&& means run the next command is the previous is successful and
source ~/.bash_profile
simply carries out the actions contained in that file.
As per the latest documentation, you need to do this:
echo 'eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"' >> /home/dhruv/.bashrc
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
Now you should be able to run brew from anywhere.
When you type in a program somewhere and click enter, it checks certain locations to see if that program exists there.
Linux brew uses locations different from the normal linux programs, so we are adding these locations to the ~/.profile file which sets the paths.
Run this in your terminal, and it will place the correct code in the .profile file, automatically.
echo "eval \$($(brew --prefix)/bin/brew shellenv)" >>~/.profile
Don't use .bash_profile because when you use something different from bash, like zsh, it may not work. .profile is the correct location.

GNU Make Error 126, C:\Program is a directory

GNU make gives me a strange error message, which I do not understand.
gao#L8470-130213 ~
$ make
echo Test
C:\Program: C:\Program: is a directory
make: *** [test] Error 126
This is what I thought of verifying:
gao#L8470-130213 ~
$ less makefile
test:
echo Test
gao#L8470-130213 ~
$ which make
/c/Programx86/GnuWin32/bin/make
gao#L8470-130213 ~
$ /c/Progra~2/GnuWin32/bin/make.exe test
echo Test
C:\Program: C:\Program: is a directory
make: *** [test] Error 126
gao#L8470-130213 ~
$ make --version
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for i386-pc-mingw32
It feels like some other program is trying to run at the end, and that its path includes some spaces. In that case, what program could it be, and how can I prevent it from running?
I have seen this thread and tried to disable my antivirus, which did not help.
I have also looked into permissions, but I am not sure if makefile needs execution rights. I can't seem to be able to change that anyway (running in bash on windows. makefile is not read-only when I check in explorer):
gao#L8470-130213 ~
$ ls -l makefile
-rw-r--r-- 1 gao Administ 21 Apr 15 14:53 makefile
gao#L8470-130213 ~
$ chmod +x makefile
gao#L8470-130213 ~
$ ls -l makefile
-rw-r--r-- 1 gao Administ 21 Apr 15 14:53 makefile
What is going on with make, what can I do?
It's not "some other program" that's trying to run, it's the echo command. Make prints the command to be run, echo test, but you never see the output (test) so that means it failed trying to find the echo program. Unfortunately I'm not very familiar with the vagaries of running GNU make on Windows: there are lots of different options. One possibility would be to get a newer version of GNU make; 3.81 is very old. 3.82 is now available and might work better for you.
Good info you added above about your environment re: using bash; that wasn't clear from the original question and on Windows there are many different ways to do things. You're using the mingw version of make; that version (as I understand it) does NOT use bash as the shell to run commands in: it's supposed to be used with native Windows environments which do not, certainly, have bash available. I believe that the version of make you have is invoking commands directly, and/or using command.com. Certainly not a UNIX shell like bash.
If you want to use bash you should set the SHELL make variable to the path of your bash.exe program. If you're using a Cygwin environment you can use the GNU make that comes with Cygwin which behaves more like a traditional make + shell.
Otherwise you'll need to write your commands using Windows command.com statements.
Again, I don't use Windows so this is mostly hearsay.
PS. The makefile does not need to be executable.
What is going on is that make doesn't like file names or directory names with spaces in them, such as Program Files. Neither do most of the utilities that makefiles typically rely on, such as the shell to execute commands with.
I create a junction from Program Files to ProgramFiles and use the latter whenever I encounter cases like this.

Custom error handling in makefile

I need to build a C program which requires a particular Linux package. I set a variable, PACKAGENOTIFICATION, to a shell command which is supposed to check if the package is installed for Ubuntu and print a notification if not:
PACKAGENOTIFICATION := if cat /etc/issue | grep Ubuntu -c >>/dev/null; then if ! dpkg -l | grep libx11-dev -c >>/dev/null; then echo "<insert notification here>"; fi; fi
[...]
maintarget: dependencies
$(PACKAGENOTIFICATION)
other_commands
Unfortunately, while making the dependencies, it runs into the files which need the package, and errors out before executing my PACKAGENOTIFICATION. An alternative formulation is to make a separate target whose only purpose is to run the notification:
maintarget: notify other_dependencies
commands
notify:
$(PACKAGENOTIFICATION)
However, since this phantom dependency always needs to be executed, make never reports that the program is up to date.
What's the best way to have make always report as up to date, but also execute my notification before it dies?
Thanks!
If your version of Make supports "order-only" prerequisites, this will do it:
# Note the pipe
maintarget: other_dependencies | notify
commands
# This should be an order-only preq of any target that needs the package
notify:
$(PACKAGENOTIFICATION)
If not, there are other approaches.

Can I use GNU make's SHELL variable to connect to a remote shell?

One of the projects I'm working on uses gnu make for testing. I would like to test on a platform that doesn't have a make implementation, but does have a POSIX shell.
Is it possible to create a script (preferably in python) that can "stitch" a remote shell to make, and put it in make's SHELL environment variable?
If not, is there another way anyone can think of to do it?
It is possible.
Create a script that forwards commands to a remote host. For example:
#!/bin/bash
shift # remove -c argument
exec ssh remote_host "$#"
And make it executable (chmod +x).
And then in Makefile:
SHELL := './my_shell.sh'
all :
#echo `hostname` is making $#
Outputs:
$ make
remote_host.peer1.net is making all