I am trying to check if the gcc version and based on the result
execute another system call (scl enable devtoolset-7 bash).
Below is what I have tried so far
gcc --version | awk '/gcc/ && ($3+0)<7.0{print "Current Version",$3,"is less than 7.5" system("scl enable devtoolset-7 bash")}'
I wish to know how to get this done correctly in awk. Also, I am open to other simpler approaches to do the above, if any.
Don't complicate things by having awk spawn a subshell to call another command if you don't have to:
if [[ gcc --version | awk '/gcc/ && ($3+0)<7.0{print "Current Version",$3,"is less than 7.5"; f=1} END{exit !f}' >&2 ]]; then
scl enable devtoolset-7 bash
fi
Note that your test is <7.0 but message says <7.5. You might want to make it this instead to ensure consistency:
if [[ gcc --version | awk -v minVer='7.0' '/gcc/ && ($3+0)<(minVer+0){print "Current Version",$3,"is less than",minVer; f=1} END{exit !f}' >&2 ]]; then
scl enable devtoolset-7 bash
fi
In general software versions can't be directly compared as either strings or numbers, though, so you might have to think about your comparison. e.g. if you wanted to compare 1.2.3 vs 1.2.4 both would be truncated to 1.2 in a numeric comparison and declared equal, but if you wanted to compare 2 different versions like 12.3 vs 5.7, the 5.7 would be considered larger in a string comparison.
I'm trying to make a custom function in a Makefile to detect the current platform and return the proper file accordingly. Here is my attempt.
UNAME := $(shell uname -s)
define platform
ifeq ($(UNAME),Linux)
$1
else ifneq ($(findstring MINGW32_NT, $(UNAME)),)
$2
else ifeq ($(UNAME),Darwin)
$3
endif
endef
all:
#echo $(call platform,linux,windows,mac)
It fails with the following error.
/bin/sh: Syntax error: "(" unexpected
[Finished]make: *** [all] Error 2
What am I doing wrong?
ifeq ... else ... endif are conditional directives in GNU Make, they can't appear inside define ... endef because the latter treats them as a literal text. (Try to remove # sign near echo command and you will see the actual result of evaluating platform function)
I'd move conditionals out of the define directive. Anyway, the target platform can't change during the execution of Make, so there is no need to parse $(UNAME) each time you call platform.
ifeq ($(UNAME),Linux)
platform = $1
else ifneq ($(findstring MINGW32_NT, $(UNAME)),)
platform = $2
else ifeq ($(UNAME),Darwin)
platform = $3
endif
Another option would be to concatenate outputs of uname to form a platform string in a certain format and have platform specific makefiles named accordingly:
ARCH := $(firstword $(shell uname -m))
SYS := $(firstword $(shell uname -s))
# ${SYS}.${ARCH} expands to Linux.x86_64, Linux.i686, SunOS.sun4u, etc..
include ${SYS}.${ARCH}.mk
For example, I have a perl script p.pl that writes "5" to stdout. I'd like to assign that output to a variable like so:
$ x = perl p.pl ! not working code
$ ! now x would be 5
The PIPE command allows you to do Unix-ish pipelining, but DCL is not bash. Getting the output assigned to a symbol is tricky. Each PIPE segment runs in a separate subprocess (like Unix) and there's no way to return a symbol from a subprocess. AFAIK, there's no bash equivalent of assigning stdout to a variable.
The typical approach is to write (redirect) the output to a file and then read it back:
$ PIPE perl p.pl > temp.txt
$ open t temp.txt
$ read t x
$ close t
Another approach is to assign the return value as a JOB logical which is shared by all subprocesses. This can be done as a one-liner using PIPE:
$ PIPE perl p.pl | DEFINE/JOB RET_VALUE #SYS$PIPE
$ x = f$logical("RET_VALUE")
Since the "RET_VALUE" is shared by all processes in the job, you have to be careful of side-effects.
Look up the PIPE command. It lets you do unix like things.
I wanted to identify a particular ACE from a file's ACL and then assign the value to a variable I could refer to later in the script. I wanted to avoid the overhead of writing to/reading from a file as I had 1000s of files to iterate over. This method worked for me.
$ PIPE DIR/SEC filename | SEARCH SYS$PIPE variable | (READ SYS$PIPE variable && DEFINE/JOB/NOLOG variable &variable)
$ SHOW LOGICAL variable
somevar := apple
export somevar
update := $(shell echo "v=$$somevar")
all:
#echo $(update)
I was hoping to apple as output of command, however it's empty, which makes me think export and := variable expansion taking place on different phases. how to overcome this?
The problem is that export exports the variable to the subshells used by the commands; it is not available for expansion in other assignments. So don't try to get it from the environment outside a rule.
somevar := apple
export somevar
update1 := $(shell perl -e 'print "method 1 $$ENV{somevar}\n"')
# Make runs the shell command, the shell does not know somevar, so update1 is "method 1 ".
update2 := perl -e 'print "method 2 $$ENV{somevar}\n"'
# Now update2 is perl -e 'print "method 2 $$ENV{somevar}\n"'
# Lest we forget:
update3 := method 3 $(somevar)
all:
echo $(update1)
$(update2)
echo $(update3)
perl -e 'print "method 4 $$ENV{somevar}\n"'
The output is:
echo method 1
method 1
perl -e 'print "method 2 $ENV{somevar}\n"'
method 2 apple
echo method 3 apple
method 3 apple
perl -e 'print "method 4 $ENV{somevar}\n"'
method 4 apple
#Beta's answer contains the crucial pointer: with GNU make, variables marked with export are only available to [the shells launched for] recipe commands (commands that are part of rules), regrettably not to invocations of $(shell ...) (they only see the environment that make itself saw when it was launched).
There is a workaround, however: explicitly pass the exported variable as an environment variable to the shell function:
update := $(shell somevar='$(somevar)' perl -e 'print "$$ENV{somevar}"')
By prepending the shell command with <var>=<val>, that definition is added as an environment variable to the environment that the command sees - this is a generic shell feature.
Caveat: #Kaz points out in a comment that this method misbehaves if $(somevar) contains certain chars., because the variable expansion is verbatim (no escaping), which can break the resulting shell command, and suggests the following variant to also work with embedded ' instances (breaks the input value into single-quoted substrings with quoted ' spliced in):
update := $(shell somevar='$(subst ','\'',$(somevar))' perl -e 'print "$$ENV{somevar}"')
This should work with all values except multi-line ones (which are rare; there is no workaround for multi-line values that I'm aware of).
On a side note, literal $ chars. in values must be represented as $$, otherwise make will interpret them as references to its own variables.
Note that I've deliberately NOT chosen the OP's original statement, update := $(shell echo "v=$$somevar"), for demonstration, because it contains a pitfall that muddles the issue: due to how the shell evaluates a command line, somevar=apple echo v=$somevar does NOT evaluate to v=apple, because the $somevar reference is expanded before somevar=apple takes effect. To achieve the desired effect in this case, you'd have to use 2 statements: update := $(shell export somevar="$(somevar)"; echo "v=$$somevar")
As for the bug-vs.-feature debate:
While it can be argued that the shell function should see the same environment as recipe commands, the documentation makes no such promise - see http://www.gnu.org/software/make/manual/make.html#Shell-Function. Conversely, http://www.gnu.org/software/make/manual/make.html#Variables_002fRecursion only mentions making exported variables available to recipe commands.
Running the makefile
foo:=apple
export foo
all:
#echo ">"$(shell echo "$$foo")
#echo ">""$$foo"
gives for me (with foo undefined in the environment)
$ make
>
>apple
$ make foo=bar
>
>apple
$ export foo=bar; make
>bar
>apple
$ export foo=bar; make foo=bar
>bar
>bar
Try using the quoted form (update := "v=$$somevar") and let the shell handle expansion when a command is run (you'll still need the export)
Although export does not play nicely with $(shell ...), there is a simple workaround. We can pass the data to the shell script via the command line.
Now of course, environment passage is robust against issues of escaping and quoting. However, the shell language has a single quote quoting method '...' which handles everything. The only problem is that there is no way to get a single quote in there; but of course that is solved by terminating the quote, backslash-escaping the needed single quote and starting a new quote: In other words:
ab'cd -> 'ab'\''cd'
In the shell script executed by $(shell ...) we just generate a variable assignment of the form var='$(...)', where $(...) is some make expression that interpolates suitably escaped material. Thus, Makefile:
somevar := apple with 'quoted' "stuff" and dollar $$signs
shell_escape = $(subst ','\'',$(1))
update := $(shell v='$(call shell_escape,$(somevar))'; echo $$v > file.txt)
.phony: all
all:
cat file.txt
Sample run:
$ make
cat file.txt
apple with 'quoted' "stuff" and dollar $signs
If we want to communicate an environment variable to a command, we can do that using the shell syntax VAR0=val0 VAR1=val1 ... VARn=valn command arg .... This can be illustrated by some minor alterations to the above Makefile:
somevar := apple with 'quoted' "stuff" and dollar $$signs
shell_escape = $(subst ','\'',$(1))
update := $(shell somevar='$(call shell_escape,$(somevar))' env > file.txt)
.phony: all
all:
grep somevar file.txt
Run:
$ make
grep somevar file.txt
somevar=apple with 'quoted' "stuff" and dollar $signs
file.txt contains a dump of environment variables, where we can see somevar. If export in GNU Make did the right thing, we would have been able to just do:
export somevar
update := $(shell env > file.txt)
but the end result is the same.
Since the end result you want is to echo $(update), you would shell_escape anyway, even if GNU Make passed exported vars to $(shell ...). That is to say, look at one more Makefile:
somevar := apple with 'quoted' "stuff" and dollar $$signs
shell_escape = $(subst ','\'',$(1))
update := $(shell v='$(call shell_escape,$(somevar))'; echo $$v)
.phony: all
all:
#echo '$(call shell_escape,$(update))'
#echo $(update)
Output:
apple with 'quoted' "stuff" and dollar $signs
apple with quoted stuff and dollar
I want to create a makefile variable that is a multi-line string (e.g. the body of an email release announcement). something like
ANNOUNCE_BODY="
Version $(VERSION) of $(PACKAGE_NAME) has been released
It can be downloaded from $(DOWNLOAD_URL)
etc, etc"
But I can't seem to find a way to do this. Is it possible?
Yes, you can use the define keyword to declare a multi-line variable, like this:
define ANNOUNCE_BODY
Version $(VERSION) of $(PACKAGE_NAME) has been released.
It can be downloaded from $(DOWNLOAD_URL).
etc, etc.
endef
The tricky part is getting your multi-line variable back out of the makefile. If you just do the obvious thing of using "echo $(ANNOUNCE_BODY)", you'll see the result that others have posted here -- the shell tries to handle the second and subsequent lines of the variable as commands themselves.
However, you can export the variable value as-is to the shell as an environment variable, and then reference it from the shell as an environment variable (NOT a make variable). For example:
export ANNOUNCE_BODY
all:
#echo "$$ANNOUNCE_BODY"
Note the use of $$ANNOUNCE_BODY, indicating a shell environment variable reference, rather than $(ANNOUNCE_BODY), which would be a regular make variable reference. Also be sure to use quotes around your variable reference, to make sure that the newlines aren't interpreted by the shell itself.
Of course, this particular trick may be platform and shell sensitive. I tested it on Ubuntu Linux with GNU bash 3.2.13; YMMV.
Another approach to 'getting your multi-line variable back out of the makefile' (noted by Eric Melski as 'the tricky part'), is to plan to use the subst function to replace the newlines introduced with define in your multi-line string with \n. Then use -e with echo to interpret them. You may need to set the .SHELL=bash to get an echo that does this.
An advantage of this approach is that you also put other such escape characters into your text and have them respected.
This sort of synthesizes all the approaches mentioned so far...
You wind up with:
define newline
endef
define ANNOUNCE_BODY
As of $(shell date), version $(VERSION) of $(PACKAGE_NAME) has been released.
It can be downloaded from $(DOWNLOAD_URL).
endef
someTarget:
echo -e '$(subst $(newline),\n,${ANNOUNCE_BODY})'
Note the single quotes on the final echo are crucial.
Assuming you only want to print the content of your variable on standard output, there is another solution :
do-echo:
$(info $(YOUR_MULTILINE_VAR))
Yes. You escape the newlines with \:
VARIABLE="\
THIS IS A VERY LONG\
TEXT STRING IN A MAKE VARIABLE"
update
Ah, you want the newlines? Then no, I don't think there's any way in vanilla Make. However, you can always use a here-document in the command part
[This does not work, see comment from MadScientist]
foo:
echo <<EOF
Here is a multiple line text
with embedded newlines.
EOF
Not completely related to the OP, but hopefully this will help someone in future.
(as this question is the one that comes up most in google searches).
In my Makefile, I wanted to pass the contents of a file, to a docker build command,
after much consternation, I decided to:
base64 encode the contents in the Makefile (so that I could have a single line and pass them as a docker build arg...)
base64 decode the contents in the Dockerfile (and write them to a file)
see example below.
nb: In my particular case, I wanted to pass an ssh key, during the image build, using the example from https://vsupalov.com/build-docker-image-clone-private-repo-ssh-key/ (using a multi stage docker build to clone a git repo, then drop the ssh key from the final image in the 2nd stage of the build)
Makefile
...
MY_VAR_ENCODED=$(shell cat /path/to/my/file | base64)
my-build:
#docker build \
--build-arg MY_VAR_ENCODED="$(MY_VAR_ENCODED)" \
--no-cache \
-t my-docker:build .
...
Dockerfile
...
ARG MY_VAR_ENCODED
RUN mkdir /root/.ssh/ && \
echo "${MY_VAR_ENCODED}" | base64 -d > /path/to/my/file/in/container
...
Why don't you make use of the \n character in your string to define the end-of-line? Also add the extra backslash to add it over multiple lines.
ANNOUNCE_BODY=" \n\
Version $(VERSION) of $(PACKAGE_NAME) has been released \n\
\n\
It can be downloaded from $(DOWNLOAD_URL) \n\
\n\
etc, etc"
Just a postscript to Eric Melski's answer: You can include the output of commands in the text, but you must use the Makefile syntax "$(shell foo)" rather than the shell syntax "$(foo)". For example:
define ANNOUNCE_BODY
As of $(shell date), version $(VERSION) of $(PACKAGE_NAME) has been released.
It can be downloaded from $(DOWNLOAD_URL).
endef
You should use "define/endef" Make construct:
define ANNOUNCE_BODY
Version $(VERSION) of $(PACKAGE_NAME) has been released.
It can be downloaded from $(DOWNLOAD_URL).
etc, etc.
endef
Then you should pass value of this variable to shell command. But, if you do this using Make variable substitution, it will cause command to split into multiple:
ANNOUNCE.txt:
echo $(ANNOUNCE_BODY) > $# # doesn't work
Qouting won't help either.
The best way to pass value is to pass it via environment variable:
ANNOUNCE.txt: export ANNOUNCE_BODY:=$(ANNOUNCE_BODY)
ANNOUNCE.txt:
echo "$${ANNOUNCE_BODY}" > $#
Notice:
Variable is exported for this particular target, so that you can reuse that environment will not get polluted much;
Use environment variable (double qoutes and curly brackets around variable name);
Use of quotes around variable. Without them newlines will be lost and all text will appear on one line.
This doesn't give a here document, but it does display a multi-line message in a way that's suitable for pipes.
=====
MSG = this is a\\n\
multi-line\\n\
message
method1:
#$(SHELL) -c "echo '$(MSG)'" | sed -e 's/^ //'
=====
You can also use Gnu's callable macros:
=====
MSG = this is a\\n\
multi-line\\n\
message
method1:
#echo "Method 1:"
#$(SHELL) -c "echo '$(MSG)'" | sed -e 's/^ //'
#echo "---"
SHOW = $(SHELL) -c "echo '$1'" | sed -e 's/^ //'
method2:
#echo "Method 2:"
#$(call SHOW,$(MSG))
#echo "---"
=====
Here's the output:
=====
$ make method1 method2
Method 1:
this is a
multi-line
message
---
Method 2:
this is a
multi-line
message
---
$
=====
With GNU Make 3.82 and above, the .ONESHELL option is your friend when it comes to multiline shell snippets. Putting together hints from other answers, I get:
VERSION = 1.2.3
PACKAGE_NAME = foo-bar
DOWNLOAD_URL = $(PACKAGE_NAME).somewhere.net
define nwln
endef
define ANNOUNCE_BODY
Version $(VERSION) of $(PACKAGE_NAME) has been released.
It can be downloaded from $(DOWNLOAD_URL).
etc, etc.
endef
.ONESHELL:
# mind the *leading* <tab> character
version:
#printf "$(subst $(nwln),\n,$(ANNOUNCE_BODY))"
Make sure, when copying and pasting the above example into your editor, that any <tab> characters are preserved, else the version target will break!
Note that .ONESHELL will cause all targets in the Makefile to use a single shell for all their commands.
GNU `make' manual, 6.8: Defining Multi-Line Variables
In the spirit of .ONESHELL, it's possible to get pretty close in .ONESHELL challenged environments:
define _oneshell_newline_
endef
define oneshell
#eval "$$(printf '%s\n' '$(strip \
$(subst $(_oneshell_newline_),\n, \
$(subst \,\/, \
$(subst /,//, \
$(subst ','"'"',$(1))))))' | \
sed -e 's,\\n,\n,g' -e 's,\\/,\\,g' -e 's,//,/,g')"
endef
An example of use would be something like this:
define TEST
printf '>\n%s\n' "Hello
World\n/$$$$/"
endef
all:
$(call oneshell,$(TEST))
That shows the output (assuming pid 27801):
>
Hello
World\n/27801/
This approach does allow for some extra functionality:
define oneshell
#eval "set -eux ; $$(printf '%s\n' '$(strip \
$(subst $(_oneshell_newline_),\n, \
$(subst \,\/, \
$(subst /,//, \
$(subst ','"'"',$(1))))))' | \
sed -e 's,\\n,\n,g' -e 's,\\/,\\,g' -e 's,//,/,g')"
endef
These shell options will:
Print each command as it is executed
Exit on the first failed command
Treat use of undefined shell variables as an error
Other interesting possibilities will likely suggest themselves.
I like alhadis's answer best. But to keep columnar formatting, add one more thing.
SYNOPSIS := :: Synopsis: Makefile\
| ::\
| :: Usage:\
| :: make .......... : generates this message\
| :: make synopsis . : generates this message\
| :: make clean .... : eliminate unwanted intermediates and targets\
| :: make all ...... : compile entire system from ground-up\
endef
Outputs:
:: Synopsis: Makefile
::
:: Usage:
:: make .......... : generates this message
:: make synopsis . : generates this message
:: make clean .... : eliminate unwanted intermediates and targets
:: make all ...... : compile entire system from ground-up
Not really a helpful answer, but just to indicate that 'define' does not work as answered by Ax (did not fit in a comment):
VERSION=4.3.1
PACKAGE_NAME=foobar
DOWNLOAD_URL=www.foobar.com
define ANNOUNCE_BODY
Version $(VERSION) of $(PACKAGE_NAME) has been released
It can be downloaded from $(DOWNLOAD_URL)
etc, etc
endef
all:
#echo $(ANNOUNCE_BODY)
It gives an error that the command 'It' cannot be found, so it tries to interpret the second line of ANNOUNCE BODY as a command.
It worked for me:
ANNOUNCE_BODY="first line\\nsecond line"
all:
#echo -e $(ANNOUNCE_BODY)
GNU Makefile can do things like the following. It is ugly, and I won't say you should do it, but I do in certain situations.
PROFILE = \
\#!/bin/sh.exe\n\
\#\n\
\# A MinGW equivalent for .bash_profile on Linux. In MinGW/MSYS, the file\n\
\# is actually named .profile, not .bash_profile.\n\
\#\n\
\# Get the aliases and functions\n\
\#\n\
if [ -f \$${HOME}/.bashrc ]\n\
then\n\
. \$${HOME}/.bashrc\n\
fi\n\
\n\
export CVS_RSH="ssh"\n
#
.profile:
echo -e "$(PROFILE)" | sed -e 's/^[ ]//' >.profile
make .profile creates a .profile file if one does not exist.
This solution was used where the application will only use GNU Makefile in a POSIX shell environment. The project is not an open source project where platform compatibility is an issue.
The goal was to create a Makefile that facilitates both setup and use of a particular kind of workspace. The Makefile brings along with it various simple resources without requiring things like another special archive, etc. It is, in a sense, a shell archive. A procedure can then say things like drop this Makefile in the folder to work in. Set up your workspace enter make workspace, then to do blah, enter make blah, etc.
What can get tricky is figuring out what to shell quote. The above does the job and is close to the idea of specifying a here document in the Makefile. Whether it is a good idea for general use is a whole other issue.
I believe the safest answer for cross-platform use would be to use one echo per line:
ANNOUNCE.txt:
rm -f $#
echo "Version $(VERSION) of $(PACKAGE_NAME) has been released" > $#
echo "" >> $#
echo "It can be downloaded from $(DOWNLOAD_URL)" >> $#
echo >> $#
echo etc, etc" >> $#
This avoids making any assumptions of on the version of echo available.
Use string substitution:
VERSION := 1.1.1
PACKAGE_NAME := Foo Bar
DOWNLOAD_URL := https://go.get/some/thing.tar.gz
ANNOUNCE_BODY := Version $(VERSION) of $(PACKAGE_NAME) has been released. \
| \
| It can be downloaded from $(DOWNLOAD_URL) \
| \
| etc, etc
Then in your recipe, put
#echo $(subst | ,$$'\n',$(ANNOUNCE_BODY))
This works because Make is substituting all occurrences of | (note the space) and swapping it with a newline character ($$'\n'). You can think of the equivalent shell-script invocations as being something like this:
Before:
$ echo "Version 1.1.1 of Foo Bar has been released. | | It can be downloaded from https://go.get/some/thing.tar.gz | | etc, etc"
After:
$ echo "Version 1.1.1 of Foo Bar has been released.
>
> It can be downloaded from https://go.get/some/thing.tar.gz
>
> etc, etc"
I'm not sure if $'\n' is available on non-POSIX systems, but if you can gain access to a single newline character (even by reading a string from an external file), the underlying principle is the same.
If you have many messages like this, you can reduce noise by using a macro:
print = $(subst | ,$$'\n',$(1))
Where you'd invoke it like this:
#$(call print,$(ANNOUNCE_BODY))
Hope this helps somebody. =)
As an alternative you can use the printf command. This is helpful on OSX or other platforms with less features.
To simply output a multiline message:
all:
#printf '%s\n' \
'Version $(VERSION) has been released' \
'' \
'You can download from URL $(URL)'
If you are trying to pass the string as an arg to another program, you can do so like this:
all:
/some/command "`printf '%s\n' 'Version $(VERSION) has been released' '' 'You can download from URL $(URL)'`"