I have the following command I want to execute in a Makefile but I'm not sure how.
The command is docker rmi -f $(docker images | grep "<none>" | awk "{print \$3}")
The command executed between $(..) should produce output which is fed to docker rmi but this is not working from within the Makefile I think that's because the $ is used specially in the Makefile but I'm not sure how to modify the command to fit in there.
Any ideas?
$ in Makefiles needs to be doubled to prevent substitution by make:
docker rmi -f $$(docker images | grep "<none>" | awk "{print \$$3}")
Also, it'd be simpler to use use a singly-quoted string in the awk command to prevent expansion of $3 by the shell:
docker rmi -f $$(docker images | grep "<none>" | awk '{print $$3}')
I really recommend the latter. It's usually better to have awk code in single quotes because it tends to contain a lot of $s, and all the backslashes hurt readability.
Related
I am trying to use awk to print the unique lines returned by a command. For simplicity, assume the command is ls -alh.
If I run the following command in my Z shell, awk shows all lines printed by ls -alh
ls -alh | awk '!seen[$0]++'
However, if I run the same command with $SHELL -c while escaping the ! with backslash, I only see the first line of the output printed.
$SHELL -c "ls -alh | awk '\!seen[$0]++'"
How can I ensure the latter command prints the exact same outputs as the former?
EDIT 1:
I initially thought the ! could be the issue. But changing the expression '!seen[$0]++' to 'seen[$0]++==0' has the same problem.
EDIT 2:
It looks like I should have escaped $ too. Since I do not know the reason behind it, I will not post an answer.
In the second form, $0 is being treated as a shell variable in the double-quoted string. The substitution creates an interestingly mangled awk command:
> print $SHELL -c "ls -alh | awk '\!seen[$0]++'"
/bin/zsh -c ls -alh | awk '!seen[-zsh]++'
The variable is not substituted in the first form since it is inside single quotes.
This answer discusses how single- and double-quoted strings are treated in bash and zsh:
Difference between single and double quotes in Bash
Escaping the $ so that $0 is passed to awk should work, but note that quoting in commands that are parsed multiple times can get really tricky.
> print $SHELL -c "ls -alh | awk '\!seen[\$0]++'"
/bin/zsh -c ls -alh | awk '!seen[$0]++'
I'm writing a shell script which shut down some services and trying to get its pid by using the following awk script.
However, this awk script can't get pid. What's wrong with that?
ps -ef | awk -v port_no=10080 '/[m]ilk.*port=port_no/{print $2}'
The result of ps -ef is like this:
username 13155 27705 0 16:06 pts/2 00:00:00 /home/username/.rbenv/versions/2.3.6/bin/ruby /home/username/.rbenv/versions/2.3.6/bin/milk web --no-browser --host=example.com --port=10080
This process is working with a different port argument as well, so I want to kill the process only working on port=10080.
The awk script below works fine, but when I specify the port no using awk -v like the above, it doesn't work well.
ps -ef | awk '/[m]ilk.*port=10080/{print $2}'
awk version: GNU Awk 4.0.2
The syntax for pattern matching with /../ does not work with variables in the regular expression. You need to use the ~ syntax for it.
awk -v port_no=10080 '$0 ~ "[m]ilk.*port="port_no{print $2}'
If you notice the regex carefully, the regex string on the r.h.s of ~ is under the double-quotes ".." except the variable name holding the port number which shouldn't be under quotes, for the expansion to happen.
This task is easily accomplished using pgrep:
$ pgrep -f '[m]ilk.*port=10080'
Have a look at man pgrep for details.
Friends,
I'm trying to extract the last part of following path in a ksh script:
TOOL_HOME=/export/fapps/mytool/mytool-V2-3-4
I want to extract the version # (i.e., 2-3-4) from the above.
awk runs fine on SuSE:
echo $TOOL_HOME | awk -F'mytool-V' '{print $2}'
#2-3-4
However, on Solaris 10, it produces the following:
#ytool
So on Solaris, awk is ignoring everything after the first character in -F'mytool-V'
What should i do to get the same output on both OS's?
On Solaris use /usr/xpg4/bin/awk, not /bin/awk (aka "old, broken awk").
Solaris awk is broken...
$ echo "$TOOL_HOME" | awk '{sub(/.*mytool-V/,"")}1'
2-3-4
or simply with sed
$ echo "$TOOL_HOME" | sed 's/.*mytool-V//'
2-3-4
No need to use awk or any other external program. ksh can do that:
echo ${TOOL_HOME##*mytool-V}
I need to find the ID of some container docker, but some containers have similar names:
$ docker images
REPOSITORY TAG IMAGE ID
app-node latest 620350b79c5a
app-node-temp latest 461c5143a985
If I run:
$ docker images | grep -w app-node-temp | awk -e '{print $3}'
461c5143a985
If I run instead:
$ docker images | grep -w app-node | awk -e '{print $3}'
620350b79c5a
461c5143a985
How can I match the exact name?
I'd say just use awk with exact string matching:
docker images | awk '$1 == "app-node" { print $3 }'
Dashes are considered non-word characters, so grep -w won't work when the difference is marked by a dash.
In context, grep '^app-node[[:space:]]' would work. It looks for the required name followed by a space.
Of course, grep | awk is an anti-pattern most of the time; it would be better to use:
docker images | awk '/^app-node[[:space:]]/ { print $3 }'
Or, an easier solution with awk again uses equality — as suggested by Tom Fenech in his answer:
for server in app-node app-node-temp
do
docker images | awk -v server="$server" '$1 == server { print $3 }'
…
done
If running docker images is too expensive, you can run it once and capture the output in a file and then scan the file. This shows how to pass a shell variable into the awk script.
The chances are the pipeline would be run to capture the container's image ID information:
image_id=$(docker images | awk -v server="$server" '$1 == server { print $3 }')
docker images -q is good for your case
i ran the following command from console it output the correct result:0,
sudo -H -u hadoop bash -c "/home/hadoop/hadoop-install/bin/hadoop dfsadmin -report | grep 'Under replicated blocks' | awk '{print \$4}'"
however if i put it in shell script and assigned it to a variable the 'awk' won't work anymore, it just output the whole result from 'grep':
replications=`sudo -H -u hadoop bash -c "/home/hadoop/hadoop-install/bin/hadoop dfsadmin -report | grep 'Under replicated blocks' | awk '{print \$4}'"`
echo "Replications: $replications"
result: Replications: Under replicated blocks: 0
how can i make the awk work again to only output 4th column which is 0 instead of the whole string?
In backtick command substitution, \ followed by $ means just $. From the POSIX standard:
Within the backquoted style of command substitution, backslash shall retain its literal meaning, except when followed by: '$', '`', or '\' (dollar sign, backquote, backslash). (...)
With the $(command) form, all characters following the open parenthesis to the matching closing parenthesis constitute the command. Any valid shell script can be used for command, except a script consisting solely of redirections which produces unspecified results.
And yet more explicitly from the bash manpage:
When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by $, `, or \. The first backquote not preceded by a backslash terminates the command substitution. When using the $(command) form, all characters between the parentheses make up the command; none are treated specially.
So the easiest way is to use
replications=$(sudo -H -u hadoop bash -c "/home/hadoop/hadoop-install/bin/hadoop dfsadmin -report | grep 'Under replicated blocks' | awk '{print \$4}'")
but
replications=`sudo -H -u hadoop bash -c "/home/hadoop/hadoop-install/bin/hadoop dfsadmin -report | grep 'Under replicated blocks' | awk '{print \\\$4}'"`
also works.