So, I'm seeing this output and I'm a bit surprised:
$ echo "a,b,c,d,e,f,g" | cut -d, -f-4
a,b,c,d
$ echo "a,b,c,d,e,f,g" | cut -d, -f6-
f,g
echo "a,b,c,d,e,f,g" | awk '{ print $0 | "cut -d, -f-4"; print $0 | "cut -d, -f6-"; }'
f,g
a,b,c,d
(As a side note, I realize this is a completely silly thing to do in awk, but it's the only command I've seen it happen for!).
As I understand it, this should pipe the record into the two commands -- in order. But for some reason, the output appears reversed. If I do this instead
$ echo "a,b,c,d,e,f,g" | awk '{ print $0 | "echo hello"; print $0 | "echo goodbye"; }'
hello
goodbye
then everything comes in the order I expected. I'm thinking this must be some sort of race condition, but I'm surprised that awk doesn't wait for the subcommand in the pipe to finish. Is this a known issue of using awk or something pecular to gawk? Is there any way to avoid such a pitfall?
EDIT:
I tried it using mawk too... same (reversed) result, and seems to happen consistently for both.
In order to ensure that an external command is completed, you must close the command.
$ echo "a,b,c,d,e,f,g" | awk 'BEGIN {cmd1 = "cut -d, -f-4"; cmd2 = "cut -d, -f6-"} { print $0 | cmd1; close(cmd1); print $0 | cmd2; close(cmd2)}'
a,b,c,d
f,g
I am surprised by this but it's clear that awk runs commands in parallel. Try this:
# time echo "a,b,c,d,e,f,g" | awk '{ print $0 | "sleep 2"; print $0 | "sleep 2"; }'
real 0m2.250s
user 0m0.030s
sys 0m0.060s
Related
In the below have I 3 grep commands that I would like to replace with awk's grep. so I have tried
! /000000000000/;
! /000000000000/ $0;
! /000000000000/ $3;
where I don't get an error, but testing with both the script below and
$ echo 000000000000 | awk '{ ! /000000000000/; print }'
000000000000
it doesn't skip the lines as expected.
Question
Can anyone explain why my "not grep" doesn't work in awk?
grep -v '^#' $hosts | grep -E '[0-9A-F]{12}\b' | grep -v 000000000000 | awk '{
print "host "$5" {"
print " option host-name \""$5"\";"
gsub(/..\B/,"&:", $3)
print " hardware ethernet "$3";"
print " fixed-address "$1";"
print "}"
print ""
}' > /etc/dhcp/reservations.conf
Could you please try changing your code to:
echo 000000000000 | awk '!/000000000000/'
Problem in your attempt: $ echo 000000000000 | awk '{ ! /000000000000/; print }' Since you are checking condition ! /000000000000/ which is having ; after it so that condition works well and DO NOT print anything. But then you have print after it which is NOT COMING under that condition so it simply prints that line.
awk works on pattern{action} if you are putting semi colon in between it means that condition ends before it and statement after ; is all together a new statements for awk.
EDIT: Adding possible solution by seeing OP's attempt here, not tested at all since no samples are shown by OP. Also I am using --re-interval since my awk version is old you could remove in case you have new version of awk in your box.
awk --re-interval '!/^#/ && !/000000000000/ && /[0-9A-Fa-f]{12}/{
print "host "$5" {"
print " option host-name \""$5"\";"
gsub(/..\B/,"&:", $3)
print " hardware ethernet "$3";"
print " fixed-address "$1";"
print "}"
print ""
}' "$host" > /etc/dhcp/reservations.conf
Taking a look at your code:
$ echo 000000000000 | awk '
{
! /000000000000/ # on given input this evaluates to false
# but since its in action, affects nothing
print # this prints the record regardless of whatever happened above
}'
Adding a print may help you understand:
$ echo 000000000000 | awk '{ print ! /000000000000/; print }'
0
000000000000
Removing the !:
$ echo 000000000000 | awk '{ print /000000000000/; print }'
1
000000000000
This is all I can help you with since there is not enough information for more.
I made a command to dynamically display how many files tar has processed:
tar zcvf some_archive.tar.gz /a/lot/of/files | \
awk 'ORS="\r"{print NR} END{print "\n"}'
In this way, I can see a growing number, as tar outputs a line for each file processed.
However, the cursor is always under the first digit. I want it to be after the last digit, so I have this:
awk 'ORS=""{print "\r"NR} END{print "\n"}'
Sadly, AWK stopped generating any output dynamically.
So how should I do it?
Not sure why, but changing to printf works for me (and then also you don't need to set ORS):
for i in {1..20}; do echo x; sleep 1; done | awk '{printf "\r" NR} END {print ""}'
This may be a more satisfying answer, adding a flush to force the output:
for i in {1..20}; do echo x; sleep 1; done | awk -v ORS="" '{print "\r" NR; fflush()} END {print "\n"}'
How would I go about converting the following bash line into perl? Could I run the system() command, or is there a better way? I'm looking for perl to print out access per day from my apache access_log file.
In bash:
awk '{print $4}' /etc/httpd/logs/access_log | cut -d: -f1 | uniq -c
Prints the following:
632 [27/Apr/2014
156 [28/Apr/2014
awk '{print $4}' /etc/httpd/logs/access_log | cut -d: -f1 | uniq -c
perl -lane'
($val) = split /:/, $F[3]; # First colon-separated elem of the 4th field
++$c{$val}; # Increment number of occurrences of val
END { print for map { "$c{$_} $_" } keys %c } # Print results in no order
' access.log
Switches:
-l automatically appends a newline to the print statement.
-l also removes the newlines from lines read by -n (and -p).
-a splits the line on whitespace into the array #F.
-n loops over the lines of the input but does not print each line.
-e execute the given script body.
Your original command translated to a Perl one-liner:
perl -lane '($k) = $F[3] =~ /^(.*?):/; $h{$k}++ }{ print "$h{$_}\t$_" for keys %h' /etc/httpd/logs/access_log
You can change all your commands to one from:
awk '{print $4}' /etc/httpd/logs/access_log | cut -d: -f1 | uniq -c
to
awk '{split($4,a,":");b[a[1]]++} END {for (i in b) print b[i],i}' /etc/httpd/logs/access_log
I'm trying to do something pretty simple but its appears more complicated than expected...
I've lines in a text file, separated by the comma and that I want to output to another file, without the first field.
Input:
echo file1,item, 12345678 | awk -F',' '{OFS = ";";$1=""; print $0}'
Output:
;item; 12345678
As you can see the spaces before 12345678 are kind of merged into one space only.
I also tried with the cut command:
echo file1,item, 12345678 | cut -d, -f2-
and I ended up with the same result.
Is there any workaround to handle this?
Actually my entire script is as follows:
cat myfile | while read l_line
do
l_line="'$l_line'"
v_OutputFile=$(echo $l_line | awk -F',' '{print $1}')
echo $(echo $l_line | cut -d, -f2-) >> ${v_OutputFile}
done
But stills in l_line all spaces but one are removed. I also created the quotes inside the file but same result.
it has nothing to do with awk. quote the string in your echo:
#with quotes
kent$ echo 'a,b, c'|awk -F, -v OFS=";" '{$1="";print $0}'
;b; c
#without quotes
kent$ echo a,b, c|awk -F, -v OFS=";" '{$1="";print $0}'
;b; c
The problem is with your invocation of the echo command you're using to feed awk the test data above. The shell is looking at this command:
echo file1,item, 12345678
and treating file1,item, and 12345678 as two separate parameters to echo. echo just prints all its parameters, separated by one space.
If you were to quote the whitespace, as follows:
echo 'file1,item, 12345678'
the shell would interpret this as a single parameter to feed to echo, so you'd get the expected result.
Update after edit to OP - having seen your full script, you could do this entirely in awk:
awk -F, '{ OFS = "," ; f = $1 ; sub("^[^,]*,","") ; print $0 >> f }' myfile
I'm a newbie to Bourne shell and want to do simple array simulation. This works:
COLORS='FF0000 0000FF 00FF00'
i=2
color=$(echo ${COLORS} | awk '{print $2}')
echo "color selected: $color"
What I want to do is to pass $i instead of the fixed $2 parameter in print (this will later be used in a loop). I spent hours figuring out the right combination of single and double quotes to do this, no luck.
The closest I got is
color=$("echo ${COLORS} | awk '{print "$"${i}}'")
The run result is:
+ COLORS=FF0000 0000FF 00FF00
+ i=2
+ echo FF0000 0000FF 00FF00 | awk '{print $2}'
./tempgraph.sh: ./tempgraph.sh: 37: echo FF0000 0000FF 00FF00 | awk '{print $2}': not found
+ color=
+ echo color selected:
color selected:
Any help is appreciated.
Don't waste your time trying to get the shell to expand the variable correctly in the awk command, just define a variable using -v:
echo $COLORS | awk -v col=2 '{ print $col }'
In terms of your i variable, this becomes:
i=1
echo $COLORS | awk -v col=$i '{ print $col }'
You can also get at your environment directly:
export COLORS='FF0000 0000FF 00FF00'
awk 'END {split(ENVIRON["COLORS"],colors);for(col in colors) { print "Color",col,"is",colors[col]}}' /dev/null
which gives the following output on this mac:
Color 2 is 0000FF
Color 3 is 00FF00
Color 1 is FF0000
I'd do it like this:
color=$(echo ${COLORS} | awk "{print \$$i}")
If you use '...', the content is not expanded. But you want the value of $i inserted in your script. So "..." is to be used, which does variable expanding. But you also want a $ in front of the number for AWK, so you've got to escape it (\$).
Variables assigned on invokation like -v foo=bar are available in the BEGIN where variable assigned with a simple baz=qux are not.
BEGIN { print foo, bar; }
{ print foo, bar; }
see the difference:
echo Don\'t Panic! | awk -f ./hello.awk -v foo=Hello bar=World
Hello
Hello World