I am trying to list out the contents of a field 1 using a function:
help(){
if [[ $# -eq 0 ]] ; then
echo '######################################'
echo ''
echo 'Argument to run run name must be given: ./report.sh Name'
echo 'Report names are:'
ALLNAMES=$(cut -d '|' -f 1 $CONFIGFILE | awk '{printf $0"\n"}')
echo $ALLNAMES
echo '######################################'
exit 0
fi
}
The output I get is :
$ bin/report.sh
######################################
Argument to run run name must be given: ./report.sh Name
Report names are:
ItemA ItemB
######################################
Whereas I want:
$ bin/report.sh
######################################
Argument to run run name must be given: ./report.sh Name
Report names are:
ItemA
ItemB
######################################
If I run the cut command I get:
[david#kallibu]$ cut -d '|' -f 1 conf/report.conf
ItemA
ItemB
Whatdo I need to change to get my newline ?
The problem is:
echo $ALLNAMES
Should be solved with quotes:
echo "$ALLNAMES"
If you're not goint to use the var ALLNAMES in other place, just:
help(){
if [[ $# -eq 0 ]] ; then
echo '######################################'
echo ''
echo 'Argument to run run name must be given: ./report.sh Name'
echo 'Report names are:'
cut -d '|' -f 1 conf/report.conf
echo '######################################'
exit 0
fi
}
Your code would be,
help(){
if [[ $# -eq 0 ]] ; then
echo '######################################'
echo ''
echo 'Argument to run run name must be given: ./report.sh Name'
echo 'Report names are:'
ALLNAMES=$(awk -F'|' '{print $1}' $CONFIGFILE)
echo "$ALLNAMES"
echo '######################################'
exit 0
fi
}
You could try this awk -F'|' '{print $1}' $CONFIGFILE command to get the value of first column where | as delimiter.
You need to put ALLNAMES inside double quotes. So that only, the ALLNAMES variable got expanded.
#Tiago provided the answer to your specific problem, but overall your script should either be the shell script #klashxx posted or this awk script:
help(){
if [[ $# -eq 0 ]] ; then
awk '
BEGIN {
FS = "|"
print "######################################\n"
print "Argument to run run name must be given: ./report.sh Name"
print "Report names are:"
}
{ print $1 }
END {
print "######################################"
}
' "$CONFIGFILE"
exit 0
fi
}
or similar.
Related
Using gawk 5, we are able to launch shell commands with something like:
command = "echo toto"
command | getline
However, this hangs until the command is complete, for example:
BEGIN {
command = "{ echo start >> test.log ; sleep 10 ; echo stop >> test.log ;}"
command | getline
print "terminated"
}
I tried using a & at the end of the command, without success:
BEGIN {
command = "{ echo start >> test.log ; sleep 10 ; echo stop >> test.log ;} &"
command | getline
print "terminated"
}
I would like to be able to launch shell commands as new processes and "forget" about them. For example:
BEGIN {
command1 = "dosomething.sh &"
command1 | getline
command2 = "dosomething2.sh &"
command2 | getline
print "terminated"
}
Can this be achieved using gawk 5 only ?
| getline waits for something to read. So output something so it can read it.
$ awk 'BEGIN { cmd="{ echo something; ( echo start >&2 ; sleep 1; echo stop >&2 ) & }"; cmd | getline; print "Awk end"; }' <<<'' ; echo "Awk end"; sleep 2
Awk end
start
After awk
stop
gawk 'BEGIN { FS="|"; OFS="|" }NR ==1 {print} NR >=2 {cmd1="echo -n "$2" | base64 -w 0";cmd1 | getline d1;close(cmd1); print $1,d1 }' dummy2.txt
input:
id|dummy
1|subhashree:1;user=phn
2|subha:2;user=phn
Expected output:
id|dummy
1|c3ViaGFzaHJlZToxO3VzZXI9cGhuCg==
2|c3ViaGE6Mjt1c2VyPXBobgo=
output produced by script:
id|dummy
1|subhashree:1
2|subha:2
I have understood that the double quote around $2 is causing the issue. It does not work hence not encoding the string properly and just stripping off the string after semi colon.Because it does work inside semicolon and gives proper output in terminal.
echo "subhashree:1;user=phn" | base64
c3ViaGFzaHJlZToxO3VzZXI9cGhuCg==
[root#DERATVIV04 encode]# echo "subha:2;user=phn" | base64
c3ViaGE6Mjt1c2VyPXBobgo=
I have tried with different variation with single and double quote inside awk but it does not work.Any help will be highly appreciated.
Thanks a lot in advance.
Your existing cmd1 producing
echo -n subhashree:1;user=phn | base64 -w 0
^ semicolon is there
So if you execute below would produce
$ echo -n subhashree:1;user=phn | base64 -w 0
subhashree:1
With quotes
$ echo -n 'subhashree:1;user=phn' | base64 -w 0
c3ViaGFzaHJlZToxO3VzZXI9cGhu
Solution is just to use quotes before echo -n '<your-string>' | base64 -w 0
$ cat file
id|dummy
1|subhashree:1;user=phn
2|subha:2;user=phn
$ gawk -v q="'" 'BEGIN { FS="|"; OFS="|" }NR ==1 {print} NR >=2 {cmd1="echo -n " q $2 q" | base64 -w 0"; cmd1 | getline d1;close(cmd1); print $1,d1 }' file
id|dummy
1|c3ViaGFzaHJlZToxO3VzZXI9cGhu
2|c3ViaGE6Mjt1c2VyPXBobg==
It can be simplified as below
gawk -v q="'" 'BEGIN {
FS=OFS="|"
}
NR==1{
print;
next
}
{
cmd1="echo -n " q $2 q" | base64 -w 0";
print ((cmd1 | getline d1)>0)? $1 OFS d1 : $0;
close(cmd1);
}
' file
Based on Ed Morton recommendation http://awk.freeshell.org/AllAboutGetline
if/while ( (getline var < file) > 0)
if/while ( (command | getline var) > 0)
if/while ( (command |& getline var) > 0)
The problem is because of lack of quotes, when trying to run the echo command in shell context. What you are trying to do is basically converted into
echo -n subhashree:1;user=phn | base64 -w 0
which the shell has executed as two commands separated by ; i.e. user=phn | base64 -w 0 means an assignment followed by a pipeline, which would be empty because the assignment would not produce any result over standard input for base64 for encode. The other segment subhashree:1 is just echoed out, which is stored in your getline variable d1.
The right approach fixing your problem should be using quotes
echo -n "subhashree:1;user=phn" | base64 -w 0
When you said, you were using quotes to $2, that is not actually right, the quotes are actually used in the context of awk to concatenate the cmd string i.e. "echo -n ", $2 and " | base64 -w 0" are just joined together. The proposed double quotes need to be in the context of the shell.
SO with that and few other fixes, your awk command should be below. Added gsub() to remove trailing spaces, which were present in your input shown. Also used printf over echo.
awk -v FS="|" '
BEGIN {
OFS = FS
}
NR == 1 {
print
}
NR >= 2 {
gsub(/[[:space:]]+/, "", $2)
cmd = "printf \"%s\" \"" $2 "\" | base64 -w 0"
if ((cmd | getline result) > 0) {
$2 = result
}
close(cmd)
print
}
' file
So with the command above, your command is executed as below, which would produce the right result.
printf "%s" "subhashree:1;user=phn" | base64 -w 0
You already got answers explaining how to use awk for this but you should also consider not using awk for this. The tool to sequence calls to other commands (e.g. bas64) is a shell, not awk. What you're trying to do in terms of calls is:
shell { awk { loop_on_input { shell { base64 } } } }
whereas if you call base64 directly from shell it'd just be:
shell { loop_on_input { base64 } }
Note that the awk command is spawning a new subshell once per line of input while the direct call from shell isn't.
For example:
#!/usr/bin/env bash
file='dummy2.txt'
head -n 1 "$file"
while IFS='|' read -r id dummy; do
printf '%s|%s\n' "$id" "$(base64 -w 0 <<<"$dummy")"
done < <(tail -n +2 "$file")
Here's the difference in execution speed for an input file that has each of your data lines duplicated 100 times created by awk -v n=100 'NR==1{print; next} {for (i=1;i<=n;i++) print}' dummy2.txt > file100
$ ./tst.sh file100
Awk:
real 0m23.247s
user 0m3.755s
sys 0m10.966s
Shell:
real 0m14.512s
user 0m1.530s
sys 0m4.776s
The above timing was produced by running this command (both awk scripts posted in answers will have about the same timeing so I just picked one at random):
#!/usr/bin/env bash
doawk() {
local file="$1"
gawk -v q="'" 'BEGIN {
FS=OFS="|"
}
NR==1{
print;
next
}
{
cmd1="echo -n " q $2 q" | base64 -w 0";
print ((cmd1 | getline d1)>0)? $1 OFS d1 : $0;
close(cmd1);
}
' "$file"
}
doshell() {
local file="$1"
head -n 1 "$file"
while IFS='|' read -r id dummy; do
printf '%s|%s\n' "$id" "$(base64 -w 0 <<<"$dummy")"
done < <(tail -n +2 "$file")
}
# Use 3rd-run timing to eliminate cache-ing as a factor
doawk "$1" >/dev/null
doawk "$1" >/dev/null
echo "Awk:"
time doawk "$1" >/dev/null
echo ""
doshell "$1" >/dev/null
doshell "$1" >/dev/null
echo "Shell:"
time doshell "$1" >/dev/null
I have the following script i am building to send an alert about java heap space.
PID=$(ps -ef | grep [t]omcat | awk '{print $2}')
echo $PID
/usr/java/bin/jstat -gcutil $PID | awk 'FNR == 2 {print $3, $4}' | while read output; do
espace=$(echo $output | awk '{print $1}' | sed 's/.\{3\}$//' )
ospace=$(echo $output | awk '{print $2}' | sed 's/.\{3\}$//' )
echo $espace
echo $ospace
if [ "$opsace" -ge "85" ]; then
echo "Old Gen Heap getting full on $(hostname) on $(date)" >> /tmp/$$
fi
done
if [ -s /tmp/$$ ]
then
sendEmail -f blah#blah.com \
-t $EMAILTO \
-u "Disk Space Alert at $(hostname)" \
-s $SMTP \
-xu $SMTP_USER \
-xp $SMTP_PASS \
-l /tmp/$$.log \
-o tls=yes -q \
< /tmp/$$ &
fi
Here is the output when i run this command.
[root in /i/admin]# sh checkheapspace.sh
4306
80
77
checkheapspace.sh: line 20: [: : integer expression expected
I am not sure why it thinks 77 is not an integer. It stock form it is a decimal, so i use sed to take the last 3 characters off the variables.
Any ideas would be greatly appreciated. Thanks
if [ "$opsace" -ge "85" ]; then
# ^^
...is not...
if [ "$ospace" -ge "85" ]; then
# ^^
Thus, this is running:
if [ '' -ge 85 ]
...and an empty string is indeed not an integer.
I'm having trouble making a script to replace a string that is inside double quotes. The files sections looks like this:
"regA~1" : "FF_NODE~94"
"regA~2" : "FF_NODE~105"
"regA~3" : "FF_NODE~116"
"regA~4" : "FF_NODE~127"
"regA~5" : "FF_NODE~138"
"regA~6" : "FF_NODE~149"
"regA~7" : "FF_NODE~154"
"regA~8" : "FF_NODE~155"
"regA~9" : "FF_NODE~156"
"regA~1" : "FF_NODE~95"
"regA~11" : "FF_NODE~96"
It works if I do
awk '/"regA~1"/{c++;if(c==2){sub("regA~1","regA~10");}}1' file > file_out
but when trying to make this a script where I pass a variable regA~1 and the value for c it doesn't work.
s="regA~1";
r="regA~10";
n=2;
awk -v search="$s" -v replace="$r" -v count=$n '/search/{c++;if(c==count){sub(search,replace);}}1' file > file_out
I also tried
awk -v search=$s -v replace=$r -v count=$n '/search/{c++;if(c==count){sub(search,replace);}}1' file > file_out
The syntax you need to match an RE that's stored as a string in a variable is
$0 ~ var
not
/var/
Thanks to Ed Morton for the tip. Here's the bash script in case anyone needs something like this. Not very sophisticated but it works for me.
#!/bin/bash
# Replaces a specific occurrence of a search string with a replace string
if [ $# -lt 4 ] ; then
echo -e "Wrong number of parameters."
echo -e "Usage:"
echo -e "repnthstr file search replace n"
echo -e "repnthstr fileext search replace n"
exit 1
fi
for file in $1
do
if [ -f $file -a -r $file ]; then
awk -v search=$2 -v replace=$3 -v cnt=$4 '$0 ~ search{c++;if(c==cnt){sub(search,replace);}}1' "$file" > temp && mv temp "$file"
else
echo "Error: Cannot read $file"
fi
done
it seems that I keep getting -gt or == error with the following. Can someone help?
flag= echo $flightSeatBooked | awk -F[,] '{print match($flightSeatBooked, $orderSeats)}'
if $flag == 0; then
echo "Success";
else
echo "fail";
Given:
flightSeatBooked= 9;,A1,A2,A3,A4,B2,E4,C3,B3,D3,D2,E1,E2,C2,B4,C4,D4,C1,D1,E3,B1
orderSeats= B2 (not found in the variable)
Expected output:
Success
Quite a few mistakes. Change it like this:
flag=$(echo $flightSeatBooked | awk -v flseat=$flightSeatBooked -v orseat=$orderSeats '{print match(flseat, orseat)}')
if [ $flag -eq 0 ]; then
echo "Success";
else
echo "fail";
fi
Command substitution has been done using the $(...) notation.
It is not a good practice to use the shell variables directly in awk, and hence passed shell variables to awk using -v.
The syntax of if used was incorrect, updated to correct it.
This is how to do what you ask:
flag=$(awk -v flseat="$flightSeatBooked" -v orseat="$orderSeats" 'BEGIN{print index(flseat, orseat)}')
if [ $flag -eq 0 ]; then
echo "Success"
else
echo "fail"
fi
BUT I don't think what you ask is a good idea. It at least should be something like this:
awk -v flseat="$flightSeatBooked" -v orseat="$orderSeats" 'BEGIN{exit index(flseat, orseat)}')
if [ $? -eq 0 ]; then
echo "Success"
else
echo "fail"
fi
and all you probably really need is something like this:
case "$flightSeatBooked" in
*"$orderSeats"* ) echo "fail";;
* ) echo "Success" ;;
esac
Check the logic (as I haven't!), but hopefully you get the approach.
You can also use this below to check whether $orderSeats is in $flightSeatBooked. If it is in then it return the length of string that matched or 0 is returned.
expr "$flightSeatBooked" : ".*,${orderSeats},"