I have an awk command to extract information from mount points (see the accepted answer in How to extract NFS information from mount on Linux and Solaris?):
awk -F'[: ]' '{if(/^\//)print $3,$4,$1;else print $1,$2,$4}
I would like to include a dig lookup in this awk command to lookup the IP of hostnames. Unfortunately, the mount command sometimes include an IP and sometimes a hostname. I tried the following, but it has an unwanted newline, unwanted return code and does not work if there is an IP address:
For hostnames
echo "example.com:/remote/export on /local/mountpoint otherstuff" | awk -F'[: ]' '{if(/^\//)print system("dig +short " $3),$4,$1;else print system("dig +short " $1),$2,$4}'
Returns
93.184.216.119
0 /remote/export /local/mountpoint
For IPs
echo "93.184.216.119:/remote/export on /local/mountpoint otherstuff" | awk -F'[: ]' '{if(/^\//)print system("dig +short " $3),$4,$1;else print system("dig +short " $1),$2,$4}'
Returns
0 /remote/export /local/mountpoint
I would like to retrieve the following in both cases
93.184.216.119 /remote/export /local/mountpoint
Update:
It seems that some versions of dig return the IP when an IP is provided as query and others return nothing.
Solution:
Based on the accepted answer I used the following adapted awk command:
awk -F'[: ]' '{if(/^\//) { system("dig +short "$3" | grep . || echo "$3" | tr -d \"\n\""); print "",$4,$1 } else { system("dig +short "$1" | grep . || echo "$1" | tr -d \"\n\"");print "",$2,$4 };}'
The additional grep . || echo "$3" takes care that the input IP/hostname is returned if dig returns nothing.
The system command in awk executes a command returns its status. Consider this:
$ awk 'END { print "today is " system("date") " and sunny" }' < /dev/null
Tue Jan 7 20:19:28 CET 2014
today is 0 and sunny
The date command outputs the date and a newline. When running from awk the same thing happens. In this example the system finishes before printf itself, so first we see the line with date, and on the next line our text with the return value 0 of system.
To get what we want we need to split this into multiple commands and we don't need the return value of system:
$ awk 'END { printf "today is "; system("date | tr -d \"\n\""); print " and sunny" }' < /dev/null
today is Tue Jan 7 20:24:01 CET 2014 and sunny
To prevent the newline after date, we piped its output to tr -d "\n".
Long story short, change from this:
print system(...), $2, $4
to this:
system(... | tr -d \"\n\"); print "", $2, $4
Related
I am trying to get only the first and third column of the following output into linux terminal. How can I do this?
my actual output:
akamai-1576314300-xhf78 0/1 Completed 0 5d4h
akamai-1576400700-6m84q 0/1 Completed 0 4d4h
output I need after using awk
akamai-1576314300-xhf78 Completed
akamai-1576400700-6m84q Completed
i am using kubectl get pods | awk '{print $1 print $3}'
but it is not woking...
This is what you are looking for :
kubectl get pods | awk '{ if ($3 == "Completed") { print $1 " " $3 }}'
Hope it helps!
Edit (to create an array of values) :
IFS=$'\n' read -d '' -a myResults <<< "$( kubectl get pods | awk NF | awk '{ if ($3 == "Completed") { print $1 " " $3 }}' )"
And then :
$ echo "${myResults[1]}"
akamai-1576400700-6m84q Completed
$ echo "${myResults[0]}"
akamai-1576314300-xhf78 Completed
I was trying to do masking of file with command 'tr' and 'awk' but failing with error fatal: cannot open pipe ( Too many open pipes) error. FILE has approx 1000000 records quite a huge number.
Below is the code I am trying :-
awk - F "|" - v OFS="|" '{ "echo \""$1"\" | tr \" 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ\" \" QWERTYUIOPASDFGHJKLZXCVBNM9876543210mnbvcxzlkjhgfdsapoiuytrewq\"" | get line $1}1' FILE.CSV > test.CSV
It is showing error :-
awk: (FILENAME=- FNR=1019) fatal: cannot open pipe `echo ""TTP_123"" | tr "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ" "QWERTYUIOPASDFGHJKLZXCVBNM9876543210mnbvcxzlkjhgfdsapoiuytrewq"' (Too many open pipes)
Please let me know what I am doing wrong here
Also a Note any number of columns could be used for masking and can be at any positions in this example I have taken 1 and 2 column positions but it could be 3 and 10 or 5,7,25 columns
Thanks
AJ
First things first, you can't have a space between - and F or v.
I was going to suggest sed, but as you only want to translate the first column, that's not as easy.
Unfortunately, awk doesn't have built-in tr functionality, so you'd have to use the shell like you are and just close the pipe:
awk -F "|" -v OFS="|" '{
command="echo \"\\"$1"\\\" | tr \" 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ\" \" QWERTYUIOPASDFGHJKLZXCVBNM9876543210mnbvcxzlkjhgfdsapoiuytrewq\""
command | getline $1
close(command)
}1' FILE.CSV > test.CSV
However, I suggest using perl, which can do field splitting and character translation:
perl -F'\|' -lane '$F[0] =~ tr/0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ/QWERTYUIOPASDFGHJKLZXCVBNM9876543210mnbvcxzlkjhgfdsapoiuytrewq/; print join("|", #F)' FILE.CSV > test.CSV
Or, for a shorter command line, just put the program into a file, drop the e in -lane and use the file name instead of the '...' command.
you can do the mapping in awk instead of making a system call for each line, or perhaps simply
paste -d'|' <(cut -d'|' -f1 file | tr '0-9' 'a-z') <(cut -d'|' -f2- file)
replace the tr arguments with yours.
This does not answer your question, but you can implement tr as an awk function that would save having to spawn lots of external processes
$ cat tr.awk
function tr(str, from, to, s,i,c,idx) {
s = ""
for (i=1; i<=length($str); i++) {
c = substr(str, i, 1)
idx = index(from, c)
s = s (idx == 0 ? c : substr(to, idx, 1))
}
return s
}
{
print $1, tr($1,
" 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ",
" QWERTYUIOPASDFGHJKLZXCVBNM9876543210mnbvcxzlkjhgfdsapoiuytrewq")
}
Example:
$ printf "%s\n" hello wor-ld | awk -f tr.awk
hello KGCCN
wor-ld 3N8-CF
Good day all,
I am running below command:
netstat -an | awk '/:25/{ print $4 }' | sed 's/:25//' | paste -sd ',' -
which produces
192.168.2.22,127.0.0.1
I would like to amend the result to something like below (to be parsed as a csv by an application)
Manuallyaddedtext 192.168.2.22,127.0.0.1
Many thanks
echo -n "Mytext " ; netstat...
I have these command lines:
grep -e "[0-9] ERROR" /home/aa/lab/utb/cic/nova-all.log | awk '{ print $6 }' | awk -F'-' '{print $3""$2""$1}' | cut -c 1-4,7-8 > part1date.txt
grep -e "[0-9] ERROR" /home/aa/lab/utb/cic/nova-all.log | awk '{ print $3" "$4" "$5" "$9 }' > part1rest.txt
grep -e "[0-9] ERROR" /home/aa/lab/utb/cic/nova-all.log | awk '{ s = ""; for (i = 15; i <= NF; i++) s = s $i " "; print s}' > part1end.txt
paste -d \ part1date.txt part1rest.txt part1end.txt > temp.txt
rm part1*
cat temp.txt
The first 3 lines will save its output in a text file.
Then I merged the columns of these texts in one file to show the output.
Can someone help me to use same command in one line without saving them in textfile?
This command used to change the standard output:
sep 10 11:13:55 node-20 nova-scheduler 2014-10-12 10:36:55.675 3817 ERROR nova.scheduler....
to this format:
ddmmyy hh:mm:ss node-xx PROCESS LOGLEVEL MESSAGE
that means change place of columns and change the format of the date.
awk '/[0-9] ERROR/{gsub("-","",$6);$2=$6;$6=$9;for(i=0;++i<=NF;)$i=i<6?$(i+1):$(i+9);NF-=9;print}' file
I have been trying to parse a Paypal Email and insert the resultant info into a Database of mine. I have most of the code working but I cannot get a Variable to insert into my awk code to create the sql insert query.
if [ -f email-data.txt ]; then {
grep -e "Transaction ID:" -e "Receipt No: " email-data.txt \
>> ../temp
cat ../temp \
| awk 'NR == 1 {printf("%s\t",$NF)} NR == 2 {printf("%s\n",$NF)}' \
>> ../temp1
awk '{print $1}' $email-data.txt \
| grep # \
| grep -v \( \
| grep -v href \
>> ../address
email_addr=$(cat ../address)
echo $email_addr
cat ../temp1 \
| awk '{print "INSERT INTO users (email,paid,paypal_tran,CCReceipt) VALUES"; print "(\x27"($email_addr)"\x27,'1',\x27"$2"\x27,\x27"$3"\x27);"}' \
> /home/linux014/opt/post-new-member.sql
The output looks like the following
INSERT INTO users (email,paid,paypal_tran,CCReceipt) VALUES('9MU013922L4775929 9MU013922L4775929',1,'9MU013922L4775929','');
Should look like
INSERT INTO users (email,paid,paypal_tran,CCReceipt) VALUES('dogcat#gmail.com',1,'9MU013922L4775929','1234-2345-3456-4567');
(Names changed to protect the innocent)
The trial data I am using is set out below
Apr 18, 2014 10:46:17 GMT-04:00 | Transaction ID: 9MU013922L4775929
You received a payment of $50.00 USD from Dog Cat (dogcat#gmail.com)
Buyer:
Dog Cat
dogcat#gmail.com
Purchase Details
Receipt No: 1234-2345-3456-4567
I cannot figure out why the email-addr is not being inserted properly.
You are calling a shell variable inside awk. The right way to do that is by creating an awk variable using -v option.
For example, say $email is your shell variable, then
... | awk -v awkvar="$email" '{do something with awkvar}' ...
Read this for more details.
However, having said that, here is how I would try and parse the text file:
awk '
/Transaction ID:/ { tran = $NF }
/Receipt No:/ { receipt = $NF }
$1 ~ /#/ { email = $1 }
END {
print "INSERT INTO users (email,paid,paypal_tran,CCReceipt) VALUES";
print "("q email q","1","q tran q","q receipt q");"
}' q="'" data.txt
Output:
INSERT INTO users (email,paid,paypal_tran,CCReceipt) VALUES
('dogcat#gmail.com',1,'9MU013922L4775929','1234-2345-3456-4567');