Is that possible to find files reside several level down in awk - awk

May I know is there a way to find certain synctax of words in awk using one liner command whereby the files I'm finding is on deeper hier. Wondering if 'find" cmd can apply here but I tried with find it is not working for me, probably my usage is wrong.
My command as below and it works beautiful if I placed the command on the same level where the *.abc reside.
awk '/hdl_file/{printf "%s", $0}/input|output/{printf "%s", $0}' *.abc | sed s/\.aux// | sed s/,//g | sed s/input// | sed s/output// | sed s/=// | sed s/\"//g | sed s/\.ac// | sed s/hdl_file// | awk '{print $NF $0}' | awk '{$NF="";print $0}'
However, I need to apply this command to looks for *.abc file on 2 hierarchy deeper. Tried command below with find but it gave error.
find . -mindepth 2 | awk '/hdl_file/{printf "%s", $0}/input|output/{printf "%s", $0}' *.abc | sed s/\.aux// | sed s/,//g | sed s/input// | sed s/output// | sed s/=// | sed s/\"//g | sed s/\.ac// | sed s/hdl_file// | awk '{print $NF $0}' | awk '{$NF="";print $0}'
awk: No match.
Pls help. Thanks.

You could also use
awk '....' */*/*.abc
if the files are exactly 2 subdirectories deep. For arbitrary depth, a recent bash can do
shopt -s globstar
awk '....' **/*.abc
Your pipeline of awks and seds can be reduced to
awk '
/hdl_file/ || /input|output/ {
gsub(/[,"]/, "")
sub(/\.aux|\.ac|input|output|hdl_file|=/, "")
last = $NF
$NF = ""
print last, $0
}
' *.abc

I'm not sure if that's your intention, but -mindepth will make find search in directories that are more than 2 levels down. If you want files that are at most 2 levels down, you need to use -maxdepth.
If you want to use the output of one command (in this case find) as arguments for another command (awk), you can use xargs, for example:
find . -maxdepth 2 | xargs awk '[your awk script goes here]'
Or, you could use -exec to read the files, if you don't care about their names:
find . -maxdepth 2 -exec cat '{}' \; | awk '[your awk script goes here]'
# Or possibly:
find . -maxdepth 2 | xargs cat | awk '[your awk script goes here]'
Note also, that you can avoid spawning multiple instances of sed and piping them together, here's two ways to do that:
sed -e 's/foo/bar/' -e 's/something/somethingelse/'
sed -f sed_script # sed_script is a text file with commands for sed

Related

Is using awk at least 'awk -F' always will be fine?

What is the difference on Ubuntu between awk and awk -F? For example to display the frequency of the cpu core 0 we use the command
cat /proc/cpuinfo | grep -i "^ cpu MHz" | awk -F ":" '{print $ 2}' | head -1
But why it uses awk -F? We could put awk without the -F and it would work of course (already tested).
Because without -F , we couldn't find from wath separator i will begin the calculation and print the right result. It's like a way to specify the kind of separator for this awk's using. Without it, it will choose the trivial separator in the line like if i type on the terminal: ps | grep xeyes | awk '{print $1}' ; in this case it will choose the space ' ' as a separator to print the first value: pid OF the process xeyes. I found it in https://www.shellunix.com/awk.html. Thanks for all.

AWK print between two characters

When I try this command:
/usr/bin/curl -s sketch*.zip "https://www.sketch.com/downloads/mac/" |\
grep 'download.sketchapp.com/sketch-' | awk 'NR==1{print $3}'
The output is:
content="0;URL='https://download.sketchapp.com/sketch-68.2-102594.zip
what I am looking to get is:
68.2
Any help would be appreciated.
It seems you want to extract the number after your pattern, only for the first matcing row. You can use one grep command:
... | grep -oPm1 '(?<=download.sketchapp.com/sketch-)[^-]+' file
or as this is the 3rd field of your 1st curl output row you want, you can use one awk command (split field using hyphen as separator to array and print the element in the middle):
awk '/download.sketchapp.com/sketch-/ && NR==1 {split($3,a,"-"); print a[2]; exit}'
Using sed:
/usr/bin/curl -s sketch*.zip "https://www.sketch.com/downloads/mac/" | \
sed -n 's!.*download.sketchapp.com/sketch-\([^-]*\).*!\1!p;' | \
head -1
head is to get rid of multiple matches. sed command extracts non-hyphen characters after download.sketchapp.com/sketch-.

AWK pipe output to SED to replace

I'm trying to replace a string using AWK pipe out to SED
grep pdo_user /html/data/_default_/configs/application.ini | awk '{print $3}' | sed -i 's/$1/"username"/g' /html/data/_default_/configs/application.ini
but found string is not replaced
Output for
grep pdo_user /html/data/_default_/configs/application.ini | awk '{print $3}'
is
"root"
Any tips on that?
I suggest to use awk and mv:
awk '/pdo_user/ && $3=="\"root\"" {$3="\"username\""}1' /path/to/application.ini > /path/to/application.tmp
mv /path/to/application.tmp /path/to/application.ini
Working solution based on Shelter's tip using AWK and SED
sed -i 's/'$(awk '/pdo_user/{print $3}' /path/to/application.ini)'/"username"/' /path/to/application.ini

use awk to print a column, adding a comma

I have a file, from which I want to retrieve the first column, and add a comma between each value.
Example:
AAAA 12345 xccvbn
BBBB 43431 fkodks
CCCC 51234 plafad
to obtain
AAAA,BBBB,CCCC
I decided to use awk, so I did
awk '{ $1=$1","; print $1 }'
Problem is: this add a comma also on the last value, which is not what I want to achieve, and also I get a space between values.
How do I remove the comma on the last element, and how do I remove the space? Spent 20 minutes looking at the manual without luck.
$ awk '{printf "%s%s",sep,$1; sep=","} END{print ""}' file
AAAA,BBBB,CCCC
or if you prefer:
$ awk '{printf "%s%s",(NR>1?",":""),$1} END{print ""}' file
AAAA,BBBB,CCCC
or if you like golf and don't mind it being inefficient for large files:
$ awk '{r=r s $1;s=","} END{print r}' file
AAAA,BBBB,CCCC
awk {'print $1","$2","$3'} file_name
This is the shortest I know
Why make it complicated :) (as long as file is not too large)
awk '{a=NR==1?$1:a","$1} END {print a}' file
AAAA,BBBB,CCCC
For better porability.
awk '{a=(NR>1?a",":"")$1} END {print a}' file
You can do this:
awk 'a++{printf ","}{printf "%s", $1}' file
a++ is interpreted as a condition. In the first row its value is 0, so the comma is not added.
EDIT:
If you want a newline, you have to add END{printf "\n"}. If you have problems reading in the file, you can also try:
cat file | awk 'a++{printf ","}{printf "%s", $1}'
awk 'NR==1{printf "%s",$1;next;}{printf "%s%s",",",$1;}' input.txt
It says: If it is first line only print first field, for the other lines first print , then print first field.
Output:
AAAA,BBBB,CCCC
In this case, as simple cut and paste solution
cut -d" " -f1 file | paste -s -d,
In case somebody as me wants to use awk for cleaning docker images:
docker image ls | grep tag_name | awk '{print $1":"$2}'
Surpised that no one is using OFS (output field separator). Here is probably the simplest solution that sticks with awk and works on Linux and Mac: use "-v OFS=," to output in comma as delimiter:
$ echo '1:2:3:4' | awk -F: -v OFS=, '{print $1, $2, $4, $3}' generates:
1,2,4,3
It works for multiple char too:
$ echo '1:2:3:4' | awk -F: -v OFS=., '{print $1, $2, $4, $3}' outputs:
1.,2.,4.,3
Using Perl
$ cat group_col.txt
AAAA 12345 xccvbn
BBBB 43431 fkodks
CCCC 51234 plafad
$ perl -lane ' push(#x,$F[0]); END { print join(",",#x) } ' group_col.txt
AAAA,BBBB,CCCC
$
This can be very simple like this:
awk -F',' '{print $1","$1","$2","$3}' inputFile
where input file is : 1,2,3
2,3,4 etc.
I used the following, because it lists the api-resource names with it, which is useful, if you want to access it directly. I also use a label "application" to find specific apps in a namespace:
kubectl -n ops-tools get $(kubectl api-resources --no-headers=true --sort-by=name | awk '{printf "%s%s",sep,$1; sep=","}') -l app.kubernetes.io/instance=application

Convert bash line to use in perl

How would I go about converting the following bash line into perl? Could I run the system() command, or is there a better way? I'm looking for perl to print out access per day from my apache access_log file.
In bash:
awk '{print $4}' /etc/httpd/logs/access_log | cut -d: -f1 | uniq -c
Prints the following:
632 [27/Apr/2014
156 [28/Apr/2014
awk '{print $4}' /etc/httpd/logs/access_log | cut -d: -f1 | uniq -c
perl -lane'
($val) = split /:/, $F[3]; # First colon-separated elem of the 4th field
++$c{$val}; # Increment number of occurrences of val
END { print for map { "$c{$_} $_" } keys %c } # Print results in no order
' access.log
Switches:
-l automatically appends a newline to the print statement.
-l also removes the newlines from lines read by -n (and -p).
-a splits the line on whitespace into the array #F.
-n loops over the lines of the input but does not print each line.
-e execute the given script body.
Your original command translated to a Perl one-liner:
perl -lane '($k) = $F[3] =~ /^(.*?):/; $h{$k}++ }{ print "$h{$_}\t$_" for keys %h' /etc/httpd/logs/access_log
You can change all your commands to one from:
awk '{print $4}' /etc/httpd/logs/access_log | cut -d: -f1 | uniq -c
to
awk '{split($4,a,":");b[a[1]]++} END {for (i in b) print b[i],i}' /etc/httpd/logs/access_log