Make work find pipe awk command in Makefile - awk

I have this find awk line to get python code analyse::
$ find ./ -name '*.py' -exec wc -l {} \; | sort -n| awk '{print $0}{s+=$0}END{print s}'
12 ./gb/__init__.py
23 ./gb/value_type.py
40 ./setup.py
120 ./gb/libcsv.py
200
$
I try to put it in a Makefile::
$ cat Makefile
python_count_lines: clean
#find ./ -name '*.py' -exec wc -l {} \; | sort -n| awk '{print \$0}{s+=\$0}END{print s}'
But this did not work::
$ make python_count_lines
awk: line 1: syntax error at or near }
Makefile:12: recipe for target 'python_count_lines' failed
make: *** [python_count_lines] Error 2
$

Bertrand Martel is correct that you need to escape dollar signs from make by doubling them, not prefixing them with backslashes (see info here).
However, the rest of that suggestion is not right and won't work; first, you should almost never use the shell function in a recipe. Second, using the info function here cannot work because in the first line you've set a shell variable RES equal to some value, then you try to print the make variable RES in the second line; not only that but each line is run in a separate shell, and also all make variable and function references are expanded up-front, before any part of the recipe is passed to the shell.
You just need to do this:
python_count_lines: clean
#find ./ -name '*.py' -exec wc -l {} \; | sort -n| awk '{print $$0}{s+=$$0}END{print s}'

Related

Running an awk command with $SHELL -c returns different results

I am trying to use awk to print the unique lines returned by a command. For simplicity, assume the command is ls -alh.
If I run the following command in my Z shell, awk shows all lines printed by ls -alh
ls -alh | awk '!seen[$0]++'
However, if I run the same command with $SHELL -c while escaping the ! with backslash, I only see the first line of the output printed.
$SHELL -c "ls -alh | awk '\!seen[$0]++'"
How can I ensure the latter command prints the exact same outputs as the former?
EDIT 1:
I initially thought the ! could be the issue. But changing the expression '!seen[$0]++' to 'seen[$0]++==0' has the same problem.
EDIT 2:
It looks like I should have escaped $ too. Since I do not know the reason behind it, I will not post an answer.
In the second form, $0 is being treated as a shell variable in the double-quoted string. The substitution creates an interestingly mangled awk command:
> print $SHELL -c "ls -alh | awk '\!seen[$0]++'"
/bin/zsh -c ls -alh | awk '!seen[-zsh]++'
The variable is not substituted in the first form since it is inside single quotes.
This answer discusses how single- and double-quoted strings are treated in bash and zsh:
Difference between single and double quotes in Bash
Escaping the $ so that $0 is passed to awk should work, but note that quoting in commands that are parsed multiple times can get really tricky.
> print $SHELL -c "ls -alh | awk '\!seen[\$0]++'"
/bin/zsh -c ls -alh | awk '!seen[$0]++'

Make 'awk' exit if it is given an empty file list from a subshell

I run:
find mydir -type f -name "the_thing.txt"
And I get nothing (the file is not there).
Then I run:
awk '{print $0}' $(find mydir -type f -name "the_thing.txt")
And I get the shell stuck in awk (because the input file was not specified, and awk is now waiting for standard input).
How can I make awk (or cat) just print nothing and exit in case find does not output anything?
Your previous post included the -maxdepth 1 option which uniquifies the file path.
That is why I've asked about that. Now the option is removed and I've understood what you mean by some subdirectory.
Then would you please try:
find mydir -type f -name "the_thing.txt" -print0 | xargs -0 -r awk '{print $0}'
Please note that the -r option to xargs suppresses the execution if the input is empty.
If you want to limit the file up to one with head command, you can say:
find mydir -type f -name "the_thing.txt" -print0 | head -n1 -z | xargs -0 -r awk '{print $0}'
The -z option to head was introduced in coreutils 8.25 (around January 2016).
If your head command does not support the option, please say alternatively:
find mydir -type f -name "the_thing.txt" | head -n1 | xargs -r awk '{print $0}'
which is less robust against the filenames which contain blank characters.

How to locate code in PHP inside a directory and edit it

I've been having problems with multiple hidden infected PHP files which are encrypted (ClamAV can't see them) in my server.
I would like to know how can you run an SSH command that can search all the infected files and edit them.
Up until now I have located them by the file contents like this:
find /home/***/public_html/ -exec grep -l '$tnawdjmoxr' {} \;
Note: $tnawdjmoxr is a piece of the code
How do you locate and remove this code inside all PHP files in the directory /public_html/?
You can add xargs and sed:
find /home/***/public_html/ -exec grep -l '$tnawdjmoxr' {} \; | xargs -d '\n' -n 100 sed -i 's|\$tnawdjmoxr||g' --
You may also use sed immediately than using grep -but- it can alter the modification time of that file and may also give some unexpected modifications like perhaps some line endings, etc.
-d '\n' makes it sure that every argument is read line by line. It's helpful if filenames has spaces on it.
-n 100 limits the number of files that sed would process in one instance.
-- makes sed recognize filenames starting with a dash. It's also commendable that grep would have it: grep -l -e '$tnawdjmoxr' -- {} \;
File searching may be faster with grep -F.
sed -i enables inline editing.
Besides using xargs it would also be possible to use Bash:
find /home/***/public_html/ -exec grep -l '$tnawdjmoxr' {} \; | while IFS= read -r FILE; do sed -i 's|\$tnawdjmoxr||g' -- "$FILE"; done
while IFS= read -r FILE; do sed -i 's|\$tnawdjmoxr||g' -- "$FILE"; done < <(exec find /home/***/public_html/ -exec grep -l '$tnawdjmoxr' {} \;)
readarray -t FILES < <(exec find /home/***/public_html/ -exec grep -l '$tnawdjmoxr' {} \;)
sed -i 's|\$tnawdjmoxr||g' -- "${FILES[#]}"

A script to change file names

I am new to awk and shell based programming. I have a bunch of files name file_0001.dat, file_0002.dat......file_1000.dat. I want to change the file names such as the number after file_ will be a multiple of 4 in comparison to previous file name. SO i want to change
file_0001.dat to file_0004.dat
file_0002.dat to file_0008.dat
and so on.
Can anyone suggest a simple script to do it. I have tried the following but without any success.
#!/bin/bash
a=$(echo $1 sed -e 's:file_::g' -e 's:.dat::g')
b=$(echo "${a}*4" | bc)
shuf file_${a}.dat > file_${b}.dat
This script will do that trick for you:
#!/bin/bash
for i in `ls -r *.dat`; do
a=`echo $i | sed 's/file_//g' | sed 's/\.dat//g'`
almost_b=`bc -l <<< "$a*4"`
b=`printf "%04d" $almost_b`
rename "s/$a/$b/g" $i
done
Files before:
file_0001.dat file_0002.dat
Files after first execution:
file_0004.dat file_0008.dat
Files after second execution:
file_0016.dat file_0032.dat
Here's a pure bash way of doing it (without bc, rename or sed).
#!/bin/bash
for i in $(ls -r *.dat); do
prefix="${i%%_*}_"
oldnum="${i//[^0-9]/}"
newnum="$(printf "%04d" $(( 10#$oldnum * 4 )))"
mv "$i" "${prefix}${newnum}.dat"
done
To test it you can do
mkdir tmp && cd $_
touch file_{0001..1000}.dat
(paste code into convert.sh)
chmod +x convert.sh
./convert.sh
Using bash/sed/find:
files=$(find -name 'file_*.dat' | sort -r)
for file in $files; do
n=$(sed 's/[^_]*_0*\([^.]*\).*/\1/' <<< "$file")
let n*=4
nfile=$(printf "file_%04d.dat" "$n")
mv "$file" "$nfile"
done
ls -r1 | awk -F '[_.]' '{printf "%s %s_%04d.%s\n", $0, $1, 4*$2, $3}' | xargs -n2 mv
ls -r1 list file in reverse order to avoid conflict
the second part will generate new filename. For example: file_0002.dat will become file_0002.dat file_0008.dat
xargs -n2 will pass two arguments every time to mv
This might work for you:
paste <(seq -f'mv file_%04g.dat' 1000) <(seq -f'file_%04g.dat' 4 4 4000) |
sort -r |
sh
This can help:
#!/bin/bash
for i in `cat /path/to/requestedfiles |grep -o '[0-9]*'`; do
count=`bc -l <<< "$i*4"`
echo $count
done

Get total number of lines of code?

Does anyone know if it is possible to get the total number of lines of code from all the classes in my project in Objective-C.
Right now I am guessing that this is not possible but I just wanted to make sure.
If it is possible does anyone know how to do it?
If you like the terminal and have all your files in the same folder, try:
$ wc *.m
To get at the number in your code, you could run it as a shell script build phase that generates a header file for you. E.g.
cd source_folder
wc -l *.m \
| tail -1 \
| awk '{ print "#define kNumberOfLines " $1 }' \
> lines_of_code_header.h
Then include that file and use the constant as you like.
find . -type f -name "*.[mh]" -exec wc -l '{}' \; | awk '{sum+=$1} END {print sum}'