awk not printing output to secondary file, syntax error? - awk

Question

More with my radio logs... So far the work I have completed is great, push notifications are going out perfectly, too well in fact. I am getting bombarded with every accidental connection. I want to filter the log to only send notifications if the mic is keyed up longer than 1.5 seconds (the 4th field), cutting the notifications down to real communication only. I am trying to filter the original "completelog.txt" file over to the "filteredlog.txt" file. The original looks like this:
W7MIT,TG 91,4601149,0.5,0,0.00
W7MIT,TG 91,4601149,0.8,0,0.00
VA3GJB,TG 91,4601149,1.2,0,0.00
G0IOR,TG 91,2345614,11.6,0,0.00
G0IOR,TG 91,2345614,10.6,0,0.00
G4NHP,TG 91,2345614,6.2,2,0.00
GM0TKC,TG 91,2345614,0.5,0,0.00
EA8CTB,TG 91,2345614,2.3,0,0.00
EA8CTB,TG 91,2345614,1.6,0,0.00
GM1RCP,TG 91,2345614,0.5,12,0.00
The code I have tried using is here:
#!/bin/bash
awk -v x=1.5 '$4 >= x' completelog.txt >> filteredlog.txt
Then I tried not appending (I think)
#!/bin/bash
awk -v x=1.5 '$4 >= x' completelog.txt > filteredlog.txt
Since neither worked I added sudo (don't think that is needed)
#!/bin/bash
sudo awk -v x=1.5 '$4 >= x' completelog.txt >> filteredlog.txt
And again not appending (still thinking that is what that means).
#!/bin/bash
sudo awk -v x=1.5 '$4 >= x' completelog.txt > filteredlog.txt
Then I just started guessing random things.
awk -v x=1.5 '$4 >= x' completelog.txt
And again...
awk '$4>=1.5{print}' completelog.txt
All to no avail. I did get few permission errors even though the files are chmoded and even tried with sudo. Please tell me what am I doing wrong?

Answer 1

The issue is that you aren't setting the delimiter (Field Separator) in awk so it's expecting whitespace delimitation by default. Instead:
awk -F"," -v x=1.5 '$4 >= x' completelog.txt >> filteredlog.txt
You can also specify this in the BEGIN block of the awk script itself by setting the FS built-in variable:
awk -v x=1.5 'BEGIN{OFS=FS=","}$4>=x' completelog.txt >> filteredlog.txt
That sets the Field Separator FS and the Output Field Separator OFS to a comma.

Answer 2

(not an answer, an extended comment)
The sudo [command ...] > file redirection is kind of misleading: you run the command with sudo privs, but the file redirection is done with your privs. This is because the shell processes redirections before processing the actual command.
Two workarounds:
use tee with sudo privs:
some command with args | sudo tee -a privileged_file
This runs the command as you, but appends to the file as sudo.
wrap everything in a shell with sudo privs:
sudo sh -c 'some command with args >> privileged_file'

Answer 3

Permission error noticed after posting, but the comma delimiter was also a great catch, thank you both.
pi-star#pi-star(rw):~$ awk -F"," '$4>=1.6{print}' completelog.txt >
filteredlog.txt
-bash: filteredlog.txt: Permission denied
pi-star#pi-star(rw):~$ touch filteredlog.txt
touch: cannot touch ‘filteredlog.txt’: Permission denied
pi-star#pi-star(rw):~$ ls -l filteredlog.txt
-rwxr-xr-x 1 root root 0 Jun 10 08:09 filteredlog.txt
pi-star#pi-star(rw):~$ sudo chmod 777 filteredlog.txt
pi-star#pi-star(rw):~$ ls -l filteredlog.txt
-rwxrwxrwx 1 root root 0 Jun 10 08:09 filteredlog.txt
pi-star#pi-star(rw):~$ awk -F"," '$4>=1.6{print}' completelog.txt >
filteredlog.txt
pi-star#pi-star(rw):~$ tail -f -n2 filteredlog.txt
PD2RAL,TG 91,2042416,6.7,1,0.00
N1AJW,TG 91,2042416,2.2,2,6.70