How to run an .awk file without 'awk -f' command? - awk

I am new to awk script. I am trying to figure out how to run an awk file without awk -f command. I see people keep saying add "#!bin/awk -f" for the first line of an awk file. But this didn't for my awk. It still gives me "no file or directory" error.
I question is what does "#!bin/awk -f" really mean, and what does it do?

Its #!/bin/awk -f not #!bin/awk. That will probably work, but theres no guaranty. If someone who has awk installed in a different location runs your script, it won't work. What you want is this: #!/usr/bin/env awk -f.
#! is what tells bash what to use to interpret your script. It should go at the very top of your file. It's called a Shebang. Right after that, you put the path to the interpreter.
/usr/bin/env finds where awk is located, and uses that script as the interpreter. So if they installed awk into somewhere else like /usr/local/bin then it'll find it. This probably won't matter for you, but it's a good habit to get into. It's more portable, and can be shared easier.
The -f says that awk is gonna read from a file. You could do awk -f yourfilename.awk in bash, but in the shebang, -f means the rest of the code will be the file it reads from.
I hope this helped. Feel free to ask me any questions if it doesn't work, or isn't clear enough.
UPDATE
If you get the error message:
/usr/bin/env: ‘awk -f’: No such file or directory
/usr/bin/env: use -[v]S to pass options in shebang lines
then change the first line of your script to #!/usr/bin/env -S awk -f (tested with GNU bash, version 4.4.23)

You probably want
#!/bin/awk -f
(The first slash after the #! is important).
This tells unix what program it should use to 'run' the script with.
It is usually called the 'shebang' which comes from hash + bang.
If you want to run your script like this you need to make sure it is executable (chmod +x <script>).
Otherwise you can just run your script by typing the command /bin/awk -f <script>

The Shebang for Awk Explained
#! is the start of a shebang line, which tells the shell which interpreter to use for the script.
/bin/awk is the path to your awk executable. You may need to change this is your awk is installed elsewhere, or if you want to use a different version of awk.
-f is a flag to awk to tell it to interpret the flag's argument as an awk script. In a shebang, it tells some awks to interpret the remainder of the script instead of a file.
Your Shebang is (Probably) Broken
You are using #!bin/awk -f which is unlikely to work, unless you have awk installed as $PWD/bin/awk. You probably meant to use #!/bin/awk instead.
In some instances, passing a flag on the shebang line may not work with your shell or your awk. If you have the rest of the shebang line correct, you might try removing the -f flag and see if that works for you.

Related

Introducing command line option arguments in awk

I have a file with some bash functions that I can call with user-defined options like this
transfer -R --src /opstk/ --dst /media/hagbard/hc1/
I also have a number of awk functions in an awk file. Would one be able to to the same with awk scripts, calling the awk file with options to do something rather than another.
awk -f replace.awk -R --src /opstk/ --dst /media/hagbard/hc1/
I understand that it is possible to include command line options for awk commands, in what context would they be useful?

How to modify sed awk command to work with relative path

Context
I had a SO question successfully answered at https://stackoverflow.com/a/59244265/80353
I have successfully used the command that was given.
cap()(cd /tmp;rm -f *.vtt;youtube-dl --skip-download --write-auto-sub "$1";\
sed '1,/^$/d' *.vtt|sed 's/<[^>]*>//g'|awk -F. 'NR%8==1{printf"%s ",$1}NR%8==3'\
|tee -a "$2")
What does this command do?
This command will download captions for a youtube video as a .vtt file from $1 parameter
then print out the simplified version of the .vtt file into another file that's stated as parameter $2
This works as advertised.
How to call the command
In the terminal I will run the above command once and then run cap $youtube_url $full_path_to_output_file
What changes I would like
Currently, the $2 parameter must be a full path. Also currently, if the $2 parameter doesn't exist, an actual file will be created. What I would like is this behavior remains even for relative path. So hopefully for relative path, this behavior of creating a new empty file still works.
Update
I see that comments are such that there's nothing wrong with the command.
However, I did try running
cap $youtube_url $relative_path_to_a_text_file and it definitely did not work for me in macOS
Perhaps I am missing something else?
Update 2
This is a video of me running the awk sed command . First I did it with just a relative path. No output file shows up in the current working directory. The second shows me typing the full path and it works.
https://www.loom.com/share/1c179506fa5b48b4a3d62c81a9d2a411
I hope this clarifies the question i am raising and the commenters would kindly update their comments based on this video.
EDIT: Adding a solution after OP's comment which do checks inside OP's function itself, warning not tested it though.
cap()(
user_path=$(echo "$path_details" | awk 'match($0,/.*\//){print substr($0,RSTART,RLENGTH)}')
path_details="$2"
PWD=`pwd`
cd "$PWD"
user_path=$(echo "$path_details" | awk 'match($0,/.*\//){print substr($0,RSTART,RLENGTH)}')
if [[ -d "$user_path" ]]
then
echo "Present path $user_path."
##Call your program here....##
cd /tmp;rm -f *.vtt;youtube-dl --skip-download --write-auto-sub "$1";\
sed '1,/^$/d' *.vtt|sed 's/<[^>]*>//g'|awk -F. 'NR%8==1{printf"%s ",$1}NR%8==3'\
|tee -a "$2"
else
echo "NOT present path $user_path."
##Can exit from here. if needed.##
fi
)
I believe OP wants to check directory of relative path passed as 2nd argument, is present or not, if this is the case then one could try following.
cat file.ksh
path_details="$2"
PWD=`pwd`
##Why I am going to your path is, in case you are running this from cron, so in that case you can mention complete path here, rather than pwd as mentioned above.
cd "$PWD"
user_path=$(echo "$path_details" | awk 'match($0,/.*\//){print substr($0,RSTART,RLENGTH)}')
if [[ -d "$user_path" ]]
then
echo "Present path $user_path."
##Call your program here....##
else
echo "NOT present path $user_path."
##Can exit from here. if needed.##
fi
Explanation: Adding detailed explanation for above code.
cat file.ksh ##For OP reference to show content I am using cat script_name here.
path_details="$2" ##Creating variable path_details whose value is $2(2nd argument passed to script)
PWD=`pwd` ##Creating variable PWD whose value is pwd(current working directory).
##Why I am going to your path is, in case you are running this from cron, so in that case you can mention complete path here, rather than pwd as mentioned above.
cd "$PWD" ##Going to current directory, why I did is you can set PWD above variable value as per your need and navigate to that path, this will help in case of script is running from Cron.
user_path=$(echo "$path_details" | awk 'match($0,/.*\//){print substr($0,RSTART,RLENGTH)}') ##Now getting path details from passed 2nd argument for script.
if [[ -d "$user_path" ]] ##Checking if user_path(path value is existing on system)
then
echo "Present path $user_path."
##Call your program here....## ##If path existing then call your program.
else ##If path NOT existing then exit from program or print message up to you :)
echo "NOT present path $user_path."
##Can exit from here. if needed.##
fi ##Closing if condition here.

Redirecting files from a directory using awk

I am running an awk command for every text file in a directory. As of now it displays to stdout. I will like it to save those changes to the actual files themselves. My command is
awk{ORS=(/^\- **\ **/?"":RS)}1 *.txt >> *.txt
Every time I redirect the command it saves everything into one file. Is there anyway I can save the changes back to the files themselves?
-s and blanks aren't regexp metacharacters outside of bracket expressions so no need to escape them in your regexp. You do need to enclose your script in single quote delimiters though. This will do what your script is apparently trying to do:
awk '{ORS=(/^- ** **/?"":RS)}1'
You cannot write to the same file you are reading. If you try to do that with any command (awk, sed, grep, whatever):
command file > file
then the shell can do whatever it likes, including executing > file before command file and so emptying the file before your command opens it.
To overwrite the input file with GNU awk 4.* would be:
awk -i inplace '{ORS=(/^- ** **/?"":RS)}1' *.txt
and with other awks you'd need something like:
for file in *.txt; do
awk '{ORS=(/^- ** **/?"":RS)}1' "$file" > tmp && mv tmp "$file"
done

Csh Variable Expansion Malfunction

I have created a tcsh script with a series of gawk commands in the following form:
gawk -f InputFileName > OutputFileName
After the standard call-in (#!/bin/csh -f), I utilized the following command:
set a = $<
In the InputFileName, I proceed to use ${a}.txt, but it does not even use $a when looking up the input file.
They were initially running fine on Cygwin (Windows); now they are running on a Linux and are presenting problems.
It was a text file compatibility issue.
Since I did not have dos2unix on my computer, I used the following one line of code.
awk '{ sub("\r$", ""); print }' winfile.txt > unixfile.txt

How can I cat back exact formatting regardless of shell?

While trying to write a script, I found an interesting issue with cat today. If I do the following at the command line, everything works properly:
var=$(ssh user#server "cat /directory/myfile.sh")
echo $var > ~/newfile.sh
This works and I have a script file with all the proper formatting and can run it. However, if I do the EXACT same thing in a script:
#!/bin/sh
var=$(ssh user#server "cat /directory/myfile.sh")
echo $var > ~/newfile.sh
The file is mangled with carriage returns and weird formatting.
Does anyone know why this is happening? My goal is to ultimately cat a script from a server and run it locally on my machine.
EDIT
I now know that this is happening because of my invoking #!/bin/sh in my shell script. The command line works because I'm using zsh and it is preserving the formatting.
Is there a way to cat back the results regardless of the shell?
As you seem to have figured out, word splitting is off by default on zsh, but on in sh, bash, etc. You can prevent word splitting in all shells by quoting the variable:
echo "$var" > ~/newfile.sh
Note that echo appends a newline to its output by default, which you can suppress (on most echo implementations and builtins) with -n.