I have a file with some bash functions that I can call with user-defined options like this
transfer -R --src /opstk/ --dst /media/hagbard/hc1/
I also have a number of awk functions in an awk file. Would one be able to to the same with awk scripts, calling the awk file with options to do something rather than another.
awk -f replace.awk -R --src /opstk/ --dst /media/hagbard/hc1/
I understand that it is possible to include command line options for awk commands, in what context would they be useful?
Related
I am running an awk command for every text file in a directory. As of now it displays to stdout. I will like it to save those changes to the actual files themselves. My command is
awk{ORS=(/^\- **\ **/?"":RS)}1 *.txt >> *.txt
Every time I redirect the command it saves everything into one file. Is there anyway I can save the changes back to the files themselves?
-s and blanks aren't regexp metacharacters outside of bracket expressions so no need to escape them in your regexp. You do need to enclose your script in single quote delimiters though. This will do what your script is apparently trying to do:
awk '{ORS=(/^- ** **/?"":RS)}1'
You cannot write to the same file you are reading. If you try to do that with any command (awk, sed, grep, whatever):
command file > file
then the shell can do whatever it likes, including executing > file before command file and so emptying the file before your command opens it.
To overwrite the input file with GNU awk 4.* would be:
awk -i inplace '{ORS=(/^- ** **/?"":RS)}1' *.txt
and with other awks you'd need something like:
for file in *.txt; do
awk '{ORS=(/^- ** **/?"":RS)}1' "$file" > tmp && mv tmp "$file"
done
I have created a tcsh script with a series of gawk commands in the following form:
gawk -f InputFileName > OutputFileName
After the standard call-in (#!/bin/csh -f), I utilized the following command:
set a = $<
In the InputFileName, I proceed to use ${a}.txt, but it does not even use $a when looking up the input file.
They were initially running fine on Cygwin (Windows); now they are running on a Linux and are presenting problems.
It was a text file compatibility issue.
Since I did not have dos2unix on my computer, I used the following one line of code.
awk '{ sub("\r$", ""); print }' winfile.txt > unixfile.txt
I have the following command I want to execute in a Makefile but I'm not sure how.
The command is docker rmi -f $(docker images | grep "<none>" | awk "{print \$3}")
The command executed between $(..) should produce output which is fed to docker rmi but this is not working from within the Makefile I think that's because the $ is used specially in the Makefile but I'm not sure how to modify the command to fit in there.
Any ideas?
$ in Makefiles needs to be doubled to prevent substitution by make:
docker rmi -f $$(docker images | grep "<none>" | awk "{print \$$3}")
Also, it'd be simpler to use use a singly-quoted string in the awk command to prevent expansion of $3 by the shell:
docker rmi -f $$(docker images | grep "<none>" | awk '{print $$3}')
I really recommend the latter. It's usually better to have awk code in single quotes because it tends to contain a lot of $s, and all the backslashes hurt readability.
I am new to awk script. I am trying to figure out how to run an awk file without awk -f command. I see people keep saying add "#!bin/awk -f" for the first line of an awk file. But this didn't for my awk. It still gives me "no file or directory" error.
I question is what does "#!bin/awk -f" really mean, and what does it do?
Its #!/bin/awk -f not #!bin/awk. That will probably work, but theres no guaranty. If someone who has awk installed in a different location runs your script, it won't work. What you want is this: #!/usr/bin/env awk -f.
#! is what tells bash what to use to interpret your script. It should go at the very top of your file. It's called a Shebang. Right after that, you put the path to the interpreter.
/usr/bin/env finds where awk is located, and uses that script as the interpreter. So if they installed awk into somewhere else like /usr/local/bin then it'll find it. This probably won't matter for you, but it's a good habit to get into. It's more portable, and can be shared easier.
The -f says that awk is gonna read from a file. You could do awk -f yourfilename.awk in bash, but in the shebang, -f means the rest of the code will be the file it reads from.
I hope this helped. Feel free to ask me any questions if it doesn't work, or isn't clear enough.
UPDATE
If you get the error message:
/usr/bin/env: ‘awk -f’: No such file or directory
/usr/bin/env: use -[v]S to pass options in shebang lines
then change the first line of your script to #!/usr/bin/env -S awk -f (tested with GNU bash, version 4.4.23)
You probably want
#!/bin/awk -f
(The first slash after the #! is important).
This tells unix what program it should use to 'run' the script with.
It is usually called the 'shebang' which comes from hash + bang.
If you want to run your script like this you need to make sure it is executable (chmod +x <script>).
Otherwise you can just run your script by typing the command /bin/awk -f <script>
The Shebang for Awk Explained
#! is the start of a shebang line, which tells the shell which interpreter to use for the script.
/bin/awk is the path to your awk executable. You may need to change this is your awk is installed elsewhere, or if you want to use a different version of awk.
-f is a flag to awk to tell it to interpret the flag's argument as an awk script. In a shebang, it tells some awks to interpret the remainder of the script instead of a file.
Your Shebang is (Probably) Broken
You are using #!bin/awk -f which is unlikely to work, unless you have awk installed as $PWD/bin/awk. You probably meant to use #!/bin/awk instead.
In some instances, passing a flag on the shebang line may not work with your shell or your awk. If you have the rest of the shebang line correct, you might try removing the -f flag and see if that works for you.
Given this script
#!/bin/awk -f
{
print $1
}
It can be called like so
foo.awk foo.txt
However I would like the script to always call foo.txt. So I would like to modify the script so that it can be called without the input file, like this
foo.awk
#!/bin/awk -f
BEGIN {ARGV[ARGC++] = "foo.txt"}
{print $1}
This will add foo.txt to the end of the arguments list, as if you had put it there on the command line. This has the added bonus of allowing you to extend your script to do more than just print, without having to put everything in the BEGIN block.
I would use a shell wrapper for this:
#!/bin/bash
foo.awk foo.txt
ok, do this trick maybe? (cheat?)
#!/bin/sh
awk '{print $1}' foo.txt
you could name it as foo.awk
and then
chmod +x foo.awk
now try ./foo.awk under same directory.
EDIT
#!/bin/awk -f
BEGIN{while(getline < "/path/to/foo.txt")print $1 }
strange requirement. why bash wrapper doesn't fit your requirement? You did tag the question with shell
anyway, the above script should be what you need.