This question already has answers here:
How do I use shell variables in an awk script?
(7 answers)
Closed 2 years ago.
I want to write a script which replaces 'gene' feature from the 3rd column of the $1 file into 'quant'.
#!/bin/bash
awk -F "\t" '{gsub("gene","quant",$3);print}' $1
The code works well, however I would like to read "gene" as an argument, so how can I specify argument $2 instead of 'gene' in the above code?
Thanks!
Use -v awkvar="$value" to create an awk variable with a given value. Thus:
#!/bin/bash
awk -v orig="$2" -F '\t' '{gsub(orig,"quant",$3);print}' "$1"
Related
This question already has an answer here:
awk: function to escape regex operators from a string
(1 answer)
Closed 6 months ago.
Trying to replace | by , using awk
$ awk '{gsub("|",","); print}' sample.txt | tee sample.txt
sample file contains ||| characters and target is to replace with ,,, when fired above command the output is ,|,|,| where it should be ,,,
Try awk '{gsub(/\|/,","); print}' sample.txt | tee output.txt. Note that "|" need to be escaped with "\", and the result is then "tee"ed to another file. Writing back to the same file may not be safe.
This question already has answers here:
How do I use shell variables in an awk script?
(7 answers)
Closed 1 year ago.
I have a csv file that I need to add a number of columns at the end. The new columns are variables taken from other files.
STNO=3389
STNNAME=SWORDS
awk -F "," '{ stnname='"$STNNAME"';stno='"$STNO"';print $0","stnname","stno }' infile
example of the output.
992501062,-6.278983000,202105210736,,3389
The stno is written fine but the stnname is blank. It seems like I can use numeric variables but not text.
any help appreciated.
thanks.
You are interpolating the literal symbol SWORDS where apparently you were hoping to interpolate a quoted string. Awk has no variable named SWORDS so it resolves to an empty string.
Better hygiene and stability altogether is to avoid interpolating junk right into your Awk script. Use -v on the Awk command line to pass in values.
awk -v stnname="$STNNAME" -v stno="$STNO" 'BEGIN {FS=OFS=","}
{ print $0, stnname , stno }' infile
Tangentially, avoid upper case for your private shell variables.
It is very easy to get lost in a sea of quotes. Maybe catch the env variables using -v like this:
awk -v stnname="$STNNAME" -v stno="$STNO" -F "," '{ print $0,stnname,stno }' infile
then you can use them in the command directly without trying to piece together a string
I'm curious about how to set command-line options in awk script, like -F for field separator. I try to write the shebang line like
#!/usr/bin/awk -F ":" -f
and get the following error:
awk: 1: unexpected character '.'
For this example, I can do with
BEGIN {FS=":"}
but I still want to know a way to set all those options. Thanks in advance.
EDIT:
let's use another example that should be easy to test.
inputfile:
1
2
3
4
test.awk:
#!/usr/bin/awk -d -f
{num += $1}
END { print num}
run
/usr/bin/awk -d -f test.awk inputfile
will get 10 and generate a file called awkvars.out with some awk global variables in it.
but
./test.awk inputfile
will get
awk: cmd. line:1: ./test.awk
awk: cmd. line:1: ^ syntax error
awk: cmd. line:1: ./test.awk
awk: cmd. line:1: ^ unterminated regexp
if I remove '-d' from shebang line,
./test.awk inputfile
will normally output 10.
My question is that whether there is a way to write "-d" in test.awk file to generate awkvars.out file?
Answering for the OP question, beyond the setting of FS.
Short Answer: you can not use multiple options with '#!', and since you need to tell awk to read the program from stdin (-f-), you are out of luck.
Long Answer:
When using shebang (#!), there is a limit of single argument (which is passed to the named programs as the 1st argument. So in general:
#! /path/to/prog arg1
input-1
input-2
Will execute /path/to/prog arg1, with the content of the file (including the leading shebang) available as stdin. This is oversimplification, actual rules are more complex., see https://unix.stackexchange.com/questions/87560/does-the-shebang-determine-the-shell-which-runs-the-script
Given this limitation of one argument, when executing awk, the only valid and required parameter is '-f', which indicates that the awk programs is provided on STDIN. You can prepend few other options that do NOT take any argument, for example 'traditional' (e.g., '-Pf-' will force POSIX behavior).
As much as I can tell, all the 'interesting' options (setting FS, RS, ORS, ...) need to be separated from the '-f-' with a space, making it impossible to embed them into the command line, other then using the 'BEGIN { ... }' or similar in the script.
Bottom line, trying #! /usr/bin/awk -f- -F, will attempt to look for program is the same as awk -f' -F', and will look for a file named '- -F`. Usually not very useful, and will not set the FS.
Let's say following is our Input_file, which we are going to use for all mentioned solutions here.
cat Input_file
a,b,c,d
ab,c
1st way of setting Field separator: 1st simple way will be setting FS value in BEGIN section of awk program file. Following is our .awk file.
cat file1.awk
BEGIN{
FS=","
}
{
print $1"..."$2
}
Now when we run the code following output will come:
/usr/local/bin/awk -f file1.awk Input_file
a...b
ab...c
2nd way of setting field separator: 2nd way will be pass FS value before reading Input_file like as follows.
/usr/local/bin/awk -f file.awk FS="," Input_file
Example: Now following is the file.awk file which has awk code.
cat file.awk
{
print $1".."$2
}
Now when we run awk file with awk -f .. command as follows will be result.
/usr/local/bin/awk -f file.awk FS="," Input_file
a..b
ab..c
Which means it is picking up the field separator as , in this above program.
3rd way of setting field separator: We can set field separator in awk -f programs like how we do for usual awk programs using -F',' option as follows.
/usr/local/bin/awk -F',' -f file.awk Input_file
a..b
ab..c
4th way of setting field separator: We could mention field separator as a variable by using -v option on command line while running file.awk script as follows.
/usr/local/bin/awk -v FS=',' -f file.awk Input_file
Never use a shebang to call awk as it robs you of the ability to separate shell arguments into awk arguments and awk variables and do anything else that's better done in shell (e.g. arg parsing with getopts) before calling awk. Just call awk from inside your shell script.
Also, don't name your shell script test.awk as it's a shell script. The fact it's implemented in awk is irrelevant. There's no reason to create a file that you sometimes call as awk file to have awk interpret and other times as just file to have the shell interpret.
This question already has answers here:
How do I use shell variables in an awk script?
(7 answers)
Closed 4 years ago.
I have a following file.
cat test.txt
NE|East
OR|East
WB|East
HP|North
HR|North
JK|North
NR|North
PB|North
I have a variable circle which stores the following value.
circle="JK"
Now, I want the value matching my variable. I have used the following code, but it doesn't provide me any output. However, when I manually writes "JK", it shows me the desired result.
awk -F '|' '{if($1==$circle) print $2;}' test.txt
awk -F '|' '{if($1 == "JK") print $2;}' test.txt
North
Please suggest. Help is much appreciated.
Could you please try following.
val="$JK"
awk -v var="$val" -F'|' '$1==var{print $2}' Input_file
This question already has answers here:
Awk with a variable
(2 answers)
Closed 6 years ago.
Having this line
Doc=$(awk '/1516001/ { print substr($0,15,11) }' /home/data.txt)
want to change the 1616001 with a variable.
for example:
Var='1516001'
Doc=$(awk '/$Var/ { print substr($0,15,11) }' /home/data.txt)
But it doesn't work
#Pedro, in awk a variable's value doesn't work like shell's variable, so we have to assign shell variable's value to an awk's variable and then use it.
Doc=$(awk -vvar="$Var" '{if($0 ~ var){print substr($0,15,11)} }' /home/data.txt)
Let me know if this helps you.
You can use awk -v
Var='1516001'
Doc=$(awk -v pat="$Var" ' $0~pat{ print substr($0,15,11) }' /home/data.txt)