i have an awk file, which i read each words from a file into an array, there is no print command in it, but after i run it, the whole content of the file is printed,
#!/bin/awk -f
{
for(i=1;i<=NF;i++)
used[$i]=1
}
after i run this awk file like this
awk 1.awk 2
the whole content of file 2 is printed on the screen, i am confused,
i tried this directly from command line, there is nothing printed out, so i think there is something wrong with the file or the way to run this file,
You missed the -f option: awk -f 1.awk 2
What you provided is, instead of the contents of "1.awk" as the awk commands, you're providing the literal string 1.awk as the awk command.
You can essentially done this: awk '"1.awk"' 2
And since that is a "true" value, the default action is to print each record of the data contained in file "2".
Related
This question already has answers here:
AWK - how to add the first line of the file?
(2 answers)
How to insert a text at the beginning of a file?
(19 answers)
Closed last month.
I would like to get some explanation here!
I'm new to awk and I'd like to know how do I insert a line at to of a file.
This is what I've tried so far
file.txt
content line
another line
awk command
awk 'BEGIN {print "first line" } {print}' file.txt
the output
first line
content line
another line
however, when run with -i inplace doesn't write to the file only gives me this output
first line
I would like to know what I am doing wrong and if you guys can explain it I'd really appreciate.
The BEGIN{} block is processed before any files are processed which means any output generated by the BEGIN{} block has nowhere to go but to stdout.
To get the line inserted into the file you need to move the print "first line" into the main body of the script where it can be processed along with the file.
One idea based on inserting the new row while reading the first line of the input file:
$ awk -i inplace 'FNR==1 {print "first line"}1' file.txt
$ cat file.txt
first line
content line
another line
NOTES:
FNR==1 will apply to each file if you happen to feed multiple files to awk, otherwise NR==1 will also suffice if it's just the one input file
the stand-alone 1 is non-zero and thus considered by awk as 'true'; here it applies against all input lines and the default behavior for a 'true' is to pass the input line through to stdout (effectively the same as {print})
The -i inplace includes the built-in inplace.awk include file to emulate sed -i in-place editing but there are some caveats that come from the method used for this emulation.
Because it works by fiddling with the processing of each input file (by using the BEGINFILE pattern), anything printed in the BEGIN selector still goes to the start-up output stream rather than to the "fiddled" output stream. That's because the first input file has yet to begin processing at that point.
So what you'll see is first line going to standard output then the two lines from the input file being printed in-place back to that file. You just may not have realised that last bit since you don't change the lines in the file when you write them back.
This was no doubt a difficult implementation decision for the creators of inplace.awk since in-place editing over multiple files needs to be catered for. The question is: where should output go in BEGIN for in-place editing?
You have a couple of options here.
First, if you know you'll only ever process one file, you can use the normal trick, with BEGIN but without inplace:
awk 'BEGIN {print "first line" } {print}' file.txt > /tmp/$$ && mv /tmp/$$ file.txt
Second, using inplace but not BEGIN, you need to first decide which of the input files it should affect. If you want it to affect all input files, that means something like:
awk -i inplace 'FNR==1 {print "first line";print;next} {print}' file1.txt file2.txt
If you want it to affect just the first input file, use NR rather than FNR (the former never decreases, the latter resets to one for each new input file).
Finally, for the case where all files should be affected, you can use the same method that inplace itself uses.
As special patterns like BEGIN are executed in order of definition (and -i comes before processing of your script), simply use BEGINFILE rather than BEGIN, as per the following transcript:
=====
pax#styx:~$ cat xx.001
Original line from xx.001
=====
pax#styx:~$ cat xx.002
Original line from xx.002
=====
pax#styx:~$ cat xx.awk
BEGINFILE {
print "inserted line"
}
{
print
}
=====
pax#styx:~$ awk -i inplace -f xx.awk xx.001 xx.002
=====
pax#styx:~$ cat xx.001
inserted line
Original line from xx.001
=====
pax#styx:~$ cat xx.002
inserted line
Original line from xx.002
The BEGINFILE from inplace will first weave its magic to capture output (per input file), then your BEGINFILE will print to that capture area.
Non-awk version:
I was excited to see the -i inplace option, but it's not available in macOS and BSDs. So in addition to #paxdiablo's solution of a tmp file, here's how you can prepend lines with /bin/ed.
ed -s file.txt << EOF
1i
first line
.
w
EOF
What I would do:
$ awk -i inplace 'NR==1{print "first line";print;next} {print}' file ; cat file
first line
original content line 1
original content line 2
I am using the awk command in terminal on my Mac.
I want to print the contents of an already existing file and give a title to each column which i'll separate using a tab then I want to send the output to another file. What line of code would I use to give titles to the columns? Im hoping to use simple awk commands and preferably if I can complete the task in as little lines as possible.
So far I have tried to use the BEGIN command. (The titles I want to give are first name, second name and score)
BEGIN { print "First Name\tSecond Name\tScore}**
then I want to print the entire contents of the file.
{print} filename.txt
Finally I want to save the output to another file
End{print} filename.txt > output.txt
to do this all all together
awk 'BEGIN {print "First Name\tSecond Name\tScore";}
{print}
End{print}' filename.txt > output.txt
However, this only saved the titles to the output file and not the contents of the original file under the columns.
awk processes files line by line. Before it starts processing the file you can have it do something. We use the BEGIN keyword to note that the following block of code is to be executed before processing. Same with END running after the processing of each line of the file is complete.
While your code has some superfluous bits in it, like the unnecessary END block, it still should do exactly what you are wanting to do, assuming you have data in your filename.txt.
A more succinct awk code would be:
awk 'BEGIN {print "First Name\tSecond Name\tScore";}1' filename.txt > output.txt
In action (using commas instead of tabs because it's easier and I'm lazy):
$ echo "1,2,3" > filename.txt
$ awk 'BEGIN {print "c1,c2,c3"}1' filename.txt > output.txt
$ cat output.txt
c1,c2,c3
1,2,3
I'm having trouble finding out how to read in my file into my awk script.
This is what I have so far. Basically, I want to print out the header, and then read in the roster file which then I will edit to the necessary format. However, my problem is just figuring out how to read in the file.
#!/bin/awk -f
BEGIN {print "Last Name:First Name:Student ID:School – Major:Academic Level:ASURITE:Email" "\n" } {print $1,$2} roster
On running this
awk -f script.awk
Last Name:First Name:Student ID:School – Major:Academic Level:ASURITE:Email
^C
This is what I end up with - the file doesn't read in and I have to CTRL-C my way out since it doesn't close.
The idea is right, but the place where you have mentioned the input file roster is wrong. Move it out of the script. You need to understand that awk syntax is always as below
awk <action> <file>
The <action> part could be directly given in the command line or provided from a script using the -f flag. But the <file> argument still needs to be given no-matter which way. Moving it inside the script, makes awk wait for an input to read its standard input but it doesn't get any.
awk -f script.awk roster
You could modify the script.awk to just use awk without -f and use the /usr/bin/env for the shell to get the location of awk to execute
#!/usr/bin/env awk
BEGIN {
print "Last Name:First Name:Student ID:School – Major:Academic Level:ASURITE:Email" "\n"
}
{
print $1,$2
}
I have a couple of hundreds of files which I want to process with xargs. They all need a fix of their first column.
Therefore I need an awk command to append the prefix "ID_" to the first column of a file (except for the first header line). Can anyone help me with this?
Something along the line:
gawk -f ';' "{$1='ID_' $1; print $0}" file.csv > file_processed.csv
I am not expert for the command, though. And I would rather like to have some inplace processing instead of making a copy of each file. Beforehand, I made it in VIM, but then I only had one file.
:%s/^-/ID_/
I hope someone can help me here.
gawk 'BEGIN{FS=";"; OFS=";"} {if(NR>1) $1="ID_"$1; print}' file.csv > file_processed.csv
FS and OFS set the input and output field separators, respectively.
NR>1 checks whether current line number is larger than 1, so we don't modify the header line.
You can also modify the file in place with -i inplace option:
gawk -i inplace 'BEGIN{FS=";"; OFS=";"} {if(NR>1) $1="ID_"$1; print}' file.csv
Edit
After elaborating the original question, here's the final version:
gawk -i inplace 'BEGIN{FS=OFS=";"} NR>1{sub(/^-/,"ID_",$2)} 1' file.csv
which substitutes - in the beginning of second column with ID_.
NR>1 action applies for all but first (header) line. 1 invokes the default default print action.
If you just want to do something, particularly adding a prefix, on the first field, it is not different from adding the prefix to the whole line.
So you can just awk '$0 = "ID_" $0' file.csv it should do the work. If you want to make it "change in place", you can:
awk '$0="ID_"$0' csv >/tmp/foo && mv /tmp/foo file.csv
You can also make use of sed:
sed -i 's/^/ID_/' file
The -i does "in-place modification"
You mentioned vim, and gave s/^-/ID_/ cmd, it doesn't add the prefix (ID_), it will replace the leading - by the ID_, they are different.
I am very new to using linux and I am trying to find/replace some of the text in my file.
I have successfully been able to find and replace "0/0" using gsub:
awk '{gsub(/0\/0/,"0")}; 1' filename
However, if I try to replace "./." using the same idea
awk '{gsub(/\.\/\./,"U")}; 1' filename
the output is truncated and stops at the location of the first "./." in the file. I know that "." is a special wildcard character, but I thought that having the "\" in front of it would neutralize it. I have searched but have been unable to find an explanation why the formula I used would truncate the file.
Any thoughts would be much appreciated. Thank you.
Recall that the basic outline of an awk is:
awk 'pattern { action }'
The most common patterns are regexes or tests against line counts:
awk '/FOO/ { do_something_with_a_line_with_FOO_in_it }'
awk 'FNR==10'
The last one has no action so the default is to print the line.
But functions that return a value are also useable as patterns. gsub is a function and returns the number of substitutions.
So given:
$ echo "$txt"
abc./.def line 1
ghk/lmn won't get printed
abc./.def abc./.def printed
To print only lines that have a successful substitution you can do:
$ echo "$txt" | awk 'gsub(/\.\/\./,"U")'
abcUdef line 1
abcUdef abcUdef printed
You do not need to put gsub into an action block since you want to run it on every line and the return tells you something about what happened. The lines that successfully are matched are printed since gsub returns the number of substitutions.
If you want every line printed regardless if there is a match:
$ echo "$txt" | awk 'gsub(/\.\/\./,"U") || 1'
abcUdef line 1
ghk/lmn won't get printed
abcUdef abcUdef printed
Or, you can use the function as an action with an empty pattern and then a 1 with an empty action:
$ echo "$txt" | awk '{gsub(/\.\/\./,"U")} 1'
abcUdef line 1
ghk/lmn won't get printed
abcUdef abcUdef printed
In either case, 1 as a pattern with no action prints the line regardless if there is a match and the gsub makes the substitution if any.
The second awk is what you have. Why it is not working on your input data is probably related to you input data.
Your awk script is fine, your input contains control-Ms, probably from being created by a Windows program. You can see them with cat -v file and use dos2unix or similar to remove them.