Can someone explain what I'm doing wrong and how to do it better.
I have a file consisting of records with field separator "-" and record separator "\t" (tab). I want to put each record on a line, followed by the line number, separated by a tab. The input file is called foo.txt.
$ cat foo.txt
a-b-c e-f-g x-y-z
$ < foo.txt tr -cd "\t" | wc -c
2
$ wc foo.txt
1 3 18 foo.txt
My awk script is in the file foo.awk
BEGIN { RS = "\t" ; FS = "-" ; OFS = "\t" }
{
print $1 "-" $2 "-" $3, NR
}
And here is what I get when I run it:
$ gawk -f foo.awk foo.txt
a-b-c 1
e-f-g 2
x-y-z
3
The last record is directly followed by a newline, a tab, and the last number. What is going on?
well I don't know your exact goal, but since you have built the thing with awk, you can just add \n to FS to reach your goal to remove the trailing \n and without starting another process, like tr, sed or awk
BEGIN { RS = "\t" ; FS = "-|\n" ; OFS = "\t" }
There is an newline character at the end of your data that is also output when printing $3.
In particular, it looks like this:
$1 = "x"
$2 = "y"
$3 = "z\n"
You can remove the trailing separator with tr before passing everything to awk:
tr -d '\n' < foo.txt | awk -f foo.awk
or alternatively add \n to the list of field separators (as shown in the answer by Kent), since awk will strip any separators from the fields.
awk 'BEGIN { RS = "\t"; FS = OFS = "-" } { sub(/\n/, ""); print $0 "\t" NR }' file
Output:
a-b-c 1
e-f-g 2
x-y-z 3
ORS = "\n" was not necessary.
And with GNU Awk or Mawk, you can just have RS = "[\t\n]+":
awk 'BEGIN { RS = "[\t\n]+"; FS = OFS = "-" } { print $0 "\t" NR }' file
Related
I have a file.file content is:
20210126000880000003|3|33.00|20210126|15:30
1|20210126000000000000000000002207|1220210126080109|1000|100000000000000319|100058110000000325|402041000012|402041000012|PT07|621067000000123645|收款方户名|2021-01-26|2021-01-26|10.00|TN|NCS|12|875466
2|20210126000000000000000000002208|1220210126080110|1000|100000000000000319|100058110000000325|402041000012|402041000012|PT06|621067000000123645|收款方户名|2021-01-26|2021-01-26|20.00|TN|NCS|12|875466
3|20210126000000000000000000002209|1220210126080111|1000|100000000000000319|100058110000000325|402041000012|402041000012|PT08|621067000000123645|收款方户名|2021-01-26|2021-01-26|3.00|TN|NCS|12|875466
I use awk command:
awk -F"|" 'NR==1{print $1};FNR==2{print $2,$3}' testfile
Get the following result：
20210126000880000003
20210126000000000000000000002207 1220210126080109
I want the number to auto-increase:
awk -F"|" 'NR==1{print $1+1};FNR==2{print $2+1,$3+1}' testfile
But get follow result:
20210126000880001024
20210126000000000944237587726336 1220210126080110
have question:
I want to the numer is auto-increase: hope the result is：
20210126000880000003
20210126000000000000000000002207|1220210126080109
-------------------------------------------------
20210126000880000004
20210126000000000000000000002208|1220210126080110
--------------------------------------------------
20210126000880000005
20210126000000000000000000002209|1220210126080111
How to auto_increase?
Thanks!
You may try this gnu awk command:
awk -M 'BEGIN {FS=OFS="|"} NR == 1 {hdr = $1; next} NF>2 {print ++hdr; print $2, $3; print "-------------------"}' file
20210126000880000004
20210126000000000000000000002207|1220210126080109
-------------------
20210126000880000005
20210126000000000000000000002208|1220210126080110
-------------------
20210126000880000006
20210126000000000000000000002209|1220210126080111
-------------------
A more readable version:
awk -M 'BEGIN {
FS=OFS="|"
}
NR == 1 {
hdr = $1
next
}
NF > 2 {
print ++hdr
print $2, $3
print "-------------------"
}' file
Here is a POSIX awk solution that doesn't need -M:
awk 'BEGIN {FS=OFS="|"} NR == 1 {hdr = $1; next} NF>2 {"echo " hdr " + 1 | bc" | getline hdr; print hdr; print $2, $3; print "-------------------"}' file
20210126000880000004
20210126000000000000000000002207|1220210126080109
-------------------
20210126000880000005
20210126000000000000000000002208|1220210126080110
-------------------
20210126000880000006
20210126000000000000000000002209|1220210126080111
-------------------
Anubhava has the best solution but for older versions of GNU awk that don't support -M (big numbers) you can try the following:
awk -F\| 'NR==1 { print $1;hed=$1;hed1=substr($1,(length($1)-1));next; } !/^$/ {print $2" "$3 } /^$/ { print "--------------------------------------------------";printf "%s%s\n",substr(hed,1,((length(hed))-(length(hed1)+1))),++hed1 }' testfile
Explanation:
awk -F\| 'NR==1 { # Set field delimiter to | and process the first line
print $1; # Print the first field
hed=$1; # Set the variable hed to the first field
hed1=substr($1,(length($1)-1)); # Set a counter variable hed1 to the last digit in hed ($1)
next;
}
!/^$/ {
print $2" "$3 # Where there is no blank line, print the second field, a space and the third field
}
/^$/ {
print "--------------------------------------------------"; # Where there is a blank field, process
printf "%s%s\n",substr(hed,1,((length(hed))-(length(hed1)+1))),++hed1 # print the header extract before the counter, followed by the incremented counter
}' testfile
I have two files. I need to print information like the example, when the first field exist and is equal, in two files.
file 1
20;"aaaaaa";99292929
24;"fsfdfa";42933294
30;"fsdsff";23832299
38;"fjsdjl";62673777
file 2
13;"fsdffsdfs";2272777
20;"ffuiiii";23728877
30;"wdwfsdh";8882817
40;"sfjslll";82371111
expect result:
file1;20;"aaaaaa";99292929;file2;20;"ffuiiii";23728877
file1,30;"fsdsff";23832299;file2;30;"wdwfsdh";8882817
I tried with:
awk 'FNR==NR{a[$1]=$1;next} $1 in a' file2 file1 > newfile
logical it's ok, but I can't show fields that I want.
awk will help:
awk -F ';' 'NR==FNR{rec[$1]=FILENAME FS $0}
NR>FNR{
if($1 in rec){
print rec[$1] FS FILENAME FS $0
}
}' file{1..2}
should do.
$ cat tst.awk
BEGIN { FS=OFS=";" }
{ $0 = FILENAME FS $0 }
NR==FNR { a[$2] = $0; next }
$2 in a { print a[$2], $0 }
$ awk -f tst.awk file1 file2
file1;20;"aaaaaa";99292929;file2;20;"ffuiiii";23728877
file1;30;"fsdsff";23832299;file2;30;"wdwfsdh";8882817
i am running a following awk script
awk 'BEGIN { FS="|" ; OFS="|" }; { printf $0, $1 "_" $2 }' .someFile
unfortunatley the concatention of fields 1 and 2 is printed on new line, looks like the last field contains a new line character
how can i trim it ?
If you want to use printf (which may have been accidental), I think you can use this:
awk 'BEGIN { FS = OFS = "|" } { printf "%s%s%s_%s", $0, OFS, $1, $2 }' .someFile
printf should always be used with a format string. printf doesn't add the Output Record Separator to the end of what it prints, so you have to do that yourself using \n in the format string or by adding %s and passing ORS as the last argument to printf.
In this case, I think you can just use print though:
awk 'BEGIN { FS = OFS = "|" } { print $0, $1 "_" $2 }' .someFile
My input file has a plain-text representation of the newline character in it separating the fields:
First line\nSecond line\nThird line
I would expect the following to replace that text \n with a newline:
$ awk 'BEGIN { FS = "\\n"; OFS = "\n" } { print $1 }' test.txt
First line\nSecond line\nThird line
But it doesn't (gawk 4.0.1 / OpenBSD nawk 20110810).
I'm allowed to separate on just the \:
$ awk 'BEGIN { FS = "\\"; OFS = "\n" } { print $1, $2 }' test.txt
First line
nSecond line
I can also use a character class in gawk:
$ awk 'BEGIN { FS = "[[:punct:]]n"; OFS = "\n" } { $1 = $1; print $0 }' test.txt
First line
Second line
Third line
But I feel like I should be able to specify the exact separator.
A field separator is a type of regexp and when using a dynamic regexp you need to double escape everything:
$ awk 'BEGIN { FS = "\\\\n"; OFS = "\n" } { print $1 }' file
First line
See the man page for details.
Here sed might be a better tool for this task
sed 's/\\n/\n/g'
In the case where type is "" print the 3rd field out of sequence and then print the whole line with the exception of the 3rd field.
Given a tab separated line a b c d e the idea is to print ab<tab>c<tab>a<tab>b<tab>d<tab>e
Setting $3="" seems to cause the subsequent print statement to lose the tab field separators and so is no good.
# $1 = year $2 = movie
BEGIN {FS = "\t"}
type=="" {printf "%s\t%s\t", $2 $1,$3; $3=""; print}
type!="" {printf "%s\t<%s>\t", $2 $1,type; print}
END {print ""}
Sticking in a for loop which I like a lot less as a solution results in a blank file.
# $1 = year $2 = movie
BEGIN {FS = "\t"}
type=="" {printf "%s\t%s\t%s\t%s\t", $2 $1,$3,$1,$2; for (i=4; i<=NF;i++) printf "%s\t",$i}
type!="" {printf "%s\t<%s>\t", $2 $1,type; print}
END {print ""}
You need to set the OFS to a tab instead of it's default single blank char and you don't want to just set $3 to a bank char as then you'll get 2 tabs between $2 and $4.
$ cat tst.awk
BEGIN {FS = OFS = "\t"}
{
if (type == "") {
val = $3
for (i=3; i<NF; i++) {
$i = $(i+1)
}
NF--
}
else {
val = "<" type ">"
}
print $2 $1, val, $0
}
$
$ awk -f tst.awk file | tr '\t' '-'
ba-c-a-b-d-e
$
$ awk -v type="foo" -f tst.awk file | tr '\t' '-'
ba-<foo>-a-b-c-d-e
The |tr '\t' '-' is obviously just added to make visible where the tabs are.
If decrementing NF doesn't work in your awk to delete the last field in the record, replace it with sub(/\t[^\t]+$/,"").
One way
awk '{$3=""}1' OFS="\t" infile|column -t
explanation
{$3=""} set column to nil
1 same as print, print the line.
OFS="\t"set Output Field Separator Variable to tab, maybe you needn't it, next commandcolumn -t` make the format again.
column -t columnate lists with tabs.