How to join two files using awk? - awk

Question

Have two files: 1.txt and 2.txt
1.txt has items and their order in this form:
item-code|order-value|label
2.txt has items and their properties in this form:
item-code|property-A|property-B| ... |property-Z
For example, 1.txt looks like this:
ITEM-CODE|_o_o_|prefLabel-EN-ANSI
6|8719|disparlure
7|3300|acids,-bases,-and-salts
8|3299|chemical-compounds
2.txt looks like this:
ITEM-CODE|TERM|AV-FTC|DB-PEDIA-IRI|LCSH-1|LCSH-2|LCSH-3|LCSH-4|LCSH-5|LCSH-6|LCSH-7|GACS-IRI
2|positive-sense,-single-stranded-RNA-viruses|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|http://id.agrisemantics.org/gacs/C4028
4|negative-sense,-single-stranded-RNA-viruses|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|http://id.agrisemantics.org/gacs/C3806
6|disparlure|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_
7|acids,-bases,-and-salts|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_
8|chemical-compounds|c_49870|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|http://id.agrisemantics.org/gacs/C29686
sample 3.txt (result - see below) looks liken this:
ITEM-CODE|TERM|AV-FTC|DB-PEDIA-IRI|LCSH-1|LCSH-2|LCSH-3|LCSH-4|LCSH-5|LCSH-6|LCSH-7|GACS-IRI|_o_o_
2|positive-sense,-single-stranded-RNA-viruses|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|http://id.agrisemantics.org/gacs/C4028|NULL
4|negative-sense,-single-stranded-RNA-viruses|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|http://id.agrisemantics.org/gacs/C3806|NULL
6|disparlure|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|_0_|8719
this awk function:
BEGIN { FS=OFS="|" }
NR==FNR{
a[$1]=$2
next
}
{
if ($1 in a)
$(NF+1)=a[$1]
else
$(NF+1)="NULL"
print
}
generates:
item-code|label|property-A|property-B| ... |property-Z|order-value
if no item-code from 1.txt matches item-code in 2.txt, NULL is substituted for the missing order-value
How to modify the awk function to keep 1.txt on the left (the "constant") and 2.txt on the right (the "variables") and generate a result like this:
item-code|order-value|label|property-A|property-B| ... |property-Z
or, if no property-value is available for item-code, then
item-code|order-value|label|NULL
the command looks like this:
C:\gnu\GnuWin32\bin\awk.exe -f a.awk 1.txt 2.txt > 3.txt
where a.awk is the awk function above.
Am running awk on Win10 and using double quotes

Answer 1

Could you please try following.
awk '
BEGIN{
FS=OFS="|"
}
FNR==1 && ++count==1{
val=$2
next
}
FNR==1 && ++count==2{
print $0,val
next
}
FNR==NR{
a[$1]=$2
next
}
{
print $0,a[$1]?a[$1]:"NULL"
}
' 1.txt 2.txt
Explanation: Adding explanation for above code too now.
awk ' ##Starting awk program here.
BEGIN{ ##Starting BEGIN section for awk program here.
FS=OFS="|" ##Setting field separator and output field separator as pipe here.
} ##Closing BEGIN section here.
FNR==1 && ++count==1{ ##Checking condition if FNR==1 and variable count value is 1 means first Input_file header is being read.
val=$2 ##Creating variable val and setting its value as $2 here.
next ##Next will skip all further statements from here onwards.
} ##Closing this condition block.
FNR==1 && ++count==2{ ##Checking condition where FNR==1 and count variable value is 2 here.
print $0,val ##Printing current line with variable val here.
next ##Next will skip all further statements from here.
}
FNR==NR{ ##Checking condition FNR==NR which will be TRUE when 1.txt is being read.
a[$1]=$2 ##Creating an array named a whose index is $1 and value is $2.
next ##next will skip all further statements from here.
}
{
print $0,a[$1]?a[$1]:"NULL" ##Printing current line and printing value of a[$1] if a[$1] is having no value then print NULL.
}
' 1.txt 2.txt ##Mentioning Input_file names here.

Answer 2

You can do that with join.
1.txt
1|48000|first
2|67500|second
3|81990|third
4|55000|fourth
2.txt
1|fred|sara|anthony
3|steve|jane|mike
4|tim
Then run:
join -a 1 -e "NULL" -t '|' -o 1.1,1.2,1.3,2.2,2.3,2.4 1.txt 2.txt
Sample Result
1|48000|first|fred|sara|anthony
2|67500|second|NULL|NULL|NULL
3|81990|third|steve|jane|mike
4|55000|fourth|tim|NULL|NULL