awk to skip pattern and compare lines and output match and mismatch between files - awk

Question

In the below awk I am skipping the lines that start with # and trying to find the matching lines between two files based on 4 specific columns, and if a line does not match it outputs the file and line it is missing from. Thank you :).
file1
##....
#....
chr1 1013466 . T TA 11438.1 PASS
chr1 1013490 . C G 14137 PASS
chr1 1013600 . T TAA 1140 PASS
file2
##...
#....
chr1 1013466 . T TA 10914 PASS
chr1 1013490 . C G 13785.1 PASS
chr1 1014600 . C A 2000 PASS
awk
f1=1234.txt
f2=5678.txt
awk -F'\t' '/^[^#]/ FNR==1 { next }
FNR == NR { $f1[$1,$2,$4,$5] = $1 FS $2 FS $4 FS $5 }
FNR != NR { $f2[$1,$2,$4,$5] = $1 FS $2 FS $4 FS $5 }
END { print "Match:"; for (k in $f1) if (k in $f2) print $f1[k] # Or $f2[k]
print "Not Found:"; for (k in $f2) if (!(k in $f1)) print $f2[k]
}' OFS="\t" $f1 $f2
desired tab-delimited
Match between 1234.txt and 5678.txt:
chr1 1013466 . T TA 11438.1 PASS
chr1 1013490 . C G 14137 PASS
Not Found in 1234.txt:
chr1 1014600 . C A 2000 PASS
Not Found in 5678.txt:
chr1 1013600 . T TAA 1140 PASS

Answer 1

A few issues:
the incorrect prefacing of awk variables with a $; in awk the $ is used to preface column references but not variable references
need to capture the entire input line (not just the 4 index columns)
the missing set of logic for the 3rd set of data
need to capture the input filenames (to be included in the output headers)
One awk idea:
f1=1234.txt
f2=5678.txt
awk '
BEGIN { FS=OFS="\t" }
/^#/ { next }
{ ndx=$1 FS $2 FS $4 FS $5 }
FNR == NR { if (!fname1) fname1=FILENAME; f1[ndx]=$0 }
FNR != NR { if (!fname2) fname2=FILENAME; f2[ndx]=$0 }
END { printf "Match between %s and %s:\n", fname1, fname2
for (ndx in f1)
if (ndx in f2)
print f1[ndx]
printf "Not Found in %s:\n",fname1
for (ndx in f2)
if ( !(ndx in f1) )
print f2[ndx]
printf "Not Found in %s:\n",fname2
for (ndx in f1)
if ( !(ndx in f2) )
print f1[ndx]
}
' $f1 $f2
This generates:
Match between 1234.txt and 5678.txt:
chr1 1013490 . C G 14137 PASS
chr1 1013466 . T TA 11438.1 PASS
Not Found in 1234.txt:
chr1 1014600 . C A 2000 PASS
Not Found in 5678.txt:
chr1 1013600 . T TAA 1140 PASS
If it's necessary to maintain the input ordering of the rows:
awk '
BEGIN { FS=OFS="\t" }
/^#/ { next }
{ ndx=$1 FS $2 FS $4 FS $5 }
FNR == NR { if (!fname1) fname1=FILENAME; indices1[++cnt1]=ndx; f1[ndx]=$0 }
FNR != NR { if (!fname2) fname2=FILENAME; indices2[++cnt2]=ndx; f2[ndx]=$0 }
END { printf "Match between %s and %s:\n", fname1, fname2
for (i=1;i<=cnt1;i++) {
ndx=indices1[i]
if (ndx in f1 && ndx in f2)
print f1[ndx]
}
printf "Not Found in %s:\n",fname1
for (i=1;i<=cnt2;i++) {
ndx=indices2[i]
if ( !(ndx in f1) )
print f2[ndx]
}
printf "Not Found in %s:\n",fname2
for (i=1;i<=cnt1;i++) {
ndx=indices1[i]
if ( !(ndx in f2) )
print f1[ndx]
}
}
' $f1 $f2
This generates:
Match between 1234.txt and 5678.txt:
chr1 1013466 . T TA 11438.1 PASS
chr1 1013490 . C G 14137 PASS
Not Found in 1234.txt:
chr1 1014600 . C A 2000 PASS
Not Found in 5678.txt:
chr1 1013600 . T TAA 1140 PASS

Answer 2

Assuming each key is unique in each input file and you don't are about the order of lines per file in the output:
$ cat tst.awk
BEGIN { FS=OFS="\t" }
/^#/ { next }
{ key = $1 FS $2 FS $4 FS $5 }
NR==FNR {
file1[key] = $0
next
}
{
if ( key in file1 ) {
both[key] = file1[key]
delete file1[key]
}
else {
file2[key] = $0
}
}
END {
printf "Match between %s and %s\n", ARGV[1], ARGV[2]
for ( key in both ) {
print both[key]
}
printf "Not found in %s\n", ARGV[1]
for ( key in file2 ) {
print file2[key]
}
printf "Not found in %s\n", ARGV[2]
for ( key in file1 ) {
print file1[key]
}
}
$ awk -f tst.awk 1234.txt 5678.txt
Match between 1234.txt and 5678.txt
chr1 1013466 . T TA 11438.1 PASS
chr1 1013490 . C G 14137 PASS
Not found in 1234.txt
chr1 1014600 . C A 2000 PASS
Not found in 5678.txt
chr1 1013600 . T TAA 1140 PASS