awk :: how to find matching words in two files - awk

Question

Some good folk here on StackOverflow helped me find common lines in two files using awk:
awk 'NR==FNR{a[tolower($0)]; next} tolower($0) in a' 1.txt 2.txt
But how to find common words in two files where words are in line?
For example, let's say that I have 1.txt with these words:
apple
orange
butter
flower
And then 2.txt with these words:
dog cat Butter tower
How to return butter or Butter?
I just want to find the common words.

Answer 1

This grep should do the job:
grep -oiwFf 1.txt 2.txt
Butter
Or else this simple gnu awk would also work:
awk -v RS='[[:space:]]+' 'NR==FNR {w[tolower($1)]; next} tolower($1) in w' 1.txt 2.txt
Butter

Answer 2

Given:
$ cat file1
apple
orange
butter
flower
$ cat file2
dog cat Butter tower
I would write it this way:
awk 'FNR==NR{for(i=1;i<=NF;i++) words[tolower($i)]; next}
{for (i=1;i<=NF;i++) if (tolower($i) in words) print $i}
' file1 file2
Note there is a field by field loop in the case of FNR==NR that handles files that may have more than one word per line. If you know that that is not the case, you can simplify to:
awk 'FNR==NR{words[tolower($1)]; next}
{for (i=1;i<=NF;i++) if (tolower($i) in words) print $i}
' file1 file2
If this is not working on Windows it may be an issue with \r\n line endings. If awk is using a RS=[\n] value then the the \r is left on all words at the end of a line; butter\r does not match butter.
Try:
awk -v RS='[ \r\n\t]' 'FNR==NR{words[tolower($0)]; next}
tolower($0) in words' file1 file2
Comments on your WSL comments in the link:
Your workarounds for Unix files on DOS are many.
Create file1 with DOS line endings this way:
$ printf 'apple\r\norange\r\nbutter\r\nflower\r\n' >file1
Now you can test / see the file has those line endings with cat -v:
$ cat -v file1
apple^M
orange^M
butter^M
flower^M
You can also remove those line endings with sed, perl, awk, etc. Here is a awk removing the \r from the files:
$ cat -v <(awk 1 RS='\r\n' ORS='\n' file1)
apple
orange
butter
flower
A sed and perl:
$ cat -v <(sed 's/\r$//' file1)
#same
or
$ cat -v <(perl -0777 -lpe 's/\r\n/\n/g' file1)
etc. Then use that same construct with awk-on-windows:
awk 'your_awk_program' <(awk 1 RS='\r\n' ORS='\n' file1) <(awk 1 RS='\r\n' ORS='\n' file2)
The downside: While each input is treated as a different logical file, so the FNR==NR awk test still works, the awk special variable FILENAME is lost in the process. If you want to keep FILENAME associated with the actual file, you need to preprocess the files prior to feeding to awk or deal with the \r inside your awk script.

Answer 3

You need to loop over every field per line (of 2.txt) and check:
awk 'NR==FNR{a[tolower($0)];next}{for(i=1;i<=NF;i++){if(tolower($i) in a){print $i}}}' \
1.txt 2.txt
An alternative way to do this in awk would be to add whitespace to the input record separator when processing the 2nd file:
awk 'NR==FNR{a[tolower($0)];next} tolower($0) in a' 1.txt RS="[\n ]" 2.txt