View duplicate based on column value - awk

Question

I have a CSV file with multiple records :
$ cat input.csv
Name;region;latitude;longitude;X;Y;code;N
USE;Cal;78676;8576;FDG;265;V;763
UTYE;Cal;56;8899;FDG;265;V;763
UKV;Cal;46;8576;FDG;265;R;763
UMV;Cal;785754;763;FDG;67;V;763
UBE;Cal;78676;8576;FDG;2651;V;763
UMV;mag;785754;763;FDG;67;V;763
UBE;mag;78676;8576;FDG;265;V;763
I need to view lines that have the same value in the second column and different values in 6th and 7th column.
For example the First two rows should be deleted a duplicate $2;$6;$7
USE;Cal;78676;8576;FDG;265;V;763
UTYE;Cal;56;8899;FDG;265;V;763
Rows 3,4,5,6 and 7 should be kept duplicate value in $2 but different value in $6 and/or $7
UKV;Cal;46;8576;FDG;265;R;763
UMV;Cal;785754;763;FDG;67;V;763
UBE;Cal;78676;8576;FDG;2651;V;763
UMV;mag;785754;763;FDG;67;V;763
UBE;mag;78676;8576;FDG;265;V;763
Expected output :
$ cat output.csv
Name;region;latitude;longitude;X;Y;code;N
UKV;Cal;46;8576;FDG;265;R;763
UMV;Cal;785754;763;FDG;67;V;763
UBE;Cal;78676;8576;FDG;2651;V;763
UMV;mag;785754;763;FDG;67;V;763
UBE;mag;78676;8576;FDG;265;V;763
I tried something like this :
awk -F\; 'NR==1; $NF in a{if (a[$NF]!=0){print a[$NF];a[$NF]=0}print;next}{a[$NF]=$0}' input.csv
It didn't work properly it doubled the records i have and i couldn't filter the result based on $6 and $7

Answer 1

Seems like awk -F';' 'NR==1{print;next} FNR==NR{a[$2,$6,$7]+=1;next;} a[$2,$6,$7]==1' input.csv input.csv should do the trick, e.g.
$ cat input.csv
Name;region;latitude;longitude;X;Y;code;N
USE;Cal;78676;8576;FDG;265;V;763
UTYE;Cal;56;8899;FDG;265;V;763
UKV;Cal;46;8576;FDG;265;R;763
UMV;Cal;785754;763;FDG;67;V;763
UBE;Cal;78676;8576;FDG;2651;V;763
UMV;mag;785754;763;FDG;67;V;763
UBE;mag;78676;8576;FDG;265;V;763
$ awk -F';' 'NR==1{print;next} FNR==NR{a[$2$,6,$7]+=1;next;} a[$2,$6,$7]==1' input.csv input.csv
Name;region;latitude;longitude;X;Y;code;N
UKV;Cal;46;8576;FDG;265;R;763
UMV;Cal;785754;763;FDG;67;V;763
UBE;Cal;78676;8576;FDG;2651;V;763
UMV;mag;785754;763;FDG;67;V;763
UBE;mag;78676;8576;FDG;265;V;763
This works logically by reading the file twice, the first time creating a map of the distinct 2nd/6th/7th field values to a count of their occurrences, then the second only printing those lines with a count of 1. More specifically -
-F';' tells awk the delimiter is a semi (not a comma like you provided, or spaces which is awks default.)
NR==1{print;next;} NR is the current line number for all files being read, so the block of code here will only execute for the first line of the first file. In this case, we print the header and go to the next line.
FNR==NR{a[$2,$6,$7]+=1;next;} FNR is the current files line number, so FNR==NR means this will only execute for the first file, as once the second file begins being read, FNR will reset. For the first file, this maps the count of the distinct 2nd/6th/7th fields. a[$2,$6,$7] is using $2,$6,$7 as a key in the array a and setting or adding 1 to it's value. Following this, we immediately go to the next line.
a[$2,$6,$7]==1 this can implicitly be read as a[$2,$6,$7]==1{print} and it will only be reached for the second files rows, as for all of the first file, we've called next before this could be ran. This looks up the mapped count of $2,$6,$7 and prints the row if it is equal to 1, thus only printing the desired rows.
edit:
Re-reading your question, it could be that you only want the rows with distinct $2/$6/$7 values that also have more than one row with the 2nd field value.
That is, given this input with the 6th adn 8th rows added, which have "unq" and "ant" for their second field, you would not want those rows since there is only one row with that 2nd field value.
$ cat input.csv
Name;region;latitude;longitude;X;Y;code;N
USE;Cal;78676;8576;FDG;265;V;763
UTYE;Cal;56;8899;FDG;265;V;763
UKV;Cal;46;8576;FDG;265;R;763
UKV;Unq;46;8576;FDG;265;R;763
UMV;Cal;785754;763;FDG;67;V;763
UBE;Cal;78676;8576;FDG;2651;V;763
UMV;mag;785754;763;FDG;67;V;763
UMV;Ant;785754;763;FDG;67;V;763
UBE;mag;78676;8576;FDG;265;V;763
If that is the case, just adding a bit to the awk works - adding a second array, b, that we keep count of the 2nd field value in. Then we only print if $2,$6,$7 is unique AND the count of $2 in the file is >1.
$ awk -F';' 'NR==1{print;next} FNR==NR{b[$2]+=1;a[$2,$6,$7]+=1;next;} a[$2,$6,$7]==1&&b[$2]>1' input.csv input.csv
Name;region;latitude;longitude;X;Y;code;N
UKV;Cal;46;8576;FDG;265;R;763
UMV;Cal;785754;763;FDG;67;V;763
UBE;Cal;78676;8576;FDG;2651;V;763
UMV;mag;785754;763;FDG;67;V;763
UBE;mag;78676;8576;FDG;265;V;763