How can I use sed or awk to delete lines matching certain field criteria? - scripting

Question

I have following kind of data :
1 abc xyz - - 2 mno
2 lnm dse - - 3 pqr
3 ebe aaa xhd asw 4 pow
4 abc fww wrw ffp 3 ffw
I would like to delete the lines which satisfies the following two conditions:
4th & 5th column are blank
The row number of the corresponding line is not contained in the 6th column of any other line
In this case line 1 should be deleted. How could I do it in sed/awk or most suitable scripting language for this case.

Answer 1

May be something like this could work -
awk 'NR==FNR{a[$6];next}
($4 ~ /[- ]/ && $5 ~ /[- ]/) && !($1 in a){next}1' file file
Condition:
If Column 4 and Column 5 are blank AND Index not present in Column 6, we skip that line and we print everything else.
Explanation:
We use NR and FNR built-in variables and pass the same file twice. In the first run, we scan thru the file and store Column 6 in an array. next is used to prevent the second pattern{action} statement from running until the first file is being read. Once, the file is completely read, we test the same file against your condition. If the Column 4 and Column 5 are blank, we look at the index and if it is not in the array then we skip the line using next, else we print it.
Test:
[jaypal:~/Temp] cat file
1 abc xyz - - 2 mno
2 lnm dse - - 3 pqr
3 ebe aaa xhd asw 4 pow
4 abc fww wrw ffp 3 ffw
[jaypal:~/Temp] awk 'NR==FNR{a[$6];next} ($4 ~ /[- ]/ && $5 ~ /[- ]/) && !($1 in a){next}1' file file
2 lnm dse - - 3 pqr
3 ebe aaa xhd asw 4 pow
4 abc fww wrw ffp 3 ffw

Answer 2

A possible solution using perl:
Content of script.pl:
use warnings;
use strict;
## Accept one argument, the input file.
#ARGV == 1 or die qq[Usage: perl $0 input-file\n];
my ($lines, %hash);
## Process file.
while ( <> ) {
## Remove leading and trailing spaces for each line.
s/^\s*//;
s/\s*$//;
## Get both indexes.
my ($idx1, $idx2) = (split)[0,5];
## Save line and index1.
push #{$lines}, [$_, $idx1];
## Save index2.
$hash{ $idx2 } = 1;
}
## Process file for second time.
for ( #{$lines} ) {
## Get fields of the line.
my #f = split /\s+/, $_->[0];
## If fourth and fifth fields are empty (-) and first index exists as second
## index, go to next line without printing.
if ( $f[3] eq qq[-] && $f[4] eq qq[-] && ! exists $hash{ $_->[1] } ) {
next;
}
## Print line.
printf qq[%s\n], $_->[0];
}
Run the script (infile has the data to process):
perl script.pl infile
And results:
2 lnm dse - - 3 pqr
3 ebe aaa xhd asw 4 pow
4 abc fww wrw ffp 3 ffw

Answer 3

This might work for you:
sed -rn 's/^.*(\S+)\s+\S+$/\1/;H;${x;s/^|\n/:/gp}' file |
sed -r '1{h;d};/^(\s*\S*){3}\s*-\s*-/{G;/^\s*(\S*).*:\1:/!d;s/\n.*//}' - file
2 lnm dse - - 3 pqr
3 ebe aaa xhd asw 4 pow
4 abc fww wrw ffp 3 ffw
Explanation:
Read the file and build a look-up table from column 6 delimited by :
Read the table (first line) into the hold space (HS) and then read the file again .
When columns 5 and 6 contain - only.
Append the look-up table to the pattern space (PS)
Do a look up using the first column as the key and if it fails delete that
line.
For all remaining lines remove the look-up table.