Eliminating NOT IN from query - sql

Question

I'm trying to eliminate the need to use NOT IN in my query:
select count(*)
FROM TABLE1 T1
LEFT OUTER JOIN TABLE2 T2
ON T1.DATAID = T2.EXISTING_DOCUMENT
AND T1.ownerid = -2000
AND T1.SUBTYPE = 144
AND T1.dataid NOT IN (SELECT T3.dataid
FROM TABLE3 T3
WHERE T3.ID = 123)
Reason: I read that NOT IN is slow (+500k rows) and doesn't use indices
I tried:
select count(*)
FROM TABLE1 T1
LEFT OUTER JOIN TABLE2 T2
ON T1.DATAID = T2.EXISTING_DOCUMENT
AND T1.ownerid = -2000
AND T1.SUBTYPE = 144
left outer join TABLE3 T3
on T3.ancestorid = T1.dataid
where T3.ID = 123

Answer 1

NOT IN does use indices, at least in a competent database such as Oracle. However, you can write this using joins if you prefer.
But, why doesn't this do what you want?
select count(*)
FROM TABLE1 T1
WHERE T1.ownerid = -2000 AND T1.SUBTYPE = 144;
You are using a LEFT JOIN, so the only difference is that your version counts duplicates in TABLE2. But that may not really apply.
Your query doesn't really make sense, because you are comparing T1.dataid to T1.dataid. But, further, the comparison to Table3 has no impact on the result. So, you can just remove it:
select count(*)
FROM TABLE1 T1 LEFT OUTER JOIN
TABLE2 T2
ON T1.DATAID = T2.EXISTING_DOCUMENT AND
T1.ownerid = -2000 AND
T1.SUBTYPE = 144 ;
Because of the LEFT JOIN, filtering in the ON clause will not remove any rows. And because it is NOT IN, there is no possibility of duplication.

Answer 2

Use a WHERE x IS NULL filter to emulate a NOT IN.
SQL Fiddle
Oracle 11g R2 Schema Setup:
CREATE TABLE t1 ( ownerid int, subtype int, dataid int, note varchar(100) ) ;
INSERT INTO t1 ( ownerid, subtype, dataid, note )
SELECT 1 as ownerid, 1 as subtype, 1 as dataid, 'WHERE Filter' as note FROM DUAL UNION ALL
SELECT -2000, 1,1, 'IN WHERE Filter' FROM DUAL UNION ALL
SELECT -2000,144,1, 'IN WHERE, NOT IN t3' FROM DUAL UNION ALL
SELECT -2000,144,2, 'IN WHERE, IN t3' FROM DUAL UNION ALL
SELECT -2000,144,3, 'IN WHERE, NOT IN t3' FROM DUAL
;
CREATE TABLE t2 ( existing_document int, note varchar(100) ) ;
INSERT INTO t2 (existing_document, note)
SELECT 1 as existing_document, 'JOIN t1' as note FROM DUAL UNION ALL
SELECT 2, 'JOIN t1' FROM DUAL UNION ALL
SELECT 2, 'JOIN t1, DUPE' FROM DUAL UNION ALL
SELECT 3, 'JOIN t1' FROM DUAL UNION ALL
SELECT 3, 'JOIN t1, DUPE' FROM DUAL UNION ALL
SELECT 4, 'NOT JOIN t1' FROM DUAL
;
CREATE TABLE t3 ( id int, dataid int, note varchar(100) ) ;
INSERT INTO t3 (id, dataid, note)
SELECT 1 as id, 1 as dataid, 'No filter. No match.' as note FROM DUAL UNION ALL
SELECT 1, 4, 'No filter. No match t1.' FROM DUAL UNION ALL
SELECT 123,2,'Match JOIN filter. Match t1' FROM DUAL
;
Read the notes in the setup to view how I'm building up the data. It's very simple and not a lot to count, but it should give you an idea on how this data works together.
Query:
SELECT * /* Not counting here so you can see what's supposed to be counted. */
FROM t1
INNER JOIN t2 ON t1.dataid = t2.EXISTING_DOCUMENT
LEFT OUTER JOIN t3 ON t1.dataid = t3.dataid
AND t3.ID = 123
WHERE t1.ownerid = -2000
AND t1.subtype = 144
AND t3.dataid IS NULL /* This is the NOT IN */
Results:
| OWNERID | SUBTYPE | DATAID | NOTE | EXISTING_DOCUMENT | NOTE | ID | DATAID | NOTE |
|---------|---------|--------|---------------------|-------------------|---------------|--------|--------|--------|
| -2000 | 144 | 1 | IN WHERE, NOT IN t3 | 1 | JOIN t1 | (null) | (null) | (null) |
| -2000 | 144 | 3 | IN WHERE, NOT IN t3 | 3 | JOIN t1 | (null) | (null) | (null) |
| -2000 | 144 | 3 | IN WHERE, NOT IN t3 | 3 | JOIN t1, DUPE | (null) | (null) | (null) |
The optimizer usually runs very well with the WHERE x IS NULL syntax and indexes should still apply, but if Oracle is able to make use of the indexes in the NOT IN, that is a big plus. If you're dealing with a lot of data, the IS NULL method can be a lot faster. The best check is to just test it with your actual data.

Answer 3

how about NOT EXISTS?
select count(*)
FROM TABLE1 T1
LEFT OUTER JOIN TABLE2 T2
ON T1.DATAID = T2.EXISTING_DOCUMENT
AND T1.ownerid = -2000
AND T1.SUBTYPE = 144
AND NOT EXISTS (SELECT 1
FROM TABLE3 T3
WHERE T3.ID = 123
AND T3.dataid = T1.dataid)