Best way to eliminate duplicates rows after multiple joins - sql

I'll consider three simple tables. A, B are my entity tables and C is an intermediate table that creates a many-to-many relationship between A & B.
Schemas:
A: (id INTEGER PRIMARY KEY)
B: (id INTEGER PRIMARY KEY)
C: (
A_id INTEGER,
B_id INTEGER,
FOREIGN KEY(A_id) REFERENCES A(id),
FOREIGN KEY(B_id) REFERENCES B(id)
)
Now, consider the below query
SELECT
A.id
FROM A
LEFT OUTER JOIN C
ON (A.id = C.A_id)
LEFT OUTER JOIN B
ON (C.B_id = B.id)
WHERE ...;
This query would result in duplicate values of A.id, which is expected because C might have multiple rows associated with each row of A. My question is what's the best way to eliminate these duplicates and get the A records. I only need the A records.
I am aware of two ways,
-- Using DISTINCT
SELECT
DISTINCT(A.id), ...
FROM A
LEFT OUTER JOIN C
ON (A.id = C.A_id)
LEFT OUTER JOIN B
ON (C.B_id = B.id)
WHERE ...
ORDER BY A.id;
And
-- Or using A.id IN (above query)/ A.id = Any(above query)
SELECT
...
FROM A
WHERE A.id IN (
SELECT
A.id
FROM A
LEFT OUTER JOIN C
ON (A.id = C.A_id)
LEFT OUTER JOIN B
ON (C.B_id = B.id)
WHERE ...
);
I'm using PostgreSQL. I need to include all the tables for filtering, so not joining a table cannot be considered as an improvement. I've analyzed both the queries but I still feel there might be a better way to do this(in terms of performance).
Any help is really appreciated!

I would suggest exists:
SELECT A.id
FROM A
WHERE EXISTS (SELECT 1
FROM C JOIN
B
ON C.B_id = B.id
WHERE A.id = C.A_id AND . . .
)

You can also try following query:
SELECT
a.* -- or whatever columns you need of a
FROM a
WHERE EXISTS(
SELECT 1
FROM c
WHERE c.a_id = a.id
)
Note, that there is no need to join table b as the existence of the row in c always guarantees for the row in b and you do not need any information contained in this row/table.
Perhaps even more clean might be:
SELECT DISTINCT
a.* -- or whatever columns you need of a
FROM a
LEFT JOIN c
You can have a look at the query plans and execution times using EXPLAIN ANALYZE <query>. Perhaps this gives you a hint on what to use best.
But be aware of caching, repeat both queries multiple times this way to see comparable results.

Related

What is a good way to make multiple full outer join?

I'd like to know if anyone would know an elegant and scalable method to full outer join multiple tables, given that I might want to regularly add new tables to the join?
For now my method consists in full joining table A with table B, store the result as a cte, then full joining the cte to table C, store the result as a cte2, full joining cte2 to table D... you got it.
Creating a new cte every time i want to add another table to the join is not very practical, but every other solutions i found so far have the same issue, there's always some kind of infinite looping either on ctes or in selects (like SELECT blabla FROM (SELECT blabla2 FROM..)).
Is there any way that i don't know that would help me perform this multiple full join without falling in an infinite recursive loop of ctes?
Thanks
EDIT: Sorry it seems it wasn't clear enough
When i perform a multiple full join in one query like:
SELECT
a.*, b.*, c.*
FROM
tableA a
FULL JOIN
tableB b
ON
a.id = b.id
FULL JOIN
tableC c
ON
a.id = c.id
If the id is present in tableB and tableC but not tableA, my result will create two lines where there should be one, because i joined b to a and c to a but not b to c. That's why i need to full join the result of the full join of a and b to c.
So if i have let's say five table instead of three, i need to full join the result of the full join of the result of the full join of the result of the full join... x)
This fiddle illustrates the problem.
If you want the rows from tables B and C to join, you need to accomodate the fact that maybe the data comes from table B and not A. The easiest is probably to use COALESCE.
Your join should therefore look like:
SELECT a.*, b.*, c.*
FROM tableA a
FULL JOIN tableB b ON a.id = b.id
FULL JOIN tableC c ON COALESCE(a.id, b.id) = c.id
-- FULL JOIN tableD d ON COALESCE(a.id, b.id, c.id) = d.id
-- FULL JOIN tableE e ON COALESCE(a.id, b.id, c.id, d.id) = e.id
Most databases that support FULL JOIN also support USING, which is the simplest way to do what you want:
SELECT *
FROM tableA a FULL JOIN
tableB b
USING (id) FULL JOIN
tableC c
USING (id);
The semantics of USING mean that only non-NULL values are used, if such a value is available.

SQL Get rows that doesn't appear in another table

I have this SQL problem: I have tables A and B. Table A has columns id and name, Table B amount and id which is a foreign key to table A.id.
I need to return all table A rows that don't have their id stored in table B. Any ideas?
So the complete opposite is:
SELECT *
FROM a
LEFT OUTER JOIN b ON a.id = b.id;
Here row what I need is left out of result
Just add a where clause:
SELECT a.*
FROM a LEFT OUTER JOIN
b
ON a.id = b.id
WHERE b.id IS NULL;
You can also use NOT EXISTS:
select a.*
from a
where not exists (select 1 from b where b.id = a.id);
In most databases, the two methods typically have similar performance.

Joining in SQL on more than 2 tables Using ORACLE

Suppose I have three tables, A, B and C.
I did Join on table A and table B, now I wanted to do Join on result AB and table c.
Do I need to create view and then do Join or need to do it by Nested query?
You don't say which DB you're using, so the syntax could be wrong, but multi-table joins aren't any different, really, than joining two tables:
SELECT ...
FROM a
JOIN b ON ...
JOIN c ON ...
JOIN d ON ...
No, you will do it as follows
SELECT *
FROM A [INNER/LEFT/RIGHT/OUTER] JOIN
B ON [a/b].IDCols = [a/b].IDCols [INNER/LEFT/RIGHT/OUTER] JOIN
C ON [a/b/c].IDCols = [a/b/c].IDCols
The specific joins (INNER/LEFT/RIGT/OUTER) will depend on what your requirements are.
Have a look at Introduction to JOINs – Basic of JOINs for an overview
The criteria for the JOIN ONs will also depend on how the tables relate to one another.
You can either use something like this:
select *
from A, B, C
where A.id = B.id
and A.id = C.id
or you can use something like this:
select *
from A INNER JOIN B ON (A.id = B.id)
INNER JOIN C ON (A.id = C.id)
How you join will of course depend on how the tables relate to each other, so what's the primary key and foreign key of A,B,C.
You might also use OUTER JOIN instead of INNER JOIN, depending on our data.

Is there alternative way to write this query?

I have tables A, B, C, where A represents items which can have zero or more sub-items stored in C. B table only has 2 foreign keys to connect A and C.
I have this sql query:
select * from A
where not exists (select * from B natural join C where B.id = A.id and C.value > 10);
Which says: "Give me every item from table A where all sub-items have value less than 10.
Is there a way to optimize this? And is there a way to write this not using exists operator?
There are three commonly used ways to test if a value is in one table but not another:
NOT EXISTS
NOT IN
LEFT JOIN ... WHERE ... IS NULL
You have already shown code for the first. Here is the second:
SELECT *
FROM A
WHERE id NOT IN (
SELECT b.id
FROM B
NATURAL JOIN C
WHERE C.value > 10
)
And with a left join:
SELECT *
FROM A
LEFT JOIN (
SELECT b.id
FROM B
NATURAL JOIN C
WHERE C.value > 10
) BC
ON A.id = BC.id
WHERE BC.id IS NULL
Depending on the database type and version, the three different methods can result in different query plans with different performance characteristics.

How do I find records that are not joined?

I have two tables that are joined together.
A has many B
Normally you would do:
select * from a,b where b.a_id = a.id
To get all of the records from a that has a record in b.
How do I get just the records in a that does not have anything in b?
select * from a where id not in (select a_id from b)
Or like some other people on this thread says:
select a.* from a
left outer join b on a.id = b.a_id
where b.a_id is null
select * from a
left outer join b on a.id = b.a_id
where b.a_id is null
The following image will help to understand SQL LET JOIN :
Another approach:
select * from a where not exists (select * from b where b.a_id = a.id)
The "exists" approach is useful if there is some other "where" clause you need to attach to the inner query.
SELECT id FROM a
EXCEPT
SELECT a_id FROM b;
You will probably get a lot better performance (than using 'not in') if you use an outer join:
select * from a left outer join b on a.id = b.a_id where b.a_id is null;
SELECT <columnns>
FROM a WHERE id NOT IN (SELECT a_id FROM b)
In case of one join it is pretty fast, but when we are removing records from database which has about 50 milions records and 4 and more joins due to foreign keys, it takes a few minutes to do it.
Much faster to use WHERE NOT IN condition like this:
select a.* from a
where a.id NOT IN(SELECT DISTINCT a_id FROM b where a_id IS NOT NULL)
//And for more joins
AND a.id NOT IN(SELECT DISTINCT a_id FROM c where a_id IS NOT NULL)
I can also recommended this approach for deleting in case we don't have configured cascade delete.
This query takes only a few seconds.
The first approach is
select a.* from a where a.id not in (select b.ida from b)
the second approach is
select a.*
from a left outer join b on a.id = b.ida
where b.ida is null
The first approach is very expensive. The second approach is better.
With PostgreSql 9.4, I did the "explain query" function and the first query as a cost of cost=0.00..1982043603.32.
Instead the join query as a cost of cost=45946.77..45946.78
For example, I search for all products that are not compatible with no vehicles. I've 100k products and more than 1m compatibilities.
select count(*) from product a left outer join compatible c on a.id=c.idprod where c.idprod is null
The join query spent about 5 seconds, instead the subquery version has never ended after 3 minutes.
Another way of writing it
select a.*
from a
left outer join b
on a.id = b.id
where b.id is null
Ouch, beaten by Nathan :)
This will protect you from nulls in the IN clause, which can cause unexpected behavior.
select * from a where id not in (select [a id] from b where [a id] is not null)