We have records (Table1), who have a workflow (submitted, approved, etc.) that occurs through inserting records in a child table (Table2). For example if the record in Table1 is submitted, a corresponding record in Table2 is inserted that has the workflow status (StatusField), date, and person submitting. When I then try to query Table1 based on its status (e.g. only submitted records), I'm trying to figure out the most efficient way and am unsure. I've tried creating functions where the PK of Table1 is the parameter and then it kicks out the most recent Statusfield from Table2 with a matching FK. I've also tried making a View with the PK of Table2 for the Max PK grouped by FK and then linked via LEFT OUTER JOIN (as there may be no corresponding records in Table2, meaning the status is pending). The way that seems faster is to do a subquery like so:
SELECT a.*
,(SELECT TOP 1 StatusField
FROM Table2 b
WHERE b.FK=a.PK
ORDER BY b.DateField DESC) StatusField
FROM Table1 a
WHERE (SELECT TOP 1 StatusField
FROM Table2 b
WHERE b.FK=a.PK
ORDER BY b.DateField DESC)='Submitted'
I still feel like there is a better way. Any thoughts? If there is a similar question and answer someone can point me to, that would also be awesome. Thank you!
One possible solution (question is tagged MySQL) - use a derived table to obtain the maximum datefield for each fk, then join back to table2 to see if statusfield is submitted....
select a.*
, b.StatusField
from table1 a
join (select fk
, max(datefield) datefield
from table2
group by fk) max on (max.fk = a.fk)
join table2 b on (b.fk = a.fk and b.datefield = max.datefield) and b.StatusField = 'Submitted');
Related
Query
SELECT ID, Name, Phone
FROM Table1
LEFT JOIN Table2 ON Table1.ID = Table2.ID
WHERE Table2.ID IS NULL
Problem
Finding it hard to understand why someone would left join on an ID
and then set it to NULL in the where clause?
Am I missing something here? Is there any significance to this?
Could we just omit the Table2 altogether? As in not join at all?
Any help would be much appreciated.
The query you have in the question is basically equivalent to the following query:
SELECT ID, Name, Phone
FROM Table1
WHERE NOT EXISTS
(
SELECT 1
FROM Table2
WHERE Table1.ID = Table2.ID
)
Meaning it selects all the records in Table1 that does not have a correlated record in Table2.
The execution plan for both queries will most likely be the same (Personally, I've never seen a case when they produce a different execution plan, but I don't rule that out), so both queries should be equally efficient, and it's up to you to decide whether the left join or the exists syntax is more readable to you.
I think you should have an alias for you table and specify which table each column is coming from.
Assuming Name is from table one and Phone is form table two and ID is common in both, then the Left join mentioned above may help get all users that do not have phone numbers.
Table 1
Id Name
1 John Smith
2 Jane Doe
Table 2
Id Phone
2 071 555 0863
Left Join without the where clause
ID Name Phone
1 John Smith NULL
2 Jane Doe 071 555 0863
Left Join with the where clause
ID Name Phone
1 John Smith NULL
This is one of the ways to implement the relational database operation of antijoin, called anti semi join within sql server's terminology. This is essentially "bring rows from one table that are not in another table".
The ways I cant think of doing this are:
select cols from t1 left join t2 on t1.key=t2.key where t2.key is null
select cols from t1 where key not in (select key from t2)
select cols from t1 where not exists (select 1 from t2 where t1.key=t2.key)
and even
select * from t1 where key in (select key from t1 except select key from t2)
There are some differences between these methods (most notably, the danger of null handling in the case of not in), but they generally do the same.
To address your points:
Finding it hard to understand why someone would left join on an ID and
then set it to NULL in the where clause?
As mentioned, in order to exclude results from t1 that are present in t2
Could we just omit the Table2 altogether? As in not join at all?
If you don't use the join (or any of its equivelant alternatives), you will get more results, as the rows in table1 that have the same id with any rows in table2 will be returned, too.
If joining condition column is having null value specifically ID then it is bad database design per my understanding.
As per your query below. Here are the possible scnario why where clause make sense
I am assuming that your name and phone number are coming from table2 and then you are trying to find the name and phone number whose ID is null.
If name and phone number is coming from table1 and table 2 is just having ID join and not selecting anything from table 2 then where clause is total waste.
SELECT
ID,
Name,
Phone
FROM
Table1
LEFT JOIN
Table2
ON
Table1.ID = Table2.ID
WHERE
Table2.ID IS NULL
Essentially in the above common business scenario, developers put where clause filter criteria in left join when any value is coming from right side is having non relevance data and not required to be the part of dataset then filter it out.
table1 (id, name)
table2 (id, name)
Query:
SELECT name
FROM table2
-- that are not in table1 already
SELECT t1.name
FROM table1 t1
LEFT JOIN table2 t2 ON t2.name = t1.name
WHERE t2.name IS NULL
Q: What is happening here?
A: Conceptually, we select all rows from table1 and for each row we attempt to find a row in table2 with the same value for the name column. If there is no such row, we just leave the table2 portion of our result empty for that row. Then we constrain our selection by picking only those rows in the result where the matching row does not exist. Finally, We ignore all fields from our result except for the name column (the one we are sure that exists, from table1).
While it may not be the most performant method possible in all cases, it should work in basically every database engine ever that attempts to implement ANSI 92 SQL
You can either do
SELECT name
FROM table2
WHERE name NOT IN
(SELECT name
FROM table1)
or
SELECT name
FROM table2
WHERE NOT EXISTS
(SELECT *
FROM table1
WHERE table1.name = table2.name)
See this question for 3 techniques to accomplish this
I don't have enough rep points to vote up froadie's answer. But I have to disagree with the comments on Kris's answer. The following answer:
SELECT name
FROM table2
WHERE name NOT IN
(SELECT name
FROM table1)
Is FAR more efficient in practice. I don't know why, but I'm running it against 800k+ records and the difference is tremendous with the advantage given to the 2nd answer posted above. Just my $0.02.
SELECT <column_list>
FROM TABLEA a
LEFTJOIN TABLEB b
ON a.Key = b.Key
WHERE b.Key IS NULL;
https://www.cloudways.com/blog/how-to-join-two-tables-mysql/
This is pure set theory which you can achieve with the minus operation.
select id, name from table1
minus
select id, name from table2
Here's what worked best for me.
SELECT *
FROM #T1
EXCEPT
SELECT a.*
FROM #T1 a
JOIN #T2 b ON a.ID = b.ID
This was more than twice as fast as any other method I tried.
Watch out for pitfalls. If the field Name in Table1 contain Nulls you are in for surprises.
Better is:
SELECT name
FROM table2
WHERE name NOT IN
(SELECT ISNULL(name ,'')
FROM table1)
You can use EXCEPT in mssql or MINUS in oracle, they are identical according to :
http://blog.sqlauthority.com/2008/08/07/sql-server-except-clause-in-sql-server-is-similar-to-minus-clause-in-oracle/
That work sharp for me
SELECT *
FROM [dbo].[table1] t1
LEFT JOIN [dbo].[table2] t2 ON t1.[t1_ID] = t2.[t2_ID]
WHERE t2.[t2_ID] IS NULL
You can use following query structure :
SELECT t1.name FROM table1 t1 JOIN table2 t2 ON t2.fk_id != t1.id;
table1 :
id
name
1
Amit
2
Sagar
table2 :
id
fk_id
email
1
1
amit#ma.com
Output:
name
Sagar
All the above queries are incredibly slow on big tables. A change of strategy is needed. Here there is the code I used for a DB of mine, you can transliterate changing the fields and table names.
This is the strategy: you create two implicit temporary tables and make a union of them.
The first temporary table comes from a selection of all the rows of the first original table the fields of which you wanna control that are NOT present in the second original table.
The second implicit temporary table contains all the rows of the two original tables that have a match on identical values of the column/field you wanna control.
The result of the union is a table that has more than one row with the same control field value in case there is a match for that value on the two original tables (one coming from the first select, the second coming from the second select) and just one row with the control column value in case of the value of the first original table not matching any value of the second original table.
You group and count. When the count is 1 there is not match and, finally, you select just the rows with the count equal to 1.
Seems not elegant, but it is orders of magnitude faster than all the above solutions.
IMPORTANT NOTE: enable the INDEX on the columns to be checked.
SELECT name, source, id
FROM
(
SELECT name, "active_ingredients" as source, active_ingredients.id as id
FROM active_ingredients
UNION ALL
SELECT active_ingredients.name as name, "UNII_database" as source, temp_active_ingredients_aliases.id as id
FROM active_ingredients
INNER JOIN temp_active_ingredients_aliases ON temp_active_ingredients_aliases.alias_name = active_ingredients.name
) tbl
GROUP BY name
HAVING count(*) = 1
ORDER BY name
See query:
SELECT * FROM Table1 WHERE
id NOT IN (SELECT
e.id
FROM
Table1 e
INNER JOIN
Table2 s ON e.id = s.id);
Conceptually would be: Fetching the matching records in subquery and then in main query fetching the records which are not in subquery.
First define alias of table like t1 and t2.
After that get record of second table.
After that match that record using where condition:
SELECT name FROM table2 as t2
WHERE NOT EXISTS (SELECT * FROM table1 as t1 WHERE t1.name = t2.name)
I'm going to repost (since I'm not cool enough yet to comment) in the correct answer....in case anyone else thought it needed better explaining.
SELECT temp_table_1.name
FROM original_table_1 temp_table_1
LEFT JOIN original_table_2 temp_table_2 ON temp_table_2.name = temp_table_1.name
WHERE temp_table_2.name IS NULL
And I've seen syntax in FROM needing commas between table names in mySQL but in sqlLite it seemed to prefer the space.
The bottom line is when you use bad variable names it leaves questions. My variables should make more sense. And someone should explain why we need a comma or no comma.
I tried all solutions above but they did not work in my case. The following query worked for me.
SELECT NAME
FROM table_1
WHERE NAME NOT IN
(SELECT a.NAME
FROM table_1 AS a
LEFT JOIN table_2 AS b
ON a.NAME = b.NAME
WHERE any further condition);
I am trying to count the number of points (stored in table2) that are found in each polygon of table 1. The query works but I have tried to alter it to add the valus generated to a blank column in table 1.
So far it only works by appending the results to the bottom of the table. Any help? To summarise I am trying to add values generated from this query into and add them into table1. At the moment the query inserts them into the blank column in table 1, but no matched against the ID, but appended at the bottom.
INSERT INTO table1(field3)
SELECT COUNT(table2.id) AS count1
FROM table1 LEFT JOIN table2
ON ST_Contains(table1.geom,table2.geom)
GROUP BY table1.id;
The only change I made here was to switch your left join to an inner join. In the case where a geometry in table1 contains no geometries in table2, the value of field3 will stay null, so you might want to start by doing an "update table1 set field3 = 0" first (it can turn out to be a bit faster doing that in two steps depending on how many features you have and how many points each geometry has).
update table1 a
set field3 = b.count1
from
(
SELECT table1.id,
COUNT(table2.id) AS count1
FROM table1
JOIN table2
ON ST_Contains(table1.geom,table2.geom)
GROUP BY table1.id
) b
where a.id = b.id
Alternative:
update table1 a
set field3 = b.count1
from
(
SELECT table1.id,
COUNT(table2.id) AS count1
FROM table1
left JOIN table2
ON ST_Contains(table1.geom,table2.geom)
GROUP BY table1.id
) b
where a.id = b.id
Also, this site just showed up on reddit this morning. I haven't spent much time digging through it but it looks promising as (yet another) resource for learning sql (in a postgres-specific environment).
Edit: I'm starting to doubt myself with regards to the two step approach that I first posted - I think it's almost entirely wrong about the performance, so I included an alternative query.
Here is my query:
Select a.* from Table1 a, Table2 b
Where
a.tid=b.tid and
b.createddate=(Select max(createddate) from Table2) and
a.tid not in (Select distinct tid from Table3);
The problem is I know this should return some valid output but it does not. The issue us with the last line in the a.tid not in (Select distinct tid from Table3); if I replace Select distinct tid from Table3 with hard coded values like ('T001','T002','T003','T004') then it works fine and returns data.
Whats wrong? Am I missing something? Please help.
Try this:
Select a.* from Table1 a, Table2 b
Where
a.tid=b.tid and
b.createddate=(Select max(createddate) from Table2) and
a.tid not in (Select tid from Table3 where tid is not null);
As all the people in the comments mentioned, if there is at least one row with a null value for tid in table3 you will get no rows returned. This is because to Oracle null is like saying "I don't know what this value is". Oracle can't say with certainty that the value you are searching for is definitely not in your sub-select because it doesn't know what this "not-known" value actually is. Also, the documentation says it works that way:
http://docs.oracle.com/cd/B28359_01/server.111/b28286/conditions013.htm
Another option would be to write the query as:
Select a.* from Table1 a, Table2 b
Where
a.tid=b.tid and
b.createddate=(Select max(createddate) from Table2) and
not exists (Select null from Table3 t3 where t3.tid = a.tid);
The handling of nulls is one of the major differences between not exists and not in.
Your query, slightly rewritten:
Select a.*
from Table1 a join
Table2 b
on a.tid=b.tid
where b.createddate=(Select max(createddate) from Table2) and
a.tid not in (Select distinct tid from Table3)
What this tells me is that the tid with the maximum create date from Table2 is in Table3.
To test this, get the maximum create date from table2. Then get all records in table1 that correspond to this max. You will find that these are also in table3.
If I had to speculate, you might want the max create date per table in Table2, rather than the overall max.
By the way, in Oracle (and most other databases) the distinct in the last subquery is redundant. The database should be smart enough to remove duplicates in this case.
I am building an application which dynamically generates sql to search for rows of a particular Table (this is the main domain class, like an Employee).
There are three tables Table1, Table2 and Table1Table2Map.
Table1 has a many to many relationship with Table2, and is mapped through Table1Table2Map table. But since Table1 is my main table the relationship is virtually like a one to many.
My app generates a sql which basically gives a result set containing rows from all these tables. The select clause and joins dont change whereas the where clause is generated based on user interaction. In any case I dont want duplicate rows of Table1 in my result set as it is the main table for result display. Right now the query that is getting generated is like this:
select distinct Table1.Id as Id, Table1.Name, Table2.Description from Table1
left outer join Table1Table2Map on (Table1Table2Map.Table1Id = Table1.Id)
left outer join Table2 on (Table2.Id = Table1Table2Map.Table2Id)
For simplicity I have excluded the where clause. The problem is when there are multiple rows in Table2 for Table1 even though I have said distinct of Table1.Id the result set has duplicate rows of Table1 as it has to select all the matching rows in Table2.
To elaborate more, consider that for a row in Table1 with Id = 1 there are two rows in Table1Table2Map (1, 1) and (1, 2) mapping Table1 to two rows in Table2 with ids 1, 2. The above mentioned query returns duplicate rows for this case. Now I want the query to return Table1 row with Id 1 only once. This is because there is only one row in Table2 that is like an active value for the corresponding entry in Table1 (this information is in Mapping table).
Is there a way I can avoid getting duplicate rows of Table1.
I think there is some basic problem in the way I am trying to solve the problem, but I am not able to find out what it is. Thanks in advance.
Try:
left outer join (select distinct YOUR_COLUMNS_HERE ...) SUBQUERY_ALIAS on ...
In other words, don't join directly against the table, join against a sub-query that limits the rows you join against.
You can use GROUP BY on Table1.Id ,and that will get rid off the extra rows. You wouldn't need to worry about any mechanics on join side.
I came up with this solution in a huge query and it this solution didnt effect the query time much.
NOTE : I'm answering this question 3 years after its been asked but this may help someone i believe.
You can re-write your left joins to be outer applies, so that you can use a top 1 and an order by as follows:
select Table1.Id as Id, Table1.Name, Table2.Description
from Table1
outer apply (
select top 1 *
from Table1Table2Map
where (Table1Table2Map.Table1Id = Table1.Id) and Table1Table2Map.IsActive = 1
order by somethingCol
) t1t2
outer apply (
select top 1 *
from Table2
where (Table2.Id = Table1Table2Map.Table2Id)
) t2;
Note that an outer apply without a "top" or an "order by" is exactly equivalent to a left outer join, it just gives you a little more control. (cross apply is equivalent to an inner join).
You can also do something similar using the row_number() function:
select * from (
select distinct Table1.Id as Id, Table1.Name, Table2.Description,
rowNum = row_number() over ( partition by table1.id order by something )
from Table1
left outer join Table1Table2Map on (Table1Table2Map.Table1Id = Table1.Id)
left outer join Table2 on (Table2.Id = Table1Table2Map.Table2Id)
) x
where rowNum = 1;
Most of this doesn't apply if the IsActive flag can narrow down your other tables to one row, but they might come in useful for you.
To elaborate on one point: you said that there is only one "active" row in Table2 per row in Table1. Is that row not marked as active such that you could put it in the where clause? Or is there some magic in the dynamic conditions supplied by the user that determines what's active and what isn't.
If you don't need to select anything from Table2 the solution is relatively simply in that you can use the EXISTS function but since you've put TAble2.Description in the clause I'll assume that's not the case.
Basically what separates the relevant rows in Table2 from the irrelevant ones? Is it an active flag or a dynamic condition? The first row? That's really how you should be removing duplicates.
DISTINCT clauses tend to be overused. That may not be the case here but it sounds like it's possible that you're trying to hack out the results you want with DISTINCT rather than solving the real problem, which is a fairly common problem.
You have to include activity clause into your join (and no need for distinct):
select Table1.Id as Id, Table1.Name, Table2.Description from Table1
left outer join Table1Table2Map on (Table1Table2Map.Table1Id = Table1.Id) and Table1Table2Map.IsActive = 1
left outer join Table2 on (Table2.Id = Table1Table2Map.Table2Id)
If you want to display multiple rows from table2 you will have duplicate data from table1 displayed. If you wanted to you could use an aggregate function (IE Max, Min) on table2, this would eliminate the duplicate rows from table1, but would also hide some of the data from table2.
See also my answer on question #70161 for additional explanation