Related
I have this query in Oracle11g :
SELECT DOC_ID,DOC_NAME,DESC
FROM TABLE1
WHERE DOC_ID NOT IN(
SELECT DOC_ID FROM TABLE2
)
The sql query above run very slow since i have so many data in the tables.
Is there any solution to get the same result with better performance and run faster?
Any help much appreciated.
Thanks.
Using WHERE EXISTS may have better performance:
SELECT DOC_ID,DOC_NAME,DESCr
FROM TABLE1 t1
WHERE not exists (
SELECT 1 FROM TABLE2 where
doc_id = t1.doc_id
);
Example: http://sqlfiddle.com/#!4/4b59e/3
I wouldn't use the in statement for that. If you join on what I imagine is one of your keys it should be much much faster:
select tb1.DOC_ID, tb1.DOC_NAME, tb1.DESC
from table1 tb1
left join table2 tb2
on tb1.DOC_ID = tb2.DOC_ID
where tab2.DOC_ID is not null
when i run SELECT stataement on my view it takes around 1.5 hours to run, what can i do to optimize it.
Below is the sample structure of how my view looks like
CREATE VIEW SCHEMANAME.VIEWNAME
{
COL, COL1, COL2, COL3 }
AS SELECT
COST.ETA,
CASE
WHEN VOL.CURR IS NOT NULL
THEN COALESCE {VOL.COMM,0}
END CASE,
CASE
WHEN...
END CASE
FROM TABLE1 t1 inner join TABLE2 t2 ON t1.ETA=t2.ETA
INNER JOIN TABLE3 t3 on t2.ETA=t3.ETA
LEFT OUTER JOIN TABLE4 t4 on t2.ETA=t4.ETA
This is your query:
SELECT COST.ETA,
(CASE WHEN VOL.CURR IS NOT NULL THEN COALESCE {VOL.COMM,0}
END) as ??,
. . .
FROM TABLE1 t1 inner join
TABLE2 t2
ON t1.ETA = t2.ETA INNER JOIN
TABLE3 t3
on t2.ETA = t3.ETA LEFT OUTER JOIN
TABLE4 t4
on t2.ETA = t4.ETA;
First, I will the fact that the select clause references tables that are not in the from clause. I assume this is a typo.
Second, you should be able to use indexes to improve this query: table1(eta), table2(eta),table3(eta), andtable4(eta).
Third, I am highly suspicious on seeing the same column used for joining so many tables. I suspect that you might have cartesian products occurring, because there are multiple values of any given eta in several tables. If that is the case, you need to fix the query to better reflect what you really need. If so, ask another question with sample data and desired results, because your query is probably not correct.
I can't seem to ever remember this query!
I want to delete all rows in table1 whose ID's are the same as in Table2.
So:
DELETE table1 t1
WHERE t1.ID = t2.ID
I know I can do a WHERE ID IN (SELECT ID FROM table2) but I want to do this query using a JOIN if possible.
DELETE t1
FROM Table1 t1
JOIN Table2 t2 ON t1.ID = t2.ID;
I always use the alias in the delete statement as it prevents the accidental
DELETE Table1
caused when failing to highlight the whole query before running it.
DELETE Table1
FROM Table1
INNER JOIN Table2 ON Table1.ID = Table2.ID
There is no solution in ANSI SQL to use joins in deletes, AFAIK.
DELETE FROM Table1
WHERE Table1.id IN (SELECT Table2.id FROM Table2)
Later edit
Other solution (sometimes performing faster):
DELETE FROM Table1
WHERE EXISTS( SELECT 1 FROM Table2 Where Table1.id = Table2.id)
PostgreSQL implementation would be:
DELETE FROM t1
USING t2
WHERE t1.id = t2.id;
Try this:
DELETE Table1
FROM Table1 t1, Table2 t2
WHERE t1.ID = t2.ID;
or
DELETE Table1
FROM Table1 t1 INNER JOIN Table2 t2 ON t1.ID = t2.ID;
I think that you might get a little more performance if you tried this
DELETE FROM Table1
WHERE EXISTS (
SELECT 1
FROM Table2
WHERE Table1.ID = Table2.ID
)
This will delete all rows in Table1 that match the criteria:
DELETE Table1
FROM Table2
WHERE Table1.JoinColumn = Table2.JoinColumn And Table1.SomeStuff = 'SomeStuff'
Found this link useful
Copied from there
Oftentimes, one wants to delete some records from a table based on criteria in another table. How do you delete from one of those tables without removing the records in both table?
DELETE DeletingFromTable
FROM DeletingFromTable INNER JOIN CriteriaTable
ON DeletingFromTable.field_id = CriteriaTable.id
WHERE CriteriaTable.criteria = "value";
The key is that you specify the name of the table to be deleted from as the SELECT. So, the JOIN and WHERE do the selection and limiting, while the DELETE does the deleting. You're not limited to just one table, though. If you have a many-to-many relationship (for instance, Magazines and Subscribers, joined by a Subscription) and you're removing a Subscriber, you need to remove any potential records from the join model as well.
DELETE subscribers, subscriptions
FROM subscribers INNER JOIN subscriptions
ON subscribers.id = subscriptions.subscriber_id
INNER JOIN magazines
ON subscriptions.magazine_id = magazines.id
WHERE subscribers.name='Wes';
Deleting records with a join could also be done with a LEFT JOIN and a WHERE to see if the joined table was NULL, so that you could remove records in one table that didn't have a match (like in preparation for adding a relationship.) Example post to come.
Since the OP does not ask for a specific DB, better use a standard compliant statement.
Only MERGE is in SQL standard for deleting (or updating) rows while joining something on target table.
merge table1 t1
using (
select t2.ID
from table2 t2
) as d
on t1.ID = d.ID
when matched then delete;
MERGE has a stricter semantic, protecting from some error cases which may go unnoticed with DELETE ... FROM. It enforces 'uniqueness' of match : if many rows in the source (the statement inside using) match the same row in the target, the merge must be canceled and an error must be raised by the SQL engine.
To Delete table records based on another table
Delete From Table1 a,Table2 b where a.id=b.id
Or
DELETE FROM Table1
WHERE Table1.id IN (SELECT Table2.id FROM Table2)
Or
DELETE Table1
FROM Table1 t1 INNER JOIN Table2 t2 ON t1.ID = t2.ID;
I often do things like the following made-up example. (This example is from Informix SE running on Linux.)
The point of of this example is to delete all real estate exemption/abatement transaction records -- because the abatement application has a bug -- based on information in the real_estate table.
In this case last_update != nullmeans the account is not closed, and res_exempt != 'p' means the accounts are not personal property (commercial equipment/furnishings).
delete from trans
where yr = '16'
and tran_date = '01/22/2016'
and acct_type = 'r'
and tran_type = 'a'
and bill_no in
(select acct_no from real_estate where last_update is not null
and res_exempt != 'p');
I like this method, because the filtering criteria -- at least for me -- is easier to read while creating the query, and to understand many months from now when I'm looking at it and wondering what I was thinking.
Referencing MSDN T-SQL DELETE (Example D):
DELETE FROM Table1
FROM Tabel1 t1
INNER JOIN Table2 t2 on t1.ID = t2.ID
This is old I know, but just a pointer to anyone using this ass a reference. I have just tried this and if you are using Oracle, JOIN does not work in DELETE statements.
You get a the following message:
ORA-00933: SQL command not properly ended.
While the OP doesn't want to use an 'in' statement, in reply to Ankur Gupta, this was the easiest way I found to delete the records in one table which didn't exist in another table, in a one to many relationship:
DELETE
FROM Table1 as t1
WHERE ID_Number NOT IN
(SELECT ID_Number FROM Table2 as t2)
Worked like a charm in Access 2016, for me.
delete
table1
from
t2
where
table1.ID=t2.ID
Works on mssql
Here is my issue, I am selecting and doing multiple joins to get the correct items...it pulls in a fair amount of rows, above 100,000. This query takes more than 5mins when the date range is set to 1 year.
I don't know if it's possible but I am afraid that the user might extend the date range to like ten years and crash it.
Anyone know how I can speed this up? Here is the query.
SELECT DISTINCT t1.first_name, t1.last_name, t1.email
FROM table1 AS t1
INNER JOIN table2 AS t2 ON t1.CU_id = t2.O_cid
INNER JOIN table3 AS t3 ON t2.O_ref = t3.I_oref
INNER JOIN table4 AS t4 ON t3.I_pid = t4.P_id
INNER JOIN table5 AS t5 ON t4.P_cat = t5.C_id
WHERE t1.subscribe =1
AND t1.Cdate >= $startDate
AND t1.Cdate <= $endDate
AND t5.store =2
I am not the greatest with mysql so any help would be appreciated!
Thanks in advance!
UPDATE
Here is the explain you asked for
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE t5 ref PRIMARY,C_store_type,C_id,C_store_type_2 C_store_type_2 1 const 101 Using temporary
1 SIMPLE t4 ref PRIMARY,P_cat P_cat 5 alphacom.t5.C_id 326 Using where
1 SIMPLE t3 ref I_pid,I_oref I_pid 4 alphacom.t4.P_id 31
1 SIMPLE t2 eq_ref O_ref,O_cid O_ref 28 alphacom.t3.I_oref 1
1 SIMPLE t1 eq_ref PRIMARY PRIMARY 4 alphacom.t2.O_cid 1 Using where
Also I added an index to table5 rows and table4 rows because they don't really change, however the other tables get around 500-1000 entries a month... I heard you should add an index to a table that has that many new entries....is this true?
I'd try the following:
First, ensure there are indexes on the following tables and columns (each set of columns in parentheses should be a separate index):
table1 : (subscribe, CDate)
(CU_id)
table2 : (O_cid)
(O_ref)
table3 : (I_oref)
(I_pid)
table4 : (P_id)
(P_cat)
table5 : (C_id, store)
Second, if adding the above indexes didn't improve things as much as you'd like, try rewriting the query as
SELECT DISTINCT t1.first_name, t1.last_name, t1.email FROM
(SELECT CU_id, t1.first_name, t1.last_name, t1.email
FROM table1
WHERE subscribe = 1 AND
CDate >= $startDate AND
CDate <= $endDate) AS t1
INNER JOIN table2 AS t2
ON t1.CU_id = t2.O_cid
INNER JOIN table3 AS t3
ON t2.O_ref = t3.I_oref
INNER JOIN table4 AS t4
ON t3.I_pid = t4.P_id
INNER JOIN (SELECT C_id FROM table5 WHERE store = 2) AS t5
ON t4.P_cat = t5.C_id
I'm hoping here that the first sub-select would cut down significantly on the number of rows to be considered for joining, hopefully making the subsequent joins do less work. Ditto the reasoning behind the second sub-select on table5.
In any case, mess with it. I mean, ultimately it's just a SELECT - you can't really hurt anything with it. Examine the plans that are generated by each different permutation and try to figure out what's good or bad about each.
Share and enjoy.
Make sure your date columns and all the columns you are joining on are indexed.
Doing an unequivalence operator on your dates means it checks every row, which is inherently slower than an equivalence.
Also, using DISTINCT adds an extra comparison to the logic that your optimizer is running behind the scenes. Eliminate that if possible.
Well, first, make a subquery to decimate table1 down to just the records you actually want to go to all the trouble of joining...
SELECT DISTINCT t1.first_name, t1.last_name, t1.email
FROM (
SELECT first_name, last_name, email, CU_id FROM table1 WHERE
table1.subscribe = 1
AND table1.Cdate >= $startDate
AND table1.Cdate <= $endDate
) AS t1
INNER JOIN table2 AS t2 ON t1.CU_id = t2.O_cid
INNER JOIN table3 AS t3 ON t2.O_ref = t3.I_oref
INNER JOIN table4 AS t4 ON t3.I_pid = t4.P_id
INNER JOIN table5 AS t5 ON t4.P_cat = t5.C_id
WHERE t5.store = 2
Then start looking at modifying the directionality of the joins.
Additionally, if t5.store is only very rarely 2, then flip this idea around: construct the t5 subquery, then join it back and back and back.
At present, your query is returning all matching rows on table2-table5, just to establish whether t5.store = 2. If any of table2-table5 have a significantly higher row count than table1, this may be greatly increasing the number of rows processed - consequently, the following query may perform significantly better:
SELECT DISTINCT t1.first_name, t1.last_name, t1.email
FROM table1 AS t1
WHERE t1.subscribe =1
AND t1.Cdate >= $startDate
AND t1.Cdate <= $endDate
AND EXISTS
(SELECT NULL FROM table2 AS t2
INNER JOIN table3 AS t3 ON t2.O_ref = t3.I_oref
INNER JOIN table4 AS t4 ON t3.I_pid = t4.P_id
INNER JOIN table5 AS t5 ON t4.P_cat = t5.C_id AND t5.store =2
WHERE t1.CU_id = t2.O_cid);
Try adding indexes on the fields that you join. It may or may not improve the performance.
Moreover it also depends on the engine that you are using. If you are using InnoDB check your configuration params. I had faced a similar problem, as the default configuration of innodb wont scale much as myisam's default configuration.
As everyone says, make sure you have indexes.
You can also check if your server is set up properly so it can contain more of, of maybe the entire, dataset in memory.
Without an EXPLAIN, there's not much to work by. Also keep in mind that MySQL will look at your JOIN, and iterate through all possible solutions before executing the query, which can take time. Once you have the optimal JOIN order from the EXPLAIN, you could try and force this order in your query, eliminating this step from the optimizer.
It sounds like you should think about delivering subsets (paging) or limit the results some other way unless there is a reason that the users need every row possible all at once. Typically 100K rows is more than the average person can digest.
I can't seem to ever remember this query!
I want to delete all rows in table1 whose ID's are the same as in Table2.
So:
DELETE table1 t1
WHERE t1.ID = t2.ID
I know I can do a WHERE ID IN (SELECT ID FROM table2) but I want to do this query using a JOIN if possible.
DELETE t1
FROM Table1 t1
JOIN Table2 t2 ON t1.ID = t2.ID;
I always use the alias in the delete statement as it prevents the accidental
DELETE Table1
caused when failing to highlight the whole query before running it.
DELETE Table1
FROM Table1
INNER JOIN Table2 ON Table1.ID = Table2.ID
There is no solution in ANSI SQL to use joins in deletes, AFAIK.
DELETE FROM Table1
WHERE Table1.id IN (SELECT Table2.id FROM Table2)
Later edit
Other solution (sometimes performing faster):
DELETE FROM Table1
WHERE EXISTS( SELECT 1 FROM Table2 Where Table1.id = Table2.id)
PostgreSQL implementation would be:
DELETE FROM t1
USING t2
WHERE t1.id = t2.id;
Try this:
DELETE Table1
FROM Table1 t1, Table2 t2
WHERE t1.ID = t2.ID;
or
DELETE Table1
FROM Table1 t1 INNER JOIN Table2 t2 ON t1.ID = t2.ID;
I think that you might get a little more performance if you tried this
DELETE FROM Table1
WHERE EXISTS (
SELECT 1
FROM Table2
WHERE Table1.ID = Table2.ID
)
This will delete all rows in Table1 that match the criteria:
DELETE Table1
FROM Table2
WHERE Table1.JoinColumn = Table2.JoinColumn And Table1.SomeStuff = 'SomeStuff'
Found this link useful
Copied from there
Oftentimes, one wants to delete some records from a table based on criteria in another table. How do you delete from one of those tables without removing the records in both table?
DELETE DeletingFromTable
FROM DeletingFromTable INNER JOIN CriteriaTable
ON DeletingFromTable.field_id = CriteriaTable.id
WHERE CriteriaTable.criteria = "value";
The key is that you specify the name of the table to be deleted from as the SELECT. So, the JOIN and WHERE do the selection and limiting, while the DELETE does the deleting. You're not limited to just one table, though. If you have a many-to-many relationship (for instance, Magazines and Subscribers, joined by a Subscription) and you're removing a Subscriber, you need to remove any potential records from the join model as well.
DELETE subscribers, subscriptions
FROM subscribers INNER JOIN subscriptions
ON subscribers.id = subscriptions.subscriber_id
INNER JOIN magazines
ON subscriptions.magazine_id = magazines.id
WHERE subscribers.name='Wes';
Deleting records with a join could also be done with a LEFT JOIN and a WHERE to see if the joined table was NULL, so that you could remove records in one table that didn't have a match (like in preparation for adding a relationship.) Example post to come.
Since the OP does not ask for a specific DB, better use a standard compliant statement.
Only MERGE is in SQL standard for deleting (or updating) rows while joining something on target table.
merge table1 t1
using (
select t2.ID
from table2 t2
) as d
on t1.ID = d.ID
when matched then delete;
MERGE has a stricter semantic, protecting from some error cases which may go unnoticed with DELETE ... FROM. It enforces 'uniqueness' of match : if many rows in the source (the statement inside using) match the same row in the target, the merge must be canceled and an error must be raised by the SQL engine.
To Delete table records based on another table
Delete From Table1 a,Table2 b where a.id=b.id
Or
DELETE FROM Table1
WHERE Table1.id IN (SELECT Table2.id FROM Table2)
Or
DELETE Table1
FROM Table1 t1 INNER JOIN Table2 t2 ON t1.ID = t2.ID;
I often do things like the following made-up example. (This example is from Informix SE running on Linux.)
The point of of this example is to delete all real estate exemption/abatement transaction records -- because the abatement application has a bug -- based on information in the real_estate table.
In this case last_update != nullmeans the account is not closed, and res_exempt != 'p' means the accounts are not personal property (commercial equipment/furnishings).
delete from trans
where yr = '16'
and tran_date = '01/22/2016'
and acct_type = 'r'
and tran_type = 'a'
and bill_no in
(select acct_no from real_estate where last_update is not null
and res_exempt != 'p');
I like this method, because the filtering criteria -- at least for me -- is easier to read while creating the query, and to understand many months from now when I'm looking at it and wondering what I was thinking.
Referencing MSDN T-SQL DELETE (Example D):
DELETE FROM Table1
FROM Tabel1 t1
INNER JOIN Table2 t2 on t1.ID = t2.ID
This is old I know, but just a pointer to anyone using this ass a reference. I have just tried this and if you are using Oracle, JOIN does not work in DELETE statements.
You get a the following message:
ORA-00933: SQL command not properly ended.
While the OP doesn't want to use an 'in' statement, in reply to Ankur Gupta, this was the easiest way I found to delete the records in one table which didn't exist in another table, in a one to many relationship:
DELETE
FROM Table1 as t1
WHERE ID_Number NOT IN
(SELECT ID_Number FROM Table2 as t2)
Worked like a charm in Access 2016, for me.
delete
table1
from
t2
where
table1.ID=t2.ID
Works on mssql