This is my query, I plan to run it in batches of perhaps 5000 hence the rownum < 5000
delete my_table
where rownum < 5000
and type = 'Happy'
and id not in
( select max_id
from ( select max(log_id) max_id
, object_id
, type
from my_table
where type = 'Happy'
group
by id
, type
)
)
I want to delete happy records but keeping the maximum log id, per object id
I hope that makes sense.
Should I be using some sort of join to improve performance?
I think this might run faster as a correlated subquery:
Delete
from my_table
where type = 'Happy' and
exists (select 1
from my_table t2
where t2.object_id = my_table.object_id and
t2.type = my_table.type and
t2.id > my_table.id
);
Then, an index on my_table(object_id, type, id) might also help this query.
Since you only care to delete ANY 5000 log entries for type = 'Happy', as long as its not the most recent for any object_id, then you can do something like this:
delete
from my_table
where log_id in (
select log_id from (
select log_id,
row_number() over (partition by object_id order by log_id desc) rnk
from my_table
where typ = 'Happy'
and rownum <= 5000
)
where rnk > 1
)
This is different from what you have because in your approach, you still need to calculate the max(id) per object across the entire table, which isn't necessary (and log tables can get very large). You just need to make sure you're not deleting the "newest" row (per object) of the 5000 batch rows. Personally, I prefer to setup log tables using partitions, but not everyone has this option.
Hope that helps.
You could simplify the query to:
delete my_table
where rownum < 5000
and type = 'Happy'
and id not in (select max(log_id) max_id
from my_table
where type = 'Happy'
group by object_id, type)
Related
I need to update the staging table based on the type if ZMD2 is present then update the records else update PNTP records.
UPDATE ITEMS_STAGING SET TYPE=b.TYPE,VALUE=b.VALUE
FROM ITEMS_STAGING a,ITEMS b
WHERE a.PARENT=b.PARENT
In the above statement I need to pick only ZMD2 records for the same parent if exists if not PNTP records. I tried to do UNION for the ITEMS it dint help.
Staging table Output:
Kindly help.
Thanks
You need to use analytical function row_number which will group the rows by parent column to give them numbers and then we will take only one record from each group to update staging table using merge statement as following:
MERGE INTO ITEM_STAGING M
USING (
SELECT T.*,
ROW_NUMBER() OVER(PARTITION BY T.PARENT ORDER BY T.TYPE DESC) RN
FROM ITEMS T
)
ON (M.PARENT = T.PARENT AND T.RN = 1)
WHEN MATCHED THEN
UPDATE SET M.TYPE = T.TYPE AND M.VALUE = T.VALUE;
Cheers!!
You may try below query -
SELECT *
FROM (SELECT IS.*, ROW_NUMBER() OVER(PARTITION BY ID ORDER BY TYPE DESC) RN
FROM ITEMS_STAGING)
WHERE RN = 1;
I am not sure what you want to update in this table.
Im trying to update a field in an oracle table, but I would only like to update 75 rows based on the order by creationdate field ascending.
This is what I have so far, but its not working.
UPDATE extractcandidate
SET process = 15
WHERE process IN
(Select process from extractcandidate where process = 1500 and rownum <=75 order by creationdate);
As #Gordon mentioned, you need to do the ORDER BY before the ROWNUM.
Firstly, order by creationdate:
SELECT *
FROM extractcandidate
WHERE process=1500
ORDER BY creationdate;
Secondly, limit the number to 75:
SELECT *
FROM (
SELECT *
FROM extractcandidate
WHERE process=1500
ORDER BY creationdate
)
WHERE rownum <= 75;
Now you can feed it into the UPDATE. To find the correct rows, you would normally use a primary key column. This seems to be missing in your case, so you can fall back to Oracle's internal ROWID:
UPDATE extractcandidate
SET process=15
WHERE rowid IN (
SELECT ri
FROM (
SELECT rowid as ri
FROM extractcandidate
WHERE process=1500
ORDER BY creationdate
)
WHERE rownum <= 75
);
You need an additional subquery for your method to work:
UPDATE extractcandidate
SET process = 15
WHERE ec.process = 1500 AND
creationdate IN (SELECT ec.creationdate
FROM (SELECT ec.*
FROM extractcandidate ec
WHERE ec.process = 1500
ORDER BY ec.creationdate
)
WHERE rownum <= 75
);
Notes:
You need to do the sorting before using rownum.
The comparison needs to be on creation date, not process.
If the number was more than 10 records, Old records clean.
that's mean,If the 15 records stored in the table, 5 first records to be erased.
Example:
"DELETE FROM Table WHERE ID NOT IN (SELECT ??? 10 ID FROM Table)"
In SQL Server, you can use row_number() to enumerate the values and then delete the oldest ones using where:
with todelete as (
select t.*, row_number() over (order by id desc) as seqnum
from t
)
delete from todelete
where seqnum > 10;
Your approach also works. You can do:
delete from t
where t.id not in (select top 10 t2.id from t t2 order by t2.id desc);
Note: This uses not in, so it assumes that id is never NULL. That seems reasonable in this situation.
To describe my query problem, the following data is helpful:
A single table contains the columns ID (int), VAL (varchar) and ORD (int)
The values of VAL may change over time by which older items identified by ID won't get updated but appended. The last valid item for ID is identified by the highest ORD value (increases over time).
T0, T1 and T2 are points in time where data got entered.
How do I get in an efficient manner to the Result set?
A solution must not involve materialized views etc. but should be expressible in a single SQL-query. Using Postgresql 9.3.
The correct way to select groupwise maximum in postgres is using DISTINCT ON
SELECT DISTINCT ON (id) sysid, id, val, ord
FROM my_table
ORDER BY id,ord DESC;
Fiddle
You want all records for which no newer record exists:
select *
from mytable
where not exists
(
select *
from mytable newer
where newer.id = mytable.id
and newer.ord > mytable.ord
)
order by id;
You can do the same with row numbers. Give the latest entry per ID the number 1 and keep these:
select sysid, id, val, ord
from
(
select
sysid, id, val, ord,
row_number() over (partition by id order by ord desc) as rn
from mytable
)
where rn = 1
order by id;
Left join the table (A) against itself (B) on the condition that B is more recent than A. Pick only the rows where B does not exist (i.e. A is the most recent row).
SELECT last_value.*
FROM my_table AS last_value
LEFT JOIN my_table
ON my_table.id = last_value.id
AND my_table.ord > last_value.ord
WHERE my_table.id IS NULL;
SQL Fiddle
I have a simple table where one of the field is a date column. I can select the most recent X records by doing
Select * from MyTable order by last_update desc limit X
But how do I efficiently delete those column? Is subselect the quickest?
If I recall correctly the IN clause accepts a sub select. Could be wrong.
DELETE FROM Person
WHERE
Person.ID IN (
SELECT t.ID
FROM
Person t
ORDER BY
t.joinTime DESC
LIMIT X
)
You can do "delete from where" in exactly the same way, with the same conditions as the original query.
e.g:
DELETE FROM Person WHERE Person.name = "jeff"
or
DELETE FROM Person WHERE Person.joinTime > 12001234567 LIMIT 100
If you want to delete a range (e.g most recent 10) you could try:
DELETE * FROM Person WHERE id >= ( LAST_INSERT_ID() - 10 );
Perhaps something like:
DELETE FROM MyTable WHERE rowid IN
(SELECT rowid FROM MyTable ORDER BY last_update DESC LIMIT X);
DELETE d.*
FROM mytable d
LEFT JOIN
(
SELECT id
FROM mytable
ORDER BY
last_update DESC
LIMIT 10
) q
ON d.id = q.id
WHERE q.id IS NULL
If your are using MySQL, this is the preferred solution, since IN (SELECT ... LIMIT) does not work in MySQL.
See this entry in my blog for more details:
Keeping rows