I want to make a query or some function in Oracle what can update ~24000 rows. For example 1 update query looks like this:
update numbers
set status = 1 and some_id = 123123
where number_id = 1231;
"some_id" and "number_id" are different in every query.
Problem is doing this one by one is taking too much time, I need quicker solution.
Put the varying values into a table and then use that in your query:
update numbers n
set status=1,
some_id = (select some_id from newtable t where t.number_id = n.number_id)
where number_id in (select number_id from newtable);
It looks as if you had hard-wired the predicates and update values in your query. You might be able to improve performance quite a bit by using bind variables. With bind variables, Oracle only needs one hard-parse for all 24k identical SQL UPDATE statements, instead of 24k hard-parses (and update execution plans) for 24k distinct SQL UPDATE statements.
With Java:
update numbers
set status = ?, some_id = ?
where number_id = ?;
With other tools:
update numbers
set status = :1, some_id = :2
where number_id = :3;
Write a PL-SQL stored procedure and pass in the parameters. Call it once and be done.
Or use PreparedStatement and batch your requests.
They probably take too long because you do a round trip for each one. Batch will help a great deal, because you'll only do one round trip.
Related
What I want to do is something that has the following logic:
IF EXISTS (SELECT * FROM people WHERE ID = 168)
THEN UPDATE people SET calculated_value = complex_queries_and_calculations
WHERE ID = 168
.., so to update a field of a given record if that record contains a given data, and else do nothing. To generate the data which would be used for the update, I need to query other tables for values and make some calculations. I want to avoid these queries + calculations, if there's actually nothing to update. And in this case, simply do nothing. Hence, I guess that putting for example an EXIST clause inside a WHERE clause of the UPDATE statement would end in many queries and calculations made in vain.
How can I only UPDATE conditionally and else do nothing, and make sure that all the queries + calculations needed to calculate the value used for the update are only made if the update is needed? And then, in the end, only do the update if complex_queries_and_calculations is not NULL?
My best solution so far uses a Common Table Expression (WITH clause), which makes it impossible to short-circuit. Anyway, such that you can understand the logic I'm trying to achieve, I'm showing what I've been trying so far (without success; code below is not working and I don't know why..):
-- complex queries and calculations; return result as t.result
WITH t AS(complex queries and calculations)
UPDATE target_table
SET
CASE
WHEN t.result IS NOT NULL
THEN target_table.target_column = t.result WHERE target_table.PK = 180
END;
UPDATES (Still saying syntax error, still not working)
WITH t AS(complex_queries_and_calculations AS stamp)
UPDATE target_table
SET target_column =
CASE
WHEN t.stamp IS NULL
THEN target_column
ELSE t.stamp
END
WHERE ID = 168;
Not even this is working (still reporting syntax error on UPDATE line):
WITH t AS(complex_queries_and_calculations AS stamp)
UPDATE target_table
SET target_column = target_column
WHERE ID = 168;
(eventual better approaches which avoid redundant target_column = target_column updates welcome)
With select it works, so I'm totally not understanding the syntax error #1064 it returns for my update query:
WITH t AS(complex_queries_and_calculations AS stamp)
SELECT
CASE
WHEN t.stamp IS NULL
THEN "Error!"
ELSE t.stamp
END
FROM t;
ADDITIONAL INFO
It seems like MariaDB actually does not support CTEs with UPDATE statements; correct me if I'm wrong... So I tried the following:
UPDATE people AS p
INNER JOIN (queries_and_calculations AS result) t
ON p.ID <> t.result -- just to join
SET p.target_column = t.result
WHERE p.ID = 168
AND t.result IS NOT NULL;
and now it's saying:
#4078 - Illegal parameter data types varchar and row for operation '='
Simply do the UPDATE. If there is no row with that ID, it will do nothing. This will probably be no slower than testing first.
Ditto for DELETE when the row might not exist.
"Upsert"/"IODKU" -- INSERT ... ON DUPLICATE KEY UPDATE ... is useful when you want to modify some columns when the row exists (according to some unique column), or add a new row (when it does not exist). This is better than doing a SELECT first.
Think of it this way... A big part of the UPDATE is
opening the table,
locating the block in the table that needs to be modified
loading that block into the cache ("buffer_pool")
All of that is needed for both your SELECT and UPDATE (yeah, redundantly). The UPDATE continues with:
If the row does not exist, exit.
Modify the row, and flag the block as "dirty".
In the background, the block will eventually be flushed to disk.
(I left out details about transactional integrity ("ACID"), etc.)
Even in the worst case, the whole task (for a single row) takes under 10 milliseconds. In the best case, it takes under 1ms and can be done somewhat in parallel with certain other activities.
There is no IF in SQL, since it is not needed:
UPDATE people p
SET calculated_value = c.val
FROM (
SELECT ID, val
FROM
... complex_queries_and_calculations
) c
WHERE c.ID = p.ID
AND ID = 168
AND v.val <> i.val -- maybe add this to avoid idempotent updates. Beware of NULLs, though!
;
~
GOT IT, The following query works to do exactly what I wanted in Mariadb :
UPDATE target_table
LEFT JOIN (complex_queries_and_calculations_to_get_update_value AS update_value) t
ON target_table.ID <> t.update_value -- serves just to have update value in memory,
-- because it needs to be accessed twice to create the updated column value
-- on update, sort of a workaround for CTE + UPDATE in MariaDB
SET target_column = JSON_ARRAY( FORMAT_UPDATE_VALUE(t.update_value),
FORMAT_2_UPDATE_VALUE(t.update_value) )
WHERE ID = 128 AND t.update_value IS NOT NULL;
If the record does not exist, the query takes about 0.0006 secs to execute, without doing anything to the table. If it does exits, it takes 0.0014 secs to execute, while updating the targeted record accordingly. So, it indeed seems to work and resources are saved if the targeted record is not found in target_table. Great thanks to all who helped!
So perhaps the title is a little confusing. If you can suggest better wording for that please let me know and i'll update.
Here's the issue. I've got a table with many thousands of rows and i need to update a few thousand of those many to store latest email data.
For example:
OldEmail#1.com => NewEmail#1.com
OldEmail#2.com => NewEmail#2.com
I've got a list of old emails ('OldEmail#1.com','OldEmail#2.com') and a list of the new ('NewEmail#1.com','NewEmail#2.com'). The HOPE was was to sort of do it simply with something like
UPDATE Table
SET Email = ('NewEmail#1.com','NewEmail#2.com')
WHERE Email = ('OldEmail#1.com','OldEmail#2.com')
I hope that makes sense. Any questions just ask. Thanks!
You could use a case expression:
update mytable
set email = case email
when 'OldEmail#1.com' then 'NewEmail#1.com'
when 'OldEmail#2.com' then 'NewEmail#2.com'
end
where email in ('OldEmail#1.com','OldEmail#2.com')
Or better yet, if you have a large list of values, you might create a table to store them (like myref(old_email, new_email)) and join it in your update query, like so:
update t
set t.email = r.new_email
from mytable t
inner join myref r on r.old_email = t.email
The actual syntax for update/join does vary accross databases - the above SQL Server syntax.
With accuracy to the syntax in particular DBMS:
WITH cte AS (SELECT 'NewEmail#1.com' newvalue, 'OldEmail#1.com' oldvalue
UNION ALL
SELECT 'NewEmail#2.com', 'OldEmail#2.com')
UPDATE table
SET table.email = cte.newvalue
FROM cte
WHERE table.email = cte.oldvalue
or, if CTE is not available,
UPDATE table
SET table.email = cte.newvalue
FROM (SELECT 'NewEmail#1.com' newvalue, 'OldEmail#1.com' oldvalue
UNION ALL
SELECT 'NewEmail#2.com', 'OldEmail#2.com') cte
WHERE table.email = cte.oldvalue
Consider prepared statement for rows update in large batches.
Basically it works as following :
database complies a query pattern you provide the first time, keep the compiled result for current connection (depends on implementation).
then you updates all the rows, by sending shortened label of the prepared function with different parameters in SQL syntax, instead of sending entire UPDATE statement several times for several updates
the database parse the shortened label of the prepared function , which is linked to the pre-compiled result, then perform the updates.
next time when you perform row updates, the database may still use the pre-compiled result and quickly complete the operations (so the first step above can be skipped).
Here is PostgreSQL example of prepare statement, many of SQL databases (e.g. MariaDB,MySQL, Oracle) also support it.
Not that familiar with SQL.
I have the follow SQL :
UPDATE my_Table
SET num = num + 1
WHERE id = 1
I would like limit that num column that it won't exceed a threshold, assuming 100. If num will reach 100, I would like it to stay 100 and will not increase.
I assume if statement should be included here somehow. but cant figure out how.
I'm seeing answers that would solve your problem from an update statement, all fine whatever way it is done.
Until someone creates a statement that doesn't adhere to your constraint, like a new co-worker, someone sleepy that forgets it or whatever reason. You should probably know about other options to force this rule.
You could create an SQL constraint that checks that num is never set to more than 100. That way any update or insert statement that makes this value more than 100 would result in an error rather than doing the insert or update.
Another way to force this rule "under the hood", not really beginner level, would be to create an update and insert trigger that would check for num being more than 100 and reset it to 100 in that case. That wouldn't result in an error when you run the insert or update statement. I don't have a good link for ANSI-SQL triggers, but each RDBMS has good documentation.
You can use a CASE expression:
UPDATE my_Table
SET num = CASE WHEN num+1 >= 100 THEN 100 ELSE num+1 END
WHERE id=1
try:
UPDATE my_Table SET num=num+1 WHERE id=1 AND num<=99
in that case you can use IIF for shorter version
UPDATE my_Table SET num=(IIF((num+1)>100,num,num+1)) WHERE id=1
I'm using SQL Server. I'm also relatively new to writing SQL... in a strong way. It's mostly self-taught, so I'm probably missing key ideas in terms of proper format.
I've a table called 'SiteResources' and a table called 'ResourceKeys'. SiteResources has an integer that corresponds to the placement of a string ('siteid') and a 'resourceid' which is an integer id that corresponds to 'resourceid' in ResourceKeys. ResourceKeys also contains a string for each key it contains ('resourcemessage'). Basically, these two tables are responsible for representing how strings are stored and displayed on a web page.
The best way to consistently update these two tables, is what? Let's say I have 5000 rows in SiteResources and 1000 rows in ResourceKeys. I could have an excel sheet, or a small program, which generates 5000 singular update statements, like:
update SiteResources set resoruceid = 0
WHERE siteid IS NULL AND resourceid IN (select resourceid
from ResourceKeys where resourcemessage LIKE 'FooBar')
I could have thousands of those singular update statements, with FooBar representing each string in the database I might want to change at once, but isn't there a cleaner way to write such a massive number of update statements? From what I understand, I should be wrapping all of my statements in begin/end/go too, just in-case of failure - which leads me to believe there is a more systematic way of writing these update statements? Is my hunch correct? Or is the way I'm going about this correct / ideal? I could change the structure of my tables, I suppose, or the structure of how I store data - that might simplify things - but let's just say I can't do that, in this instance.
As far as I understand, you just need to update everything in table SiteResources with empty parameter 'placement of a string'. If so, here is the code:
UPDATE a
SET resourceid = 0
FROM SiteResources a
WHERE EXISTS (select * from ResourceKeys b where a.resourceid = b.resourceid)
AND a.siteid IS NULL
For some specific things like 'FooBar'-rows you can add it like this:
UPDATE a
SET resourceid = 0
FROM SiteResources a
WHERE EXISTS (select * from ResourceKeys b where a.resourceid = b.resourceid and b.resourcemessage IN ('FooBar', 'FooBar2', 'FooBar3', ...))
AND a.siteid IS NULL
Let me see if I understood the question correctly. You'd like to update resourceid to 0 if the resourcemessage corresponds to a list of strings ? If so, you can build your query like this.
UPDATE r
SET resourceid = 0
FROM SiteResources r
JOIN ResourceKeys k ON r.resourceid = k.resourceid
WHERE k.resourcemessage IN ('FooBar', ...)
AND r.siteid IS NULL;
This is using an extended UPDATE syntax in transact-sql allowing you to use a JOIN in the UPDATE statement. But maybe it's not exactly what you want ? Why do you use the LIKE operator in your query, without wildcard (%) ?
With table-valued parameters, you can pass a table from your client app to the SQL batch that your app submits for execution. You can use this to pass a list of all the strings you need to update to a single UPDATE that updates all rows at once.
That way you don't have to worry about all of your concerns: the number of updates, transactional atomicitty, error handling. As a bonus, performance will be improved.
I recommend that you do a bit of research what TVPs are and how they are used.
I want to write a update SQL statement, but one conidtion of this statement is the result from a select SQL statement, and I also want to return the result of the select SQL statement.
Like this: update ... set ... where id = (select id from ...)
I want to return the value of id back.
Does anybody know how should I do this?
Thanks in advance!
I don't believe that's possible in one statement. Update then query (select) the new value, or query the value first, and then submit an update.
Alternative would be a stored procedure on the database, which executes the multiple queries and returns the result for you.
This is not possible in all Java database frameworks that I know. Probably you need to separate your query and update in Java.
I don't see any problem in using a subselect in a WHERE clause of an update statement.
For the second request, getting back the value of id, I know this is possible in DB2, and maybe others implement that syntax too:
SELECT id FROM FINAL TABLE (
update ... set ... where id = (select id from ...)
)
This works also for INSERT and DELETE statements. (See the documentation.)
Update statements won't return the updated datasets. The select in that case would be a subselect that isn't directly accessible.
You'd thus have to use at least two queries:
select the ids you want
call the update query passing the previously selected ids as a parameter