I noticed that concurrent execution of simple and identical queries similar to
BEGIN;
SELECT files.data FROM files WHERE files.file_id = 123 LIMIT 1 FOR UPDATE;
UPDATE files SET ... WHERE files.file_id = 123;
COMMIT;
lead to deadlock which is surprising to me since it looks like such queries should not create a deadlock. Also: it is usually takes only milliseconds to complete such request. During such deadlock situation if I run:
SELECT blockeda.pid AS blocked_pid, blockeda.query as blocked_query,
blockinga.pid AS blocking_pid, blockinga.query as blocking_query FROM pg_catalog.pg_locks blockedl
JOIN pg_stat_activity blockeda ON blockedl.pid = blockeda.pid
JOIN pg_catalog.pg_locks blockingl ON(blockingl.transactionid=blockedl.transactionid
AND blockedl.pid != blockingl.pid)
JOIN pg_stat_activity blockinga ON blockingl.pid = blockinga.pid
WHERE NOT blockedl.granted;
I see both of my identical select statements listed for blocked_pid and blockin_pid for whole duration of the deadlock.
So my question is: Is it normal and expected for queries that try to select same row FOR UPDATE to cause deadlock? And if so, what is the best strategy to avoid deadlocking in this scenario?
Your commands are contradicting.
If files.file_id is defined UNIQUE (or PRIMARY KEY), you don't need LIMIT 1. And you don't need explicit locking at all. Just run the UPDATE, since only a single row is affected in the whole transaction, there cannot be a deadlock. (Unless there are side effects from triggers or rules or involved functions.)
If files.file_id is not UNIQUE (like it seems), then the UPDATE can affect multiple rows in arbitrary order and only one of them is locked, a recipe for deadlocks. The more immediate problem would then be that the query does not do what you seem to want to begin with.
The best solution depends on missing information. This would work:
UPDATE files
SET ...
WHERE primary_key_column = (
SELECT primary_key_column
FROM files
WHERE file_id = 123
LIMIT 1
-- FOR UPDATE SKIP LOCKED
);
No BEGIN; and COMMIT; needed for the single command, while default auto-commit is enabled.
You might want to add FOR UPDATE SKIP LOCKED (or FOR UPDATE NOWAIT) to either skip or report an error if the row is already locked.
And you probably want to add a WHERE clause that avoids processing the same row repeatedly.
More here:
Postgres UPDATE … LIMIT 1
Related
I have transaction with several queries. First, a select rows with FOR UPDATE lock:
SELECT f.source_id FROM files AS f WHERE
f.component_id = $1 AND
f.archived_at IS NULL
FOR UPDATE
Next, there is an update query:
UPDATE files AS f SET archived_at = NOW()
WHERE
hw_component_id = $1 AND
f.source_id = ANY($2::text[])
And then there is an insert:
INSERT INTO files AS f (
source_id,
...
)
VALUES (..)
ON CONFLICT (component_id, source_id) DO UPDATE
SET archived_at = null,
is_valid = excluded.is_valid
I have two application instances and sometimes I see deadlock errors in PostgreSQL log:
ERROR: deadlock detected
DETAIL: Process 3992939 waits for ShareLock on transaction 230221362; blocked by process 4108096.
Process 4108096 waits for ShareLock on transaction 230221365; blocked by process 3992939.
Process 3992939: SELECT f.source_id FROM files AS f WHERE f.component_id = $1 AND f.archived_at IS NULL FOR UPDATE
Process 4108096: INSERT INTO files AS f (source_id, ...) VALUES (..) ON CONFLICT (component_id, source_id) DO UPDATE SET archived_at = null, is_valid = excluded.is_valid
CONTEXT: while locking tuple (41116,185) in relation \"files\"
I assume that it may be caused by ON CONFLICT DO UPDATE statement, which may update rows which are not locked by previous SELECT FOR UPDATE
But I can't understand how can SELECT ... FOR UPDATE query cause deadlock if it is the first query in transaction. There is not queries before it.
Can SELECT ... FOR UPDATE statement lock several rows and then wait for other rows in condition to be unlocked?
SELECT FOR UPDATE is no safeguard against deadlocks. It just locks rows. Locks are acquired along the way, in the order instructed by ORDER BY, or in arbitrary order in the absence of ORDER BY. The best defense against deadlocks is to lock rows in consistent order across the whole transaction - and doing likewise in all concurrent transactions. Or, as the manual puts it:
The best defense against deadlocks is generally to avoid them by being
certain that all applications using a database acquire locks on
multiple objects in a consistent order.
Else, this can happen (row1, row2, ... are rows numbered according to the virtual consistent order):
T1: SELECT FOR UPDATE ... -- lock row2, row3
T2: SELECT FOR UPDATE ... -- lock row4, wait for T1 to release row2
T1: INSERT ... ON CONFLICT ... -- wait for T2 to release lock on row4
--> deadlock
Adding ORDER BY to your SELECT... FOR UPDATE may already avoid your deadlocks. (It would avoid the one demonstrated above.) Or this happens and you have to do more:
T1: SELECT FOR UPDATE ... -- lock row2, row3
T2: SELECT FOR UPDATE ... -- lock row1, wait for T1 to release row2
T1: INSERT ... ON CONFLICT ... -- wait for T2 to release lock on row1
--> deadlock
Everything within the transaction must happen in consistent order to be absolutly sure.
Also, your UPDATE does not seem to be in line with the SELECT FOR UPDATE. component_id <> hw_component_id. Typo?
Also, f.archived_at IS NULL does not guarantee that the later SET archived_at = NOW() only affects these rows. You would have to add WHERE f.archived_at IS NULL to the UPDATE be in line. (Seems like a good idea in any case?)
I assume that it may be caused by ON CONFLICT DO UPDATE statement,
which may update rows which are not locked by previous SELECT FOR UPDATE.
As long as the UPSERT (ON CONFLICT DO UPDATE) sticks to the consistent order, that wouldn't be a problem. But that may be hard or impossible to enforce.
Can SELECT ... FOR UPDATE statement lock several rows and then wait for other rows in condition to be unlocked?
Yes, as explained above, locks are acquired along the way. It can have to stop and wait half way through.
NOWAIT
If all that still can't resolve your deadlocks, the slow and sure method is to use Serializable Isolation Level. Then you have to be prepared for serialization failures and retry the transaction in this case. Considerably more expensive overall.
Or it might be enough to add NOWAIT:
SELECT FROM files
WHERE component_id = $1
AND archived_at IS NULL
ORDER BY id -- whatever you use for consistent, deterministic order
FOR UPDATE NOWAIT;
The manual:
With NOWAIT, the statement reports an error, rather than waiting, if a selected row cannot be locked immediately.
You may even skip the ORDER BY clause with NOWAIT if you cannot establish consistent order with the UPSERT anyway.
Then you have to catch that error and retry the transaction. Similar to catching serialization failures, but much cheaper - and less reliable. For example, multiple transactions can still interlock with their UPSERT alone. But it gets less and less likely.
Currently, we are updating many rows at the same time using this statement:
update my_table set field = 'value' where id in (<insert ids here>);
My worry is, it might cause a deadlock with another query that we run in intervals:
select my_table where field = 'value' for update order by id;
The query above will fetch multiple rows.
Is this scenario possible?
Just a bit of background:
We added the order by id before since when we run the query above multiple times at the same time, we were having random deadlocks due to different orders by that query.
We were wondering if this applies to update statements as well.
Yes, these can deadlock. To avoid this, run the select ... for update order by id in the same transaction immediately before the update. This will lock all rows affected and avoid any other transaction from running the same select ... for update query.
I am not saying consolidate the same two tasks. I am saying use the same locking select in both.
I want to insert rows with a MERGE statement in a specified order to avoid deadlocks. Deadlocks could otherwise happen because multiple transaction will call this statement with overlapping sets of keys. Note that this code is also sensitive to duplicate value exception but I handle that by retrying so that is not my question. I was doing the following:
MERGE INTO targetTable
USING (
SELECT ...
FROM sourceCollection
ORDER BY <desiredUpdateOrder>
)
WHEN MATCHED THEN
UPDATE ...
WHEN NOT MATCHED THEN
INSERT ...
Now I'm still getting the dead lock so I'm becoming unsure whether oracle maintains the order of the sub-query. Does anyone know how to best make sure that oracle locks the rows in targetTable in the same order in this case? Do I have to do a SELECT FOR UPDATE before the merge? In which order does the SELECT FOR UPDATE lock the rows? Oracle UPDATE statement has an ORDER BY clause that MERGE seems to be missing. Is there another way to avoid dead locks other than locking the rows in the same order every time?
[Edit]
This query is used to maintain a count of how often a certain action has taken place. When the action happens the first time a row is inserted, when it happens a second time the "count" column is incremented. There are millions of different actions and they happen very often. A table lock wouldn't work.
Controlling the order in which the target table rows are modified requires that you control the query execution plan of the USING subquery. That's a tricky business, and depends on what sort of execution plans your query is likely to be getting.
If you're getting deadlocks then I'd guess that you're getting a nested loop join from the source collection to the target table, as a hash join would probably be based on hashing the source collection and would modify the target table roughly in target-table rowid order because that would be full scanned -- in any case, the access order would be consistent across all of the query executions.
Likewise, if there was a sort-merge between the two data sets you'd get consistency in the order in which target table rows are accessed.
Ordering of the source collection seems to be desirable, but the optimiser might not be applying it so check the execution plan. If it is not then try inserting your data into a global temporary table using APPEND and with an ORDER BY clause, and then selecting from there without an order by clause, and explore the us of hints to entrench a nested loop join.
I don't believe the ORDER BY will affect anything (though I'm more than willing to be proven wrong); I think MERGE will lock everything it needs to.
Assume I'm completely wrong, assume that you get row-by-row locks with MERGE. Your problem still isn't solved as you have no guarantees that your two MERGE statements won't hit the same row simultaneously. In fact, from the information given, you have no guarantees that an ORDER BY improves the situation; it might make it worse.
Despite there being no skip locked rows syntax as there is with UPDATE there is still a simple answer, stop trying to update the same row from within different transactions. If feasible, you can use some form of parallel execution, for instance the DBMS_PARALLEL_EXECUTE subprogram CREATE_CHUNKS_BY_ROWID and ensure that your transactions only work on a specific sub-set of the rows in the table.
As an aside I'm a little worried by your description of the problem. You say there's some duplicate erroring that you fix by rerunning the MERGE. If the data in these duplicates is different you need to ensure that the ORDER BY is done not only on the data to be merged but the data being merged into. If you don't then there's no guarantee that you don't overwrite the correct data with older, incorrect, data.
First locks are not really managed at row level but at block level. You may encounter an ORA-00060 error even without modifying the same row. This can be tricky. Managing this is the request developper's job.
One possible workaround is to organize your table (never do that on huge tables or table with heavy change rates)
https://use-the-index-luke.com/sql/clustering/index-organized-clustered-index
Rather than do a merge, I suggest that you try and lock the row. If successful update it, if not insert new row. By default lock will wait if another process has a lock on the same thing.
CREATE TABLE brianl.deleteme_table
(
id INTEGER PRIMARY KEY
, cnt INTEGER NOT NULL
);
CREATE OR REPLACE PROCEDURE brianl.deleteme_table_proc (
p_id IN deleteme_table.id%TYPE)
AUTHID DEFINER
AS
l_id deleteme_table.id%TYPE;
-- This isolates this procedure so that it doesn't commit
-- anything outside of the procedure.
PRAGMA AUTONOMOUS_TRANSACTION;
BEGIN
-- select the row for update
-- this will pause if someone already has the row locked.
SELECT id
INTO l_id
FROM deleteme_table
WHERE id = p_id
FOR UPDATE;
-- Row was locked, update it.
UPDATE deleteme_table
SET cnt = cnt + 1
WHERE id = p_id;
COMMIT;
EXCEPTION
WHEN NO_DATA_FOUND
THEN
-- we were unable to lock the record, insert a new row
INSERT INTO deleteme_table (id, cnt)
VALUES (p_id, 1);
COMMIT;
END deleteme_table_proc;
CREATE OR REPLACE PROCEDURE brianl.deleteme_proc_test
AUTHID CURRENT_USER
AS
BEGIN
-- This resets the table to empty for the test
EXECUTE IMMEDIATE 'TRUNCATE TABLE brianl.deleteme_table';
brianl.deleteme_table_proc (p_id => 1);
brianl.deleteme_table_proc (p_id => 2);
brianl.deleteme_table_proc (p_id => 3);
brianl.deleteme_table_proc (p_id => 2);
FOR eachrec IN ( SELECT id, cnt
FROM brianl.deleteme_table
ORDER BY id)
LOOP
DBMS_OUTPUT.put_line (
a => 'id: ' || eachrec.id || ', cnt:' || eachrec.cnt);
END LOOP;
END;
BEGIN
-- runs the test;
brianl.deleteme_proc_test;
END;
I am using MS SQL Server 2008
i have a table which is constantly in use (data is always changing and inserted to it)
it contains now ~70 Mill rows,
I am trying to run a simple query over the table with a stored procedure that should properly take a few days,
I need the table to keep being usable, now I executed the stored procedure and after a while every simple select by identity query that I try to execute on the table is not responding/running too much time that I break it
what should I do?
here is how my stored procedure looks like:
SET NOCOUNT ON;
update SOMETABLE
set
[some_col] = dbo.ufn_SomeFunction(CONVERT(NVARCHAR(500), another_column))
WHERE
[some_col] = 243
even if i try it with this on the where clause (with an 'and' logic..) :
ID_COL > 57000000 and ID_COL < 60000000 and
it still doesn't work
BTW- SomeFunction does some simple mathematics actions and looks up rows in another table that contains about 300k items, but is never changed
From my perspective your server has a serious performance problem. Even if we assume that none of the records in the query
select some_col with (nolock) where id_col between 57000000 and 57001000
was in memory, it shouldn't take 21 seconds to read the few pages sequentially from disk (your clustered index on the id_col should not be fragmented if it's an auto-identity and you didn't do something stupid like adding a "desc" to the index definition).
But if you can't/won't fix that, my advice would be to make the update in small packages like 100-1000 records at a time (depending on how much time the lookup function consumes). One update/transaction should take no more than 30 seconds.
You see each update keeps an exclusive lock on all the records it modified until the transaction is complete. If you don't use an explicit transaction, each statement is executed in a single, automatic transaction context, so the locks get released when the update statement is done.
But you can still run into deadlocks that way, depending on what the other processes do. If they modify more than one record at a time, too, or even if they gather and hold read locks on several rows, you can get deadlocks.
To avoid the deadlocks, your update statement needs to take a lock on all the records it will modify at once. The way to do this is to place the single update statement (with only the few rows limited by the id_col) in a serializable transaction like
IF ##TRANCOUNT > 0
-- Error: You are in a transaction context already
SET NOCOUNT ON
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
-- Insert Loop here to work "x" through the id range
BEGIN TRANSACTION
UPDATE SOMETABLE
SET [some_col] = dbo.ufn_SomeFunction(CONVERT(NVARCHAR(500), another_column))
WHERE [some_col] = 243 AND id_col BETWEEN x AND x+500 -- or whatever keeps the update in the small timerange
COMMIT
-- Next loop
-- Get all new records while you where running the loop. If these are too many you may have to paginate this also:
BEGIN TRANSACTION
UPDATE SOMETABLE
SET [some_col] = dbo.ufn_SomeFunction(CONVERT(NVARCHAR(500), another_column))
WHERE [some_col] = 243 AND id_col >= x
COMMIT
For each update this will take an update/exclusive key-range lock on the given records (but only them, because you limit the update through the clustered index key). It will wait for any other updates on the same records to finish, then get it's lock (causing blocking for all other transactions, but still only for the given records), then update the records and release the lock.
The last extra statement is important, because it will take a key range lock up to "infinity" and thus prevent even inserts on the end of the range while the update statement runs.
I have a rather basic and general question about atomicity of "UPDATE ... SET .. WHERE ..." statement.
having a table (without extra constraint),
+----------+
| id | name|
+----------+
| 1 | a |
+----+-----+
now, I would execute following 4 statements "at the same time" (concurrently).
UPDATE table SET name='b1' WHERE name='a'
UPDATE table SET name='b2' WHERE name='a'
UPDATE table SET name='b3' WHERE name='a'
UPDATE table SET name='b4' WHERE name='a'
is there only one UPDATE statement would be executed with table update?
or, is it possible that more than one UPDATE statements can really update the table?
should I need extra transaction or lock to let only one UPDATE write values into table?
thanks
[EDIT]
the 4 UPDATE statements are executed parallel from different processes.
[EDIT] with Postgresql
One of these statements will lock the record (or the page, or the whole table, depending on your engine and locking granularity) and will be executed.
The others will wait for the resource to be freed.
When the lucky statement will commit, the others will either reread the table and do nothing (if your transaction isolation mode is set to READ COMMITTED) or fail to serialize the transaction (if the transaction isolation level is SERIALIZABLE).
If you ran these UPDATEs in one go, it will run them in order, so it would update everything to b1 but then the other 3 would fail to update any as there will be no A's left to update.
There would be an implicit transaction around each which would hold other UPDATEs in a queue. Only the first one through would win in this case as each subsequent update will not see a name called 'a'.
Edit: I was assuming here that you were calling each UPDATE from separate processes. If they were called in a single script, they would be run consecutively in the order of appearence.
There is only ever one UPDATE statement that can access a record. Before it runs, it starts a transaction and locks the table (or more correctly, the page the record is on, or even only the record itself - this depends on many factors). Then it makes the change and unlocks the table again, commiting the transaction in the process.
Updates/deletes/inserts to a certain record are single threaded by their very nature, it is required to ensure table integrity. This does not mean that updates to many records (that are on different pages) could not run in parallel.
With the statement you posted and you would end up with an error because after the first update 'a' can't be found. What are you trying to achieve with this?
Even if you don't specify begin transaction and commit transaction, a single SQL statement is always transactional. That means only one of the updates is allowed to modify the row at the same time. The other queries will block on the update lock owned by the first query.
I believe this may be what you are looking for (in T-SQL):
UPDATE [table]
SET [table].[name] = temp.new_name
FROM
[table],
(
SELECT
id,
new_name = 'B'+ cast(row_number() over (order by [name]) as varchar)
FROM
[table]
WHERE
[table].name = 'A'
) temp
WHERE [table].id = temp.id
There should be an equivalent function to row_number in the other SQL flavors.