I have a stored procedure that is called from business code. This code uses parallelism, so multiple instances of this SP could be running at the same time depending on some conditions.
There is some logic in this SP that I want to execute only once. I have a table (let's call it HISTORY) that holds a UID for the run and a DATETIME when this portion of the code is executed. Here's my flow:
SP BEGIN
-- some logic
IF certain conditions are met, check if HISTORY does not have an entry for the UID
1. Add an entry in HISTORY for the current UID
2. Run the once only code
SP END
The issue is that, at times, the logic above still gets executed multiple times if different instances reach that part at the same time. What can I do to ensure that it only runs once?
Thank you!
BEGIN TRANSACTION;
INSERT [HISTORY](UID, ...)
SELECT #UID, ...
WHERE NOT EXISTS (
SELECT * FROM [HISTORY] WITH (HOLDLOCK) WHERE UID = #UID
);
IF ##ROWCOUNT = 1 BEGIN;
-- we inserted, do logic that should run only once
END;
COMMIT;
HOLDLOCK (equivalent to running the transaction under SERIALIZABLE, but more granular) ensures that no other transaction running in parallel can insert an entry in HISTORY for that UID; any transaction that tries so will block until the first INSERT is finished and then return (since a row already exists). Ensure that an index on UID exists, otherwise it will lock a lot more than is healthy for performance.
Getting code like this right is always tricky, so make sure to test it in practice as well by stress-testing concurrent inserts for the same (and different) UID.
Related
I have a system that's set up as a series of jobs and tasks. Each job is made up of several tasks, and there's another task_progress table that does nothing but record the current state of each task. (This information is kept separate from the main tasks table for business reasons that are not relevant to this issue.)
The jobs table has an overall status column, which needs to get updated to completed when all of the job's tasks reach a terminal state (ok or error). This is handled by a trigger function:
CREATE OR REPLACE FUNCTION update_job_status_when_progress_changes()
RETURNS trigger AS $$
DECLARE
current_job_id jobs.id%TYPE;
pending integer;
BEGIN
SELECT tasks.job_id INTO current_job_id
FROM tasks WHERE tasks.id = NEW.task_id;
SELECT COUNT(*) INTO pending
FROM task_progress
JOIN tasks ON task_progress.task_id = tasks.id
WHERE tasks.job_id = current_job_id
AND task_progress.status NOT IN ('ok', 'error');
IF pending = 0 THEN
UPDATE jobs
SET status = 'completed', updated_at = NOW() AT TIME ZONE 'utc'
WHERE jobs.id = current_job_id;
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql
CREATE TRIGGER task_progress_update_job_status
AFTER UPDATE OR DELETE
ON task_progress
FOR EACH ROW
EXECUTE PROCEDURE update_job_status_when_progress_changes()
It's been almost entirely fine. But sometimes – like, maybe once every few hundred jobs – a job will fail to flip over to completed status. The progress rows are all correct; the business logic that displays a % complete based on the contents of the task_progress table hits 100%, but the status of the job stays at processing. We've been unable to reproduce it reliably; it's just something that happens now and then. But it's frustrating, and I'd like to nail it down.
There are no transactions involved; each task progress is updated atomically by the process that completes the task.
Is it possible to hit a situation where, e.g., the last two tasks in a job complete almost simultaneously, causing the trigger for Task A to see that Task B is still pending, and vice versa? I thought FOR EACH ROW was supposed to prevent race conditions like this, but I can't explain what I'm seeing otherwise.
What's my best option here?
Yes, there is a race condition. If the last two tasks complete at about the same time, the trigger functions can run concurrently. Since the trigger runs as part of the transaction, and the transactions are both not committed yet, none of the trigger functions can see the data modifications made by the other transaction. So each believes there is still a task open.
You could use an advisory lock to make sure that that cannot happen: right before the SELECT count(*) ..., add
SELECT pg_advisory_xact_lock(42);
That makes sure that no session will execute the query while another session that has already executed the query is still not committed, because the lock is held until the end of the transaction.
SQL Table contains a table with a primary key with two columns: ID and Version. When the application attempts to save to the database it uses a command that looks similar to this:
BEGIN TRANSACTION
INSERT INTO [dbo].[FirstTable] ([ID],[VERSION]) VALUES (41,19)
INSERT INTO [dbo].[SecondTable] ([ID],[VERSION]) VALUES (41,19)
COMMIT TRANSACTION
SecondTable has a foreign key constraint to FirstTable and matching column names.
If two computers execute this statement at the same time, does the first computer lock FirstTable and the second computer waits, then when it is finished waiting it finds that the first insert statement fails and throws an error and does not execute the second statement? Is it possible for the second insert to run successfully on both computers?
What is the safest way to ensure that the second computer does not write anything and returns an error to the calling application?
Since these are transactions, all operations must be successful otherwise everything gets rolled back. Whichever computer completes the first statement of the transaction first will ultimately complete its transaction. The other computer will attempt to insert a duplicate primary key and fail, causing its transaction to roll back.
Ultimately there will be one row added to both tables, via only one of the computers.
Ofcourse if there are 2 statements then one of them is going to be executed and the other one will throw an error . In order to avoid this your best bet would be to use either "if exists" or "if not exists"
What this does is basically checks to see if there is data already present in the table if not then you insert , else just select.
Take a look at the flow shown below :
if not exists (select * from Table with (updlock, rowlock, holdlock) where
...)
/* insert */
else
/* update */
commit /* locks are released here */
The transaction did not rollback automatically if an error was encountered. It needed to have
set xact_abort on
Found this in this question
SQL Server - transactions roll back on error?
I have a table like this:
CREATE TABLE Tasks(
Name text,
Date datetime
)
In my application each person will add his/her tasks to this table.
Every night my robot will pick the first task to do, so it calls following Storedprocedure:
CREATE PROCEDURE PickTask
begin
select top (1) * from Tasks
delete top (1) from Tasks
end
The robot will call PickTask until there is no rows in the Tasks table.
My robot works multithread, so I want to know what will happened if two or more thread in my application wants to call PickTask ?
at the first I thought the select query will execute for both thread, so both threads pick a same row from Tasks, after that each one will delete one row, finally the robot do one task for two times and delete one unfinished task!!
I tried to use TRANC but I'm not sure, does my application have problem to do Tasks or not?
if you change it to:
CREATE PROCEDURE PickTask
begin transaction
select top (1) * from Tasks
delete top (1) from Tasks
commit
You can be sure that whichever program invokes this first will lock tasks until both steps are done. Then the next program that next invokes picktask will lock tasks exclusively. You probably want to wrap the whole thing in a begin try/begin catch block.
You can use application locks in your T-SQL, as noted in this link: http://www.sqlteam.com/article/application-locks-or-mutexes-in-sql-server-2005
You would acquire the lock in your stored procedure, just before the SELECT statement, then release that lock as soon as you delete the row, although you would be better off deleting the row with a WHERE clause that uses the unique key you obtained from that SELECT, since any additions that occurred (though unlikely) between the SELECT and DELETE could be deleted and never selected.
I'm curious about this. What happens if the DBA (or a grantor) revokes a user's (grantee's) privileges while that user (grantee) has an ongoing transaction that needs those privileges (obviously, when the user (grantee) issued his transaction, he had those privileges).
A trivial scenario to make this more concrete: user A grants user B privileges to insert data into a table (say Table1) in his (user A's) schema. User B goes ahead and issues a transaction that makes a lot of insertions. While the insertions are going on, user A revokes B's insert privileges. Does:
1. user B's transaction fail midway with an automatic rollback performed on Table1 (I'm assuming this is an example of an "abnormally terminated" process -- one of the conditions for a rollback; is it?)? If yes, how is data integrity handled? For example, does Oracle guarantee that the insertions won't be halted in the middle of a row?
2. user B's transaction complete before his privileges are taken away (meaning he can't make any more insertions after this)?
Logically, I'd guess it'd be 2 but I can't find any information to confirm this in Oracle docs.
Thanks.
EDIT: Adding some PL/SQL code I wrote to test this (I just cobbled this together for testing this scenario. I don't know PL/SQL so please correct me if I'm wrong).
/* Start by testing select privileges */
set serveroutput on;
declare v_emp HR.tempemp%rowtype;
begin
dbms_output.enable(300000);
for count_var in 1..200000 loop -- loop a large number of times, and BEFORE the loop exits,
-- revoke the permissions manually from the grantor's session
select * into v_emp
from HR.tempemp
where employee_id = 100;
dbms_output.put_line(count_var||' '||v_emp.employee_id); -- print count_var so we know which iteration we're in
end loop;
end;
/* Now test insert privileges */
set serveroutput on;
begin
dbms_output.enable(300000);
for count_var in 1..20000000 loop -- loop a large number of times, and BEFORE the loop exits,
-- revoke the permissions manually from the grantor's session
insert into HR.tempemp
values(100);
dbms_output.put_line(count_var); -- print count_var so we know which iteration we're in
end loop;
end;
Observations:
1. The for loop falls through as soon as the revoke takes effect. So, as Dave said, privileges are checked each time a statement is executed.
2. For the insert part of the test, once the for loop fails midway, none of the insertions (not even the ones for which the for went through are visible in the grantee's session).
I just tested this with the following scenario, here is what happened:
1-User A creates a table.
2-User A grants user B to insert.
3-User B inserts a row, but does not COMMIT.
4-User A revokes from user B.
5-User B inserts a row, but fails.
5-User B commits successfully.
This is pretty easy to test: Create empty table in schema A. Grant insert privilege to schema B. In schema B, start a long-running INSERT statement. While it's running, revoke the insert privilege.
If you do this, you will see that the insert continues running and completes successfully. Then, if you immediately try to execute it again you will get ORA-01031: insufficient privileges. So it seems clear that Oracle checks the privileges once for each statement execution. I glanced through some documentation and didn't see anything that stated this outright, but it seems like the most logical approach and the experiment supports it.
You asked:
"does Oracle guarantee that the insertions won't be halted in the middle of a row?"
As shown above, this isn't really relevant in the case of revocation of privilege; but it seems worth explaining more generally how Oracle behaves if an error occurs in the middle of processing a statement. There is no possibility, excluding bugs in Oracle, that a partial row would be inserted and left behind when an error occurred. If any error happens in the middle of processing a single SQL statement, then the changes made so far by that statement (not transaction) are rolled back internally by Oracle. For instance, if you are inserting many rows and the data segment needs to be extended but has no space available, the work done so far by the current statement would be rolled back and then an error would be returned to the code that executed that statement. This is not an "abnormally terminated process" as discussed in the other thread you referenced; the process continues running and determines how to deal with the error -- it has the option to rollback the entire transaction but it is not obliged to do so.
I'm working on a web app connected to oracle. We have a table in oracle with a column "activated". Only one row can have this column set to 1 at any one time. To enforce this, we have been using SERIALIZED isolation level in Java, however we are running into the "cannot serialize transaction" error, and cannot work out why.
We were wondering if an isolation level of READ COMMITTED would do the job. So my question is this:
If we have a transaction which involves the following SQL:
SELECT *
FROM MODEL;
UPDATE MODEL
SET ACTIVATED = 0;
UPDATE MODEL
SET ACTIVATED = 1
WHERE RISK_MODEL_ID = ?;
COMMIT;
Given that it is possible for more than one of these transactions to be executing at the same time, would it be possible for more than one MODEL row to have the activated flag set to 1 ?
Any help would be appreciated.
your solution should work: your first update will lock the whole table. If another transaction is not finished, the update will wait. Your second update will guarantee that only one row will have the value 1 because you are locking the table (it doesn't prevent INSERT statements however).
You should also make sure that the row with the RISK_MODEL_ID exists (or you will have zero row with the value '1' at the end of your transaction).
To prevent concurrent INSERT statements, you would LOCK the table (in EXCLUSIVE MODE).
You could consider using a unique, function based index to let Oracle handle the constraint of only having a one row with activated flag set to 1.
CREATE UNIQUE INDEX MODEL_IX ON MODEL ( DECODE(ACTIVATED, 1, 1, NULL));
This would stop more than one row having the flag set to 1, but does not mean that there is always one row with the flag set to 1.
If what you want is to ensure that only one transaction can run at a time then you can use the FOR UPDATE syntax. As you have a single row which needs locking this is a very efficient approach.
declare
cursor c is
select activated
from model
where activated = 1
for update of activated;
r c%rowtype;
begin
open c;
-- this statement will fail if another transaction is running
fetch c in r;
....
update model
set activated = 0
where current of c;
update model
set activated = 1
where risk_model_id = ?;
close c;
commit;
end;
/
The commit frees the lock.
The default behaviour is to wait until the row is freed. Otherwise we can specify NOWAIT, in which case any other session attempting to update the current active row will fail immediately, or we can add a WAIT option with a polling time. NOWAIT is the option to choose to absolutely avoid the risk of hanging, and it also gives us the chance to inform the user that someone else is updating the table, which they might want to know.
This approach is much more scalable than updating all the rows in the table. Use a function-based index as WW showed to enforce the rule that only one row can have ACTIVATED=1.