First of all I would like to know what is the actual root cause of deadlocks in SQL Server 2005. Is it because when two processes access the same row in a table?
Anyways, consider two tables _Table_Now_ and _Table_History_ where both having the same structure.
Suppose there is one column called NAME.
So when one process tries to UPDATE a record with NAME='BLUE' in _Table_Now_, first, it need to put the present row with NAME='BLUE' into _Table_History_ then update
_Table_Now_, and also delete previously present row from _Table_History_.
Deadlock occurs while deleting. I do not understand why?
Please guide me!
deadlock basically mean when process A is dependent on process B and process B is dependent on process A, so A will just start\continue when B finishes and B will only start\continue when A finishes
what you may be experiencing are table (or row) lock, so SQL locks the row before updating the table to make sure no other process tries to access that row while it is doing the update.
Can you be more specific on how are you doing the insert\update\delete. You shouldnt have deadlocks in this scenario.
FYI, don't use with (NOLOCK). It will yes prevent from locking but it does so by telling SQL Server to read uncommitted data, and it can end up in data inconsistencies.
Deadlock occurs when Process A is waiting for Process B to release resources and Process B is waiting for Process A to release resources.
If I understand the order of Updates correctly, it is this:
1. Read a row in Table_Now
2. Update a row in Table_History
3. Update a row in Table_Now
4. Delete a row in Table_History.
This could be a risky order if you are using transactions or locks incorrectly.
To avoid deadlocks, for each process you should execute:
1. Begin Transaction (Preferably table lock)
2. Perform all the DB operations
3. Commit the transaction (or Rollback in case any problem occurs while DB update)
This will ensure each process to lock both the tables, perform all the operations and then exit.
If you are already using transactions, what scope and level are you using? If not, introduce transactions. It should solve the problem.
Related
I am developing a server application with Spring + Hibernate + SQL-Server and i recognized that all my transactions are blocking other transaction, even if other transactions do not touch the same tables / rows and there are no relationships between this tables.
Here is a screenshot:
Transaction Report
The screenshot shows that a delete statement on table A blocks a select statement on table B. But there is no relationship between the tables.
In my understanding a transaction should only lock a table or row and another transactions that will hit the locked table or row will be blocked.
But why are all transactions blocked?
Do i missunderstand anything?
Solution was found:
The process was a copy-process. At the beginning i opened a transaction for the whole process and copy n-objects in this process. When i need to do selects while copying the selects were blocked. So i decide to open a transaction everytime i copy one object and after that copying-process close the transaction for this one object. So selects are no longer blocked. This design is like the copy-process in Windows. If you decide to cancel the copy-process the already copied files will stay. Like in my case the already copied objects will stay cause the transaction was closed and committed for every object separatly.
While using the python connector for snowflake with queries of the form
UPDATE X.TABLEY SET STATUS = %(status)s, STATUS_DETAILS = %(status_details)s WHERE ID = %(entry_id)s
, sometimes I get the following message:
(snowflake.connector.errors.ProgrammingError) 000625 (57014): Statement 'X' has locked table 'XX' in transaction 1588294931722 and this lock has not yet been released.
and soon after that
Your statement X' was aborted because the number of waiters for this lock exceeds the 20 statements limit
This usually happens when multiple queries are trying to update a single table. What I don't understand is that when I see the query history in Snowflake, it says the query finished successfully (Succeded Status) but in reality, the Update never happened, because the table did not alter.
So according to https://community.snowflake.com/s/article/how-to-resolve-blocked-queries I used
SELECT SYSTEM$ABORT_TRANSACTION(<transaction_id>);
to release the lock, but still, nothing happened and even with the succeed status the query seems to not have executed at all. So my question is, how does this really work and how can a lock be released without losing the execution of the query (also, what happens to the other 20+ queries that are queued because of the lock, sometimes it seems that when the lock is released the next one takes the lock and have to be aborted as well).
I would appreciate it if you could help me. Thanks!
Not sure if Sergio got an answer to this. The problem in this case is not with the table. Based on my experience with snowflake below is my understanding.
In snowflake, every table operations also involves a change in the meta table which keeps track of micro partitions, min and max. This meta table supports only 20 concurrent DML statements by default. So if a table is continuously getting updated and getting hit at the same partition, there is a chance that this limit will exceed. In this case, we should look at redesigning the table updation/insertion logic. In one of our use cases, we increased the limit to 50 after speaking to snowflake support team
UPDATE, DELETE, and MERGE cannot run concurrently on a single table; they will be serialized as only one can take a lock on a table at at a time. Others will queue up in the "blocked" state until it is their turn to take the lock. There is a limit on the number of queries that can be waiting on a single lock.
If you see an update finish successfully but don't see the updated data in the table, then you are most likely not COMMITting your transactions. Make sure you run COMMIT after an update so that the new data is committed to the table and the lock is released.
Alternatively, you can make sure AUTOCOMMIT is enabled so that DML will commit automatically after completion. You can enable it with ALTER SESSION SET AUTOCOMMIT=TRUE; in any sessions that are going to run an UPDATE.
I see so much information about avoiding blocks. My situation is that I WANT blocks.
We have this table with which two separate processes will be communicating with each other. The processes will run at random times and will use this control table to understand if the other process is busy. Both processes can't be busy at the same time, hence the control table.
Each job, when run, will check the control table... and based on that data will decide whether it's OK to run, and if OK, will update the control table record.
The issue is that if both processes run at the same moment, it's not clear that they won't do the following undesired actions (in this exact order):
Proc A reads the control table (table says good to go)
Proc B reads the control table (table says good to go)
Proc A updates control table (table now says "Proc A is busy")
Proc B updates control table (table now says "Proc B is busy")
<- In that scenario, both processes think they successfully updated the control table and will start their main flow (which is NOT what we want)
What I want here is for Proc B to be BLOCKED from SELECTING (not just updating) from the control table. This way, if/when Proc B's select query finally works, it will see the updated 'busy' value, not the value that existed before being changed by Proc A.
We're using SQL Server 2008 R2 I believe. I checked out SERIALIZABLE isolation but it doesn't appear to be strong enough.
For what it's worth we're trying to accomplish this with JDBC... using conn.setTransactionIsolation(Connection.TRANSACTION_SERIALIZABLE);
Which we understand to be the highest level of isolation, but I can still run selects all day from another window.
I'm 100% sure this is nowhere near a unique problem.... does anyone have any suggestion for making this work?
Your approach can work, but there are a few things to consider:
You need to open a transaction in the very beginning (before the first read) and you must only commit it after you have finished your work.
If both A and B try to read/modify the same record, this will work out of the box, even with the default transaction isolation level (READ COMMITTED). Otherwise, you need to tell SQL Server to lock the whole table (using the TABLOCK hint).
In fact, you don't need the reads at all!
This is how it will work:
P1 P2
---------------------------------
BEGIN TRANS
BEGIN TRANS
WRITE (success)
WRITE (blocked)
do work |
. |
. |
COMMIT -> block released, WRITE finishes
do work
.
.
COMMIT
PS: Note, though, that SQL server supports application locks. Thus, if you just want to synchronize two processes, you don't need to "abuse" a table:
Implementing application locks within SQL Server (Distributed Locking Pattern)
PPS: For completeness, let me also answer the question in the title ("How to force SELECT blocking on SQL server?"): For this, you can use a combination of the HOLDLOCK and the XLOCK table hint (or TABLOCKX, if you want to exclusively lock the whole table).
If you need the read (because you want to some processing) I would do the following:
Set transaction isolation level serializable
begin transaction
select from tablea
update tablea
commit
I have an ETL process that is building dimension tables incrementally in RedShift. It performs actions in the following order:
Begins transaction
Creates a table staging_foo like foo
Copies data from external source into staging_foo
Performs mass insert/update/delete on foo so that it matches staging_foo
Drop staging_foo
Commit transaction
Individually this process works, but in order to achieve continuous streaming refreshes to foo and redundancy in the event of failure, I have several instances of the process running at the same time. And when that happens I occasionally get concurrent serialization errors. This is because both processes are replaying some of the same changes to foo from foo_staging in overlapping transactions.
What happens is that the first process creates the staging_foo table, and the second process is blocked when it attempts to create a table with the same name (this is what I want). When the first process commits its transaction (which can take several seconds) I find that the second process gets unblocked before the commit is complete. So it appears to be getting a snapshot of the foo table before the commit is in place, which causes the inserts/updates/deletes (some of which may be redundant) to fail.
I am theorizing based on the documentation http://docs.aws.amazon.com/redshift/latest/dg/c_serial_isolation.html where it says:
Concurrent transactions are invisible to each other; they cannot detect each other's changes. Each concurrent transaction will create a snapshot of the database at the beginning of the transaction. A database snapshot is created within a transaction on the first occurrence of most SELECT statements, DML commands such as COPY, DELETE, INSERT, UPDATE, and TRUNCATE, and the following DDL commands :
ALTER TABLE (to add or drop columns)
CREATE TABLE
DROP TABLE
TRUNCATE TABLE
The documentation quoted above is somewhat confusing to me because it first says a snapshot will be created at the beginning of a transaction, but subsequently says a snapshot will be created only at the first occurrence of some specific DML/DDL operations.
I do not want to do a deep copy where I replace foo instead of incrementally updating it. I have other processes that continually query this table so there is never a time when I can replace it without interruption. Another question asks a similar question for deep copy but it will not work for me: How can I ensure synchronous DDL operations on a table that is being replaced?
Is there a way for me to perform my operations in a way that I can avoid concurrent serialization errors? I need to ensure that read access is available for foo so I can't LOCK that table.
OK, Postgres (and therefore Redshift [more or less]) uses MVCC (Multi Version Concurrency Control) for transaction isolation instead of a db/table/row/page locking model (as seen in SQL Server, MySQL, etc.). Simplistically every transaction operates on the data as it existed when the transaction started.
So your comment "I have several instances of the process running at the same time" explains the problem. If Process 2 starts while Process 1 is running then Process 2 has no visibility of the results from Process 1.
Using Sql Server 2005. I have a long running update that may take about 60 seconds in our production environment. The update is not part of any explicit transactions nor has any sql hints. While the update is running, what's to be expected from other requests that occur on those rows that will be updated? There's about 6 million total rows in the table that will be updated of which about 500,000 rows will be updated.
Some concurrency concerns/questions:
1) What if another select query (with nolock hint) is performed on this table among some of the rows that are being updated. Will the query wait until the update is finished?
2) What the other select query does not have a nolock hint? Will this query have to wait until the update is finished?
3) What if another update query is performing an update on one of these rows? Will this query have to wait until it's finished?
4) What about deletes?
5) What about inserts?
Thanks!
Dave
Every statement in sql server runs in a transaction. If you don't explicitly start one, the server starts one for every statement and commits it if the statement is successful, and rolls it back if it is not.
The exact locking you'll see with your update, unfortunately, depends. It will start off as row locks, but it is likely that it will be elevated to at least a few page locks based on the number of rows you're updating. Full elevation to a table lock is unlikely, but this depends in some amount on your server - SQL Server will elevate it if the page locks are using too much memory.
If your select is ran with nolock then you will get dirty reads if you happen to select any rows which are involved in the update. This means you will read the uncommitted data, and it may not be consistent with other rows (since those may not have been updated yet).
For all other cases if your statement encounters a row involved in the update, or a row on a locked page (assuming the lock has been elevated) then the statement will have to wait for the update to finish.