I am getting deadlocks occasionally in sql server. I created a function for locking non database operations (credit card processing) so duplicates cannot happen. My functions are as follows (sorry for the tcl, but the sql is clear enough). Can anyone see why a deadlock happens occasionally?????
proc ims_syn_lock_object { db object {timeout 30} {wait 1}} {
if {[catch {
while {true} {
am_dbtransaction begin $db
# read the object locks that aren't timed out
set result [am_db1cell $db "SELECT object from GranularLocks WITH (ROWLOCK,HOLDLOCK) where object = [ns_dbquotevalue $object] AND timeActionMade > DATEADD(second,-timeout, GETDATE())"]
# check to see if this object is locked and not timed out
if { [string equal "" $result] } {
break;
} else {
# another process has this object and it is not timed out.
# release the row lock
am_dbtransaction rollback $db
if { $wait } {
# sleep for between 400 and 800 miliseconds
sleep [expr [ns_rand 400] + 400]
} else {
# we aren't waiting on locked resources.
return 0;
}
}
}
# either the object lock has timed out, or the object isn't locked
# create the object lock.
ns_db dml $db "DELETE FROM GranularLocks WHERE object = [ns_dbquotevalue $object]"
ns_db dml $db "INSERT INTO GranularLocks(object,timeout) VALUES ([ns_dbquotevalue $object],[ns_dbquotevalue $timeout int])"
# releases the row lock and commits the transaction
am_dbtransaction commit $db
} errMsg]} {
ns_log Notice "Could not lock $object. $errMsg"
catch {
am_dbtransaction rollback $db
} errMsg
return 0
}
return 1
}
proc ims_syn_unlock_object {db object } {
#simply remove the objects lock
ns_db dml $db "DELETE FROM GranularLocks WHERE object = [ns_dbquotevalue $object]"
}
Try adding UPDLOCK to the 1st select to force an exclusive lock too
Try sp_getapplock which is provided for this kind of operation.
I'd prefer number 2, personally...
It would be usefull to have the deadlock graph.
SQL deadlocks happen not only because of the queries involves, but equaly important is the schema involved. For example you can get a Reader-Writer deadlocks with perfectly valid and 'correct' queries simply because the read and write choose different access paths to the data. I could see this happening in your case if an index on timeActionMade exists on GranularLocks that does not cover 'object' column. But again, the solution will depend on what the actual deadlock is on.
Related
I learned the example of apache Ignite. I just want ignite to help to solve distribute transactions. For example。 My account is in DB A, My wife account is in DB B. I want to transfer money to my wife. So the
transaction like this :
IgniteTransactions transactions = ignite.transactions();
p1.setSalary(500);
p2_1.setSalary(1500);
Transaction tx = transactions.txStart(TransactionConcurrency.PESSIMISTIC,TransactionIsolation.SERIALIZABLE);
try {
cache.put(1L, p1);
cache2.put(1L,p2_1);
tx.commit();
}catch(Exception e) {
tx.rollback();
}
But the cacheStore is like that :
public void write(Entry<? extends Long, ? extends Person> entry) throws CacheWriterException {
System.out.println(" +++++++++++ single wirte");
Long key = entry.getKey();
Person val = entry.getValue();
System.out.println(">>> Store write [key=" + key + ", val=" + val + ']');
try {
Connection conn = dataSource.getConnection();
int updated;
// Try update first. If it does not work, then try insert.
// Some databases would allow these to be done in one 'upsert' operation.
try (PreparedStatement st = conn.prepareStatement(
"update PERSON set orgId = ?, name = ?, salary=? where id = ?")) {
st.setLong(1, val.getOrgId());
st.setString(2, val.getName());
st.setLong(3, val.getSalary());
st.setLong(4, val.getId());
updated = st.executeUpdate();
}
// If update failed, try to insert.
if (updated == 0) {
try (PreparedStatement st = conn.prepareStatement(
"insert into PERSON (id, orgId,name, salary) values (?, ?, ?,?)")) {
st.setLong(1, val.getId());
st.setLong(2, val.getOrgId());
st.setString(3, val.getName());
st.setLong(4, val.getSalary());
st.executeUpdate();
}
}
}
catch (SQLException e) {
throw new CacheWriterException("Failed to write object [key=" + key + ", val=" + val + ']', e);
}
}
When the part one commit that the salary updated, the second part failed. part one can not rollback.
How could commit or rollback them simultaneously? does ignite guarantee this or you do it your self?
ps: why ignite said that : it accelerate the transaction? it seems that it only accelerate querys , not deleting or updating operations. because it simultaneously access database when the soft transaction memory happens.
Can somebody figure it out? I don't understand the principle of ignite.
Ignite does support transactions. But there are two things to consider:
You need to define your cache as TRANSACTIONAL. The default is ATOMIC, which does not support transactions
It does not currently support transactions using SQL. You need to use the key-value API
I’m not sure where you’ve seen it said that Ignite is faster for transactions, but the general principle is that by keeping everything in memory, Ignite can be a lot quicker than legacy databases.
Apache Ignite expects that Cache Store does not fail. In your case, the upsert is very fragile and will fail.
At the very least, transactional operations imply transactional cache stores. You need to observe transaction in your cache store, only COMMIT; when told to.
To handle concurrency in my database:
Client A updates a row
Client B tries to update the same row
Client B needs to wait for Client A to commit his updates
Both Client A & B instance are simulated and using this code:
using (myEntities db = new myEntities ())
{
db.Database.Connection.Open();
try
{
using (var scope = db .Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
{
{
var test = db.customer_table.Where(x => x.id == 38).FirstOrDefault();
test.bank_holder_name = "CLIENT NAME XXXX";
db.SaveChanges(); <=== CLIENT B stop here while client A still in progress. After CLIENT A finish commit, here will throw *Deadlock found error*"
scope.Commit();
}
}
}
catch (Exception ex)
{
throw;
}
}
This is not what I expected where Client B should wait and not allowed to query any data about row id=38, but somehow it can proceed until SaveChanges and throws an error in the end.
Thus, I suspected this might caused by linq (incorrect row/ table lock)
I edited my code as below:
using (myEntities db = new myEntities ())
{
db.Database.Connection.Open();
try
{
using (var scope = db .Database.BeginTransaction(System.Data.IsolationLevel.Serializable))
{
{
var test = db.Database.ExecuteSqlCommand("Update customer_table set bank_holder_name = 'CLIENT XXXXX' where pu_id = 38"); <===== Client B is stop here and proceed after Client A is completed
db.SaveChanges();
scope.Commit();
}
}
}
catch (Exception ex)
{
throw;
}
}
Finally, the transaction is working with code above (not linq function). This is so confusing, what linq have done in behind making Transaction working inconsistent behavior?
This is due to the EF code generating two SQL statements: a SELECT for the line:
var test = db.customer_table.Where(x => x.id == 38).FirstOrDefault();
...and a subsequent UPDATE for the SaveChanges() call.
With a serializable isolation level both client A and client B take a shared lock for the duration of the transaction on the record when the SELECT statement is run. Then when one or other of them first tries to perform the UPDATE they cannot get the requisite exclusive lock because the other client has a shared lock on it. The other client itself then tries to obtain an exclusive lock and you have a deadlock scenario.
The ExecuteSqlCommand only requires a single update statement and thus a deadlock does not occur.
The Serializable isolation level can massively reduce concurrency and this example shows exactly why. You'll find that less stringent isolation levels will allow the EF code to work, but at the risk of phantom records, non-repeatable reads etc. These may well however be risks you are willing to take and/or mitigate against in order to improve concurrency.
Don't fetch the entity first. Instead create a "stub entity" and update that, eg
var test = new Customer() { id = 38 };
test.bank_holder_name = "CLIENT NAME XXXX";
db.Entry(test).Property(nameof(Customer.bank_holder_name)).IsModified = true;
db.SaveChanges();
Which translates to
SET NOCOUNT ON;
UPDATE [Customers] SET [bank_holder_name] = #p0
WHERE [id] = #p1;
SELECT ##ROWCOUNT;
I have a PDO SQL script which enables a user to complete a form which captures band information. It then posts this information to my database table called 'bands'. This works fine.
Simultaneously, I would like the script to update a different table called 'users' which has a column called 'num_bands' which needs to increase by a value of +1 if the user creates more than one band.
I have tried a number of methods, but none of them work. The script seems to be able to INSERT to the 'bands' table perfectly, but I cannot UPDATE the 'users' table. Here is the 'register_band' script:
<?php
// First we execute our common code to connection to the database and start the session
require("common.php");
// At the top of the page we check to see whether the user is logged in or not
if(empty($_SESSION['user']))
{
// If they are not, we redirect them to the login page.
header("Location: ../index.php");
// Remember that this die statement is absolutely critical. Without it,
// people can view your members-only content without logging in.
die("Redirecting to ../index.php");
}
// This if statement checks to determine whether the registration form has been submitted
// If it has, then the registration code is run, otherwise the form is displayed
if(!empty($_POST))
{
// Ensure that the user has entered a non-empty username
if(empty($_POST['username']))
{
// Note that die() is generally a terrible way of handling user errors
// like this. It is much better to display the error with the form
// and allow the user to correct their mistake. However, that is an
// exercise for you to implement yourself.
die("Please enter a username.");
}
// An INSERT query is used to add new rows to a database table.
// Again, we are using special tokens (technically called parameters) to
// protect against SQL injection attacks.
$query = "
INSERT INTO bands (
member_id,
username,
bandname,
bandhometown,
bandtype
) VALUES (
:member_id,
:username,
:bandname,
:bandhometown,
:bandtype
)
";
// Here we prepare our tokens for insertion into the SQL query. We do not
// store the original password; only the hashed version of it. We do store
// the salt (in its plaintext form; this is not a security risk).
$query_params = array(
':member_id' => $_POST['member_id'],
':username' => $_POST['username'],
':bandname' => $_POST['bandname'],
':bandhometown' => $_POST['bandhometown'],
':bandtype' => $_POST['bandtype']
);
try
{
// Execute the query to create the user
$stmt = $db->prepare($query);
$result = $stmt->execute($query_params);
}
catch(PDOException $ex)
{
// Note: On a production website, you should not output $ex->getMessage().
// It may provide an attacker with helpful information about your code.
die("Failed to run query: " . $ex->getMessage());
}
$query2 = "UPDATE users
SET num_bands = num_bands + 1
WHERE id = :member_id";
$stmt2 = $db->prepare($query2);
// This redirects the user to the private page after they register
header("Location: ../gig_view.php");
// Calling die or exit after performing a redirect using the header function
// is critical. The rest of your PHP script will continue to execute and
// will be sent to the user if you do not die or exit.
die("Redirecting to ../gig_view.php");
}
?>
I'm running this in non-production mode at the moment, so the code is not 100%. How do I get the script to UPDATE the 'users' table?
$stmt->closeCursor();
$query2 = "UPDATE users
SET num_bands = num_bands + 1
WHERE id = :member_id";
$stmt2 = $db->prepare($query2);
$params = array(':member_id' => $_POST['member_id']);
$result = $stmt2->execute($params);
The code you have here is well documented, and explains how to use PDO statements, prepared queries and how to execute them with parameters.
Just follow the same pattern as you did with your SELECT, only the string of the query is meant to change here.
Sorry for my English first of all. I have a problem and need help.
I have a simple tool made by myself on c#. This tool makes connect to local or remote firebird server (v.2.5). And my tool can create specified .fdb file (database) somewhere on the server.
Also I have a file with SQL statements (create table, triggers and so on). I want to execute this file after database was created. Executing this file will fill structure of user database - not data, only structure.
But then I try to execute my SQL script - firebird server returns a
SQL error code = -104 Token unknown line xxx column xxx.
That's the line on this CREATE TABLE SQL statement, for example:
CREATE TABLE tb1
(
col1 INTEGER NOT NULL,
col2 VARCHAR(36)
);
/* This next create statement causes an error */
CREATE TABLE tb2
(
col1 INTEGER NOT NULL,
col2 VARCHAR(36)
);
If I will leave only one create statement in my file - all will be good... I don't know how I explained (it's clear or not)) - another words - why can't I execute full query with many create statements in one transaction? There is my main method which executes query:
public static string Do(string conString, string query)
{
using (FbConnection conn = new FbConnection())
{
try
{
conn.ConnectionString = conString;
conn.Open();
FbTransaction trans = conn.BeginTransaction();
FbCommand cmd = new FbCommand(query, conn, trans);
cmd.ExecuteNonQuery();
trans.Commit();
}
catch (Exception ex)
{
System.Windows.MessageBox.Show(ex.ToString());
return "Transaction Fail";
}
}
return "Transaction Commited";
}
There is a query is my SQL file.
As Victor already stated in his final comment, you can use the FBScript class for batch execution.
I was just confronted with the same task. This question pointed me in the right direction but i had to do some further digging.
I this example, the source of the statements is a external script file:
private void ExecuteScript(FbConnection myConnection, string scriptPath) {
if (!File.Exists(scriptPath))
throw new FileNotFoundException("Script not found", scriptPath);
FileInfo file = new FileInfo(scriptPath);
string script = file.OpenText().ReadToEnd();
// use FbScript to parse all statements
FbScript fbs = new FbScript(script);
fbs.Parse();
// execute all statements
FbBatchExecution fbe = new FbBatchExecution(myConnection, fbs);
fbe.Execute(true);
}
This will work fine, but you may wonder why this whole thing isn't surrounded by a transaction. Actually there is no support to "bind" FbBatchExecution to a transaction directly.
The first thing i tried was this (will not work)
private void ExecuteScript(FbConnection myConnection, string scriptPath) {
using (FbTransaction myTransaction = myConnection.BeginTransaction()) {
if (!File.Exists(scriptPath))
throw new FileNotFoundException("Script not found", scriptPath);
FileInfo file = new FileInfo(scriptPath);
string script = file.OpenText().ReadToEnd();
// use FbScript to parse all statements
FbScript fbs = new FbScript(script);
fbs.Parse();
// execute all statements
FbBatchExecution fbe = new FbBatchExecution(myConnection, fbs);
fbe.Execute(true);
myTransaction.Commit();
}
}
This will result in an exception stating: "Execute requires the Command object to have a Transaction object when the Connection object assigned to the command is in a pending local transaction. The Transaction property of the Command has not been initialized."
This means nothing more than that the commands that are executed by FbBatchExecution are not assigned to our local transaction that is surrounding the code block. What helps here is that that FbBatchExecution provides
the event CommandExecuting where we can intercept every command and assign our local transaction like this:
private void ExecuteScript(FbConnection myConnection, string scriptPath) {
using (FbTransaction myTransaction = myConnection.BeginTransaction()) {
if (!File.Exists(scriptPath))
throw new FileNotFoundException("Script not found", scriptPath);
FileInfo file = new FileInfo(scriptPath);
string script = file.OpenText().ReadToEnd();
// use FbScript to parse all statements
FbScript fbs = new FbScript(script);
fbs.Parse();
// execute all statements
FbBatchExecution fbe = new FbBatchExecution(myConnection, fbs);
fbe.CommandExecuting += delegate(object sender, CommandExecutingEventArgs args) {
args.SqlCommand.Transaction = myTransaction;
};
fbe.Execute(true);
// myTransaction.Commit();
}
}
Note that i have uncommented the myTransaction.Commit() line. I was a little bit surprised by this behavior, but if you keep that line the transaction will throw an exception stating that it has already been committed. The bool parameter fbe.Execute(true) is named "autoCommit", but changing this to false seems to have no effect.
I would like some feedback if you see any potential issues with assigning the local transaction this way, or if it has any benefits at all or could as well be omitted.
Probably error in launching two create statements in one batch. Would it work if you break it to separate queries? Does it work in your SQL tool?
I'm making a script that goes through a table that contains all the other table names on the database. As it parses each row, it checks to see if the table is empty by
select count(*) cnt from $table_name
Some tables don't exist in the schema anymore and if I do that
select count(*)
directly into the command prompt, it returns the error:
206: The specified table (adm_rpt_rec) is not in the database.
When I run it from inside Perl, it appends this to the beginning:
DBD::Informix::db prepare failed: SQL: -
How can I avoid the program quitting when it tries to prepare this SQL statement?
One option is not to use RaiseError => 1 when constructing $dbh. The other is to wrap the prepare in an eval block.
Just put the calls that may fail in an eval block like this:
for my $table (#tables) {
my $count;
eval {
($count) = $dbi->selectrow_array("select count(*) from $table");
1; #this is here so the block returns true if it succeeds
} or do {
warn $#;
next;
}
print "$table has $count rows\n";
}
Although, in this case, since you are using Informix, you have a much better option: the system catalog tables. Informix keeps metadata like this in a set of system catalog tables. In this case you want systables:
my $sth = $dbh->prepare("select nrows from systables where tabname = ?");
for my $table (#tables) {
$sth->execute($table);
my ($count) = $sth->fetchrow_array;
$sth->finish;
unless (defined $count) {
print "$table does not exist\n";
next;
}
print "$table has $count rows\n";
}
This is faster and safer than count(*) against the table. Full documentation of the system catalog tables can be found in IBM Informix Guide to SQL (warning this is a PDF).
Working code - assuming you have a 'stores' database.
#!/bin/perl -w
use strict;
use DBI;
my $dbh = DBI->connect('dbi:Informix:stores','','',
{RaiseError=>0,PrintError=>1}) or die;
$dbh->do("create temp table tlist(tname varchar(128) not null) with no log");
$dbh->do("insert into tlist values('systables')");
$dbh->do("insert into tlist values('syzygy')");
my $sth = $dbh->prepare("select tname from tlist");
$sth->execute;
while (my($tabname) = $sth->fetchrow_array)
{
my $sql = "select count(*) cnt from $tabname";
my $st2 = $dbh->prepare($sql);
if ($st2)
{
$st2->execute;
if (my($num) = $st2->fetchrow_array)
{
print "$tabname: $num\n";
}
else
{
print "$tabname: error - missing?\n";
}
}
}
$sth->finish;
$dbh->disconnect;
print "Done - finished under control.\n";
Output from running the code above.
systables: 72
DBD::Informix::db prepare failed: SQL: -206: The specified table (syzygy) is not in the database.
ISAM: -111: ISAM error: no record found. at xx.pl line 14.
Done - finished under control.
This printed the error (PrintError=>1), but continued. Change the 1 to 0 and no error appears. The parentheses in the declarations of $tabname and $num are crucial - array context vs scalar context.