I have a simple SQL insert statement of the form:
insert into MyTable (...) values (...)
It is used repeatedly to insert rows and usually works as expected. It inserts exactly 1 row to MyTable, which is also the value returned by the Delphi statement AffectedRows:= myInsertADOQuery.ExecSQL.
After some time there was a temporary network connectivity problem. As a result, other threads of the same application perceived EOleExceptions (Connection failure, -2147467259 = unspecified error). Later, the network connection was reestablished, these threads reconnected and were fine.
The thread responsible for executing the insert statement described above, however, did not perceive the connectivity problems (No exceptions) - probably it was simply not executed while the network was down. But after the network connectivity problems myInsertADOQuery.ExecSQL always returned 0 and no rows were inserted to MyTable anymore. After a restart of the application the insert statement worked again as expected.
For SQL Server, is there any defined case where an insert statment like the one above would not insert a row and return 0 as the number of affected rows? Primary key is an autogenerated GUID. There are no unique or check constraints (which should result in an exception anyway rather than not inserting a row).
Are there any known ADO bugs (Provider=SQLOLEDB.1)?
Any other explanations for this behaviour?
Thanks,
Nang.
If you does not have any exceptions, then:
When a table has triggers without SET NOCOUNT ON, then actually the operation (INSERT / UPDATE / DELETE) may be finished successfully, but a number of affected records may be returned as 0.
Depending on a transaction activity in current session, other sessions may not see changes made by current session. But current session will see own changes and a number of affected records will be (may be) not 0.
So, the exact answer may depend on your table DDL (+ triggers if any) and on how you are checking the inserted rows.
Looks like your Insert thread lost silently the connection and is not checking on it to do an auto reconnect if needed but keeps queuing the inserts without actually sending them.
I would isolate this code in a small standalone app to debug it and see how it behaves when you voluntarily disconnect the network then reconnect it.
I would not be surprised if you either found a "swallowed" exception, or some code omitting to check for success/failure.
Hope it helps...
If the values you're trying to insert are violating
a CHECK constraint
a FOREIGN KEY relationship
a NOT NULL constraint
a UNIQUE constraint
or any other constraints, then the row(s) will not be inserted.
Do you use transactions? Maybe your application has no autocommit? Some drivers do not commit data if there was error in transaction.
Related
I'm working on a big database and i'm seeking for everything that can speed up the database. The question is : when you have an unique index on somes fields, what is the faster between make a select request to check if it's ok, or try it anyway and catch the exception if the entry already exists ?
I made some researchs but nothing conclusive. Thanks.
A manual check won't do what you think it does. (See below.)
If you check first, every insert requires two round-trips to the database. It might also require serializable transactions.
And you have to trap errors anyway. A duplicate value is just one thing that can go wrong on an insert; there are a lot of other things that can go wrong.
I say just insert, and trap the errors.
The point of a SELECT before INSERT is to determine whether a value already exists in the database. But you can't rely on that to work. Here's why.
Open two terminal sessions (for example), and connect both to your database. This table already exists. It's empty.
create table test (
test_id serial primary key,
test_email varchar(15) not null unique
);
A: begin transaction;
A: select test_email
from test
where test_email = 'a#b.com';
(0 rows)
B: begin transaction;
A: insert into test (test_email)
values ('a#b.com');
INSERT 0 1
B: select test_email
from test
where test_email = 'a#b.com';
(0 rows)
B: insert into test (test_email)
values ('a#b.com');
(waiting for lock)
A: commit;
B: ERROR: duplicate key value
violates unique constraint...
you have 2 choices
Try to insert and if the query fails, you will catch and roll back transaction.
Try to query once to check if it exists or not. If it does not exist,Insert the value.
In my opinion, first one is better because you use network connection twice if you query twice. Also select is an good option when you have really big data.
In the first case you try to insert but you get DataIntegrityException. Single request and response is better than two request and two response.
Transaction Manager can handle the exception as well.
My understanding is try / catch exceptions will abruptly stop the flow of the program. Even when properly handled. The recommended practice is to use them apart from domain logic. An extra select shouldn't be that bad unless your database server is physically far away.
I'm running a stored procedure that selects values my temp table and inserts them into the database like so:
INSERT INTO emails (EmailAddress) (
SELECT
DISTINCT eit.EmailAddress
FROM #EmailInfoTemp eit
LEFT JOIN emails ea
ON eit.EmailAddress = ea.EmailAddress
WHERE ea.EmailAddressID IS NULL )
On rare cases(~ once every couple of hours on a server that handles thousands of requests a minute), I then receive a unique constraint error "Violation of UNIQUE KEY constraint�".. on an index on the EmailAddress column.
I can confirm that I am not passing in duplicate values. Even if I was, it should be caught by the DISTINCT.
-SQL Server 2008
-Stored proc + not using transactions + JDBC callablestatement
Could it happen that between the SELECT and the ensuing INSERT, there was another call to the same/different stored proc that completed an INSERT with similiar data? If so, what would be the best way to prevent that?
Some ideas: We have many duplicate instances of "clients" who communicate with this one SQL Server at once in production, so my first reaction was a concurrency issue, but I can't seem to replicate it myself. That's the best guess I had, but it's gone nowhere so far. This does not happen on our staging environment where the load is insignificant compared to the production environment. That was the main reason I started looking into concurrency issues.
The error is probably caused by two sessions executing an insert at the same time.
You can make your SQL code safer by using MERGE. As Aaron Bertrand's comment says (thanks!), you have to include a with (holdlock) hint to make merge really safe.
; merge emails e with (holdlock)
using #EmailInfoTemp eit
on e.EmailAddress = eit.EmailAddress
when not matched then insert
(EmailAddress) values (eit.EmailAddress)
The merge statement will take appropriate locks to ensure that no other session can sneak in between it's "not matched" check and the "insert".
If you can't use merge, you could solve the problem client-side. Make sure that no two inserts are running at the same time. This is typically easy to do with a mutex or other synchronization construct.
I want to run the following on a table of about 12 million records.
ALTER TABLE t1
ADD c1 int NULL;
ALTER TABLE t2
ADD c2 bit NOT NULL
DEFAULT(0);
I've done it in staging and the timing seemed fine, but before I do it in production, I wanted to know how locking works on the table during new column creation (especially when a default value is specified). So, does anyone know? Does the whole table get locked, or do the rows get locked one by one during default value insertion? Or does something different altogether happen?
Prior to SQL Server 11 (Denali) the add non-null column with default will run an update behind the scenes to populate the new default values. Thus it will lock the table for the duration of the 12 million rows update. In SQL Server 11 this is no longer the case, the column is added online and no update occurs, see Online non-NULL with values column add in SQL Server 11.
Both in SQL Server 11 and prior a Sch-M lock is acquired on the table to modify the definition (add the new column metadata). This lock is incompatible with any other possible access (including dirty reads). The difference is in the duration: prior to SQL Server 11 this lock will be hold for a size-of-data operation (update of 12M rows). In SQL Server 11 the lock is only held for a short brief. In the pre-SQL Server 11 update of the rows no row lock needs to be acquired because the Sch-M lock on the table guarantees that there cannot be any conflict on any individual row.
Yes, it will lock the table.
A table, as a whole, has a single schema (set of columns, with associated types). So, at a minimum, a schema lock would be required to update the definition of the table.
Try to think about how things would work contrariwise - if each row was updated individually, how would any parallel queries work (especially if they involved the new columns)?
And default values are only useful during INSERT and DDL statements - so if you specify a new default for 10,000,000 rows, that default value has to be applied to all of those rows.
Yes, it will lock.
DDL statements issue a Schema Lock (see this link) which will prevent access to the table until the operation completes.
There's not really a way around this, and it makes sense if you think about it. SQL needs to know how many fields are in a table, and during this operation some rows will have more fields than others.
The alternative is to make a new table with the correct fields, insert into, then rename the tables to swap them out.
I have not read how the lock mechanism works when adding a column, but I am almost 100% sure row by row is impossible.
Watch when you do these types of things in SQL Server Manager with drag and drop (I know you are not doing this here, but this is a public forum), as some changes are destructive (fortunately, SQL Server 2008, at least R2, is safer here as it tells you "no can do" rather than just do it).
You can run both column additions in a single statement, however, and reduce the churn.
I am not sure if the transaction log is what I need or what.
My first problem is that I have a stored procedure that inserts some rows. I was just checking out elmah and I see that some sql exceptions happens. They all are the same error(a PK constraint was violated).
Other than that elmah is not telling me much more. So I don't know what row caused this primary key constraint(I am guessing the same row was for some reason being added twice).
So I am not sure if the the transaction log would tell me what happened and what data was trying to be inserted. I can't recreate this error it always works for me.
My second problem is for some reason when my page loads up I have a row from that database that I don't think exists anymore(I have a hidden column with the PK in it.) When I try to find this primary key it does not exist in the database.
I am using ms sql 2005.
Thanks
I don't think transaction log will help you.
SQL 2 modes on how to insert data with uniqueness violation.
There is a setting : IGNORE_DUP_KEY. By default it is OFF. IF you turn it ON, SQL will ignire duplicate rows and your INSERT statement will succeed.
You can read about it here:
http://msdn.microsoft.com/en-us/library/ms175132.aspx
BTW, to view transaction log, you can use this command:
SELECT * FROM fn_dblog(null, null)
You can inspect the log with the (undocumented) function fn_dblog(), but it won't tell you anything in the case of duplicate key violation because the violation happens before the row is inserted, so no log record is generated. Is true though that you'll get other operations at the time of error and from those you can, possibly, recreate the actions that lead to the error condition. Note that if the database is in SIMPLE recovery model then the log gets reused and you likely lost track of anything that happened.
Have a look at this article How do checkpoints work and what gets logged for an example of fn_dblog() usage. Although is on a different topic, it shows how the function works.
If you can repeat the error I would suggest using SQL Server profiler so you can see exactly what is going on.
If you are using asp.net to load the page are you using any output caching or data caching that might be retaining the row that no longer exists in the db?
This question is very similar to SQL Server 2005: T-SQL to temporarily disable a trigger
However I do not want to disable all triggers and not even for a batch of commands, but just for one single INSERT.
I have to deal with a shop system where the original author put some application logic into a trigger (bad idea!). That application logic works fine as long as you don't try to insert data in another way than the original "administration frontend". My job is to write an "import from staging system" tool, so I have all data ready. When I try to insert it, the trigger overwrites the existing Product Code (not the IDENTITY numeric ID!) with a generated one. To generate the Code it uses the autogenerated ID of an insert to another table, so that I can't even work with the ##IDENTITY to find my just inserted column and UPDATE the inserted row with the actual Product Code.
Any way that I can go to avoid extremly awkward code (INSERT some random characters into the product name and then try to find the row with the random characters to update it).
So: Is there a way to disable triggers (even just one) for just one INSERT?
You may find this helpful:
Disabling a Trigger for a Specific SQL Statement or Session
But there is another problem that you may face as well.
If I understand the situation you are in correctly, your system by default inserts product code automatically(by generating the value).
Now you need to insert a product that was created by some staging system, and for that product its product code was created by the staging system and you want to insert it to the live system manually.
If you really have to do it you need to make sure that the codes generated by you live application in the future are not going to conflict with the code that you inserted manually - I assume they musty be unique.
Other approach is to allow the system to generate the new code and overwrite any corresponding data if needed.
You can disable triggers on a table using:
ALTER TABLE MyTable DISABLE TRIGGER ALL
But that would do it for all sessions, not just your current connection.. which is obviously a very bad thing to do :-)
The best way would be to alter the trigger itself so it makes the decision if it needs to run, whether that be with an "insert type" flag on the table or some other means if you are already storing a type of some sort.
Rather than disabling triggers can you not change the behaviour of the trigger. Add a new nullable column to the table in question called "insertedFromImport".
In the trigger change the code so that the offending bit of the trigger only runs on rows where "insertedFromImport" is null. When you insert your records set "insertedFromImport" to something non-null.
Disable the trigger, insert, commit.
SET IDENTITY_INSERT Test ON
GO
BEGIN TRAN
DISABLE TRIGGER trg_Test ON Test
INSERT INTO Test (MyId, MyField)
VALUES (999, 'foo')
ENABLE TRIGGER trg_Test ON Test
COMMIT TRAN
SET IDENTITY_INSERT Test OFF
GO
Can you check for SUSER_SNAME() and only run when in context of the administration frontend?
I see many things that could create a problem. First change the trigger to consider multiple record imports. That may probably fix your problem. DO not turn off the trigger as it is turned off for everyone not just you. If you must then put the database into single user user mode before you do it and do your task during off hours.
Next, do not under any circumstances ever use ##identity to get the value just inserted! USe scope_identity instead. ##identity will return the wrong value if there are triggers onthe table that also do inserts to other tables with identity fields. If you are using ##identity right now through your system (since we know your system has triggers), your abosolute first priority must be to immediately find and change all instances of ##identity in your code. You can have serious data integrity issues if you do not. This is a "stop all work until this is fixed" kind of problem.
As far as getting the information you just inserted back, consider creating a batchid as part of you insert and then adding a column called batchid (which is nullable so it won't affect other inserts)to the table. Then you can call back what you inserted by batchid.
If you insert using BULK INSERT, you can disable triggers just for the insert.
I'm pretty sure bulk insert will require a data file on the file system to import so you can't just use T-SQL.
To use BULK INSERT you need INSERT and ADMINISTRATOR BULK OPERATION permissions.
If you disable triggers or constraints, you'll also need ALTER TABLE permission.
If you are using windows authentication, your windows user will need read access from the file. if using Mixed Mode authentication, the SQl Server Service account needs read access from the file.
When importing using BULK IMPORT, triggers are disabled by default.
More information: http://msdn.microsoft.com/en-us/library/ms188365.aspx