I am inserting records through a query similar to this one:
insert into tbl_xyz select field1 from tbl_abc
Now I would like to retreive the newly generated IDENTITY Values of the inserted records. How do I do this with minimum amount of locking and maximum reliability?
You can get this information using the OUTPUT clause.
You can output your information to a temp target table or view.
Here's an example:
DECLARE #InsertedIDs TABLE (ID bigint)
INSERT into DestTable (col1, col2, col3, col4)
OUTPUT INSERTED.ID INTO #InsertedIDs
SELECT col1, col2, col3, col4 FROM SourceTable
You can then query the table InsertedIDs for your inserted IDs.
##IDENTITY will return you the last inserted IDENTITY value, so you have two possible problems
Beware of triggers executed when inserting into table_xyz as this may change the value of ##IDENTITY.
Does tbl_abc have more than one row. If so then ##IDENTITY will only return the identity value of the last row
Issue 1 can be resolved by using SCOPE__IDENTITY() instead of ##IDENTITY
Issue 2 is harder to resolve. Does field1 in tbl_abc define a unique record within tbl_xyz, if so you could reselect the data from table_xyz with the identity column. There are other solutions using CURSORS but these will be slow.
SELECT ##IDENTITY
This is how I've done it before. Not sure if this will meet the latter half of your post though.
EDIT
Found this link too, but not sure if it is the same...
How to insert multiple records and get the identity value?
As far as I know, you can't really do this with straight SQL in the same script. But you could create an INSERT trigger. Now, I hate triggers, but it's one way of doing it.
Depending on what you are trying to do, you might want to insert the rows into a temp table or table variable first, and deal with the result set that way. Hopefully, there is a unique column that you can link to.
You could also lock the table, get the max key, insert your rows, and then get your max key again and do a range.
Trigger:
--Use the Inserted table. This conaints all of the inserted rows.
SELECT * FROM Inserted
Temp Table:
insert field1, unique_col into #temp from tbl_abc
insert into tbl_xyz (field1, unique_col) select field1, unique_col from tbl_abc
--This could be an update, or a cursor, or whatever you want to do
SELECT * FROM tbl_xyz WHERE EXISTS (SELECT top 1 unique_col FROM #temp WHERE unique_col = tbl_xyz.unique_col)
Key Range:
Declare #minkey as int, #maxkey as int
BEGIN TRANS --You have to lock the table for this to work
--key is the name of your identity column
SELECT #minkey = MAX(key) FROM tbl_xyz
insert into tbl_xyz select field1 from tbl_abc
SELECT #maxkey = MAX(key) FROM tbl_xyz
COMMIT Trans
SELECT * FROM tbl_xyz WHERE key BETWEEN #minkey and #maxkey
Related
How do I return the row I just inserted including DB-generated identifier?
My SQL is just a standard dynamic SQL insert
insert into dbo.TableName (Col1, Col2) values (#Col1, #Col2);
I have tried select from inserted, but inserted object is not a known object
insert into dbo.TableName (Col1, Col2) values (#Col1, #Col2); select * from inserted;
I have tried using output, but I cannot do that, when there is a trigger on the table
insert into dbo.TableName (Col1, Col2) output inserted.* values (#Col1, #Col2);
Any ideas?
You could insert the row and then use SCOPE_IDENTITY() to get the ID of the row you inserted and return the row with that id from the table.
For example:
INSERT INTO tableA VALUES (a1, a2);
DECLARE #Id INT = SCOPE_IDENTITY();
SELECT * FROM tableA WHERE ID = #Id
You can use OUTPUT clause:
--Sample table
CREATE TABLE IdentityInsert
(
ID int IDENTITY,
A int
)
INSERT IdentityInsert OUTPUT inserted.* VALUES (3)
There is no way to ask SQL Server which row was inserted last unless you are doing so in the same batch as the insert. For example, if your table has an IDENTITY column, you can use SCOPE_IDENTITY() (never use ##IDENTITY, since that can be unreliable if you have or will ever add triggers to the source table):
INSERT dbo.table(column) SELECT 1;
SELECT SCOPE_IDENTITY();
More generally, you can use the OUTPUT clause, which doesn't rely on an IDENTITY column (but will still make it difficult to identify which row(s) the clause identifies if there is no PK):
INSERT dbo.table(column) OUTPUT inserted.* SELECT 1;
If you're not talking about the same batch, then the only real way to identify the last row inserted is to use a date/time column where the timestamp of insertion is recorded. Otherwise it is like you emptied a bag of marbles on the floor, then asked someone to enter the room and identify which one hit the floor last.
You may be tempted or even advised to use the IDENT_CURRENT() function, but I explain here why this is unreliable too.
You could add a column to track this going forward:
ALTER TABLE dbo.table ADD DateInserted DEFAULT CURRENT_TIMESTAMP;
Now you can find the last row(s) inserted by simply:
;WITH x AS (SELECT *, r = RANK() OVER (ORDER BY DateInserted DESC)
FROM dbo.table)
SELECT * FROM x WHERE r = 1;
(If you don't want ties, you can add a tie-breaking column to the ORDER BY, or you can simply change RANK() to ROW_NUMBER() if you don't care which of the tied rows you get.)
You might make the assumption that the last row inserted is the highest identity value, but this isn't necessarily the case. The identity can be reseeded and it can also be overridden using SET IDENTITY_INSERT ON;.
I need to get the maximum number in a column because in a insert operation I have to insert max number in that column +1 for each insert, I did this:
insert into table1(id ,.., field,...)
select newid(), ..., (SELECT CONVERT(VARCHAR(8), FORMAT((MAX(number)+1),'00000000'))
FROM table1)
It works just for the first inserted row, than I get this number for other rows too!
This is too long for a comment.
Why aren't you using an identity column? You can define the number as:
number int identity(1, 1)
If you want the value as a string padded to eight characters, then use a computed column:
number_string as (format(number, '00000000'))
EDIT:
There are strong reasons why you want an identity column and not to calculate the values yourself. You can do what you want using row_number(), where the logic looks like this:
insert into table1(id ,.., field,...)
select newid(), ...,
convert(varchar(8),
(coalesce(t1.max_number, 0) +
row_number() over (order by (select null))
)
)
from table2 t2 cross join
(select max(t1.number) as max_number from table1 t1) t1;
N
Note: I am assuming that the inserts are coming from a different table, but table2 can really be table1.
Very importantly: This is not thread safe. Two different threads can run the same code and result in the same values. The solution to this is locking the entire table. However, that can have very significant performance impacts.
There's a lot to comment on here, and like Gordon, I can't fit it into one.
Firstly, I notice you have a column id and you're inserted the value NEWID() into it. I therefore hope that id isn't your CLUSTERED INDEX, as if it is NEWID() is not doing it any favours. If you are using a uniqueindentifier for your CLUSTERED INDEX, use NEWSEQUENTIALID() instead, and don't provuide the value in the INSERT:
ALTER TABLE dbo.Table1 ADD CONSTRAINT DF_Table1_id DEFAULT NEWSEQUENTIALID() FOR id;
As for your INSERT, as I said in my comment: "Use an IDENTITY column. If you must then have sequential values, use a VIEW and ROW_NUMBER. Let SQL Server gracefully handle the incrementing value. Trying to increment the number yourself is going to only cause you problems, such as race conditions, and your data will be in a far worse position.". Unfortunately you can't change an existing column to an IDENTITY, so this is a little harder. Likely you'll want to do something like:
ALTER TABLE dbo.Table1 ADD number_new int IDENTITY(1,1);
GO
SET IDENTITY_INSERT dbo.Table1 ON;
--UPdate the new column with the existing values
UPDATE dbo.Table1
SET number_new = number;
SET IDENTITY_INSERT dbo.Table1 OFF;
GO
--Drop the old column and rename
ALTER TABLE dbo.Table1 DROP COLUMN number;
EXEC sp_rename N'dbo.Table1.number_new', N'number', N'COLUMN'
As Gordon said, if you simply then need a formatted value (with leading 0's) and no worry about gaps, use a computed column:
ALTER TABLE dbo.Table1 ADD Number_f AS RIGHT(CONCAT('00000000',number),8) PERSISTED;
If, however, you want them to be in sequential order, and update accordingly when a row is deleted, or a INSERT fails, etc, then you can use a view, with the following expression:
RIGHT(CONCAT('00000000',ROW_NUMBER() OVER (ORDER BY number ASC),8)
You can use a ROW_NUMBER() function to increment the IDs. Just replace the +1 with ROW_NUMBER() OVER(). Something like this:
insert into table1(id ,.., field,...)
select (SELECT MAX(number) FROM table1) + ROW_NUMBER() OVER(ORDER BY <field1>), ...
from table1
SQL Fiddle
I am trying to copy a record in a table and change a few values with a stored procedure in SQL Server 2005. This is simple, but I also need to copy relationships in other tables with the new primary keys. As this proc is being used to batch copy records, I've found it difficult to store some relationship between old keys and new keys.
Right now, I am grabbing new keys from the batch insert using OUTPUT INTO.
ex:
INSERT INTO table
(column1, column2,...)
OUTPUT INSERTED.PrimaryKey INTO #TableVariable
SELECT column1, column2,...
Is there a way like this to easily get the old keys inserted at the same time I am inserting new keys (to ensure I have paired up the proper corresponding keys)?
I know cursors are an option, but I have never used them and have only heard them referenced in a horror story fashion. I'd much prefer to use OUTPUT INTO, or something like it.
If you need to track both old and new keys in your temp table, you need to cheat and use MERGE:
Data setup:
create table T (
ID int IDENTITY(5,7) not null,
Col1 varchar(10) not null
);
go
insert into T (Col1) values ('abc'),('def');
And the replacement for your INSERT statement:
declare #TV table (
Old_ID int not null,
New_ID int not null
);
merge into T t1
using (select ID,Col1 from T) t2
on 1 = 0
when not matched then insert (Col1) values (t2.Col1)
output t2.ID,inserted.ID into #TV;
And (actually needs to be in the same batch so that you can access the table variable):
select * from T;
select * from #TV;
Produces:
ID Col1
5 abc
12 def
19 abc
26 def
Old_ID New_ID
5 19
12 26
The reason you have to do this is because of an irritating limitation on the OUTPUT clause when used with INSERT - you can only access the inserted table, not any of the tables that might be part of a SELECT.
Related - More explanation of the MERGE abuse
INSERT statements loading data into tables with an IDENTITY column are guaranteed to generate the values in the same order as the ORDER BY clause in the SELECT.
If you want the IDENTITY values to be assigned in a sequential fashion
that follows the ordering in the ORDER BY clause, create a table that
contains a column with the IDENTITY property and then run an INSERT ..
SELECT … ORDER BY query to populate this table.
From: The behavior of the IDENTITY function when used with SELECT INTO or INSERT .. SELECT queries that contain an ORDER BY clause
You can use this fact to match your old with your new identity values. First collect the list of primary keys that you intend to copy into a temporary table. You can also include your modified column values as well if needed:
select
PrimaryKey,
Col1
--Col2... etc
into #NewRecords
from Table
--where whatever...
Then do your INSERT with the OUTPUT clause to capture your new ids into the table variable:
declare #TableVariable table (
New_ID int not null
);
INSERT INTO #table
(Col1 /*,Col2... ect.*/)
OUTPUT INSERTED.PrimaryKey INTO #NewIds
SELECT Col1 /*,Col2... ect.*/
from #NewRecords
order by PrimaryKey
Because of the ORDER BY PrimaryKey statement, you will be guaranteed that your New_ID numbers will be generated in the same order as the PrimaryKey field of the copied records. Now you can match them up by row numbers ordered by the ID values. The following query would give you the parings:
select PrimaryKey, New_ID
from
(select PrimaryKey,
ROW_NUMBER() over (order by PrimaryKey) OldRow
from #NewRecords
) PrimaryKeys
join
(
select New_ID,
ROW_NUMBER() over (order by New_ID) NewRow
from #NewIds
) New_IDs
on OldRow = NewRow
One question about how to t-sql program the following query:
Table 1
I insert 400.000 mobilephonenumbers in a table with two columns. The number to insert and identity id.
Table 2
The second table is called SendList. It is a list with 3columns, a identity id, a List id, and a phonenumberid.
Table 3
Is called ListInfo and contains PK list id. and info about the list.
My question is how should I using T-sql:
Insert large list with phonenumbers to table 1, insert the generated id from the insert of phonenum. in table1, to table 2. AND in a optimized way. It cant take long time, that is my problem.
Greatly appreciated if someone could guide me on this one.
Thanks
Sebastian
What version of SQL Server are you using? If you are using 2008 you can use the OUTPUT clause to insert multiple records and output all the identity records to a table variable. Then you can use this to insert to the child tables.
DECLARE #MyTableVar table(MyID int);
INSERT MyTabLe (field1, field2)
OUTPUT INSERTED.MyID
INTO #MyTableVar
select Field1, Field2 from MyOtherTable where field3 = 'test'
--Display the result set of the table variable.
Insert MyChildTable (myID,field1, field2)
Select MyID, test, getdate() from #MyTableVar
I've not tried this directly with a bulk insert, but you could always bulkinsert to a staging table and then use the processs, described above. Inserting groups of records is much much faster than one at a time.
I have an web application that creates printable forms, these forms have a unique number on them, the problem is I have 2 forms that separate numbers need to be created for them.
ie)
Form1- Numbered 2000000-2999999
Form2- Numbered 3000000-3999999
dbo.test2 - is my form information table
Tsel - is my autoinc table for the 3000000 series numbers
Tadv - is my autoinc table for the 2000000 series numbers
What I have done is create 2 tables with just autoinc row (one for 2000000 series numbers and one for 3000000 series numbers), I then created a trigger to add a record to the coresponding table, read back the autoinc number and add it to my table that stores the form information including the just created autoinc number for the right series of forms.
Although it does work, I'm concerned that the numbers will get messed up under load.
I'm not sure the ##IDENTITY will always return the right value when many people are using the system. (I cannot have duplicates and I need to use the numbering form show above.
See code below.
**** TRIGGER ****
CREATE TRIGGER MAKEANID2 ON dbo.test2
AFTER INSERT
AS
SET NOCOUNT ON
declare #someid int
declare #someid2 int
declare #startfrom int
declare #test1 varchar(10)
select #someid=##IDENTITY
select #test1 = (Select name1 from test2 where sysid = #someid )
if #test1 = 'select'
begin
insert into Tsel Default values
select #someid2 = ##IDENTITY
end
if #test1 = 'adv'
begin
insert into Tadv Default values
select #someid2 = ##IDENTITY
end
update test2
set name2=(#someid2) where sysid = #someid
SET NOCOUNT OFF
The best way to keep the two IDs in sync is to create a persisted Computed Column based on the actual identity column. Where Col1 is the identity column and Col2 is the persisted computed column that is the result of some formula based on Col1. You can then even Create Indexes on Computed Columns.
test this out:
CREATE TABLE YourTable
(Col1 int not null identity(2000000,1)
,Col2 AS (Col1-2000000+3000000) PERSISTED
,Col3 varchar(5)
)
GO
insert into YourTable (col3) values ('a')
insert into YourTable (col3) SELECT 'b' UNION SELECT 'c'
SELECT * FROM YourTable
OUTPUT:
Col1 Col2 Col3
----------- ----------- -----
2000000 3000000 a
2000001 3000001 b
2000002 3000002 c
(3 row(s) affected)
EDIT After OPs comments, I'm still not 100% sure what you are after.
I never used SQL Server 2000 (we skipped that version), and I don't really want to look up how to do everything in that version, it is so limited without the OUTPUT clause and ROW_NUMBER(), CTEs, etc.
I can think of three methods to do:
1) You could just create a sequence table, where you have 2 rows one for A and one for B, each time you need to insert one, look up, increment, and save the value of the type of seq you need and then insert with that value. for example if you are inserting a type "A" row, do this:
INSERT INTO test2
(col1, col2, col3,...)
SELECT
ISNULL(MAX(NextSeq),0)+1, col2, col3,...
FROM YourSequenceTable WITH (UPDLOCK, HOLDLOCK)
WHERE SequenceType='A'
UPDATE YourSequenceTable
SET NextSeq=ISNULL(NextSeq,0)+1
WHERE SequenceType='A'
2) change your table structure to just save the data in Tsel or Tadv and have a trigger insert into a third common table table where you can have your additional "common" identity. common table would be like
CommonTable
ID int not null indentity(1,1) primary key
TselID int null FK to Tsel.PK
TadvID int null FK to Tadv.PK
3) if you need a single table, try this, which is a real hack. Change your Tsel and Tadv tables to contain all the necessary columns and from the application INSERT INTO Tsel when the value is select and have a trigger grab that identity value and then INSERT that into test2, then remove the data from tsel. Then, from the application when the value is adv just INSERT INTO Tadv an have a trigger on that table insert the data into test2, and remove the data from Tadv. You need to have all data columns in Tsel and Tadv so the trigger can copy the values to test2, but the trigger will remove the rows from there (the identity will be sequential even if the original rows are removed).
your Tsel trigger would look like:
CREATE Trigger MAKEANID2_Tsel ON dbo.Tsel
AFTER INSERT
AS
--copy data from Tsel into test2., test2 can still have its own identity value
INSERT INTO test2
(PK, col1, col2, col3,...)
SELECT
col0, col1, col2, col3,....
FROM INSERTED
--remove rows from Tsel, which were just copied and not needed anymore.
DELETE Tsel
WHERE PK IN (SELECT PK FROM INSERTED)
GO
YOu are right to worry about ##identity, it is not a recommended peice of code, if somone else adds a differnet trigger that inserets an identity and that one triggers first, that is the value you will get.
But you have much bigger problems. Your trigger is deisgned to work on only one record ata time. This is a very very very bad thing to do with a trigger. Triggers operate on sets of data and must ALWAYS even if you think therer will never be more than one record inserted ata time) be set up to handle sets of data not one record. Further, you don;t need to ask for the identity, you have the identities of all records inserted inteh batch in a psuedotable availlble in triggers called inserted.
Now reading one of your comments, you say you can't have any missing values at all. Inthat case you cannot under any circustance use an identity column as it will have gaps if any transaction is rolled back. You will have to write your own process to create the numbers based onteh last number and look out for race conditions.