I would like to be able to pull the custom key value from a table, but would also like it to perform like SQL Server's IDENTITY(1,1) column on inserts.
The custom key is for another application and will need to be used by different functions so the value will need to be pulled from a table and available for other areas.
Here are some if my attempts:
Tried a trigger on the table works well on single inserts, failed on using SQL insert (forgetting the fact that a triggers are not per row but by batch)
ALTER TRIGGER [sales].[trg_NextInvoiceDocNo]
ON [sales].[Invoice]
AFTER INSERT
AS
BEGIN
DECLARE #ResultVar VARCHAR(25)
DECLARE #Key VARCHAR(25)
EXEC [dbo].[usp_GetNextKeyCounterChar]
#tcForTbl = 'docNbr', #tcForGrp = 'docNbr', #NewKey = #ResultVar OUTPUT
UPDATE sales.InvoiceRET
SET DocNbr = #ResultVar
FROM sales.InvoiceRET
JOIN inserted ON inserted.id = sales.InvoiceRET.id;
END;
Thought about a scalar function, but functions cannot exec stored procedures or update statements in order to set the new key value in the lookup table.
Thanks
You can use ROW_NUMBER() depending on the type of concurrency you are dealing with. Here is some sample data and a demo you can run locally.
-- Sample table
USE tempdb
GO
IF OBJECT_ID('dbo.sometable','U') IS NOT NULL DROP TABLE dbo.sometable;
GO
CREATE TABLE dbo.sometable
(
SomeId INT NULL,
Col1 INT NOT NULL
);
GO
-- Stored Proc to insert data
CREATE PROC dbo.InsertProc #output BIT AS
BEGIN -- Your proc starts here
INSERT dbo.sometable(Col1)
SELECT datasource.[value]
FROM (VALUES(CHECKSUM(NEWID())%100)) AS datasource([value]) -- simulating data from somewhere
CROSS APPLY (VALUES(1),(1),(1)) AS x(x);
WITH
id(MaxId) AS (SELECT ISNULL(MAX(t.SomeId),0) FROM dbo.sometable AS t),
xx AS
(
SELECT s.SomeId, RN = ROW_NUMBER() OVER (ORDER BY (SELECT NULL))+id.MaxId, s.Col1, id.MaxId
FROM id AS id
CROSS JOIN dbo.sometable AS s
WHERE s.SomeId IS NULL
)
UPDATE xx SET xx.SomeId = xx.RN;
IF #output = 1
SELECT t.* FROM dbo.sometable AS t;
END
GO
Each time I run: EXEC dbo.InsertProc 1; it returns 3 more rows with the correct ID col. Each time I execute it, it adds more rows and auto-increments as needed.
SomeId Col1
-------- ------
1 62
2 73
3 -17
Related
Is it possible to have a non-null column where the value is generated at insert by calling a stored procedure the parameters of which are values passed to insert into the row?
For example, I have table User:
| username | name | surname | id |
Insert looks like this:
INSERT INTO USER (username, name, surname)
VALUES ('myusername', 'myname', 'mysurname');
The id column is populated with an (integer) value retrieved by calling stored procedure mystoredproc with parameters myusername, myname, mysurname.
A further question is, would this stored procedure be called on each row, or can it be called in a grouped fashion. For example, I'd like my stored procedure to take the name and append a random integer to it so that that if I insert 100 users with the name 'David', they will get the same id and the stored procedure will be called only once. A bit of a bad example on the second point.
Good day,
Is it possible to have a non-null column where the value is generated at insert by calling a stored procedure
Option 1: please check if this work for you
Specify Default Value for the Column and use "NOT NULL"
create trigger on the table AFTER INSERT
Inside the trigger, you can use the virtual table "inserted" in order to get the inserted values.
Using these values (using the inserted table) you can update the column using the logic you need for all the rows at once
** there is no need to use external SP probably, but you can execute SP from trigger if needed
** All executed by a trigger is in the same transaction as the original query.
would this stored procedure be called on each row
NO! The trigger will be executed once for all rows you insert in the same statement. The inserted table includes all the rows which were inserted. In your update section (step 4) you can update all the rows which were inserted in once and no need to execute something for each row
** If you do use external SP which is executed from the trigger then you can pass it all the inserted table as one using Table-Valued Parameter
------------------- update ---------------
Here is a full example of using this logic:
drop table if exists T;
CREATE TABLE T (id int identity(2,2), c int NOT NULL default 1)
GO
CREATE TRIGGER tr ON T AFTER INSERT
AS BEGIN
SET NOCOUNT ON;
UPDATE T SET T.c = T2.C + 1
FROM inserted T2
INNER JOIN T T1 ON T1.id = T2.id
END
INSERT T(c) values (1) -- I insert the value 1 but the trigger will change it to 1+1=2
select * from T
GO
-- test multiple rows:
INSERT T(c) values (10),(20),(30),(40)
select * from T
GO
DECLARE #rc INT = 0,
#UserID INT = ABS(CHECKSUM(NEWID())) % 1000000 + 1;
WHILE #rc = 0
BEGIN
IF NOT EXISTS (SELECT 1 FROM dbo.Users WHERE UserId= #UserId)
BEGIN
INSERT dbo.Users(UserId) WHERE Username = #UserName SELECT #UserId;
SET #rc = 1;
END
ELSE
BEGIN
SELECT #UserId = ABS(CHECKSUM(NEWID())) % 1000000 + 1,
#rc = 0;
END
END
I have a table with 5 billions of rows in SQL Server 2014 (Developer Edition, x64, Windows 10 Pro x64):
CREATE TABLE TestTable
(
ID BIGINT IDENTITY(1,1),
PARENT_ID BIGINT NOT NULL,
CONSTRAINT PK_TestTable PRIMARY KEY CLUSTERED (ID)
);
CREATE NONCLUSTERED INDEX IX_TestTable_ParentId
ON TestTable (PARENT_ID);
I'm trying to apply the following patch:
-- Create non-nullable column with default (should be online operation in Enterprise/Developer edition)
ALTER TABLE TestTable
ADD ORDINAL TINYINT NOT NULL CONSTRAINT DF_TestTable_Ordinal DEFAULT 0;
GO
-- Populate column value for existing data
BEGIN
SET NOCOUNT ON;
DECLARE #BATCH_SIZE BIGINT = 1000000;
DECLARE #COUNTER BIGINT = 0;
DECLARE #ROW_ID BIGINT;
DECLARE #ORDINAL BIGINT;
DECLARE ROWS_C CURSOR
LOCAL FORWARD_ONLY FAST_FORWARD READ_ONLY
FOR
SELECT
ID AS ID,
ROW_NUMBER() OVER (PARTITION BY PARENT_ID ORDER BY ID ASC) AS ORDINAL
FROM
TestTable;
OPEN ROWS_C;
FETCH NEXT FROM ROWS_C
INTO #ROW_ID, #ORDINAL;
BEGIN TRANSACTION;
WHILE ##FETCH_STATUS = 0
BEGIN
UPDATE TestTable
SET
ORDINAL = CAST(#ORDINAL AS TINYINT)
WHERE
ID = #ROW_ID;
FETCH NEXT FROM ROWS_C
INTO #ROW_ID, #ORDINAL;
SET #COUNTER = #COUNTER + 1;
IF #COUNTER = #BATCH_SIZE
BEGIN
COMMIT TRANSACTION;
SET #COUNTER = 0;
BEGIN TRANSACTION;
END;
END;
COMMIT TRANSACTION;
CLOSE ROWS_C;
DEALLOCATE ROWS_C;
SET NOCOUNT OFF;
END;
GO
-- Drop default constraint from the column
ALTER TABLE TestTable
DROP CONSTRAINT DF_TestTable_Ordinal;
GO
-- Drop IX_TestTable_ParentId index
DROP INDEX IX_TestTable_ParentId
ON TestTable;
GO
-- Create IX_TestTable_ParentId_Ordinal index
CREATE UNIQUE INDEX IX_TestTable_ParentId_Ordinal
ON TestTable (PARENT_ID, ORDINAL);
GO
The aim of patch is to add a column, called ORDINAL, which is an ordinal number of the record within the same parent (defined by PARENT_ID). The patch is run using SQLCMD.
The patch is done is this way for a set of reasons:
Table is too large to run a single UPDATE statement on it (takes enormous amount of time and space in transaction log/tempdb).
Batch updates using a single UPDATE statement with TOP n rows are not simple to implement (if we update table in, say, 1m rows batches, 1000001st row may belong to the same PARENT_ID as 1000000th which will lead to wrong ordinal number assigned to 1000001st record). In other words, SELECT statement run in cursor should be run once (without paging) or more complicated operations (joins/conditions) should be applied.
Adding NULL column and changing it to NOT NULL later is not a good solution since I use SNAPSHOT isolation (full table update will be performed on altering column to be NOT NULL).
The patch works perfect on a small database with a few millions of rows, but, when applied to the one with billions of rows, I get:
Msg 3606, Level 16, State 2, Server XXX, Line 22
Arithmetic overflow occurred.
My first guess was ORDINAL value is too big to fit into TINYINT column, but this is not the case. I created a test database with similar structure and populated with data (more than 255 rows per parent). The error message I get is still arithmetic exception, but with different message code and different wording (explicitly saying it can't fit data into TINYINT).
Currently I have a couple of suspicions, but I haven't managed to find anything that could help me:
CURSOR is not able to handle more than MAX(INT32) rows.
SQLCMD imposed limitations.
Do you have any ideas on what could the problem be?
How about using a While loop but making sure that you keep the same parent_ids together:
DECLARE #SegmentSize BIGINT = 1000000
DECLARE #CurrentSegment BigInt = 0
WHILE 1 = 1
BEGIN
;With UpdateData As
(
SELECT ID AS ID,
ROW_NUMBER() OVER (PARTITION BY PARENT_ID ORDER BY ID ASC) AS ORDINAL
FROM TestData
WHERE ID > #CurrentSegment AND ID <= (#CurrentSegment + #SegmentSize)
)
UPDATE TestData
SET Ordinal = UpdateDate.Ordinal
FROM TestData
INNER JOIN UpdateData ON TestData.Id = UpdateData.Id
IF ##ROWCOUNT = 0
BEGIN
BREAK
END
SET #CurrentSegment = #CuurentSegment + #SegmentSize
END
EDIT - Amended to segment on Parent_Id as per request. This should be
reasonably quick as Parent_id is indexed (added Option(Recompile)
to ensure that actual value is used for the lookup.
Because you are not updating
the whole table this will limit the Transaction Log growth!
DECLARE #SegmentSize BIGINT = 1000000
DECLARE #CurrentSegment BigInt = 0
WHILE 1 = 1
BEGIN
;With UpdateData As
(
SELECT ID AS ID,
ROW_NUMBER() OVER (PARTITION BY PARENT_ID ORDER BY ID ASC) AS ORDINAL
FROM TestData
WHERE Parent_ID > #CurrentSegment AND
Parent_ID <= (#CurrentSegment + #SegmentSize)
)
UPDATE TestData
SET Ordinal = UpdateDate.Ordinal
FROM TestData
INNER JOIN UpdateData ON TestData.Id = UpdateData.Id
OPTION (RECOMPILE)
IF ##ROWCOUNT = 0
BEGIN
BREAK
END
SET #CurrentSegment = #CuurentSegment + #SegmentSize
END
I have a table called Employee. The EmpId column serves as the primary key. In my scenario, I cannot make it AutoNumber.
What would be the best way of generating the the next EmpId for the new row that I want to insert in the table?
I am using SQL Server 2008 with C#.
Here is the code that i am currently getting, but to enter Id's in key value pair tables or link tables (m*n relations)
Create PROCEDURE [dbo].[mSP_GetNEXTID]
#NEXTID int out,
#TABLENAME varchar(100),
#UPDATE CHAR(1) = NULL
AS
BEGIN
DECLARE #QUERY VARCHAR(500)
BEGIN
IF EXISTS (SELECT LASTID FROM LASTIDS WHERE TABLENAME = #TABLENAME and active=1)
BEGIN
SELECT #NEXTID = LASTID FROM LASTIDS WHERE TABLENAME = #TABLENAME and active=1
IF(#UPDATE IS NULL OR #UPDATE = '')
BEGIN
UPDATE LASTIDS
SET LASTID = LASTID + 1
WHERE TABLENAME = #TABLENAME
and active=1
END
END
ELSE
BEGIN
SET #NEXTID = 1
INSERT INTO LASTIDS(LASTID,TABLENAME, ACTIVE)
VALUES(#NEXTID+1,#TABLENAME, 1)
END
END
END
Using MAX(id) + 1 is a bad idea both performance and concurrency wise.
Instead you should resort to sequences which were design specifically for this kind of problem.
CREATE SEQUENCE EmpIdSeq AS bigint
START WITH 1
INCREMENT BY 1;
And to generate the next id use:
SELECT NEXT VALUE FOR EmpIdSeq;
You can use the generated value in a insert statement:
INSERT Emp (EmpId, X, Y)
VALUES (NEXT VALUE FOR EmpIdSeq, 'x', 'y');
And even use it as default for your column:
CREATE TABLE Emp
(
EmpId bigint PRIMARY KEY CLUSTERED
DEFAULT (NEXT VALUE FOR EmpIdSeq),
X nvarchar(255) NULL,
Y nvarchar(255) NULL
);
Update: The above solution is only applicable to SQL Server 2012+. For older versions you can simulate the sequence behavior using dummy tables with identity fields:
CREATE TABLE EmpIdSeq (
SeqID bigint IDENTITY PRIMARY KEY CLUSTERED
);
And procedures that emulates NEXT VALUE:
CREATE PROCEDURE GetNewSeqVal_Emp
#NewSeqVal bigint OUTPUT
AS
BEGIN
SET NOCOUNT ON
INSERT EmpIdSeq DEFAULT VALUES
SET #NewSeqVal = scope_identity()
DELETE FROM EmpIdSeq WITH (READPAST)
END;
Usage exemple:
DECLARE #NewSeqVal bigint
EXEC GetNewSeqVal_Emp #NewSeqVal OUTPUT
The performance overhead of deleting the last inserted element will be minimal; still, as pointed out by the original author, you can optionally remove the delete statement and schedule a maintenance job to delete the table contents off-hour (trading space for performance).
Adapted from SQL Server Customer Advisory Team Blog.
Working SQL Fiddle
The above
select max(empid) + 1 from employee
is the way to get the next number, but if there are multiple user inserting into the database, then context switching might cause two users to get the same value for empid and then add 1 to each and then end up with repeat ids. If you do have multiple users, you may have to lock the table while inserting. This is not the best practice and that is why the auto increment exists for database tables.
I hope this works for you. Considering that your ID field is an integer
INSERT INTO Table WITH (TABLOCK)
(SELECT CASE WHEN MAX(ID) IS NULL
THEN 1 ELSE MAX(ID)+1 END FROM Table), VALUE_1, VALUE_2....
Try following query
INSERT INTO Table VALUES
((SELECT isnull(MAX(ID),0)+1 FROM Table), VALUE_1, VALUE_2....)
you have to check isnull in on max values otherwise it will return null in final result when table contain no rows .
Typically when you specify an identity column you get a convenient interface in SQL Server for asking for particular row.
SELECT * FROM $IDENTITY = #pID
You don't really need to concern yourself with the name if the identity column because there can only be one.
But what if I have a table which mostly consists of temporary data. Lots of inserts and lots of deletes. Is there a simple way for me to reuse the identity values.
Preferably I would want to be able to write a function that would return say NEXT_SMALLEST($IDENTITY) as next identity value and do so in a fail-safe manner.
Basically find the smallest value that's not in use. That's not entirely trivial to do, but what I want is to be able to tell SQL Server that this is my function that will generate the identity values. But what I know is that no such function exists...
I want to...
Implement global data base IDs, I need to provide a default value that I'm in control of.
My idea was based around that I should be able to have a table with all known IDs and then every row ID from some other table that needed a global ID would reference that table. The default value would be provided by something like
INSERT INTO GlobalID
RETURN SCOPE_IDENTITY()
No; it's not unique if it can be reused.
Why do you want to re-use them? Why do you concern yourself with this field? If you want to be in control of it, don't make it an identity; create your own scheme and use that.
Don't reuse identities, you'll just shoot your self in the foot. Use a large enough value so that it never rolls over (64 bit big int).
To find missing gaps in a sequence of numbers join the table against itself with a +/- 1 difference:
SELECT a.id
FROM table AS a
LEFT OUTER JOIN table AS b ON a.id = b.id+1
WHERE b.id IS NULL;
This query will find the numbers in the id sequence for which id-1 is not in the table, ie. contiguous sequence start numbers. You can then use SET IDENTITY INSERT OFF to insert a specific id and reuse a number. The cost of doing so is overwhelming (both runtime and code complexity) compared with the an ordinary identity based insert.
If you really want to reset Identity value to the lowest,
here is the trick you can use through DBCC CHECKIDENT
Basically following sql statements resets identity value so that identity value restarts from the lowest possible number
create table TT (id int identity(1, 1))
GO
insert TT default values
GO 10
select * from TT
GO
delete TT where id between 5 and 10
GO
--; At this point, next ID will be 11, not 5
select * from TT
GO
insert TT default values
GO
--; as you can see here, next ID is indeed 11
select * from TT
GO
--; Now delete ID = 11
--; so that we can reseed next highest ID to 5
delete TT where id = 11
GO
--; Now, let''s reseed identity value to the lowest possible identity number
declare #seedID int
select #seedID = max(id) from TT
print #seedID --; 4
--; We reseed identity column with "DBCC CheckIdent" and pass a new seed value
--; But we can't pass a seed number as argument, so let's use dynamic sql.
declare #sql nvarchar(200)
set #sql = 'dbcc checkident(TT, reseed, ' + cast(#seedID as varchar) + ')'
exec sp_sqlexec #sql
GO
--; Now the next
insert TT default values
GO
--; as you can see here, next ID is indeed 5
select * from TT
GO
I guess we would really need to know why you want to reuse your identity column. The only reason I can think of is because of the temporary nature of your data you might exhaust the possible values for the identity. That is not really likely, but if that is your concern, you can use uniqueidentifiers (guids) as the primary key in your table instead.
The function newid() will create a new guid and can be used in insert statements (or other statements). Then when you delete the row, you don't have any "holes" in your key because guids are not created in that order anyway.
[Syntax assumes SQL2008....]
Yes, it's possible. You need to two management tables, and two triggers on each participating table.
First, the management tables:
-- this table should only ever have one row
CREATE TABLE NextId (Id INT)
INSERT NextId VALUES (1)
GO
CREATE TABLE RecoveredIds (Id INT NOT NULL PRIMARY KEY)
GO
Then, the triggers, two on each table:
CREATE TRIGGER tr_TableName_RecoverId ON TableName
FOR DELETE AS BEGIN
IF ##ROWCOUNT = 0 RETURN
INSERT RecoveredIds (Id) SELECT Id FROM deleted
END
GO
CREATE TRIGGER tr_TableName_AssignId ON TableName
INSTEAD OF INSERT AS BEGIN
DECLARE #rowcount INT = ##ROWCOUNT
IF #rowcount = 0 RETURN
DECLARE #required INT = #rowcount
DECLARE #new_ids TABLE (Id INT PRIMARY KEY)
DELETE TOP (#required) OUTPUT DELETED.Id INTO #new_ids (Id) FROM RecoveredIds
SET #rowcount = ##ROWCOUNT
IF #rowcount < #required BEGIN
DECLARE #output TABLE (Id INT)
UPDATE NextId SET Id = Id + (#required-#rowcount)
OUTPUT DELETED.Id INTO #output
-- this assumes you have a numbers table around somewhere
INSERT #new_ids (Id)
SELECT n.Number+o.Id-1 FROM Numbers n, #output o
WHERE n.Number BETWEEN 1 AND #required-#rowcount
END
SET IDENTITY_INSERT TableName ON
;WITH inserted_CTE AS (SELECT _no = ROW_NUMBER() OVER (ORDER BY Id), * FROM inserted)
, new_ids_CTE AS (SELECT _no = ROW_NUMBER() OVER (ORDER BY Id), * FROM #new_ids)
INSERT TableName (Id, Attr1, Attr2)
SELECT n.Id, i.Attr1, i.Attr2
FROM inserted_CTE i JOIN new_ids_CTE n ON i._no = n._no
SET IDENTITY_INSERT TableName OFF
END
You could script the triggers out easily enough from system tables.
You would want to test this for concurrency. It should work as is, syntax errors notwithstanding: The OUTPUT clause guarantees atomicity of id lookup->increment as one step, and the entire operation occurs within a transaction, thanks to the trigger.
TableName.Id is still an identity column. All the common idioms like $IDENTITY and SCOPE_IDENTITY() will still work.
There is no central table of ids by table, but you could create one easily enough.
I don't have any help for finding the values not in use but if you really want to find them and set them yourself, you can use
set identity_insert on ....
in your code to do so.
I'm with everyone else though. Why bother? Don't you have a business problem to solve?
I'm not sure if this is something I should do in T-SQL or not, and I'm pretty sure using the word 'iterate' was wrong in this context, since you should never iterate anything in sql. It should be a set based operation, correct? Anyway, here's the scenario:
I have a stored proc that returns many uniqueidentifiers (single column results). These ids are the primary keys of records in a another table. I need to set a flag on all the corresponding records in that table.
How do I do this without the use of cursors? Should be an easy one for you sql gurus!
This may not be the most efficient, but I would create a temp table to hold the results of the stored proc and then use that in a join against the target table. For example:
CREATE TABLE #t (uniqueid int)
INSERT INTO #t EXEC p_YourStoredProc
UPDATE TargetTable
SET a.FlagColumn = 1
FROM TargetTable a JOIN #t b
ON a.uniqueid = b.uniqueid
DROP TABLE #t
You could also change your stored proc to a user-defined function that returns a table with your uniqueidentifiers. You can joing directly to the UDF and treat it like a table which avoids having to create the extra temp table explicitly. Also, you can pass parameters into the function as you're calling it, making this a very flexible solution.
CREATE FUNCTION dbo.udfGetUniqueIDs
()
RETURNS TABLE
AS
RETURN
(
SELECT uniqueid FROM dbo.SomeWhere
)
GO
UPDATE dbo.TargetTable
SET a.FlagColumn = 1
FROM dbo.TargetTable a INNER JOIN dbo.udfGetUniqueIDs() b
ON a.uniqueid = b.uniqueid
Edit:
This will work on SQL Server 2000 and up...
Insert the results of the stored proc into a temporary table and join this to the table you want to update:
INSERT INTO #WorkTable
EXEC usp_WorkResults
UPDATE DataTable
SET Flag = Whatever
FROM DataTable
INNER JOIN #WorkTable
ON DataTable.Ket = #WorkTable.Key
If you upgrade to SQL 2008 then you can pass table parameters I believe. Otherwise, you're stuck with a global temporary table or creating a permanent table that includes a column for some sort of process ID to identify which call to the stored procedure is relevant.
How much room do you have in changing the stored procedure that generates the IDs? You could add code in there to handle it or have a parameter that lets you optionally flag the rows when it is called.
Use temporary tables or a table variable (you are using SS2005).
Although, that's not nest-able - if a stored proc uses that method then you can't dumpt that output into a temp table.
An ugly solution would be to have your procedure return the "next" id each time it is called by using the other table (or some flag on the existing table) to filter out the rows that it has already returned
You can use a temp table or table variable with an additional column:
DECLARE #MyTable TABLE (
Column1 uniqueidentifer,
...,
Checked bit
)
INSERT INTO #MyTable
SELECT [...], 0 FROM MyTable WHERE [...]
DECLARE #Continue bit
SET #Continue = 1
WHILE (#Continue)
BEGIN
SELECT #var1 = Column1,
#var2 = Column2,
...
FROM #MyTable
WHERE Checked = 1
IF #var1 IS NULL
SET #Continue = 0
ELSE
BEGIN
...
UPDATE #MyTable SET Checked = 1 WHERE Column1 = #var1
END
END
Edit: Actually, in your situation a join will be better; the code above is a cursorless iteration, which is overkill for your situation.