I am currently tasked with a project on a database whose schema cannot be changed. I need to insert a new row into a table that requires an ID to be unique, but the original creators of the structure did not set this value to autoincrement. To go around this, I have been using code akin to:
(SELECT TOP 1 [ID] from [Table] ORDER BY [ID] DESC) + 1
when giving the value of the ID field, basically having an inner query of sorts. Problem is that a few lines down, I need that ID I just inputted. If I could set a SQLParameter to output for this column, I could get the value it was set to, problem is I'm using SQL, and not a hard value like I do with other SQLParameters. Can't I use SQL in place of just a value?
This is a potential high volume exchange, so I'd rather not do 2 different queries (one to get id, then one to insert).
You say you cannot change the schema, but can you add an additional table to the project that does an autoincrement column? Then you could use that table to (safely) create your new IDs and return them to your code.
This is similar to how Oracle does IDs, and sometimes vendor applications for sql server that also run on Oracle will use that approach just to help minimize the differences between the two databases.
Update:
Ah, I just spotted your comment to the other answer here. In that case, the only other thing I can think that might work is to put your two statements (insert a new ID, and then read back the new ID) inside a transaction with the SERIALIZABLE isolation level. And that just kinda sucks, because it leaves you open to performance and locking gotchas.
Is it possible for you to create a stored procedure in the database to do this and the return value of the stored procedure will then return the ID that you need?
I'm a bit confused about where you need to use this ID. If it inside of the same stored proc just use this method:
DECLARE #NewId int
SELECT TOP 1 #NewId = [ID] + 1 from [Table] ORDER BY [ID] DESC
SELECT #NewId
You can put more than one SQL statement in a single SqlCommand. So you could easily do something along the lines of what Abe suggested:
DECLARE #NewId int
SELECT TOP 1 #NewId = [ID] + 1 from [Table] ORDER BY [ID] DESC
INSERT INTO [Table] (ID, ...) VALUES (#NewId, ...)
SELECT #NewId
Then you just call ExecuteScalar on your SqlCommand, and it will do the INSERT and then return the ID it used.
Related
I am having serious performance issues when using a nested loop in a WHERE clause.
When I run the below code as is, it takes several minutes. The trick is I'm using the WHERE clause to pull ALL data if the report_id is NULL, but only certain report_id's if I set them in the parameter string.
The function [fn_Parse_List] turns a VARCHAR string such as '123,456,789' into a table where each row is each number in integer form, which is then used in the IN clause.
When I run the code below with report_id = '456' (the dashed out portion), the code takes seconds, but passing the temporary table and using the SELECT statement in the WHERE clause kills it.
alter procedure dbo.p_revenue
(#report_id varchar(max) = NULL)
as
select cast(value as int) Report_ID
into #report_ID_Temp
from [fn_Parse_List] (#report_id)
SELECT *
FROM BIGTABLE
where #report_id is null
or a.report_id in (select Report_ID from #report_ID_Temp)
--Where #report_id is null or a.report_id in (456)
exec p_revenue #report_id = '456'
Is there a way to optimize this? I tried a JOIN with the table #report_ID_Temp, but it still takes just as long and doesn't work when the report_id is NULL.
You're breaking three different rules.
If you want two query plans, you need two queries: OR does not give you two query plans. IF does.
If you have a temporary table, make sure it has a primary key and any appropriate indexes. In your case, you need an ALTER TABLE statement to add the primary key clustered index. Or you can CREATE TABLE to declare the structure in the first place.
If you think fn_Parse_List is a good idea, you haven't read enough Sommarskog
If I were to write the Stored Procedure for your case, I would use a Table Valued Parameter (TVP) instead of passing multiple values as a comma-seperated string.
Something like the following:
-- Create a type for the TVP
CREATE TYPE REPORT_IDS_PAR AS TABLE(
report_id INT
);
GO
-- Use the TVP type instead of VARCHAR
CREATE PROCEDURE dbo.revenue
#report_ids REPORT_IDS_PAR READONLY
AS
BEGIN
SET NOCOUNT ON;
IF NOT EXISTS(SELECT 1 FROM #report_ids)
SELECT
*
FROM
BIGTABLE;
ELSE
SELECT
*
FROM
#report_ids AS ids
INNER JOIN BIGTABLE AS bt ON
bt.report_id=ids.report_id;
-- OPTION(RECOMPILE) -- see remark below
END
GO
-- Execute the Stored Procedure
DECLARE #ids REPORT_IDS_PAR;
-- Empty table for all rows:
EXEC dbo.revenue #ids;
-- Specific report_id's for specific rows:
INSERT INTO #ids(report_id)VALUES(123),(456),(789);
EXEC dbo.revenue #ids;
GO
If you run this procedure with a TVP with a lot of rows or a wildly varying number of rows, I suggest you add the option OPTION(RECOMPILE) to the query.
I see 2 possible things that could help improve performance. Depends on which part is taking the longest. First off, SELECT INTO is a single threaded operation until SQL Server 2014. If this is taking a long time, create an explicitly defined temp table with CREATE TABLE. Secondly, depending on the number of records inserted into the temp table, you probably need an index on the Report_ID column. That can all be done in the body of the stored procedure. If you do end up using an explicitly defined temp table, I would create the index after the data is loaded.
If that doesn't help, first check that the report_id column on the BIGTABLE is indexed. Then try splitting the select into 2 and combining with a UNION ALL like this:
ALTER PROCEDURE dbo.p_revenue
(
#report_id VARCHAR(MAX) = NULL
)
AS
SELECT CAST(value AS INT) Report_ID
INTO #report_ID_Temp
FROM fn_Parse_List(#report_id);
SELECT *
FROM BIGTABLE
WHERE #report_id IS NULL
UNION ALL
SELECT *
FROM BIGTABLE
WHERE a.report_id IN ( SELECT Report_ID
FROM #report_ID_Temp );
GO
EXEC p_revenue #report_id = '456';
Are you saying I should have two queries, one where it pulls if the report_id doesn't exists and one where there is a list of report_ids?
Yes, yes, yes. The fact, that it somehow works when You enter the numbers directly, distracts You from the core problem. You need table scan when #report_id is null and index seek when it is not and You can not have both in one execution plan. The performance would inevitably have to suffer, one way or another.
I would prefer not to, as the table i'm pulling from is actually a
view with 800 lines with an additional parameter not shown above.
I do not see where is the problem, SELECT * FROM BIGTABLE and SELECT * FROM BIGVIEW seems the same. If You need parameters You can use inline table valued function. If You have more parameters with variable selectivity like #report_id, I guess You would end up with dynamic sql anyway, sooner or later.
UNION ALL as proposed by #db_brad would help, but one of those subquery is executed even when there is no need for it.
As a quick patch You can append OPTION(RECOMPILE) to the SELECT and have table scan one time and index seek the other time, but recompiling every time would induce nontrivial overhead.
I need to update a SQL Server table periodically by inserting new records into it.
The table has an ID column in the form of Company0001 through Company0020 right now.
Let's say I added one record of a new company into the table. I want to fill the ID column with Company0021 for this new record. Can anyone suggest a way to do this?
Thank you so much!
I would strongly suggest to use an identity column. Identity is a mechanism designed and used for this actual purpose and therefore it would be much better in terms of performance.
Nevertheless, if you insist on IDs on the format 'CompanyXXX' I would suggest to use a varchar column. Then you would add a trigger on the insert and update operations. When the trigger runs, it would find out the last 'CompanyXXX' and form the new one. If you need help regarding triggers, you could check this tutorial.
Hope I helped!
My suggestion would be to have an autoincrement field, and then concatenate the company name with the ID.
If you don't want to do it with an ID field, do you want it to happen automatically, or are you going to manage it manually? If automatically, you'll need to write a trigger to intercept the INSERT and change the value there. Shouldn't be too hard to do.
I'd seriously recommend NOT doing this and going down the autoincrement field path. It's better.
Add another column to the table to hold an integer value (in this example SNo) and then write query as
declare #SNo int
select #SNo=max(SNo)+1 from Table_Name
insert into Table_Namevalues (#SNo,'company'+right('0000'+cast(#SNo as varchar(10)),4))
And then see the result
Hope this helps
In case a solution is required as, having only one column with values in desired format you can create a function as:
create table table1(id varchar(100));
Go
create function dbo.fn_GetCompanyIdentity ()
returns varchar(100)
as
begin
declare #CompanyIdentify varchar(100);
select #CompanyIdentify =
(select 'Company' +
right ('00000' + cast (
(
(
case when Not exists (select ROW_NUMBER() over( order by (select 1)) from Table1 ) then 1
else (select top 1 ROW_NUMBER() over( order by (select 1)) as currentRownumber from Table1 order by currentRownumber desc) + 1
end
)
)
as varchar(4))
,4));
return #CompanyIdentify;
end;
go
and then use the function in insert statement as :
insert into Table1 (id)
select dbo.fn_GetCompanyIdentity();
Go
Hope this helps!!
Why dont you just create an auto-increment column and then concatenate "Company" to this column in another column. And for presentation just select "Company+autoincrement" column.
I'm trying to get a the key-value back after an INSERT-statement.
Example:
I've got a table with the attributes name and id. id is a generated value.
INSERT INTO table (name) VALUES('bob');
Now I want to get the id back in the same step. How is this done?
We're using Microsoft SQL Server 2008.
No need for a separate SELECT...
INSERT INTO table (name)
OUTPUT Inserted.ID
VALUES('bob');
This works for non-IDENTITY columns (such as GUIDs) too
Use SCOPE_IDENTITY() to get the new ID value
INSERT INTO table (name) VALUES('bob');
SELECT SCOPE_IDENTITY()
http://msdn.microsoft.com/en-us/library/ms190315.aspx
INSERT INTO files (title) VALUES ('whatever');
SELECT * FROM files WHERE id = SCOPE_IDENTITY();
Is the safest bet since there is a known issue with OUTPUT Clause conflict on tables with triggers. Makes this quite unreliable as even if your table doesn't currently have any triggers - someone adding one down the line will break your application. Time Bomb sort of behaviour.
See msdn article for deeper explanation:
http://blogs.msdn.com/b/sqlprogrammability/archive/2008/07/11/update-with-output-clause-triggers-and-sqlmoreresults.aspx
Entity Framework performs something similar to gbn's answer:
DECLARE #generated_keys table([Id] uniqueidentifier)
INSERT INTO Customers(FirstName)
OUTPUT inserted.CustomerID INTO #generated_keys
VALUES('bob');
SELECT t.[CustomerID]
FROM #generated_keys AS g
JOIN dbo.Customers AS t
ON g.Id = t.CustomerID
WHERE ##ROWCOUNT > 0
The output results are stored in a temporary table variable, and then selected back to the client. Have to be aware of the gotcha:
inserts can generate more than one row, so the variable can hold more than one row, so you can be returned more than one ID
I have no idea why EF would inner join the ephemeral table back to the real table (under what circumstances would the two not match).
But that's what EF does.
SQL Server 2008 or newer only. If it's 2005 then you're out of luck.
There are many ways to exit after insert
When you insert data into a table, you can use the OUTPUT clause to
return a copy of the data that’s been inserted into the table. The
OUTPUT clause takes two basic forms: OUTPUT and OUTPUT INTO. Use the
OUTPUT form if you want to return the data to the calling application.
Use the OUTPUT INTO form if you want to return the data to a table or
a table variable.
DECLARE #MyTableVar TABLE (id INT,NAME NVARCHAR(50));
INSERT INTO tableName
(
NAME,....
)OUTPUT INSERTED.id,INSERTED.Name INTO #MyTableVar
VALUES
(
'test',...
)
IDENT_CURRENT: It returns the last identity created for a particular table or view in any session.
SELECT IDENT_CURRENT('tableName') AS [IDENT_CURRENT]
SCOPE_IDENTITY: It returns the last identity from a same session and the same scope. A scope is a stored procedure/trigger etc.
SELECT SCOPE_IDENTITY() AS [SCOPE_IDENTITY];
##IDENTITY: It returns the last identity from the same session.
SELECT ##IDENTITY AS [##IDENTITY];
##IDENTITY Is a system function that returns the last-inserted identity value.
There are multiple ways to get the last inserted ID after insert command.
##IDENTITY : It returns the last Identity value generated on a Connection in current session, regardless of Table and the scope of statement that produced the value
SCOPE_IDENTITY(): It returns the last identity value generated by the insert statement in the current scope in the current connection regardless of the table.
IDENT_CURRENT(‘TABLENAME’) : It returns the last identity value generated on the specified table regardless of Any connection, session or scope. IDENT_CURRENT is not limited by scope and session; it is limited to a specified table.
Now it seems more difficult to decide which one will be exact match for my requirement.
I mostly prefer SCOPE_IDENTITY().
If you use select SCOPE_IDENTITY() along with TableName in insert statement, you will get the exact result as per your expectation.
Source : CodoBee
The best and most sure solution is using SCOPE_IDENTITY().
Just you have to get the scope identity after every insert and save it in a variable because you can call two insert in the same scope.
ident_current and ##identity may be they work but they are not safe scope. You can have issues in a big application
declare #duplicataId int
select #duplicataId = (SELECT SCOPE_IDENTITY())
More detail is here Microsoft docs
You can use scope_identity() to select the ID of the row you just inserted into a variable then just select whatever columns you want from that table where the id = the identity you got from scope_identity()
See here for the MSDN info http://msdn.microsoft.com/en-us/library/ms190315.aspx
Recommend to use SCOPE_IDENTITY() to get the new ID value, But NOT use "OUTPUT Inserted.ID"
If the insert statement throw exception, I except it throw it directly. But "OUTPUT Inserted.ID" will return 0, which maybe not as expected.
This is how I use OUTPUT INSERTED, when inserting to a table that uses ID as identity column in SQL Server:
'myConn is the ADO connection, RS a recordset and ID an integer
Set RS=myConn.Execute("INSERT INTO M2_VOTELIST(PRODUCER_ID,TITLE,TIMEU) OUTPUT INSERTED.ID VALUES ('Gator','Test',GETDATE())")
ID=RS(0)
You can append a select statement to your insert statement.
Integer myInt =
Insert into table1 (FName) values('Fred'); Select Scope_Identity();
This will return a value of the identity when executed scaler.
* Parameter order in the connection string is sometimes important. * The Provider parameter's location can break the recordset cursor after adding a row. We saw this behavior with the SQLOLEDB provider.
After a row is added, the row fields are not available, UNLESS the Provider is specified as the first parameter in the connection string. When the provider is anywhere in the connection string except as the first parameter, the newly inserted row fields are not available. When we moved the the Provider to the first parameter, the row fields magically appeared.
After doing an insert into a table with an identity column, you can reference ##IDENTITY to get the value:
http://msdn.microsoft.com/en-us/library/aa933167%28v=sql.80%29.aspx
I have a T-SQL routine that copies user information from one table 'Radius' to another 'Tags'. However, as the rows are transfered, I would also like to include a unique randomly generated code in the INSERT (3 chars long). The code is generated by the WHILE loop below. Any way to do this?
INSERT Tags (UserID, JobID, Code)
SELECT UserID, #JobID, ?????
FROM Radius
Unique random code generator:
WHILE EXISTS (SELECT * FROM Tags WHERE Code = #code)
BEGIN
select #code=#code+char(n) from
(
select top 3 number as n from master..spt_values
where type='p' and number between 48 and 57 or number between 65 and 90
order by newid()
)
END
CLARIFICATION: The reason for doing this is that I want to keep the random code generation logic at the level of the SQL stack. Implementing this in the app code would require me to check the db everytime a potential random code is generated to see if it is unique. As the number of code records increases so will the number of calls to the db as probability increases that there will be more duplicate codes generated before a unique one is generated.
Part One, Generate a table with all possible values
DECLARE #i int
CREATE TABLE #AllChars(value CHAR(1))
SET #i=48
WHILE #i<=57
BEGIN
INSERT INTO #Allchars(value) VALUES(CHAR(#i))
SET #i=#i+1
END
SET #i=65
WHILE #i<=90
BEGIN
INSERT INTO #Allchars(value) VALUES(CHAR(#i))
SET #i=#i+1
END
CREATE TABLE AllCodes(value CHAR(3),
CONSTRAINT PK_AllChars PRIMARY KEY CLUSTERED(value))
INSERT INTO AllCodes(value)
SELECT AllChars1.Value+AllChars2.Value+AllChars3.Value
FROM #AllChars AS AllChars1,#AllChars AS AllChars2,#AllChars AS AllChars3
This is a one off operation and takes around 1 second to run on SQL Azure. Now that you have all possible values in a table any future inserts become, something along the lines of
SELECT
RadiusTable.UserID,
RadiusTable.JobID,
IDTable.Value
FROM
(
SELECT ROW_NUMBER() OVER (ORDER BY UserID,JobID) As RadiusRow,
UserID,JobID
FROM Radius
) AS RadiusTable INNER JOIN
(
SELECT ROW_NUMBER() OVER (ORDER BY newID()) As IDRow,
Value
FROM AllCodes
) AS IDTable ON RadiusTable.RadiusRow = IDTable.IDRow
Before going with any of these schemes you had better be certain that you are not going to have more than 46656 rows in your table otherwise you will run out of unique ID Values.
I do not know if this is possible and suitable for your situation, but to me it seems that a scalar-valued function would be a solution.
Well, let me start over then.
This seems kind of ugly but it might work: newid() inside sql server function
The accepted answer that is.
Ah, been there done that too. The problem with this is that I am using T-SQL Stored Procedures that are called by Asp.net Where would I put the CREATE VIEW statement? I can't add it to the function file.
I need to create a temp table, than add a new int NOT NULL AUTO_INCREMENT field to it so I can use the new field as a row number. Whats wrong with my query?
SELECT post, newid FROM ((SELECT post`test_posts`) temp
ALTER TABLE temp ADD COLUMN newid int NOT NULL AUTO_INCREMENT)
edit:
SELECT post, newid FROM ((SELECT post, newid as int NOT NULL AUTO_INCREMENT FROM `test_posts`) temp
This didn't work ether.
Not that this means it's impossible, but I haven't seen any SQL version that will allow you to modify a table from within a SELECT. Pull the alter table out and make it a separate statement. After you fix any syntax issues, you should be good.
Also, it doesn't look like you actually have a "temp table" to alter. Rather, you're looking for a solution that will let you add an arbitrary id to the result of your "SELECT post..." query. I don't know which engine you're using, but a sequence counter, rowid, rownum, or other similar feature would better fit your needs.
If you need a row number and don't want to actually create a temporary table, you can achieve it by using user variables.
SET #my_row_num =0;
SELECT #my_row_num := #my_row_num+1 as row_number, post, newid FROM ((SELECT post`test_posts`) temp;