Customized Primary Key on SQL Server 2008 R2 - sql

I have several days trying to solve this problem, but my lack of knowledge is stopping me, I don’t know if is possible what I am trying to accomplish.
I need to have a table like this:
The first field should be a custom primary key ID (auto incremented):
YYYYMMDD-99
Where YYYMMDD is the actual day and “99” is a counter that should be incremented automatically from 01 to 99 in every new row added and need to be automatically restarted to 01 the next day.
The second field is a regular NVARCHAR(40) text field called: Name
For example, I add three rows, just introducing the “Name” of the person, the ID is automatically added:
ID Name
---------------------------
20160629-01 John
20160629-02 Katie
20160629-03 Mark
Then, the next day I add two new rows:
ID Name
-------------------------
20160630-01 Bob
20160630-02 Dave
The last two digits should be restarted, after the day changes.
And, what is all this about ?
Answer: Customer requirement.
If is possible to do it in a stored procedure, it will works for me too.
Thanks in advance!!

This is pretty easy to achieve, but a bit complicated to do so it is safe with multiple clients.
What you need is a new table (for example named IndexHelper) that actually stores the parts of the index as it should be using two columns: One has the current date properly formatted as you want it in your index and one is the current index as integer. Example:
DateString CurrentIndex
-------------------------------
20160629 13
Now you need some code that helps you get the next index value atomically, i.e. in a way that also works when more than one client try to insert at the same time without getting the same index more than once.
T-SQL comes to the rescue with its UPDATE ... OUTPUT clause, which allows you to update a table, at the same time outputting the new values as an atomic operation, which can not be interrupted.
In your case, this statement could look like this:
DECLARE #curDay NVARCHAR(10)
DELCARE #curIndex INT
DECLARE #tempTable TABLE (theDay NVARCHAR(10), theIndex INT)
UPDATE IndexHelper SET CurrentIndex = CurrentIndex + 1 OUTPUT INSERTED.DateString, INSERTED.CurrentIndex INTO #temptable WHERE CurrentDate = <code that converts CURRENT_TIMESTAMP into the string format you want>
SELECT #curDay = theDay, #curIndex = theIndex FROM #tempTable
Unfortunately you have to go the temporary table way, as it is demanded by the OUTPUT clause.
This increments the CurrentIndex field in IndexHelper atomically for the current date. You can combine both into a value like this:
DECLARE #newIndexValue NVARCHAR(15)
SET #newIndexValue = #curDay + '-' + RIGHT('00' + CONVERT(NVARCHAR, #curIndex), 2)
Now the question is: How do you handle the "go back to 01 for the next day" requirement? Also easy: Add entries into IndexHelper for 2 days in advance with the respective date and index 0. You can do this safely everytime your code is called if you check that an entry for a day is actually missing. So for today your table might look like this:
DateString CurrentIndex
-------------------------------
20160629 13
20160630 0
20160701 0
The first call tomorrow would make this look like:
DateString CurrentIndex
-------------------------------
20160629 13
20160630 1
20160701 0
20160702 0
Wrap this up into a stored procedure that does the entire INSERT process into your original table, what you get is:
Add missing entries for the next two days to IndexHelper table.
Get the next ID atomically as described above
Combine date string and ID from the UPDATE command into a single string
Use this in the INSERT command for your actual data
This results in the following stored procedure you can use to insert your data:
-- This is our "work date"
DECLARE #now DATETIME = CURRENT_DATETIME
-- These are the date strings that we need
DECLARE #today NVARCHAR(10) = CONVERT(NVARCHAR, #now, 112)
DECLARE #tomorrow NVARCHAR(10) = CONVERT(NVARCHAR, DATEADD(dd, 1, #now), 112)
DECLARE #datomorrow NVARCHAR(10) = CONVERT(NVARCHAR, DATEADD(dd, 2, #now), 112)
-- We will need these later
DECLARE #curDay NVARCHAR(10)
DELCARE #curIndex INT
DECLARE #tempTable TABLE (theDay NVARCHAR(10), theIndex INT)
DECLARE #newIndexValue NVARCHAR(15)
-- Add entries for next two days into table
-- NOTE: THIS IS NOT ATOMIC! SUPPOSED YOU HAVE A PK ON DATESTRING, THIS
-- MAY EVEN FAIL! THAT'S WHY IS USE BEGIN TRY
BEGIN TRY
IF NOT EXISTS (SELECT 1 FROM IndexHelper WHERE DateString = #tomorrow)
INSERT INTO IndexHelper (#tomorrow, 0)
END TRY
BEGIN CATCH
PRINT 'hmpf'
END CATCH
BEGIN TRY
IF NOT EXISTS (SELECT 1 FROM IndexHelper WHERE DateString = #datomorrow)
INSERT INTO IndexHelper (#datomorrow, 0)
END TRY
BEGIN CATCH
PRINT 'hmpf again'
END CATCH
-- Now perform the atomic update
UPDATE IndexHelper
SET
CurrentIndex = CurrentIndex + 1
OUTPUT
INSERTED.DateString,
INSERTED.CurrentIndex
INTO #temptable
WHERE CurrentDate = #today
-- Get the values after the update
SELECT #curDay = theDay, #curIndex = theIndex FROM #tempTable
-- Combine these into the new index value
SET #newIndexValue = #curDay + '-' + RIGHT('00' + CONVERT(NVARCHAR, #curIndex), 2)
-- PERFORM THE INSERT HERE!!
...

One way to achieve customised auto increment is using INSTEAD OF trigger in SQL Server.
https://msdn.microsoft.com/en-IN/library/ms189799.aspx
I have tested this using below code.
This might be helpful.
It is written with the assumption that maximum 99 records will be inserted in a given day.
You will have to modify it to handle more than 99 records.
CREATE TABLE dbo.CustomerTb(
ID VARCHAR(50),
Name VARCHAR(50)
)
GO
CREATE TRIGGER dbo.InsertCustomerTrigger ON dbo.CustomerTb INSTEAD OF INSERT
AS
BEGIN
DECLARE #MaxID SMALLINT=0;
SELECT #MaxID=ISNULL(MAX(RIGHT(ID,2)),0)
FROM dbo.CustomerTb
WHERE LEFT(ID,8)=FORMAT(GETDATE(),'yyyyMMdd');
INSERT INTO dbo.CustomerTb(
ID,
Name
)
SELECT FORMAT(GETDATE(),'yyyyMMdd')+'-'+RIGHT('00'+CONVERT(VARCHAR(5),ROW_NUMBER() OVER(ORDER BY Name)+#MaxID),2),
Name
FROM inserted;
END
GO
TEST CASE 1
INSERT INTO dbo.CustomerTb(NAME) VALUES('A'),('B');
SELECT * FROM dbo.CustomerTb;
TEST CASE 2
INSERT INTO dbo.CustomerTb(NAME) VALUES('P'),('Q');
SELECT * FROM dbo.CustomerTb;

Related

Stored Procedure to Validate Input Data before insert

I am trying to write a stored procedure to validate data before inserted into a table.
I have a CHANGES table that a user will be inserting values into (basically if there is wrong data in certain tables/columns in our system, this user will send us the correct data to update certain columns). Before the data can be inserted into this CHANGES table, I would like to validate the input data against a LOOKUP table. This lookup table essentially shows what is the accepted data type, length of column etc before insert.
Here is the lookup table:
ID
TABLENAME
COLUMNAME
ACCEPTEDDATATYPE
ACCEPTEDLENGTH
1
EMPLOYEE
AGE
INT
2
EMPLOYEE
MIDDLENAME
VARCHAR
50
3
DEPARTMENT
BLDG
VARCHAR
10
Basically ID is the unique identifier, so if a user is inserting an ID of 1, we know they are trying to insert data for the employee table and making a correction to the AGE column, and in this case we only want to accept an int value.
Here is the structure of the CHANGES table:
ID
PKFROMTABLE
NEWVALUE
USERNAME
1
234
39
sk
1
345
john
jf
2
455
BIOL
jf
I would like to wrap this in a stored procedure: This is what I have so far
CREATE PROCEDURE [dbo].[sp_CleanupData]
(
#ID int,
#uniqueid VARCHAR(40),
#value VARCHAR(50)
)
AS
BEGIN TRANSACTION AddLoopback
DECLARE #columnlength int;
DECLARE #columndatatype varchar(30);
set #columnlength=(select acceptedlength from lookup where ID=#ID)
set #columndatatype=(select accepteddatatype from lookup where ID=#ID)
IF len(#value) <= #columnlength
BEGIN
INSERT INTO [CHANGES] (ID, pkfromtable,newvalue,USERNAME) VALUES (#ID, #uniqueid, #value,SUSER_SNAME())
PRINT 'New Record Inserted'
COMMIT TRANSACTION
END
ELSE
BEGIN
ROLLBACK TRANSACTION AddLoopback
PRINT 'id is not acceptable'
END
GO
How would I add validation for the datatype? And how would I make sure all validation passes before the insert? If any validation would fail per record, I do not want that record inserted into the CHANGES tables.
You can just check each datatype, do something like this
DECLARE #result BIT = 0;
IF #columndatatype = 'INT'
SET #result = IIF(TRY_CAST(#value AS INT) IS NULL, 0, 1)
ELSE IF #columndatatype = 'BIGINT'
SET #result = IIF(TRY_CAST(#value AS BIGINT) IS NULL, 0, 1)
ELSE IF #columndatatype = 'VARCHAR'
SET #result = IIF(len(#value) <= #columnlength, 0, 1)
SELECT
#result
If the cast fails, the result of the cast is null, and you set you #result to 0.
When you've finsihed your check you know if it is the correct format.
You could of course do this in a dynamic way, using sp_executesql as well.

SQL Server WHILE

I am trying to generating unique card number from following function. I put my query inside a while loop to prevent duplicate card number but still I am getting duplicate numbers.
Anyone can help me?
Create FUNCTION GetCardNumber ()
RETURNS varchar(20)
AS
BEGIN
Declare #NewID varchar(20);
Declare #NewID1 varchar(36) ;
Declare #Counter int = 0;
While(1=1)
Begin
Set #NewID1 = (SELECT [MyNewId] FROM Get_NewID);
Set #NewID = '2662464' + '823' + '001' +right(YEAR(GETUTCDATE()),2) +(left(convert(varchar,ABS(CAST(CAST(#NewID1 AS VARBINARY(5)) AS bigint))),5));
Set #Counter = (Select count(*) from ContactTBL where ContactMembershipID = #NewID);
If #Counter = 0
BEGIN
BREAK;
END
End
return #newID
END
Go
Update : I am getting MyNewID from View:
CREATE VIEW Get_NewID
AS
SELECT NEWID() AS MyNewID
GO
Many thanks in advance.
Won't this just return the same value every time you run it? I can't see anywhere where you're incrementing anything, or getting any kind of value that would give you unique values each time. You need to do something that changes the value each time, for example using the current exact date and time.
You're returning varchar(20) in line 2. To get your 'unique' NewId, you're doing this:
Set #NewId = (13 digit constant value) + (last 2 digits of current year) +
left(
convert(varchar,
ABS(CAST
(CAST(#NewID1 AS VARBINARY(5)) AS bigint)
)
)
,5)
which leaves you only 5 characters of uniqueness! This is almost certainly the issue. An easy fix may be increase the characters you return on line 2 e.g. RETURNS varchar(30)
What you're doing is unnecessarily complicated, and I think there is an element of overprotecting against potential duplicate values. This line is very suspect:
Set #NewID = '2662464' + '823' + '001' +right(YEAR(GETUTCDATE()),2) +(left(convert(varchar,ABS(CAST(CAST(#NewID1 AS VARBINARY(5)) AS bigint))),5));
The maximum for bigint is 2^63-1, so casting your 5-byte VARBINARY to a bigint could result in an overflow, which may also cause an issue.
I'm not sure exactly what you're trying to achieve, but you need to simplify things and make sure you have more scope for unique values!
Set #NewID1 = (SELECT [MyNewId] FROM Get_NewID);
always return the same result (if no other changes)
Set #NewID = '2662464' + '823' + '001' +right(YEAR(GETUTCDATE()),2) +(left(convert(varchar,ABS(CAST(CAST(#NewID1 AS VARBINARY(5)) AS bigint))),5));
as result #New_ID will be the same also

comparable varchar "arrays" in seperate fields but on same row

I have a table that looks like this:
memberno(int)|member_mouth (varchar)|Inspected_Date (varchar)
-----------------------------------------------------------------------------
12 |'1;2;3;4;5;6;7' |'12-01-01;12-02-02;12-03-03' [7 members]
So by looking at how this table has been structured (poorly yes)
The values in the member_mouth field is a string that is delimited by a ";"
The values in the Inspected_Date field is a string that is delimited by a ";"
So - for each delimited value in member_mouth there is an equal inspected_date value delimited inside the string
This table has about 4Mil records, we have an application written in C# that normalizes the data and stores it in a separate table. The problem now is because of the size of the table it takes a long time for this to process. (the example above is nothing compared to the actual table, it's much larger and has a couple of those string "array" fields)
My question is this: What would be the best and fastest way to normilize this data in MSSQL proc? let MSSQL do the work and not a C# app?
The best way will be SQL itself. The way followed in the below code is something which worked for me well with 2-3 lakhs of data.
I am not sure about the below code when it comes to 4 Million, but may help.
Declare #table table
(memberno int, member_mouth varchar(100),Inspected_Date varchar(400))
Insert into #table Values
(12,'1;2;3;4;5;6;7','12-01-01;12-02-02;12-03-03;12-04-04;12-05-05;12-07-07;12-08-08'),
(14,'1','12-01-01'),
(19,'1;5;8;9;10;11;19','12-01-01;12-02-02;12-03-03;12-04-04;12-07-07;12-10-10;12-12-12')
Declare #tableDest table
(memberno int, member_mouth varchar(100),Inspected_Date varchar(400))
The table will be like.
Select * from #table
See the code from here.
------------------------------------------
Declare #max_len int,
#count int = 1
Set #max_len = (Select max(Len(member_mouth) - len(Replace(member_mouth,';','')) + 1)
From #table)
While #count <= #max_len
begin
Insert into #tableDest
Select memberno,
SUBSTRING(member_mouth,1,charindex(';',member_mouth)-1),
SUBSTRING(Inspected_Date,1,charindex(';',Inspected_Date)-1)
from #table
Where charindex(';',member_mouth) > 0
union
Select memberno,
member_mouth,
Inspected_Date
from #table
Where charindex(';',member_mouth) = 0
Delete from #table
Where charindex(';',member_mouth) = 0
Update #table
Set member_mouth = SUBSTRING(member_mouth,charindex(';',member_mouth)+1,len(member_mouth)),
Inspected_Date = SUBSTRING(Inspected_Date,charindex(';',Inspected_Date)+1,len(Inspected_Date))
Where charindex(';',member_mouth) > 0
Set #count = #count + 1
End
------------------------------------------
Select *
from #tableDest
Order By memberno
------------------------------------------
Result.
You can take a reference here.
Splitting delimited values in a SQL column into multiple rows
Do it on SQl server side, if possible a SSIS package would be great.

Incrementing custom primary key values in SQL

I am asked to generate custom ID values for primary key columns. The query is as follows,
SELECT * FROM SC_TD_GoodsInward WHERE EntityId = #EntityId
SELECT #GoodsInwardId=IIF((SELECT COUNT(*) FROM SC_TD_GoodsInward)>0, (Select 'GI_'+RIGHT('00'+CONVERT(varchar,datepart(YY,getdate())),2)+RIGHT('00'+CONVERT(varchar,datepart(MM,getdate())),2)+RIGHT('00'+CONVERT(varchar,datepart(DD,getdate())),2)+'_'+CONVERT(varchar,#EntityId)+'_'+(SELECT RIGHT('0000'+CONVERT(VARCHAR,CONVERT(INT,RIGHT(MAX(GoodsInwardId),4))+1),4) from SC_TD_GoodsInward)), (SELECT 'GI_'+RIGHT('00'+CONVERT(varchar,datepart(YY,getdate())),2)+RIGHT('00'+CONVERT(varchar,datepart(MM,getdate())),2)+RIGHT('00'+CONVERT(varchar,datepart(DD,getdate())),2)+'_'+CONVERT(varchar,#EntityId)+'_0001'))
Here the SC_TD_GoodsInward is a table, GoodsInwardId is the value to be generated. I am getting the desired outputs too. Examples.
GI_131118_1_0001
GI_131212_1_0002
GI_131212_1_0003
But, the above condition fails when the last digits reach 9999. I simulated the query and the results were,
GI_131226_1_9997
GI_140102_1_9998
GI_140102_1_9999
GI_140102_1_0000
GI_140102_1_0000
GI_140102_1_0000
GI_140102_1_0000
GI_140102_1_0000
After 9999, it goes to 0000 and does not increment thereafter. So, in the future, I will eventually run into a PK duplicate error. How can i recycle the values so that after 9999, it goes on as 0000, 0001 ... etc. What am I missing in the above query?
NOTE: Please consider the #EntityId value to be 1 in the query.
I am using SQL SERVER 2012.
Before giving a solution for the question few points on your question:
As the Custom primary key consists of mainly three parts Date(140102), physical location where transaction takes place (entityID), 4 place number(9999).
According to the design on a single date in a single physical location there cannot be more than 9999 transactions -- My Solution will also contain the same limitation.
Some points on my solution
The 4 place digit is tied up to the date which means for a new date the count starts from 0000. For Example
GI_140102_1_0001,
GI_140102_1_0002,
GI_140102_1_0003,
GI_140103_1_0000,
GI_140104_1_0000
Any way the this field will be unique.
The solution compares the latest date in the record to the current date.
The Logic:
If current date and latest date in the record matches
Then it increments 4 place digit by the value by 1
If the current date and the latest date in the record does not matched
The it sets the 4 place digit by the value 0000.
The Solution: (Below code gives out the value which will be the next GoodsInwardId, Use it as per requirement to fit in to your solution)
declare #previous nvarchar(30);
declare #today nvarchar(30);
declare #newID nvarchar(30);
select #previous=substring(max(GoodsInwardId),4,6) from SC_TD_GoodsInward;
Select #today=RIGHT('00'+CONVERT(varchar,datepart(YY,getdate())),2)
+RIGHT('00'+CONVERT(varchar,datepart(MM,getdate())),2)+RIGHT('00'+CONVERT(varchar,datepart(DD,getdate())),2);
if #previous=#today
BEGIN
Select #newID='GI_'+RIGHT('00'+CONVERT(varchar,datepart(YY,getdate())),2)
+RIGHT('00'+CONVERT(varchar,datepart(MM,getdate())),2)+RIGHT('00'+CONVERT(varchar,datepart(DD,getdate())),2)
+'_'+CONVERT(varchar,1)+'_'+(SELECT RIGHT('0000'+
CONVERT(VARCHAR,CONVERT(INT,RIGHT(MAX(GoodsInwardId),4))+1),4)
from SC_TD_GoodsInward);
END
else
BEGIN
SET #newID='GI_'+RIGHT('00'+CONVERT(varchar,datepart(YY,getdate())),2)
+RIGHT('00'+CONVERT(varchar,datepart(MM,getdate())),2)+RIGHT('00'+CONVERT(varchar,datepart(DD,getdate())),2)
+'_'+CONVERT(varchar,1)+'_0000';
END
select #newID;
T-SQL to create the required structure (Probable Guess)
For the table:
CREATE TABLE [dbo].[SC_TD_GoodsInward](
[EntityId] [int] NULL,
[GoodsInwardId] [nvarchar](30) NULL
)
Sample records for the table:
insert into dbo.SC_TD_GoodsInward values(1,'GI_140102_1_0000');
insert into dbo.SC_TD_GoodsInward values(1,'GI_140101_1_9999');
insert into dbo.SC_TD_GoodsInward values(1,'GI_140101_1_0001');
**Its a probable solution in your situation although the perfect solution would be to have identity column (use reseed if required) and tie it with the current date as a computed column.
You get this problem because once the last 4 digits reach 9999, 9999 will remain the highest number no matter how many rows are inserted, and you are throwing away the most significant digit(s).
I would remodel this to track the last used INT portion value of GoodsInwardId in a separate counter table (as an INTEGER), and then MODULUS (%) this by 10000 if need be. If there are concurrent calls to the PK generator, remember to lock the counter table row.
Also, even if you kept all the digits (e.g. in another field), note that ordering a CHAR is as follows
1
11
2
22
3
and then applying MAX() will return 3, not 22.
Edit - Clarification of counter table alternative
The counter table would look something like this:
CREATE TABLE PK_Counters
(
TableName NVARCHAR(100) PRIMARY KEY,
LastValue INT
);
(Your #EntityID might be another candidate for the counter PK column.)
You then increment and fetch the applicable counter on each call to your custom PK Key generation PROC:
UPDATE PK_Counters
SET LastValue = LastValue + 1
WHERE TableName = 'SC_TD_GoodsInward';
Select
'GI_'+RIGHT('00'+CONVERT(varchar,datepart(YY,getdate())),2)
+RIGHT('00'+CONVERT(varchar,datepart(MM,getdate())),2)
+RIGHT('00'+CONVERT(varchar,datepart(DD,getdate())),2)+'_'
+CONVERT(varchar,#EntityId)+'_'
+(SELECT RIGHT('0000'+ CONVERT(NVARCHAR, LastValue % 10000),4)
FROM PK_Counters
WHERE TableName = 'SC_TD_GoodsInward');
You could also modulo the LastValue in the counter table (and not in the query), although I believe there is more information about the number of records inserted by leaving the counter un-modulo-ed.
Fiddle here
Re : Performance - Selecting a single integer value from a small table by its PK and then applying modulo will be significantly quicker than selecting MAX from a SUBSTRING (which would almost certainly be a scan)
DECLARE #entityid INT = 1;
SELECT ('GI_'
+ SUBSTRING(convert(varchar, getdate(), 112),3,6) -- yymmdd today DATE
+ '_' + CAST(#entityid AS VARCHAR(50)) + '_' --#entity parameter
+ CASE MAX(t.GI_id + 1) --take last number + 1
WHEN 10000 THEN
'0000' --reset
ELSE
RIGHT( CAST('0000' AS VARCHAR(4)) +
CAST(MAX(t.GI_id + 1) AS VARCHAR(4))
, 4)
END) PK
FROM
(
SELECT TOP 1
CAST(SUBSTRING(GoodsInwardId,11,1) AS INT) AS GI_entity,
CAST(SUBSTRING(GoodsInwardId,4,6) AS INT) AS GI_date,
CAST(RIGHT(GoodsInwardId,4) AS INT) AS GI_id
FROM SC_TD_GoodsInward
WHERE CAST(SUBSTRING(GoodsInwardId,11,1) AS INT) = #entityid
ORDER BY gi_date DESC, rowTimestamp DESC, gi_id DESC
) AS t
This should take the last GoodInwardId record, ordered by date DESC and take its numeric "id". Then add + 1 to return the NEW id and combine it with today's date and the #entityid you passed. If >9999, start again from 0000.
You need a timestamp type column tho, to order two inserted in the same date + same transaction time. Otherwise you could get duplicates.
I have simplified the answer even more and arrived with the following query.
IF (SELECT COUNT(GoodsInwardId) FROM SC_TD_GoodsInward WHERE EntityId = #EntityId)=0
BEGIN
SELECT #GoodsInwardId= 'GI_'+RIGHT('00'+CONVERT(varchar,datepart(YY,getdate())),2)+
RIGHT('00'+CONVERT(varchar,datepart(MM,getdate())),2)+
RIGHT('00'+CONVERT(varchar,datepart(DD,getdate())),2)+'_'+
CONVERT(varchar,#EntityId)+'_0001'
END
ELSE
BEGIN
SELECT * FROM SC_TD_GoodsInward WHERE EntityId = #EntityId AND CONVERT(varchar,CreatedOn,103) = CONVERT(varchar,GETDATE(),103)
SELECT #GoodsInwardId=IIF(##ROWCOUNT>0,
(Select 'GI_'+
RIGHT('00'+CONVERT(varchar,datepart(YY,getdate())),2)+
RIGHT('00'+CONVERT(varchar,datepart(MM,getdate())),2)+
RIGHT('00'+CONVERT(varchar,datepart(DD,getdate())),2)+'_'+
CONVERT(varchar,#EntityId)+'_'+
(SELECT RIGHT('0000'+CONVERT(VARCHAR,CONVERT(INT,RIGHT(MAX(GoodsInwardId),4))+1),4) from SC_TD_GoodsInward WHERE CONVERT(varchar,CreatedOn,103) = CONVERT(varchar,GETDATE(),103))),
(SELECT 'GI_'+RIGHT('00'+CONVERT(varchar,datepart(YY,getdate())),2)+
RIGHT('00'+CONVERT(varchar,datepart(MM,getdate())),2)+
RIGHT('00'+CONVERT(varchar,datepart(DD,getdate())),2)+'_'+
CONVERT(varchar,#EntityId)+'_0001'))
END
select * from SC_TD_GoodsInward

sql server cursor

I want to copy data from one table (rawdata, all columns are VARCHAR) to another table (formatted with corresponding column format).
For copying data from the rawdata table into formatted table, I'm using cursor in order to identify which row is affected. I need to log that particular row in an error log table, skip it, and continue copying remaining rows.
It takes more time to copying. Is there any other way to achieve this?
this is my query
DECLARE #EntityId Varchar(16) ,
#PerfId Varchar(16),
#BaseId Varchar(16) ,
#UpdateStatus Varchar(16)
DECLARE CursorSample CURSOR FOR
SELECT EntityId, PerfId, BaseId, #UpdateStatus
FROM RawdataTable
--Returns 204,000 rows
OPEN CursorSample
FETCH NEXT FROM CursorSample INTO #EntityId,#PerfId,#BaseId,#UpdateStatus
WHILE ##FETCH_STATUS = 0
BEGIN
BEGIN TRY
--try insertting row in formatted table
Insert into FormattedTable
(EntityId,PerfId,BaseId,UpdateStatus)
Values
(Convert(int,#EntityId),
Convert(int,#PerfId),
Convert(int,#BaseId),
Convert(int,#UpdateStatus))
END TRY
BEGIN CATCH
--capture Error EntityId in errorlog table
Insert into ERROR_LOG
(TableError_Message,Error_Procedure,Error_Log_Time)
Values
(Error_Message()+#EntityId,’xxx’, GETDATE())
END CATCH
FETCH NEXT FROM outerCursor INTO #EntityId, #BaseId
END
CLOSE CursorSample
DEALLOCATE CursorSampler –cleanup CursorSample
You should just be able to use a INSERT INTO statement to put the records directly into the formatted table. INSERT INTO will perform much better than using a cursor.
INSERT INTO FormattedTable
SELECT
CONVERT(int, EntityId),
CONVERT(int, PerfId),
CONVERT(int, BaseId),
CONVERT(int, UpdateStatus)
FROM RawdataTable
WHERE
IsNumeric(EntityId) = 1
AND IsNumeric(PerfId) = 1
AND IsNumeric(BaseId) = 1
AND IsNumeric(UpdateStatus) = 1
Note that IsNumeric can sometimes return 1 for values that will then fail on CONVERT. For example, IsNumeric('$e0') will return 1, so you may need to create a more robust user defined function for determining if a string is a number, depending on your data.
Also, if you need a log of all records that could not be moved into the formatted table, just modify the WHERE clause:
INSERT INTO ErrorLog
SELECT
EntityId,
PerfId,
BaseId,
UpdateStatus
FROM RawdataTable
WHERE
NOT (IsNumeric(EntityId) = 1
AND IsNumeric(PerfId) = 1
AND IsNumeric(BaseId) = 1
AND IsNumeric(UpdateStatus) = 1)
EDIT
Rather than using IsNumeric directly, it may be better to create a custom UDF that will tell you if a string can be converted to an int. This function worked for me (albeit with limited testing):
CREATE FUNCTION IsInt(#value VARCHAR(50))
RETURNS bit
AS
BEGIN
DECLARE #number AS INT
DECLARE #numeric AS NUMERIC(18,2)
SET #number = 0
IF IsNumeric(#value) = 1
BEGIN
SET #numeric = CONVERT(NUMERIC(18,2), #value)
IF #numeric BETWEEN -2147483648 AND 2147483647
SET #number = CONVERT(INT, #numeric)
END
RETURN #number
END
GO
The updated SQL for the insert into the formatted table would then look like this:
INSERT INTO FormattedTable
SELECT
CONVERT(int, CONVERT(NUMERIC(18,2), EntityId)),
CONVERT(int, CONVERT(NUMERIC(18,2), PerfId)),
CONVERT(int, CONVERT(NUMERIC(18,2), BaseId)),
CONVERT(int, CONVERT(NUMERIC(18,2), UpdateStatus))
FROM RawdataTable
WHERE
dbo.IsInt(EntityId) = 1
AND dbo.IsInt(PerfId) = 1
AND dbo.IsInt(BaseId) = 1
AND dbo.IsInt(UpdateStatus) = 1
There may be a little weirdness around handling NULLs (my function will return 0 if NULL is passed in, even though an INT can certainly be null), but that can be adjusted depending on what is supposed to happen with NULL values in the RawdataTable.
You can put a WHERE clause in your cursor definition so that only valid records are selected in the first place. You might need to create a function to determine validity, but it should be faster than looping over them.
Actually, you might want to create a temp table of the invalid records, so that you can log the errors, then define the cursor only on the rows that are not in the temp table.
Insert into will work much more better than Cursor.
As Cursor work solely in Memory of your PC and slows down the optimization of SQL Server. We should avoid using Cursors but (of course) there are situations where usage of Cursor cannot be avoided.