Here's what I have to do :
I have a text file which has 3 columns: PID, X, Y.
Now I have two tables in my database:
Table 1 contains 4 columns: UID, PID, X, Y
Table 2 contains multiple columns, required ones being UID, X, Y
I need to update Table 2 with corresponding X and Y values.
I think we can use BULK INSERT for updating table 1, then some WHILE loop or something.
But I can't figure out exact thing.
CREATE PROCEDURE [dbo].[BulkInsert]
(
#PID int ,
#x int,
#y int,
)
AS
BEGIN
SET NOCOUNT ON;
declare #query varchar(max)
CREATE TABLE #TEMP
(
[PID] [int] NOT NULL ,
[x] int NOT NULL,
[y] int NOT NULL,
)
SET #query = 'BULK INSERT #TEMP FROM ''' + PathOfYourTextFile + ''' WITH ( FIELDTERMINATOR = '','',ROWTERMINATOR = ''\n'')'
--print #query
--return
execute(#query)
BEGIN TRAN;
MERGE TableName AS Target
USING (SELECT * FROM #TEMP) AS Source
ON (Target.YourTableId = Source.YourTextFileFieldId)
-- In the above line we are checking if the particular row exists in the table(Table1) then update the Table1 if not then insert the new row in Table-1.
WHEN MATCHED THEN
UPDATE SET
Target.PID= Source.PID, Target.x= Source.x, Target.y= Source.y
WHEN NOT MATCHED BY TARGET THEN
-- Insert statement
You can use this above approach to solve your problem. Hope this helps. :)
How are you going to run it ? From a stored procedure ?
To save some performance, I would have done BULK INSERT to temp table, then insert from temp table to Table 1 & 2.
It should look like this
INSERT INTO Table1 ( PID, X, Y)
SELECT PID, X, Y
FROM #tempTable
Some will tell that temp table are not good, but it really depend - if you file is big, reading it from disk will take time and you don't want to do it twice.
You don't need any loop to update table 2; all you need is insert from table 1.
Or, if you are trying to update existing rows in table 2, use an update query that joins on table 1. See this question for an example.
However, you should consider changing your database design, as it appears to be incorrect: you are storing X and Y in two places; they should only be stored in one table, and you should join to this table if you need to use them in conjunction with other data. If you did this, you wouldn't have to worry about messy issues of keeping the two tables in sync.
Related
I would like to be able to pull the custom key value from a table, but would also like it to perform like SQL Server's IDENTITY(1,1) column on inserts.
The custom key is for another application and will need to be used by different functions so the value will need to be pulled from a table and available for other areas.
Here are some if my attempts:
Tried a trigger on the table works well on single inserts, failed on using SQL insert (forgetting the fact that a triggers are not per row but by batch)
ALTER TRIGGER [sales].[trg_NextInvoiceDocNo]
ON [sales].[Invoice]
AFTER INSERT
AS
BEGIN
DECLARE #ResultVar VARCHAR(25)
DECLARE #Key VARCHAR(25)
EXEC [dbo].[usp_GetNextKeyCounterChar]
#tcForTbl = 'docNbr', #tcForGrp = 'docNbr', #NewKey = #ResultVar OUTPUT
UPDATE sales.InvoiceRET
SET DocNbr = #ResultVar
FROM sales.InvoiceRET
JOIN inserted ON inserted.id = sales.InvoiceRET.id;
END;
Thought about a scalar function, but functions cannot exec stored procedures or update statements in order to set the new key value in the lookup table.
Thanks
You can use ROW_NUMBER() depending on the type of concurrency you are dealing with. Here is some sample data and a demo you can run locally.
-- Sample table
USE tempdb
GO
IF OBJECT_ID('dbo.sometable','U') IS NOT NULL DROP TABLE dbo.sometable;
GO
CREATE TABLE dbo.sometable
(
SomeId INT NULL,
Col1 INT NOT NULL
);
GO
-- Stored Proc to insert data
CREATE PROC dbo.InsertProc #output BIT AS
BEGIN -- Your proc starts here
INSERT dbo.sometable(Col1)
SELECT datasource.[value]
FROM (VALUES(CHECKSUM(NEWID())%100)) AS datasource([value]) -- simulating data from somewhere
CROSS APPLY (VALUES(1),(1),(1)) AS x(x);
WITH
id(MaxId) AS (SELECT ISNULL(MAX(t.SomeId),0) FROM dbo.sometable AS t),
xx AS
(
SELECT s.SomeId, RN = ROW_NUMBER() OVER (ORDER BY (SELECT NULL))+id.MaxId, s.Col1, id.MaxId
FROM id AS id
CROSS JOIN dbo.sometable AS s
WHERE s.SomeId IS NULL
)
UPDATE xx SET xx.SomeId = xx.RN;
IF #output = 1
SELECT t.* FROM dbo.sometable AS t;
END
GO
Each time I run: EXEC dbo.InsertProc 1; it returns 3 more rows with the correct ID col. Each time I execute it, it adds more rows and auto-increments as needed.
SomeId Col1
-------- ------
1 62
2 73
3 -17
Let's say I have two table variables declared as below:
DECLARE #Table1 TABLE (
A INT,
B NVARCHAR(100)
)
DECLARE #Table2 TABLE (
A INT,
C NVARCHAR(100)
)
Here are the contents of #Table1:
1, 'Hello'
2, 'Hi'
3, 'Ola'
These are the contents of #Table2:
1, 'my old friend'
1, 'sweetheart'
2, 'buddy'
4, 'the end'
Now I want to insert #Table1 into a table X and #Table2 into a table Y. The scenario is that I have to maintain transaction integrity for the insertion into both X and Y for every same value of column A.
For instance, let's say I am inserting the first row (1,'Hello') of #Table1 into X. This means I must also insert the first two rows ((1,'my old friend'), (1,'sweetheart')) of #Table2 into Y in the same transaction. So if any insert of Y fails for A=1, X also fails for A=1. For any value of column A that is not in both #Table1 and #Table2, they are individual transactions by themselves (e.g. A=3 in #Table1 and A=4 in #Table2).
Here are the ways I see to deal with this problem:
I fetch all values of A in both #Table1 and #Table2, run a cursor over it and then for each value of A, I insert into tables X and Y in a single transaction. The issue here is first of all, I don't want to use cursors as much as possible and also, this would mean a super large number of individual inserts.
I pre-validate my #Table1 and #Table2 values and then do one single insert of #Table1 on X and #Table2 on Y. This will be much faster than the above method. But the issues I see here are that first of all, not putting it in a 'transaction' somehow doesn't seem right and also, there could be a small chance I might have missed a validation somewhere (unlikely, yet still).
Which approach should I go for? Is there a better solution?
P.S. Please also note that I do not want to fail the entire insert on X and Y if there is an issue in inserting for only one or few values of A. Also, going back and deleting tables from my DB based on the failed inserts is also not an option as it messes with the running id continuity which I am trying to avoid.
A DML statement is executed completely or not executed at all.
You can do a mix of your two options
First add as much validations as possible, if it fails run the second one using a temp table instead of a cursor
BEGIN TRY
BEGIN TRAN Opt2
--Option 2
COMMIT TRAN Opt2
END TRY
BEGIN CATCH
ROLLBACK TRAN Opt2
DECLARE #A INT, #B VARCHAR(100)
SELECT * INTO #TMP FROM #Table1
WHILE EXISTS (SELECT 1 FROM #TMP)
BEGIN
SELECT TOP 1 #A = A, #B = B FROM #Table1
-- Option 1
DELETE #Temp WHERE A = #A
END
END CATCH
I'm trying to accomplish the following thing:
I receive a DocumentID. Find all the records in a table that match the specific DocumentID, for example I have 10 records matching and every record is with different DocumentAttachmentID.
I update all the records with the new data.
The problem comes that I need to insert some of the information from these ten records + other information received to a new table, which is History table, i.e. I need to insert ten new records there.
I've succeeded to this with Cursor, but it looks like that the Cursor isn't really good, because of the performance.
Is there a way to loop throught the 10 records that I selected from this table and for every record to take some information, add some additional info and then insert this in the other table ?
EDIT:
I tried to do this withoud looping(thanks you all for the answers)
I will try it tomorrow, do you think this is gonna work ?:
With the first Update, I update all documentAttachments,
The second block is INSERT TO, which should insert all document attachments in the other table with some extra columns.
UPDATE [sDocumentManagement].[tDocumentAttachments]
SET DeletedBy = #ChangedBy,
DeletedOn = #CurrentDateTime,
IsDeleted = 1,
WHERE DocumentID = #DocumentID;
INSERT INTO [sDocumentManagement].[tDocumentHistory] ( DocumentAttachmentID, DocumentID, ActivityCodeID, ChangedOn, ChangedBy, AdditionalInformation )
SELECT DocumentAttachmentID,
#DocumentID, [sCore].[GetActivityCodeIDByName] ( 'DeletedDocument' ),
#CurrentDateTime,
#ChangedBy,
#AdditionalInformation
FROM [sDocumentManagement].[tDocumentAttachments]
WHERE DocumentID = #DocumentID;
for looping without a cursor I quite often use the following technique:
DECLARE #items TABLE(id INT, val INT);
DECLARE #id INT;
DECLARE #val INT;
WHILE EXISTS(SELECT * FROM #items) BEGIN
SELECT TOP(1) #id = id, #val = val FROM #items;
DELETE FROM #items WHERE (id = #id);
--do what is needed with the values here.
SELECT #id, #val;
END
this treats the #items table as a queue pulling the rows off one at a time till it is empty.
I have a table called Employee. The EmpId column serves as the primary key. In my scenario, I cannot make it AutoNumber.
What would be the best way of generating the the next EmpId for the new row that I want to insert in the table?
I am using SQL Server 2008 with C#.
Here is the code that i am currently getting, but to enter Id's in key value pair tables or link tables (m*n relations)
Create PROCEDURE [dbo].[mSP_GetNEXTID]
#NEXTID int out,
#TABLENAME varchar(100),
#UPDATE CHAR(1) = NULL
AS
BEGIN
DECLARE #QUERY VARCHAR(500)
BEGIN
IF EXISTS (SELECT LASTID FROM LASTIDS WHERE TABLENAME = #TABLENAME and active=1)
BEGIN
SELECT #NEXTID = LASTID FROM LASTIDS WHERE TABLENAME = #TABLENAME and active=1
IF(#UPDATE IS NULL OR #UPDATE = '')
BEGIN
UPDATE LASTIDS
SET LASTID = LASTID + 1
WHERE TABLENAME = #TABLENAME
and active=1
END
END
ELSE
BEGIN
SET #NEXTID = 1
INSERT INTO LASTIDS(LASTID,TABLENAME, ACTIVE)
VALUES(#NEXTID+1,#TABLENAME, 1)
END
END
END
Using MAX(id) + 1 is a bad idea both performance and concurrency wise.
Instead you should resort to sequences which were design specifically for this kind of problem.
CREATE SEQUENCE EmpIdSeq AS bigint
START WITH 1
INCREMENT BY 1;
And to generate the next id use:
SELECT NEXT VALUE FOR EmpIdSeq;
You can use the generated value in a insert statement:
INSERT Emp (EmpId, X, Y)
VALUES (NEXT VALUE FOR EmpIdSeq, 'x', 'y');
And even use it as default for your column:
CREATE TABLE Emp
(
EmpId bigint PRIMARY KEY CLUSTERED
DEFAULT (NEXT VALUE FOR EmpIdSeq),
X nvarchar(255) NULL,
Y nvarchar(255) NULL
);
Update: The above solution is only applicable to SQL Server 2012+. For older versions you can simulate the sequence behavior using dummy tables with identity fields:
CREATE TABLE EmpIdSeq (
SeqID bigint IDENTITY PRIMARY KEY CLUSTERED
);
And procedures that emulates NEXT VALUE:
CREATE PROCEDURE GetNewSeqVal_Emp
#NewSeqVal bigint OUTPUT
AS
BEGIN
SET NOCOUNT ON
INSERT EmpIdSeq DEFAULT VALUES
SET #NewSeqVal = scope_identity()
DELETE FROM EmpIdSeq WITH (READPAST)
END;
Usage exemple:
DECLARE #NewSeqVal bigint
EXEC GetNewSeqVal_Emp #NewSeqVal OUTPUT
The performance overhead of deleting the last inserted element will be minimal; still, as pointed out by the original author, you can optionally remove the delete statement and schedule a maintenance job to delete the table contents off-hour (trading space for performance).
Adapted from SQL Server Customer Advisory Team Blog.
Working SQL Fiddle
The above
select max(empid) + 1 from employee
is the way to get the next number, but if there are multiple user inserting into the database, then context switching might cause two users to get the same value for empid and then add 1 to each and then end up with repeat ids. If you do have multiple users, you may have to lock the table while inserting. This is not the best practice and that is why the auto increment exists for database tables.
I hope this works for you. Considering that your ID field is an integer
INSERT INTO Table WITH (TABLOCK)
(SELECT CASE WHEN MAX(ID) IS NULL
THEN 1 ELSE MAX(ID)+1 END FROM Table), VALUE_1, VALUE_2....
Try following query
INSERT INTO Table VALUES
((SELECT isnull(MAX(ID),0)+1 FROM Table), VALUE_1, VALUE_2....)
you have to check isnull in on max values otherwise it will return null in final result when table contain no rows .
I have been asked to keep track of how many times each item comes up within the results of a particular query. My thought is to just store the result of a query into a tracking table and then spit the results back to the caller. I am wondering what the most efficient method of storing these results would be since the result set could include up to 1000 records.
My plan is to pull the query results into a temp table and insert those results into the tracking table then return the temp table as the result of the SPROC. Something like this:
DECLARE #QueryTime datetime
SET #QueryTime = GETDATE()
DECLARE #Results TABLE (X nvarchar(255), Y nvarchar(255))
INSERT INTO #Results
SELECT X,Y FROM TableA
INSERT INTO TableB
SELECT X, #QueryTime FROM #Results
SELECT X, Y FROM #Results
Does anyone have a more efficient way to post the bulk result set into a tracking table?
You don't need the #Results table.
Directly inserting and selecting from TableA does the job and will most likely be the most efficient way.
DECLARE #QueryTime DATETIME
SET #QueryTime = GetDate()
INSERT INTO TableB
SELECT X, #QueryTime FROM TableA
SELECT X, Y FROM #TableA
If you are using Sql 2008, you could return the rows as XML. Then you would just have 1 row to insert in your tracking table.