Dynamically display rows as columns - sql

I couldn't think of a good way to word the title, if anyone can come up with something better please feel free. Basically there is an old VB6 app that pulls data from a db that I have more or less completely restructured and gives the user a dump of all of the product information at once. So I need to do some inner joins to get all of these tables together. I know how to do basic inner joins but I am stuck on one detail. There are a few tables where there are multiple entries for each item. For example, the CrossReference table may have multiple cross reference numbers for an item, or it may only have one, or it may have none at all. Is it possible to have those placed dynamically into separate columns. so
this:
Item CrossReferenceNumber
XXXXX crossref1
XXXXX crossref2
XXXXX crossref3
could become this (after a join with some other tables):
Item BasePart Size CrossReferenceNumber1 CrossReferenceNumber2 CrossReferenceNumber3
XXXX XXXX Large crossref1 crossref2 crossref3
But if there were no cross references, there would be no cross reference columns. Is something like that possible or am I dreaming?

Oracle 11g and Sql Server 2005+ both contain a pivot command that will accomplish what you want.
http://www.orafaq.com/wiki/PIVOT
http://msdn.microsoft.com/en-us/library/ms177410.aspx
Otherwise you would need to build a dynamic sql statement to achieve this.
Edit - Here you go (SQL Server version).
/* Begin Set up of test data */
IF EXISTS (SELECT 1 from sys.tables WHERE name = N'Item')
DROP TABLE Item
GO
IF EXISTS (SELECT 1 from sys.tables WHERE name = N'CrossReference')
DROP TABLE CrossReference
GO
CREATE TABLE Item
(
Item varchar(20),
BasePart varchar(20),
Size varchar(20)
);
CREATE Table CrossReference
(
Item varchar(20),
CrossReferenceNumber varchar(20)
);
INSERT INTO Item VALUES ('item1', 'b1', 'Large');
INSERT INTO Item VALUES ('item2', 'bxx1', 'Large');
INSERT INTO Item VALUES ('item3', 'bddf1', 'Small');
INSERT INTO Item VALUES ('item4', 'be3f1', 'Small');
INSERT INTO Item VALUES ('item5', 'b13vx1', 'Small');
INSERT INTO CrossReference VALUES( 'item1', 'crossRef1')
INSERT INTO CrossReference VALUES('item1', 'crossRef2')
INSERT INTO CrossReference VALUES('item1', 'crossRef3')
INSERT INTO CrossReference VALUES('item1', 'crossRef4')
INSERT INTO CrossReference VALUES('item2', 'crossRef1')
INSERT INTO CrossReference VALUES('item2', 'crossRef1')
INSERT INTO CrossReference VALUES('item3', 'crossRef1')
INSERT INTO CrossReference VALUES('item4', 'crossRef2')
INSERT INTO CrossReference VALUES('item5', 'crossRef5')
INSERT INTO CrossReference VALUES('item5', 'crossRef1')
INSERT INTO CrossReference VALUES('item5', 'crossRef2')
INSERT INTO CrossReference VALUES('item5', 'crossRef3')
/* End of test data setup */
/* Begin of actual query */
DECLARE #xRefs VARCHAR(2000),
#query VARCHAR(8000)
SELECT #xRefs = STUFF((SELECT DISTINCT '],[' + ltrim(CrossReferenceNumber)
FROM CrossReference
ORDER BY '],[' + ltrim(CrossReferenceNumber)
FOR XML PATH('')
), 1, 2, '') + ']'
SET #query =
'SELECT *
FROM Item i
INNER JOIN
(
SELECT *
FROM
(
SELECT Item, CrossReferenceNumber
FROM CrossReference
) t
PIVOT (MAX(CrossReferenceNumber) FOR CrossReferenceNumber IN (' + #xRefs + ')) as pvt
) xRefs
ON i.Item = xRefs.Item
ORDER BY i.Item'
EXECUTE (#query)
/* end */

Related

Update Id on INSERT INTO SELECT FROM statement

i want to copy specific columns from Table [From] to Table [To] but also want to insert the foreign key from [To] in [From]
Table definitions:
[From]
Id (int)
pic (varbinary(MAX))
picId (int)
[To]
Id (int)
pic (varbinary(MAX))
My copy query works perfectly but i dont know how to update the "picId" column inside of the Table [From]
INSERT INTO dbo.[To] (Id,pic)
SELECT
isnull(T.m, 0) + row_number() over (order by F.pic),
F.pic
FROM dbo.[From] AS F
outer apply (select max(pic) as m from dbo.[To]) as T;
Now i want to copy the inserted [To].Id to [From].picId.
Can anyone help me please?
2 changes would make your solution much easier, but assuming you can't control anything about the dbo.[TO] table, here is a solution that will work for you.
The first improvement would be making dbo.[to].ID an identity column. Then you could drop your whole "row_number() over" line and let SQL manage the ID. What you're doing works, but it's like cutting wood with a chain saw by dragging the (not running) saw back and forth over the wood. You can do it, but it's a lot easier if you start the engine and let it do the work.
The second change is adding a column dbo.[to].FromID and populating it when you insert the row. The OUTPUT statement can only reference fields in the row being inserted (or deleted, but that's not relevant here) so you can't get the ID of the row in dbo.[from] that you want to update unless you have it in the row in dbo.[to] you just inserted. If you do this, you can use a plain old INSERT with an OUTPUT clause. The trick here is using a MERGE statement, and you can see a full explanation here: Is it possible to for SQL Output clause to return a column not being inserted? I strongly urge you to upvote that answer if you find this useful. I could not have provided you with this without that answer!
Anyway, here is the solution:
--Create some fake data, you'll already have dbo.[From]
CREATE TABLE #TestFrom (FromID INT, Pic nvarchar(1000), ToID INT NULL)
CREATE TABLE #TestTo (ToID INT, Pic nvarchar(1000))
--It would be much easier if your TO used an IDENTITY column instead of managing the ID manually
CREATE TABLE #TestToIdent (ToID INT IDENTITY(1,1), Pic nvarchar(1000))
INSERT INTO #TestFrom
VALUES (1, 'Test 1', NULL)
, (3, 'Test 3', NULL)
, (7, 'Test 7', NULL)
, (13, 'Test 13', NULL)
--Define a table variable to hold your OUTPUT, you'll need this
DECLARE #Mapping as table (FromID INT, ToID INT);
with cteIns as (
SELECT
isnull(T.m, 0) + row_number() over (order by F.pic) as ToID
, F.pic, F.FromID
FROM #TestFrom AS F
outer apply (select max(ToID) as m from #TestTo ) as T
) --From https://stackoverflow.com/questions/10949730/is-it-possible-to-for-sql-output-clause-to-return-a-column-not-being-inserted
MERGE INTO #TestTo as T --Put results here
USING cteIns as F --Source is here
on 0=1 --But this is never true so never merge, just INSERT
WHEN NOT MATCHED THEN --ALWAYS because 0 never = 1
INSERT (ToID, pic)
VALUES (F.ToID, F.pic)
OUTPUT F.FromID, inserted.ToID
INTO #Mapping (FromID, ToID );
--SELECT * FROM #Mapping --Test the mapping if you're curious
--SELECT * FROM #TestTo --Test the insedert if you're curious
--Update the dbo.[FROM] with the ID of the [TO] that got inserted
UPDATE #TestFrom SET ToID = M.ToID
FROM #TestFrom as F
INNER JOIN #Mapping as M
ON M.FromID = F.FromID
--View the results
SELECT * FROM #TestFrom
DROP TABLE #TestFrom
DROP TABLE #TestTo
DROP TABLE #TestToIdent

Replacing a 3 level deep nested cursors in SQL Server

I have three SQL Server tables that I need to loop trhough and update. I did it successfully with a cursor but it is so slow that it is pretty pointless sincethe main table with all the data to loop through is over 1,000 rows long.
The tables are (with some sample data):
-- The PK is InvoiceId and the IsMajorPart is '0' or '1'.
-- The MajorPartId and SubPartId1 to 4 are "technically" FKs for PartId but aren't hooked up and will not be ever due to some external issues outside of scope.
-- The part Id's can be NULL or empty.
-- This table exists elsewhere and is loaded with Id's being varchars but in transfering they will be going in as int's which is the proper way.
CREATE TABLE dbo.Invoices(
InvoicdeId varchar(50),
PartName varchar(255),
IsMajorPart varchar(1)
MajorPartId varchar(50),
SubPartId1 varchar(50),
SubPartId2 varchar(50),
SubPartId3 varchar(50),
SubPartId4 varchar(50));
-- Sampe inserts
INSERT INTO dbo.Invoices VALUES ('1', 'A Part', '0', '', '100', '105', '' ,''):
INSERT INTO dbo.Invoices VALUES ('5', 'E Part', '1', '101', '110', '', '' ,''):
INSERT INTO dbo.Invoices VALUES ('11', 'Z Part', '1', '201', '100', '115', '' ,''):
-- Essentially the old table above is being moved into a normalized, correct tables below.
- The PK is the PartId
CREATE TABLE dbo.Parts
PartsId int,
PartName varchar(255)
-- Sampe inserts (that will be updated or inserted by looping through the first table)
INSERT INTO dbo.Parts VALUES (100,'A Part'):
INSERT INTO dbo.Parts VALUES (110,'B Part'):
INSERT INTO dbo.Parts VALUES (201,'C Part'):
-- The PK is the combination of InvoiceId and PartId
CREATE TABLE dbo.InvoiceToParts
InvoiceId int,
PartsId int,
IsMajorPart bit);
-- Sampe inserts (that will be inserted from the first table but conflicts might occur if an InvoiceId from the first table has 2 PartId's that are the same)
INSERT INTO dbo.Parts VALUES (1, 100, 0):
INSERT INTO dbo.Parts VALUES (5, 100, 1):
INSERT INTO dbo.Parts VALUES (17, 201, 0):
The sample INSERTs above are just samples of the data for seeing what is in the tables.
The rules to move Invoices (I don't care what happens to this table), into the correct tables of Parts and InvoiceToParts are below (and these last two tables are the only ones that I care about.
Loop through Invoices and get all the data.
First, find out if IsMajorPart is '1' and then get the MajorPartId.
Push the MajorPartId with PartName in Parts table if it DOESN'T already exist.
Next check InvoiceToParts to see if the PK of InvoiceId and PartId exist.
If they do, update IsMaorPart to '1'.
If they don't exist, INSERT it.
Next do the same process for all SubPartId1 to SubPartId4.
I have a nested 3-level cursor which performance-wise ran for over 30min before I stopped it as it wasn't even close to finishing and was sucking up all the resources. I am trying to look for a faster way to do this. The Invoices table can have up to about 5,000 rows in it.
You need to unpivot your data and then just do what is called an UPSERT, which has two steps:
If exists, update record(s)
If not exists, insert record(s)
Plenty of examples if you search for examples online for UPSERT
Table Setup
DROP TABLE IF EXISTS #Invoice
DROP TABLE IF EXISTS #Unpivot
DROP TABLE IF EXISTS #InvoiceToParts
DROP TABLE IF EXISTS #Parts
CREATE TABLE #Parts(
PartsId int,
PartName varchar(255)
)
CREATE TABLE #InvoiceToParts(
InvoiceId int,
PartsId int,
IsMajorPart bit
);
CREATE TABLE #Invoice(
InvoiceId varchar(50),
PartName varchar(255),
IsMajorPart varchar(1),
MajorPartsID varchar(50),
SubPartsID1 varchar(50),
SubPartsID2 varchar(50),
SubPartsID3 varchar(50),
SubPartsID4 varchar(50)
);
INSERT INTO #Invoice
VALUES ('1', 'A Part', '0', '', '100', '105', '' ,'')
,('5', 'E Part', '1', '101', '110', '', '' ,'')
,('11', 'Z Part', '1', '201', '100', '115', '' ,'')
SQL to Process Data
Will first unpivot the data, then load into Parts table first so the ID's can be referenced before inserting into the junction table InvoicetoParts
SELECT A.InvoiceId
,B.*
INTO #Unpivot
FROM #Invoice AS A
CROSS APPLY (
VALUES
(NULLIF(MajorPartsID,''),PartName,IsMajorPart)
,(NULLIF(SubPartsID1,''),NULL,0)
,(NULLIF(SubPartsID2,''),NULL,0)
,(NULLIF(SubPartsID3,''),NULL,0)
,(NULLIF(SubPartsID4,''),NULL,0)
) AS B(PartsID,PartName,IsMajorPart)
WHERE B.PartsID IS NOT NULL /*If not data, filter out*/
/*INSERT into table Parts if not exists*/
INSERT INTO #Parts
SELECT PartsID,PartName
FROM #Unpivot AS A
WHERE A.IsMajorPart = 1
AND NOT EXISTS (
SELECT *
FROM #Parts AS DTA
WHERE A.PartsID = DTA.PartsID
)
GROUP BY PartsID,PartName
/*UPSERT into table dbo.InvoiceParts*/
UPDATE #InvoiceToParts
SET IsMajorPart = B.IsMajorPart
FROM #InvoiceToParts AS A
INNER JOIN #Unpivot AS B
ON A.InvoiceId = B.InvoiceId
AND A.PartsId = B.PartsID
INSERT INTO #InvoiceToParts(InvoiceId,PartsId,IsMajorPart)
SELECT InvoiceId
,PartsId
,IsMajorPart
FROM #Unpivot AS A
WHERE NOT EXISTS (
SELECT *
FROM #InvoiceToParts AS DTA
WHERE A.InvoiceId = DTA.InvoiceID
AND A.PartsID = DTA.PartsID
)
SELECT *
FROM #InvoiceToParts
SELECT *
FROM #Parts

SQL Server: return joined data from insert select

I perform steps:
Create temporal table and fill it with data and unique order column [_oid]
Insert everything from temporal table into real table except fictional [_oid], outputting generated [id]'s
Return those generated [id]'s along with corresponding [_oid]
SQL:
CREATE TABLE #temp
(
[Hash] INT NOT NULL,
[Size] INT NOT NULL,
[Data] NVARCHAR(MAX),
[_oid] INT NOT NULL
)
--here insert data into #temp--
INSERT [dbo].[TestObjects]
OUTPUT INSERTED.[Id]
SELECT [Hash], [Size], [Data]
FROM #temp
DROP TABLE #temp
How I can return ([Id], [_oid]) rows ? ....Or at least return [Id] ordered by [_oid] ?
I know insert does not preserve order of inserted items in it's output, but still...
I think you what you are asking for is INSERT INTO, as so:
INSERT INTO [dbo].[TestObjects]
SELECT Hash, Size, Data FROM #temp
ORDER BY _oid
But as you say, there's no guarantee about order when you select from TestObjects, so if it's important can you not have a field in TestObjects you can ORDER BY when you SELECT from it?
IF your insert into #temp is such that both o_id and (hash,size,data) are unique for each row (ie keys), then you could retrieve the inserted o_id from #temp:
select t.[_oid],to.[Id]
from #temp t
inner join [dbo].[TestObjects] to
on t.Hash=to.Hash and t.Size=to.Size and t.data=to.data
As noted by George Menoutis, I did merge:
MERGE [dbo].[TestObjects] AS T_Base
USING #temp AS T_Source
ON (0<>0)
WHEN NOT MATCHED THEN INSERT ([Hash],[Size],[Data]) VALUES (T_Source.[Hash],T_Source.[Size],T_Source.[Data])
OUTPUT INSERTED.[Id], T_Source.[_oid];
If anyone have better approach - feel free to contribute to this answer.

INSERT inside an INSERT statement and use its ID in the outer INSERT [duplicate]

Very simplified, I have two tables Source and Target.
declare #Source table (SourceID int identity(1,2), SourceName varchar(50))
declare #Target table (TargetID int identity(2,2), TargetName varchar(50))
insert into #Source values ('Row 1'), ('Row 2')
I would like to move all rows from #Source to #Target and know the TargetID for each SourceID because there are also the tables SourceChild and TargetChild that needs to be copied as well and I need to add the new TargetID into TargetChild.TargetID FK column.
There are a couple of solutions to this.
Use a while loop or cursors to insert one row (RBAR) to Target at a time and use scope_identity() to fill the FK of TargetChild.
Add a temp column to #Target and insert SourceID. You can then join that column to fetch the TargetID for the FK in TargetChild.
SET IDENTITY_INSERT OFF for #Target and handle assigning new values yourself. You get a range that you then use in TargetChild.TargetID.
I'm not all that fond of any of them. The one I used so far is cursors.
What I would really like to do is to use the output clause of the insert statement.
insert into #Target(TargetName)
output inserted.TargetID, S.SourceID
select SourceName
from #Source as S
But it is not possible
The multi-part identifier "S.SourceID" could not be bound.
But it is possible with a merge.
merge #Target as T
using #Source as S
on 0=1
when not matched then
insert (TargetName) values (SourceName)
output inserted.TargetID, S.SourceID;
Result
TargetID SourceID
----------- -----------
2 1
4 3
I want to know if you have used this? If you have any thoughts about the solution or see any problems with it? It works fine in simple scenarios but perhaps something ugly could happen when the query plan get really complicated due to a complicated source query. Worst scenario would be that the TargetID/SourceID pairs actually isn't a match.
MSDN has this to say about the from_table_name of the output clause.
Is a column prefix that specifies a table included in the FROM clause of a DELETE, UPDATE, or MERGE statement that is used to specify the rows to update or delete.
For some reason they don't say "rows to insert, update or delete" only "rows to update or delete".
Any thoughts are welcome and totally different solutions to the original problem is much appreciated.
In my opinion this is a great use of MERGE and output. I've used in several scenarios and haven't experienced any oddities to date.
For example, here is test setup that clones a Folder and all Files (identity) within it into a newly created Folder (guid).
DECLARE #FolderIndex TABLE (FolderId UNIQUEIDENTIFIER PRIMARY KEY, FolderName varchar(25));
INSERT INTO #FolderIndex
(FolderId, FolderName)
VALUES(newid(), 'OriginalFolder');
DECLARE #FileIndex TABLE (FileId int identity(1,1) PRIMARY KEY, FileName varchar(10));
INSERT INTO #FileIndex
(FileName)
VALUES('test.txt');
DECLARE #FileFolder TABLE (FolderId UNIQUEIDENTIFIER, FileId int, PRIMARY KEY(FolderId, FileId));
INSERT INTO #FileFolder
(FolderId, FileId)
SELECT FolderId,
FileId
FROM #FolderIndex
CROSS JOIN #FileIndex; -- just to illustrate
DECLARE #sFolder TABLE (FromFolderId UNIQUEIDENTIFIER, ToFolderId UNIQUEIDENTIFIER);
DECLARE #sFile TABLE (FromFileId int, ToFileId int);
-- copy Folder Structure
MERGE #FolderIndex fi
USING ( SELECT 1 [Dummy],
FolderId,
FolderName
FROM #FolderIndex [fi]
WHERE FolderName = 'OriginalFolder'
) d ON d.Dummy = 0
WHEN NOT MATCHED
THEN INSERT
(FolderId, FolderName)
VALUES (newid(), 'copy_'+FolderName)
OUTPUT d.FolderId,
INSERTED.FolderId
INTO #sFolder (FromFolderId, toFolderId);
-- copy File structure
MERGE #FileIndex fi
USING ( SELECT 1 [Dummy],
fi.FileId,
fi.[FileName]
FROM #FileIndex fi
INNER
JOIN #FileFolder fm ON
fi.FileId = fm.FileId
INNER
JOIN #FolderIndex fo ON
fm.FolderId = fo.FolderId
WHERE fo.FolderName = 'OriginalFolder'
) d ON d.Dummy = 0
WHEN NOT MATCHED
THEN INSERT ([FileName])
VALUES ([FileName])
OUTPUT d.FileId,
INSERTED.FileId
INTO #sFile (FromFileId, toFileId);
-- link new files to Folders
INSERT INTO #FileFolder (FileId, FolderId)
SELECT sfi.toFileId, sfo.toFolderId
FROM #FileFolder fm
INNER
JOIN #sFile sfi ON
fm.FileId = sfi.FromFileId
INNER
JOIN #sFolder sfo ON
fm.FolderId = sfo.FromFolderId
-- return
SELECT *
FROM #FileIndex fi
JOIN #FileFolder ff ON
fi.FileId = ff.FileId
JOIN #FolderIndex fo ON
ff.FolderId = fo.FolderId
I would like to add another example to add to #Nathan's example, as I found it somewhat confusing.
Mine uses real tables for the most part, and not temp tables.
I also got my inspiration from here: another example
-- Copy the FormSectionInstance
DECLARE #FormSectionInstanceTable TABLE(OldFormSectionInstanceId INT, NewFormSectionInstanceId INT)
;MERGE INTO [dbo].[FormSectionInstance]
USING
(
SELECT
fsi.FormSectionInstanceId [OldFormSectionInstanceId]
, #NewFormHeaderId [NewFormHeaderId]
, fsi.FormSectionId
, fsi.IsClone
, #UserId [NewCreatedByUserId]
, GETDATE() NewCreatedDate
, #UserId [NewUpdatedByUserId]
, GETDATE() NewUpdatedDate
FROM [dbo].[FormSectionInstance] fsi
WHERE fsi.[FormHeaderId] = #FormHeaderId
) tblSource ON 1=0 -- use always false condition
WHEN NOT MATCHED
THEN INSERT
( [FormHeaderId], FormSectionId, IsClone, CreatedByUserId, CreatedDate, UpdatedByUserId, UpdatedDate)
VALUES( [NewFormHeaderId], FormSectionId, IsClone, NewCreatedByUserId, NewCreatedDate, NewUpdatedByUserId, NewUpdatedDate)
OUTPUT tblSource.[OldFormSectionInstanceId], INSERTED.FormSectionInstanceId
INTO #FormSectionInstanceTable(OldFormSectionInstanceId, NewFormSectionInstanceId);
-- Copy the FormDetail
INSERT INTO [dbo].[FormDetail]
(FormHeaderId, FormFieldId, FormSectionInstanceId, IsOther, Value, CreatedByUserId, CreatedDate, UpdatedByUserId, UpdatedDate)
SELECT
#NewFormHeaderId, FormFieldId, fsit.NewFormSectionInstanceId, IsOther, Value, #UserId, CreatedDate, #UserId, UpdatedDate
FROM [dbo].[FormDetail] fd
INNER JOIN #FormSectionInstanceTable fsit ON fsit.OldFormSectionInstanceId = fd.FormSectionInstanceId
WHERE [FormHeaderId] = #FormHeaderId
Here's a solution that doesn't use MERGE (which I've had problems with many times I try to avoid if possible). It relies on two memory tables (you could use temp tables if you want) with IDENTITY columns that get matched, and importantly, using ORDER BY when doing the INSERT, and WHERE conditions that match between the two INSERTs... the first one holds the source IDs and the second one holds the target IDs.
-- Setup... We have a table that we need to know the old IDs and new IDs after copying.
-- We want to copy all of DocID=1
DECLARE #newDocID int = 99;
DECLARE #tbl table (RuleID int PRIMARY KEY NOT NULL IDENTITY(1, 1), DocID int, Val varchar(100));
INSERT INTO #tbl (DocID, Val) VALUES (1, 'RuleA-2'), (1, 'RuleA-1'), (2, 'RuleB-1'), (2, 'RuleB-2'), (3, 'RuleC-1'), (1, 'RuleA-3')
-- Create a break in IDENTITY values.. just to simulate more realistic data
INSERT INTO #tbl (Val) VALUES ('DeleteMe'), ('DeleteMe');
DELETE FROM #tbl WHERE Val = 'DeleteMe';
INSERT INTO #tbl (DocID, Val) VALUES (6, 'RuleE'), (7, 'RuleF');
SELECT * FROM #tbl t;
-- Declare TWO temp tables each with an IDENTITY - one will hold the RuleID of the items we are copying, other will hold the RuleID that we create
DECLARE #input table (RID int IDENTITY(1, 1), SourceRuleID int NOT NULL, Val varchar(100));
DECLARE #output table (RID int IDENTITY(1,1), TargetRuleID int NOT NULL, Val varchar(100));
-- Capture the IDs of the rows we will be copying by inserting them into the #input table
-- Important - we must specify the sort order - best thing is to use the IDENTITY of the source table (t.RuleID) that we are copying
INSERT INTO #input (SourceRuleID, Val) SELECT t.RuleID, t.Val FROM #tbl t WHERE t.DocID = 1 ORDER BY t.RuleID;
-- Copy the rows, and use the OUTPUT clause to capture the IDs of the inserted rows.
-- Important - we must use the same WHERE and ORDER BY clauses as above
INSERT INTO #tbl (DocID, Val)
OUTPUT Inserted.RuleID, Inserted.Val INTO #output(TargetRuleID, Val)
SELECT #newDocID, t.Val FROM #tbl t
WHERE t.DocID = 1
ORDER BY t.RuleID;
-- Now #input and #output should have the same # of rows, and the order of both inserts was the same, so the IDENTITY columns (RID) can be matched
-- Use this as the map from old-to-new when you are copying sub-table rows
-- Technically, #input and #output don't even need the 'Val' columns, just RID and RuleID - they were included here to prove that the rules matched
SELECT i.*, o.* FROM #output o
INNER JOIN #input i ON i.RID = o.RID
-- Confirm the matching worked
SELECT * FROM #tbl t

SQL Server, Select/Output/Insert - need to select value for output but not insert [duplicate]

Very simplified, I have two tables Source and Target.
declare #Source table (SourceID int identity(1,2), SourceName varchar(50))
declare #Target table (TargetID int identity(2,2), TargetName varchar(50))
insert into #Source values ('Row 1'), ('Row 2')
I would like to move all rows from #Source to #Target and know the TargetID for each SourceID because there are also the tables SourceChild and TargetChild that needs to be copied as well and I need to add the new TargetID into TargetChild.TargetID FK column.
There are a couple of solutions to this.
Use a while loop or cursors to insert one row (RBAR) to Target at a time and use scope_identity() to fill the FK of TargetChild.
Add a temp column to #Target and insert SourceID. You can then join that column to fetch the TargetID for the FK in TargetChild.
SET IDENTITY_INSERT OFF for #Target and handle assigning new values yourself. You get a range that you then use in TargetChild.TargetID.
I'm not all that fond of any of them. The one I used so far is cursors.
What I would really like to do is to use the output clause of the insert statement.
insert into #Target(TargetName)
output inserted.TargetID, S.SourceID
select SourceName
from #Source as S
But it is not possible
The multi-part identifier "S.SourceID" could not be bound.
But it is possible with a merge.
merge #Target as T
using #Source as S
on 0=1
when not matched then
insert (TargetName) values (SourceName)
output inserted.TargetID, S.SourceID;
Result
TargetID SourceID
----------- -----------
2 1
4 3
I want to know if you have used this? If you have any thoughts about the solution or see any problems with it? It works fine in simple scenarios but perhaps something ugly could happen when the query plan get really complicated due to a complicated source query. Worst scenario would be that the TargetID/SourceID pairs actually isn't a match.
MSDN has this to say about the from_table_name of the output clause.
Is a column prefix that specifies a table included in the FROM clause of a DELETE, UPDATE, or MERGE statement that is used to specify the rows to update or delete.
For some reason they don't say "rows to insert, update or delete" only "rows to update or delete".
Any thoughts are welcome and totally different solutions to the original problem is much appreciated.
In my opinion this is a great use of MERGE and output. I've used in several scenarios and haven't experienced any oddities to date.
For example, here is test setup that clones a Folder and all Files (identity) within it into a newly created Folder (guid).
DECLARE #FolderIndex TABLE (FolderId UNIQUEIDENTIFIER PRIMARY KEY, FolderName varchar(25));
INSERT INTO #FolderIndex
(FolderId, FolderName)
VALUES(newid(), 'OriginalFolder');
DECLARE #FileIndex TABLE (FileId int identity(1,1) PRIMARY KEY, FileName varchar(10));
INSERT INTO #FileIndex
(FileName)
VALUES('test.txt');
DECLARE #FileFolder TABLE (FolderId UNIQUEIDENTIFIER, FileId int, PRIMARY KEY(FolderId, FileId));
INSERT INTO #FileFolder
(FolderId, FileId)
SELECT FolderId,
FileId
FROM #FolderIndex
CROSS JOIN #FileIndex; -- just to illustrate
DECLARE #sFolder TABLE (FromFolderId UNIQUEIDENTIFIER, ToFolderId UNIQUEIDENTIFIER);
DECLARE #sFile TABLE (FromFileId int, ToFileId int);
-- copy Folder Structure
MERGE #FolderIndex fi
USING ( SELECT 1 [Dummy],
FolderId,
FolderName
FROM #FolderIndex [fi]
WHERE FolderName = 'OriginalFolder'
) d ON d.Dummy = 0
WHEN NOT MATCHED
THEN INSERT
(FolderId, FolderName)
VALUES (newid(), 'copy_'+FolderName)
OUTPUT d.FolderId,
INSERTED.FolderId
INTO #sFolder (FromFolderId, toFolderId);
-- copy File structure
MERGE #FileIndex fi
USING ( SELECT 1 [Dummy],
fi.FileId,
fi.[FileName]
FROM #FileIndex fi
INNER
JOIN #FileFolder fm ON
fi.FileId = fm.FileId
INNER
JOIN #FolderIndex fo ON
fm.FolderId = fo.FolderId
WHERE fo.FolderName = 'OriginalFolder'
) d ON d.Dummy = 0
WHEN NOT MATCHED
THEN INSERT ([FileName])
VALUES ([FileName])
OUTPUT d.FileId,
INSERTED.FileId
INTO #sFile (FromFileId, toFileId);
-- link new files to Folders
INSERT INTO #FileFolder (FileId, FolderId)
SELECT sfi.toFileId, sfo.toFolderId
FROM #FileFolder fm
INNER
JOIN #sFile sfi ON
fm.FileId = sfi.FromFileId
INNER
JOIN #sFolder sfo ON
fm.FolderId = sfo.FromFolderId
-- return
SELECT *
FROM #FileIndex fi
JOIN #FileFolder ff ON
fi.FileId = ff.FileId
JOIN #FolderIndex fo ON
ff.FolderId = fo.FolderId
I would like to add another example to add to #Nathan's example, as I found it somewhat confusing.
Mine uses real tables for the most part, and not temp tables.
I also got my inspiration from here: another example
-- Copy the FormSectionInstance
DECLARE #FormSectionInstanceTable TABLE(OldFormSectionInstanceId INT, NewFormSectionInstanceId INT)
;MERGE INTO [dbo].[FormSectionInstance]
USING
(
SELECT
fsi.FormSectionInstanceId [OldFormSectionInstanceId]
, #NewFormHeaderId [NewFormHeaderId]
, fsi.FormSectionId
, fsi.IsClone
, #UserId [NewCreatedByUserId]
, GETDATE() NewCreatedDate
, #UserId [NewUpdatedByUserId]
, GETDATE() NewUpdatedDate
FROM [dbo].[FormSectionInstance] fsi
WHERE fsi.[FormHeaderId] = #FormHeaderId
) tblSource ON 1=0 -- use always false condition
WHEN NOT MATCHED
THEN INSERT
( [FormHeaderId], FormSectionId, IsClone, CreatedByUserId, CreatedDate, UpdatedByUserId, UpdatedDate)
VALUES( [NewFormHeaderId], FormSectionId, IsClone, NewCreatedByUserId, NewCreatedDate, NewUpdatedByUserId, NewUpdatedDate)
OUTPUT tblSource.[OldFormSectionInstanceId], INSERTED.FormSectionInstanceId
INTO #FormSectionInstanceTable(OldFormSectionInstanceId, NewFormSectionInstanceId);
-- Copy the FormDetail
INSERT INTO [dbo].[FormDetail]
(FormHeaderId, FormFieldId, FormSectionInstanceId, IsOther, Value, CreatedByUserId, CreatedDate, UpdatedByUserId, UpdatedDate)
SELECT
#NewFormHeaderId, FormFieldId, fsit.NewFormSectionInstanceId, IsOther, Value, #UserId, CreatedDate, #UserId, UpdatedDate
FROM [dbo].[FormDetail] fd
INNER JOIN #FormSectionInstanceTable fsit ON fsit.OldFormSectionInstanceId = fd.FormSectionInstanceId
WHERE [FormHeaderId] = #FormHeaderId
Here's a solution that doesn't use MERGE (which I've had problems with many times I try to avoid if possible). It relies on two memory tables (you could use temp tables if you want) with IDENTITY columns that get matched, and importantly, using ORDER BY when doing the INSERT, and WHERE conditions that match between the two INSERTs... the first one holds the source IDs and the second one holds the target IDs.
-- Setup... We have a table that we need to know the old IDs and new IDs after copying.
-- We want to copy all of DocID=1
DECLARE #newDocID int = 99;
DECLARE #tbl table (RuleID int PRIMARY KEY NOT NULL IDENTITY(1, 1), DocID int, Val varchar(100));
INSERT INTO #tbl (DocID, Val) VALUES (1, 'RuleA-2'), (1, 'RuleA-1'), (2, 'RuleB-1'), (2, 'RuleB-2'), (3, 'RuleC-1'), (1, 'RuleA-3')
-- Create a break in IDENTITY values.. just to simulate more realistic data
INSERT INTO #tbl (Val) VALUES ('DeleteMe'), ('DeleteMe');
DELETE FROM #tbl WHERE Val = 'DeleteMe';
INSERT INTO #tbl (DocID, Val) VALUES (6, 'RuleE'), (7, 'RuleF');
SELECT * FROM #tbl t;
-- Declare TWO temp tables each with an IDENTITY - one will hold the RuleID of the items we are copying, other will hold the RuleID that we create
DECLARE #input table (RID int IDENTITY(1, 1), SourceRuleID int NOT NULL, Val varchar(100));
DECLARE #output table (RID int IDENTITY(1,1), TargetRuleID int NOT NULL, Val varchar(100));
-- Capture the IDs of the rows we will be copying by inserting them into the #input table
-- Important - we must specify the sort order - best thing is to use the IDENTITY of the source table (t.RuleID) that we are copying
INSERT INTO #input (SourceRuleID, Val) SELECT t.RuleID, t.Val FROM #tbl t WHERE t.DocID = 1 ORDER BY t.RuleID;
-- Copy the rows, and use the OUTPUT clause to capture the IDs of the inserted rows.
-- Important - we must use the same WHERE and ORDER BY clauses as above
INSERT INTO #tbl (DocID, Val)
OUTPUT Inserted.RuleID, Inserted.Val INTO #output(TargetRuleID, Val)
SELECT #newDocID, t.Val FROM #tbl t
WHERE t.DocID = 1
ORDER BY t.RuleID;
-- Now #input and #output should have the same # of rows, and the order of both inserts was the same, so the IDENTITY columns (RID) can be matched
-- Use this as the map from old-to-new when you are copying sub-table rows
-- Technically, #input and #output don't even need the 'Val' columns, just RID and RuleID - they were included here to prove that the rules matched
SELECT i.*, o.* FROM #output o
INNER JOIN #input i ON i.RID = o.RID
-- Confirm the matching worked
SELECT * FROM #tbl t