Destination tables look like this:
Source table look like this:
Customer
CustomerId FirstName LastName Email Address1 Address2 City Zip
Person table in destination is a base table (which will later be inherited by new customer table). So I am trying to export a row from one table and populate 3 tables in destination.
I managed to do this in following way:
Get records from source table (Customer)
Create empty AddressId field
Populate Address table using OLE DB Command task (it calls stored procedure which returns SCOPE_IDENTITY() that's mapped to AddressId field)
Repeat step 3 for populating Person table (and retrieving PersonId
Populate cross reference table PersonAddress using PersonId and AddressId fields
Screenshot of this package is below.
Biggest issue with this approach is that OLE DB Command task is inserting row by row and it makes the whole package extremely slow. Is it possible to achieve the same thing but using fast load?
I am able to do it using OLE DB Command task which calls the stored procedure and then
I don't think you need SSIS.
You can use OUTPUT clause of INSERT which returns all identity keys to a temporary table
Lets try reproduce your scenario...
set nocount on
go
create table Customer (CustomerId int, CustomerName varchar(100) null, Address1 varchar(100) null, Address2 varchar(100) )
create table [Person] (PersonId int identity, PersonName varchar(100) null)
create table [Address] (AddressId int identity, AddressLine varchar(100) null)
create table [PersonAddress] (AddressId int, PersonId int )
go
-- create some data...
insert into Customer (CustomerId) values ( 1000000 + convert(int, RAND() * 1000000) )
go 1000
update Customer
set CustomerName = 'CustomerName ' + convert(varchar, CustomerId),
Address1 = 'Address1 ' + convert(varchar, CustomerId),
Address2 = 'Address2 ' + convert(varchar, CustomerId)
go
declare #identities_Person table ([rownumber] int identity, id int)
declare #identities_Address table ([rownumber] int identity, id int)
insert into Person (PersonName)
output inserted.PersonId into #identities_Person
select
c.CustomerName
from Customer c
order by c.CustomerId
insert into [Address] (AddressLine)
output inserted.AddressId into #identities_Address
select
c.Address1
from Customer c
order by c.CustomerId
insert into [PersonAddress] (PersonId, AddressId)
select p.id, a.id
from #identities_Address a
inner join #identities_Person p on p.rownumber = a.rownumber
select *
from PersonAddress pa
inner join [Address] a on a.AddressId = pa.AddressId
inner join [Person] p on p.PersonId = pa.PersonId
Related
I have an old table with lots of columns that i want to split into 3 tables with many to many relation.
The old table have no identity column.
Old table:
CustomerNumber
FirstName
LastName
Address
Postal
City
....
New tables
Customer:
Id
Customernumber
Firstname
Lastname
... ect
Address:
Id
Address
Postal
City
... ect
CustomerAddress
Id
CustomerId
AddressId
Now how can I spilt the old tabel into the new ones using SQL?
I have tried with MERGE but that can't handle more than one table at the time. One option is using CURSOR, but I read that it is a bad idea, to use that or iteration, but for now that is the only solution I have found for this.
declare
#CustomerId bigint,
#CustomerNumber float,
#Status int,
#Address varchar(50),
#RoadNumber int,
#LastEdited datetime,
#AddressId bigint
declare my_cursor cursor
local static read_only forward_only
for
select CustomerNumber, Address, Housenumber, Status, Date
from [db1].dbo.OldCustomer k
where
FIRMANR in (1, 40, 60, 80, 90, 120, 180, 400)
open my_cursor
fetch next from my_cursor into #CustomerNumber, #Address, #RoadNumber, #Status, #LastEdited
while ##FETCH_STATUS = 0
begin
--if the customer already exists we get the identity
if exists (select Id from [db2].dbo.Customers where CustomerNumber = #CustomerNumber)
select #CustomerId = Id from [db2].dbo.Customers where CustomerNumber = #CustomerNumber
--if the customer does not exit we need to insert and retrieve the new Identity value
else
begin
-- insert the customer
insert into [db2].dbo.Customers (CustomerNumber, [Status], LastEdited) values (#CustomerNumber, #Status, #LastEdited)
set #CustomerId = SCOPE_IDENTITY()
end
-- get address if it already exists
if exists (select Id from [db2].dbo.Addresses where Road = #Address and Roadnumber = #RoadNumber)
select #AddressId = Id from [db2].dbo.Addresses where Road = #Address and Roadnumber = #RoadNumber
else
begin
-- insert new addresses
insert into [db2].dbo.Addresses (Road,Roadnumber) values (#Address, #RoadNumber)
set #AddressId = SCOPE_IDENTITY()
end
-- insert customer => address reference if it does not exist
if not exists (select Id from [db2].dbo.CustomerAddress where CustomerId = #CustomerId and AddressId = #AddressId)
-- insert customer => address reference
insert into [db2].dbo.CustomerAddress(CustomerId,AddressId) values (#CustomerId, #AddressId)
fetch next from my_cursor into #CustomerNumber, #Address, #RoadNumber, #Status, #LastEdited
end
close my_cursor
deallocate my_cursor
It would probably be easiest to:
Add a column to old customer called AddressId
Populate it with unique IDs (you can reuse the Customer ID, or if its GUID, use NewID())
Create new tables for address and customeraddress
Insert part of the old customer data into each new table
Drop columns from Customer that relate to address
--populate new Address table
INSERT INTO Address(id,col1,col2...)
SELECT AddressID, col1, col2... FROM Customer
--populate new CustomerAddress table
INSERT INTO CustomerAddress(CustomerId,AddressId)
SELECT Id, AddressID FROM Customer
With SQLS you can do this as a simple script, with a transaction if you want.. No need for stored procedures, cursors, merge etc..
Don't give CustomerAddress its own Id column; the primary key of CustomerAddress is the combination of CustomerId and AddressId; make a composite PK, not a separate one
Tbh, I'd probably not have a CustomerAddress table and instead just have a BillingAddressId, WorkAddressId, HomeAddressId, ShippingAddressId column in customer, but it's up to you how to manage this; If you have lots and variable types of addresses then sure, have a M:M breakdown, but if realistically your customers are only ever going to have up to 3 addresses etc, I'd stick with having a named column stating what the address was for, in customer
If you do go with a CustomerAddress table, consider adding a column declaring the type/reason for the address
I have a temp table that looks like this:
FirstName
LastName
DOB
Sex
Age
ExternalID
In my stored procedure I'm inserting these values into a regular table that has the following structure:
ID identity(1,1)
FirstName
LastName
So, I do this:
Insert into myTable
select FirstName, LastName from TempTable
During the insert I need to insert primary key from main table back into temp table "ExternalID" column. How can this be achieved?
I tried using OUTPUT statement but it only allows to insert to a separate table and then I have no way to map back to temp table
I need to insert generated IDs to column ExternalID in temp table right after the insert. FirstName and LastName are not unique.
One possible solution would be to use loop and insert one row at a time. This way, I can update temp table row with scope_identity(). But I want to avoid using loops.
Try using MERGE instead of INSERT.
MERGE allows you to output a column you didn't insert, such as an identifier on your temp table. Using this method, you can build another temporary table that maps your temp table to the inserted rows (named #TempIdTable in the sample below).
First, give #TempTable its own primary key. I'll call it TempId. I'll also assume you have a column on #TempTable to store the returned primary key from MyTable, ID.
--Make a place to store the associated ID's
DECLARE #TempIdTable TABLE
([TempId] INT NOT NULL
,[ID] INT NOT NULL)
--Will only insert, as 1 never equals 0.
MERGE INTO myTable
USING #TempTable AS tt
ON 1 = 0
WHEN NOT MATCHED
THEN
INSERT ([FirstName]
,[LastName])
VALUE (t.[FirstName]
,t.[LastName])
OUTPUT tt.[TempId], inserted.[ID] --Here's the magic
INTO #TempIdTable
--Associate the new primary keys with the temp table
UPDATE #TempTable
SET [ID] = t.[ID]
FROM #TempIdTable t
WHERE #TempTable.[TempId] = t.[TempId]
I was working on a similar issue and found this trick over here: Is it possible to for SQL Output clause to return a column not being inserted?
Here's the full code I used in my own testing.
CREATE TABLE [MQ]
([MESSAGEID] INT IDENTITY PRIMARY KEY
,[SUBJECT] NVARCHAR(255) NULL);
CREATE TABLE [MR]
([MESSAGESEQUENCE] INT IDENTITY PRIMARY KEY
,[TO] NVARCHAR(255) NOT NULL
,[CC] NVARCHAR(255) NOT NULL
,[BCC] NVARCHAR(255) NOT NULL);
CREATE TABLE #Messages (
[subject] nvarchar(255) NOT NULL
,[to] nvarchar(255) NOT NULL
,[cc] nvarchar(255) NULL
,[bcc] nvarchar(255) NULL
,[MESSAGEID] INT NULL
,[sortKey] INT IDENTITY PRIMARY KEY
);
INSERT INTO #Messages
VALUES ('Subject1','to1','cc1','bcc1', NULL)
,('Subject2','to2', NULL, NULL, NULL);
SELECT * FROM #Messages;
DECLARE #outputSort TABLE (
[sortKey] INT NOT NULL
,[MESSAGEID] INT NOT NULL
,[subject] NVARCHAR(255)
);
MERGE INTO [MQ]
USING #Messages M
ON 1 = 0
WHEN NOT MATCHED
THEN
INSERT ([SUBJECT])
VALUES (M.[subject])
OUTPUT M.[SORTKEY]
,inserted.[MESSAGEID]
,inserted.[SUBJECT]
INTO #outputSort;
SELECT * FROM #outputSort;
SELECT * FROM [MQ];
UPDATE #Messages
SET MESSAGEID = O.[MESSAGEID]
FROM #outputSort O
WHERE #Messages.[sortKey] = O.[sortKey];
SELECT * FROM #Messages;
DROP TABLE #Messages;
As you said, FirstName and LastName are not unique. This means you cannot use a trigger because there can be the same FirstName + LastName so you cannot join on them.
But you can do the inverse thing: first update your temp table ExternalID (I suggest you to use sequence object and just do update #t set ExternalID = next value for dbo.seq1;) and then just insert your rows including ExternalID into myTable. To be able to insert into identity field you can use set identity_insert myTable on or you can re-design your destination table to contain no identity at all as now you use sequence for the same purpose.
We need a unique column for able to make the comparison at the update operation after the insert. That's why we are using ExternalID column temporarily. ExternalID updated by row_nubmber.
;WITH CTE AS
(
SELECT *, RN = ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) FROM #TempTable
)
UPDATE CTE SET ExternalID = RN
We are keeping the output of the insert operation in a temp table. The trick is order by with ExternalID, it will help us for making the unique row number for same first and last name
DECLARE #output TABLE (
ID INT,
FirstName VARCHAR(10),
LastName VARCHAR(10))
Insert into #myTable
OUTPUT inserted.ID, inserted.FirstName, inserted.LastName INTO #output(ID, FirstName, LastName)
select FirstName, LastName from #TempTable T
order by ExternalID
For replacing the ExternalID column with inserted id value, we are making comparing with first name, last name and row number.
;WITH TMP_T AS(
SELECT *, RN = ROW_NUMBER() OVER(PARTITION BY FirstName, LastName ORDER BY ExternalID) FROM #TempTable )
,OUT_T AS(
SELECT *, RN = ROW_NUMBER() OVER(PARTITION BY FirstName, LastName ORDER BY ID) FROM #output )
UPDATE TMP_T SET ExternalID = OUT_T.ID
FROM
TMP_T INNER JOIN OUT_T ON
TMP_T.FirstName = OUT_T.FirstName
AND TMP_T.LastName = OUT_T.LastName
AND TMP_T.RN = OUT_T.RN
Sample Data:
DECLARE #TempTable TABLE (
FirstName VARCHAR(10),
LastName VARCHAR(10),
DOB VARCHAR(10),
Sex VARCHAR (10),
Age VARCHAR(10),
ExternalID INT)
INSERT INTO #TempTable VALUES
('Serkan1', 'Arslan1', 'A','M','1',NULL),
('Serkan2', 'Arslan2', 'B','M','1',NULL),
('Serkan3', 'Arslan', 'C','M','1',NULL),
('Serkan3', 'Arslan', 'D','M','1',NULL)
DECLARE #myTable TABLE (
ID INT identity(100,1), -- started from 100 for see the difference
FirstName VARCHAR(10),
LastName VARCHAR(10))
Result:
MyTable
ID FirstName LastName
----------- ---------- ----------
100 Serkan1 Arslan1
101 Serkan2 Arslan2
102 Serkan3 Arslan
103 Serkan3 Arslan
TempTable
FirstName LastName DOB Sex Age ExternalID
---------- ---------- ---------- ---------- ---------- -----------
Serkan1 Arslan1 A M 1 100
Serkan2 Arslan2 B M 1 101
Serkan3 Arslan C M 1 102
Serkan3 Arslan D M 1 103
One way to do this is by duplicating the data into a second temp table like so:
SELECT *
INTO #TEMPTABLE
FROM (VALUES (1, 'Adam'), (2, 'Kate'), (3, 'Jess')) AS X (Id, Name)
SELECT TOP 0 CAST(NULL AS INT) AS IdentityValue, *
INTO #NEWTEMPTABLE
FROM #TEMPTABLE
CREATE TABLE #TABLEFORINSERT (
IdentityColumn INT IDENTITY(1,1),
Id INT,
Name VARCHAR(255)
)
INSERT INTO #TABLEFORINSERT (Id, Name)
OUTPUT INSERTED.IdentityColumn, INSERTED.Id, Inserted.Name INTO #NEWTEMPTABLE
SELECT Id, Name FROM #TEMPTABLE
--New temp table with identity values
SELECT * FROM #NEWTEMPTABLE
I'm designing a simple in-office ticket system, and would like to include a field for the party responsible for the next action. To do so right this moment I'm thinking of using tableName and tableID as specifiers for the specific responsible party (could be a technician, customer, or third party, all in different tables)
It would be fine to pull that data in and run another select call using the name of the table as a parameter, but the extra data flow slows things down significantly.
Is there a way to use a single join statement to return the details of the party with a column for the table name and one for the individual table id or is there a better way to store the data from multiple potential tables?
You can use left join to achieve your requirement :-
Set Nocount On;
Declare #OfficeTickets Table
(
Id Int Identity(1,1)
,Column1 Varchar(100)
,PartyType Varchar(1)
,TechnicianId Int Null
,CustomerId Int Null
,ThirdPartyId Int Null
)
Declare #OfficeTickets1 Table
(
Id Int Identity(1,1)
,Column1 Varchar(100)
,TableName Varchar(100)
,TableId Int Null
)
Declare #Technician Table
(
Id Int Identity(1,1)
,TechnicianName Varchar(100)
)
Declare #Customers Table
(
Id Int Identity(1,1)
,CustomerName Varchar(100)
)
Declare #ThirdParty Table
(
Id Int Identity(1,1)
,ThirdPartyName Varchar(100)
)
Insert Into #Technician(TechnicianName) Values
('Technician_1')
,('Technician_2')
,('Technician_3')
Insert Into #Customers(CustomerName) Values
('Customer_1')
,('Customer_2')
,('Customer_3')
Insert Into #ThirdParty(ThirdPartyName) Values
('ThirdParty_1')
,('ThirdParty_2')
,('ThirdParty_3')
,('ThirdParty_4')
Insert Into #OfficeTickets(Column1,PartyType,TechnicianId,CustomerId,ThirdPartyId) Values
('ABC','T',3,Null,Null)
,('XYZ','C',Null,2,Null)
,('PUQ','P',Null,Null,4)
Insert Into #OfficeTickets1(Column1,TableName,TableId) Values
('ABC','Technician',3)
,('XYZ','Customers',2)
,('PUQ','ThirdParty',4)
---- taken separate columns for parties
Select ot.Id
,ot.Column1
,t.TechnicianName
,c.CustomerName
,tp.ThirdPartyName
From #OfficeTickets As ot
Left Join #Technician As t On ot.PartyType = 'T' And ot.TechnicianId = t.Id
Left Join #Customers As c On ot.PartyType = 'C' And ot.CustomerId = c.Id
Left Join #ThirdParty As tp On ot.PartyType = 'P' And ot.ThirdPartyId = tp.Id
---- by TableName and TableId
Select ot.Id
,ot.Column1
,t.TechnicianName
,c.CustomerName
,tp.ThirdPartyName
From #OfficeTickets1 As ot
Left Join #Technician As t On ot.TableName = 'Technician' And ot.TableId = t.Id
Left Join #Customers As c On ot.TableName = 'Customers' And ot.TableId = c.Id
Left Join #ThirdParty As tp On ot.TableName = 'ThirdParty' And ot.TableId = tp.Id
output:-
I am using staging tables to perform validation and insert into live.
Suppose I have a table PERSONS
TABLE Persons
(
Id int NOT NULL,
LastName varchar(255) NOT NULL,
FirstName varchar(255),
HouseNumber int,
)
and a STAGING TABLE as follows
TABLE Persons_Staging
(
Id int NOT NULL,
LastName varchar(255) NOT NULL,
FirstName varchar(255),
HouseNumber varchar(255),
)
I need to write a procedure to transfer data from the staging table to the live table while ensuring no duplicates are inserted. How can I achieve that?
Thanks in advance
Use the MERGE command.
Something like this:
MERGE
INTO Persons AS TARGET
USING Persons_Staging AS SOURCE
ON TARGET.ID = SOURCE.ID
--WHEN MATCHED
-- THEN UPDATE???
WHEN NOT MATCHED BY TARGET
THEN INSERT (Id , LastName , FirstName, HouseNumber)
VALUES (SOURCE.Id , SOURCE.LastName , SOURCE.FirstName, SOURCE.HouseNumber)
-- WHEN NOT MATCHED BY SOURCE
-- THEN DELETE???
;
If you want to update existing records you uncomment the UPDATE part and add a suitable update clause. The same with the delete part.
you could use this with a left outer join on both tables to get all the data that isn't the same. That data you then can insert into your column
INSERT INTO Tab1(front,end,number)
SELECT first,last,nr from tab2 LEFT OUTER JOIN tab1 ON front = first AND last = end AND convert(int,number) = CONVERT(int,nr)
WHERE tab1.ID is null
this could work, on the other hand there are tools made for this kind of stuff
What is the simplest way to insert a Car for each user in Users and set the users CarID to the ID of the inserted Car?
[Users]
- ID
- Name
- CarID
[Cars]
- ID (Auto increment)
- Name
Sorry if this might be a duplicate question but I can't find any simple solutions. Everything I've found is using complicated cursors, pointers etc.
A simple and reusable syntax for this would save me hours while migrating data during system upgrades etc.
If you are on SQL Server 2008 or later you can use merge and output something like this.
Sample tables and data:
declare #Users table
(
ID int identity primary key,
Name varchar(10) not null,
CarID int null
);
declare #Cars table
(
ID int identity primary key,
Name varchar(10) not null
);
insert into #Users(Name) values ('User1'),('User2'),('User3');
Add one care for each user and move the auto-generated CarID back to Users.
declare #ID table(CarID int, UserID int)
merge #Cars as C
using #Users as U
on 0 = 1
when not matched then
insert (Name) values ('CarName')
output inserted.ID, U.ID into #ID;
update U
set CarID = I.CarID
from #Users as U
inner join #ID as I
on U.ID = I.UserID
Try it out on SE Data.
More info on the merge/output trick can be found here.
I'm assuming the code you're using lets you call stored procedures?
create procedure dbo.CarUserInsert_sp
(
#p_UserID int,
#p_UserName varchar(100),
#p_CarID int,
#p_CarName varchar(100)
)
as
if not exists ( select 1 from Cars where ID = #p_CarID )
insert Cars values ( #p_CarID, #p_CarName )
if exists (select 1 from Users where ID = #p_UserID )
update Users set Name = #p_UserName, CarID = #p_CarID where ID = #p_UserID
else
insert Users values ( #p_UserID, #p_UserName, #p_CarID )
go
try this:
insert into cars (name)
select distinct(name) from users
update user
set carId = (select ID from cars where cars.name=user.Name)