I got a performance issue with an import task (user need to import new data into the system) and could use little help.
The data within the database looks like this:
Table Categories:
Columns:
Id int primary key identity(1,1)
Code nvarchar(20) not null
RootId int not null
ParentId int not null
Table CategoryNames:
Columns:
CategoryId int not null foreign key references Category (Id)
LanguageId int not null foreign key references Language (Id)
Name nvarchar(100)
Currently it's working like this: for each row
Create connection to database
Call stored procedure with data[i]
Close connection
I already got rid of the creating and closing connection for each row of data. But it's still not good enough.
Currently the task needs round about 36 minutes to insert ~22000 categories.
The stored procedure looks like this:
procedure Category_Insert
(#code nvarchar(20),
#root int,
#parent int,
#languageId int,
#name nvarchar(100),
#id int output)
as
set nocount on
insert into Categories (Code, RootId, ParentId)
values (#code, #root, #parent)
select id = scope_identity()
insert into CategoryNames (CategoryId, LanguageId, Name)
values (#id, #languageId, #name)
Got any advice how I can speed up the performance of that task?
I would love to use bulk insert or something like that, but how would I realize the logic of the stored procedure with bulk insert?
Or is the any other way to speed this up?
Related
I have two related tables something like this:
CREATE TABLE test.items
(
id INT identity(1,1) PRIMARY KEY,
type VARCHAR(max),
price NUMERIC(6,2)
);
CREATE TABLE test.books
(
id INT PRIMARY KEY REFERENCES test.items(id),
title VARCHAR(max),
author VARCHAR(max)
);
Is it possible to insert into both tables using a single SQL statement?
In PostgreSQL, I can use something like this:
-- PostgreSQL:
WITH item AS (INSERT INTO test.items(type,price) VALUES('book',12.5) RETURNING id)
INSERT INTO test.books(id,title) SELECT id,'Good Omens' FROM item;
but apparently SQL Server limits CTEs to SELECT statements, so that won’t work.
In principle, I could use the OUTPUT clause this way:
-- SQL Server:
INSERT INTO test.items(type, price)
OUTPUT inserted.id, 'Good Omens' INTO test.books(id,title)
VALUES ('book', 12.5);
but this doesn’t work if there’s a foreign key involved, as above.
I know about using variables and procedures, but I wondered whether there is a simple single-statement approach.
You can using dynamic sql as follows. Although its awkward to construct query like this.
CREATE TABLE dbo.items (
id INT identity(1,1) PRIMARY KEY,
type VARCHAR(max),
price NUMERIC(6,2)
);
CREATE TABLE dbo.books (
id INT PRIMARY KEY REFERENCES dbo.items(id),
title VARCHAR(max),
author VARCHAR(max)
);
insert into dbo.books(id,title)
exec ('insert into dbo.items(type,price) output inserted.id,''Good Omen'' VALUES(''book'',12.5)')
I created a new database called sample1. In it I created 3 tables: "address, contact and buyer"
CREATE TABLE address
(
address_id int NOT NULL identity(1,1) primary key,
street_name varchar(30)
);
CREATE TABLE contact
(
contact_id int NOT NULL identity(1,1) primary key,
phone_num varchar(20) NOT NULL,
address_id int NOT NULL foreign key REFERENCES address(address_id)
);
CREATE TABLE buyer
(
username char(20) NOT NULL primary key,
name char(40) NOT NULL,
contact_id int NOT NULL foreign key REFERENCES contact(contact_id)
);
The tables were created successfully, now I'm trying to create a stored procedure as follows:
create procedure csc
(
#username char(20),
#address_id int,
#contact_id int
)
as
set nocount on;
insert into [dbo].[buyer] values (#username);
insert into [dbo].[address_id] values (#address_id);
insert into [dbo].[contact_id] values (#contact_id);
go
But I get the following errors:
Msg 137, Level 15, State 2, Line 4
Must declare the scalar variable "#username".
Msg 137, Level 15, State 2, Line 10
Must declare the scalar variable "#username".
Msg 137, Level 15, State 2, Line 11
Must declare the scalar variable "#address_id".
Msg 137, Level 15, State 2, Line 12
Must declare the scalar variable "#contact_id".
My questions / errors:
Are my tables logical? Should I be using buyer username as the PK for address / contact ID (my teacher suggested this, but I think it's bad practice)
When creating my procedure, I get red lines on [dbo].[buyer], [dbo].
[address_id], [dbo].[contact_id], can I ignore these?
How in the world do I fix the errors?
Thanks in advance.
1, are my tables logical? should I be using buyer username as the PK
for address / contact ID (my teacher suggested this, but I think it's
bad practice)
It depends whether a buyer is always defined by their username and that won't get updated. This is an age old debate between natural and surrogate keys, you can have a google and find some pretty convincing arguments for either side. If you think it's bad practice then you'll need to be able to explain why you think this. I would certainly avoid using the char datatype unless you really wanted to use it, it will blank pad your data up to it's specified length.
2, when creating my procedure, I get red lines on [dbo].[buyer],
[dbo]. [address_id], [dbo].[contact_id], can I ignore these?
That's because when SSMS last looked, those tables do not exist. If you press CTRL+SHIFT+R, you will refresh intelligence and it will check again. It'll still fail though because address_id nor contact_id are tables.
3, how in the world do I fix the errors?
You fix the table names. But once you've done that you'll reach another error as you aren't specifying what column your values should be inserted into and there are multiple it can use. The problem is that your code wants to insert your primary key columns but with no additional data - so really you need to complete your code so that all the required data is input and the columns they are inserting into are explicitly mentioned.
e.g.
insert into [dbo].[buyer] (username, name, contact_id) values (#username, #name, #contact_id);
But.. you are inserting the contact row at the same time and you've declared it as an identity column, should you really know the value of this column already or should you be inserting the other values into the table and generating a new contact_id with it?
Perhaps your procedure should really start
create procedure csc(
#username char(20),
#name char(40),
#street_name varchar(30),
#phone_num varchar(20)
)
And then build up your insert statements from that data. Something like:
as
set nocount on;
declare #address_id int;
declare #contact_id int;
insert into [dbo].[address] (street_name) values (#street_name);
set #address_id = SCOPE_IDENTITY() ;
insert into [dbo].contact (phone_num, address_id) values (#phone_num, #address_id);
set #contact_id = SCOPE_IDENTITY() ;
insert into [dbo].[buyer] (username, name, contact_id) values (#username, #name, #contact_id);
go
And to demo:
exec csc 'andy','andy','Nice Road',42
select * from buyer;
select * from contact;
select * from address;
username name contact_id
-------------------- ---------------------------------------- -----------
andy andy 3
(1 row affected)
contact_id phone_num address_id
----------- -------------------- -----------
3 42 3
(1 row affected)
address_id street_name
----------- ------------------------------
3 Nice Road
(1 row affected)
It looks like you've got a couple of typos. Try it like this
create procedure csc
#username char(20),
#address_id int,
#contact_id int
as
set nocount on;
insert into [dbo].[buyer] values (#username);
insert into [dbo].[address] values (#address_id);
insert into [dbo].[contact] values (#contact_id);
go
Use a begin/end block:
create procedure csc (
#username char(20),
#address_id int,
#contact_id int
) as
begin
set nocount on;
insert into [dbo].[buyer] values (#username);
insert into [dbo].[address_id] values (#address_id);
insert into [dbo].[contact_id] values (#contact_id);
end;
go
I'm having problem creating a trigger. The database model look something like this:
CREATE TABLE Unit (
SerialNO VARCHAR(3)
PRIMARY KEY (serialNO)
);
CREATE TABLE PackageConfig (
PartNO VARCHAR(15),
SomeValue VARCHAR(20),
PRIMARY KEY (PartNO)
);
CREATE TABLE Package (
ID INT IDENTITY(1,1),
Config VARCHAR(15),
PRIMARY KEY (ID),
FOREIGN KEY (Config) REFERENCES PackageConfig(PartNO)
);
CREATE TABLE UnitInPackage (
Package INT,
Unit VARCHAR(3),
PRIMARY KEY (Package,Unit),
FOREIGN KEY (Package) REFERENCES Package(ID),
FOREIGN KEY (Unit) REFERENCES Unit(SerialNO)
);
There are Units, PackageConfigurations and Packages. A package has exactly one PackageConfiguration and the relation between Package and Unit is a many (units) to at most one (package).
What I'm trying to accomplish is a trigger on a view that would let me do the following (assuming that the PackageConfig and Units exist):
INSERT INTO v_MultiUnitPackage (PackageConfig, Unit1, Unit2, Unit3)
VALUES ('SomeConfig','abc','bcd','cde');
Which inside the trigger would do something like:
DECLARE #last INT;
INSERT INTO Package (Config) VALUES ('SomeConfig');
SET #last = SCOPE_IDENTITY();
INSERT INTO UnitInPackage (Package,Unit)
VALUES (#last,'abc'),(#last,'bcd'),(#last,'cde');
Or for more columns:
INSERT INTO Package (Config) VALUES ('SomeConfig');
SET #last = SCOPE_IDENTITY();
INSERT INTO UnitInPackage (Package,Unit)
VALUES (#last,'hgf'),(#last,'gfe'),(#last,'fed'),(#last,'edc'),(#last,'dcb'),(#last,'cba');
To summarize, I'd like to create a trigger that takes a PackageConfig and N-number of Units and insert them in the corresponding tables.
I've been looking into creating a dummy view that simply has the correct data types and a great enough number of columns to allow for the number of Units I want but didn't find a solution.
I also looked into something like grouping by Package.ID and for each group selecting the first unit that hasn't already been selected to a previous column. Since GROUP BY is only usable with aggregate functions I'm unsure how to realize this idea.
Realistically I don't see ever needing more than 5 or so Units but would prefer a generic solution to my problem.
Perhaps there is a really simple solution that I'm just not seeing. Any help is greatly appreciated.
Insert with a view is one way of doing it, so you just need to have your dummy view update with the number of units you need. Here is how the trigger should looks like:
create view v_MultiUnitPackage
as
select 'Some-Config' as PackageConfig, 'abc' as Unit1, 'bcd' as Unit2, 'cde' as Unit3
go
create TRIGGER tg_I_v_MultiUnitPackage
ON v_MultiUnitPackage
INSTEAD OF INSERT
AS
BEGIN
DECLARE #last INT;
INSERT INTO Package (Config)
SELECT inserted.PackageConfig
FROM inserted
SET #last = SCOPE_IDENTITY();
INSERT INTO UnitInPackage (Package,Unit)
SELECT #last, inserted.Unit1
FROM inserted;
INSERT INTO UnitInPackage (Package,Unit)
SELECT #last, inserted.Unit2
FROM inserted;
INSERT INTO UnitInPackage (Package,Unit)
SELECT #last, inserted.Unit3
FROM inserted;
END
A better option in my opinion, would be passing the units to a stored procedure as XML and handle the insert there. Here is how the procedured should looks like:
CREATE PROCEDURE usp_i_PackageUnits
(
#PackageConfig varchar(15),
#Units xml -- <root><unit>abc</unit><unit>bcd</unit><unit>cde</unit>
)
AS
BEGIN
DECLARE #last INT;
INSERT INTO Package (Config) VALUES (#PackageConfig);
SET #last = SCOPE_IDENTITY();
INSERT INTO UnitInPackage (Package,Unit)
select node.value('(.)[1]', 'VARCHAR(3)') from #xml.nodes('/root/unit')as result(node)
END
I have two tables:
Rooms:
ID (auto-incrementing primary key, int)
Topic (varchar(50))
MangerId (varchar(50))
Rooms_Users:
UserId (varchar(50))
RoomId (varchar(50))
both fields together are the primary key
I want to insert a room but I also must insert the manger to the table rooms_users.
Here is what I have so far:
ALTER PROCEDURE [dbo].[Creat_Room] #MangerId varchar(50) ,#Topic varchar(50)
AS
BEGIN
SET NOCOUNT
insert into Rooms(ManagerId,Topic) values(#MangerId,#Topic)
insert into Rooms_Users(UserId,RoomId) values(#MangerId,?????????????)
END
The ????????????? is the problem: I don't know what to put here i want to put the roomid i insert above.
You can use the output clause. Look at MSDN here: OUTPUT Clause (Transact-SQL)
Example:
declare #tbl table
(
NewID int
)
insert into Rooms(ManagerId,Topic)
output inserted.ID into #tbl
values(#MangerId,#Topic)
Then the table variable will contains the new id given to the row you inserted
Use the SCOPE_IDENTITY() function:
ALTER PROCEDURE [dbo].[Create_Room]
#ManagerId varchar(50),
#Topic varchar(50)
AS
BEGIN
DECLARE #NewRoomID INT
insert into Rooms(ManagerId, Topic) values(#MangerId, #Topic)
SELECT #NewRoomID = SCOPE_IDENTITY()
insert into Rooms_Users(UserId, RoomId) values(#ManagerId, #NewRoomID)
END
This function will return the last inserted IDENTITY value in this particular scope - the scope of your stored procedure.
I've created a insert stored procedure with two tables like in the exapmle:
Table NameAge
CREATE TABLE [dbo].[Assignment3_NameAge]
(
userID int PRIMARY KEY IDENTITY(1,1),
Name varchar(255) NOT NULL,
Age int NOT NULL
)
Table Hobbies
CREATE TABLE [dbo].[Assignment3_Hobbies]
(
hobbiesID int Identity(1,1) Primary Key,
userID int Foreign Key references Assignment3_NameAge(userID),
hobbies varchar(255) NOT NULL,
)
Insert Stored Procedure
CREATE PROCEDURE [dbo].p_Assignment3Join_ins
#Name nvarchar(100),
#Age int,
#Hobbies nvarchar(100)
AS
INSERT INTO [TABLE].[dbo].[Assignment3_NameAge]
([Name]
,[Age])
VALUES (#Name,#Age)
INSERT INTO [TABLE].[dbo].[Assignment3_Hobbies]
([Hobbies])
VALUES (#Hobbies)
The problem is that when i run the stored procedure the table Hobbies has a null value for userid(the foreign key)
What am i doing wrong?
You should provide the key of the Assignment3_NameAge value you want to insert into Assignment3_Hobbies.
If you want the last inserted you can use SCOPE_IDENTITY() from SQL Server(if you're using SQL Server) or equivalent. It will give you the last inserted value from Assignment3_NameAge
I am guessing this is SQL Server based on the IDENTITY column. Correct?
The first insert creates a user, but there is no user ID being set on the insert of the hobby. You need to capture the identity value from the first insert to be used in the second insert. Have you gon over the system functions available?
You're not supplying a value for it, SQL won't automagically fill the value in for you even though you've created a Foreign Key relationship. It's your job to populate the tables.