I have a SQL Server database. I am looking to insert some values into multiple tables, where the data does not already exist.
Example:
Table 1
ID
Name
Material
Other
1
Aluminum
2014
v1
2
Magnesium
2013
v2
I want to develop a stored procedure such that it will insert into a table the following information:
Aluminum | 2013
My current stored procedure does not let me do this as it recognizes Magnesium | 2013 and rejects 2013 because it is being duplicated.
Also how would we compare multiple column values, for example:
INSERT WHERE NOT EXISTS (Material = 2014 AND Other = v3)
Current stored procedure:
IF EXISTS(SELECT *
FROM dbo.[01_matcard24]
WHERE NOT EXISTS (SELECT [01_matcard24].Element
FROM dbo.[01_matcard24]
WHERE dbo.[01_matcard24].Element = #new_element)
AND NOT EXISTS (SELECT [01_matcard24].Material
FROM dbo.[01_matcard24]
WHERE dbo.[01_matcard24].Material = #new_material)
)
INSERT INTO dbo.[15_matcard24_basis-UNUSED] (Element, Material)
VALUES (#new_element, #new_material)
Rather than using an IF statement, keep it set-based as part of the INSERT
Create your EXISTS sub-query to detect unique rows, not individual columns
INSERT INTO dbo.[15_matcard24_basis-UNUSED] (Element, Material)
SELECT #new_element, #new_material
WHERE NOT EXISTS (
SELECT 1
FROM dbo.[15_matcard24_basis-UNUSED]
WHERE Element = #new_element AND Material = #new_material
-- AND Other = #Other
);
Related
I have the following SQL query I want to use to import data into Power BI. It involves the creation of a temp table and using that table as the main data source. How can I do this in Power BI? I tried using this query in the editor when loading data from a database, but I keep getting an error like so
I basically used this dataset https://www.kaggle.com/kyanyoga/sample-sales-data and loaded it into a postgressql database.
-- 1. Create temp table to house temporary results
DROP TABLE IF EXISTS product_quantity;
CREATE TEMP TABLE product_quantity
(product_line varchar, this_month_quantity integer, last_month_quantity integer)
--2. Quantity ordered for each Product line for current month is inserted into temporary table.
INSERT INTO product_quantity (product_line, this_month_quantity, last_month_quantity)
SELECT "productline", SUM("quantityordered"), 0
FROM test_schema.sales_data_sample
where "month_id" = 3 and "year_id" = 2003
GROUP BY "productline";
--3. Quantity ordered for each Product line for last month is inserted into temporary table.
INSERT INTO product_quantity (product_line, this_month_quantity, last_month_quantity)
SELECT "productline", 0, SUM("quantityordered")
FROM test_schema.sales_data_sample
where "month_id" = 2 and "year_id" = 2003
GROUP BY "productline";
--4. Retrieve required results.
select
"product_line",
sum("this_month_quantity") as "this_month_quantity",
sum("last_month_quantity") as "last_month_quantity"
FROM product_quantity
group by "product_line"
Does this query run without an error?
I've converted your query to one big inline query.
select
ST."product_line",
sum(ST."this_month_quantity") as "this_month_quantity",
sum(ST."last_month_quantity") as "last_month_quantity"
FROM
(
SELECT "productline",
SUM("quantityordered") as this_month_quantity,
0 as last_month_quantity
FROM test_schema.sales_data_sample
where "month_id" = 3 and "year_id" = 2003
GROUP BY "productline"
UNION ALL
SELECT "productline",
0,
SUM("quantityordered")
FROM test_schema.sales_data_sample
where "month_id" = 2 and "year_id" = 2003
GROUP BY "productline"
) as ST
group by ST."product_line"
(note I've just taken a guess at conversion - I don't have a postgresql to test on)
I have a table, known as Fruit_Veg_Product_Table which is used to contain the characteristics of certain fruit and vegetable stock.
The table has the following columns:
Product_ID
Product_Type
Product_Name
Product_Colour
Product_Price
Product_In_Sale
Product_Stock_Level
Product_Height
Product_Width
Product_Depth
Product_Package_Height
Product_Package_Width
Product_Package_Depth
When a new product is inserted into the table, sometimes the product is inserted without any dimensions (the columns from Product_Height all the way to Product_Package_Depth). In this circumstance, the dimensions are entered as NULL.
I am in need of a SQL Server trigger that will replace all the NULL values from the attempted insert with the values corresponding to products that are already stored in the table which share a common Product_Type with the product that is being entered.
Any help with this problem is greatly appreciated.
Triggers have an INSERTED logical table that can be used to join the inserted row data back to the physical table. Here is an example:
CREATE TRIGGER Fruit_Veg_Product_Table_Trg
ON dbo.Fruit_Veg_Product_Table
FOR INSERT
AS
UPDATE dbo.Fruit_Veg_Product_Table
SET Product_Package_Height = ca.Product_Package_Height,
Product_Package_Width = ca.Product_Package_Width,
Product_Package_Depth = ca.Product_Package_Depth
FROM dbo.Fruit_Veg_Product_Table
CROSS APPLY
(
SELECT TOP 1
Product_Package_Height,
Product_Package_Width,
Product_Package_Depth
FROM dbo.Fruit_Veg_Product_Table AS fvpt
WHERE dbo.Fruit_Veg_Product_Table.Product_Type = fvpt.Product_Type
AND Product_Package_Height IS NOT NULL
AND Product_Package_Width IS NOT NULL
AND Product_Package_Depth IS NOT NULL
) AS ca
WHERE EXISTS
(
SELECT *
FROM INSERTED
WHERE INSERTED.Product_ID = dbo.Fruit_Veg_Product_Table.Product_ID
AND INSERTED.Product_Package_Height IS NULL
AND INSERTED.Product_Package_Width IS NULL
AND INSERTED.Product_Package_Depth IS NULL
);
GO
I'm given a list of transaction records from a remote server, some of which already exist in our database and some of which are new. My task is to update the ones that already exist and insert the ones that don't. Assume the transactions have remote IDs that aren't dependent on my local database. The size of the list can be anywhere from 1 to ~500.
Database is postgresql.
My initial thought was something like this:
BEGIN
CREATE TEMP TABLE temp_transactions (LIKE transactions) ON COMMIT DROP;
INSERT INTO temp_transactions(...) VALUES (...);
WITH updated_transactions AS (...update statement...)
DELETE FROM temp_transactions USING updated_transactions
WHERE temp_transactions.external_id = updated_transactions.external_id;
INSERT INTO transactions SELECT ... FROM temp_transactions;
COMMIT;
In other words:
Create a temp table that exists only for the life of the transaction.
Dump all my records into the temp table.
Do all the updates in a single statement that also deletes the updated records from the temp table.
Insert anything remaining in the temp table in to the permanent table because it wasn't an update.
But then I began to wonder whether it might be more efficient to use a per-session temp table and not wrap all the operations in a single transaction. My database sessions are only ever going to be used by a single thread, so this should be possible:
CREATE TEMP TABLE temp_transactions IF NOT EXISTS (LIKE transactions);
INSERT INTO temp_transactions(...) VALUES (...);
WITH updated_transactions AS (...update statement...)
DELETE FROM temp_transactions USING updated_transactions
WHERE temp_transactions.external_id = updated_transactions.external_id;
INSERT INTO transactions SELECT ... FROM temp_transactions;
TRUNCATE temp_transactions;
My thinking:
This avoids having to create the temp table each time a new batch of records is received. Instead, if a batch has already been processed using this database session (which is likely) the table will already exist.
This saves rollback space since I'm not stringing together multiple operations within a single transaction. It isn't a requirement that the entire update/insert operation be atomic; the only reason I was using a transaction is so the temp table would be automatically dropped upon commit.
Is the latter method likely to be superior to the former? Does either method have any special "gotchas" I should be aware of?
What you're describing is commonly known as upsert. Even the official documentation mentions it, here: http://www.postgresql.org/docs/current/static/plpgsql-control-structures.html#PLPGSQL-UPSERT-EXAMPLE
The biggest problem with upserts are concurrency problems, as described here: http://www.depesz.com/2012/06/10/why-is-upsert-so-complicated/ and here: http://johtopg.blogspot.com.br/2014/04/upsertisms-in-postgres.html
I think your approach is good, although I wouldn't use a temporary table at all, and put the VALUES part into the UPDATE part, to make the whole thing a single statement.
Like this:
CREATE TABLE test (id int, data int);
CREATE TABLE
WITH new_data (id, data) AS (
VALUES (1, 2), (2, 6), (3, 10)
),
updated AS (
UPDATE test t
SET data = v.data
FROM new_data v
WHERE v.id = t.id
RETURNING t.id
)
INSERT INTO test
SELECT *
FROM new_data v
WHERE NOT EXISTS (
SELECT 1
FROM updated u
WHERE u.id = v.id
);
INSERT 0 3
SELECT * FROM test;
id | data
----+------
1 | 2
2 | 6
3 | 10
(3 rows)
WITH new_data (id, data) AS (
VALUES (1, 20), (2, 60), (4, 111)
),
updated AS (
UPDATE test t
SET data = v.data
FROM new_data v
WHERE v.id = t.id
RETURNING t.id
)
INSERT INTO test
SELECT *
FROM new_data v
WHERE NOT EXISTS (
SELECT 1
FROM updated u
WHERE u.id = v.id
);
INSERT 0 1
SELECT * FROM test;
id | data
----+------
3 | 10
1 | 20
2 | 60
4 | 111
(4 rows)
PG 9.5+ will support concurrent upserts out of the box, with the INSERT ... ON CONFLICT DO NOTHING/UPDATE syntax.
I would like to insert records in a table below (structure of table with example data). I have to use TSQL to achieve this:
MasterCategoryID MasterCategoryDesc SubCategoryDesc SubCategoryID
1 Housing Elderly 4
1 Housing Adult 5
1 Housing Child 6
2 Car Engine 7
2 Car Engine 7
2 Car Window 8
3 Shop owner 9
So for example if I enter in a new record with MasterCategoryDesc = 'Town' it will insert '4' in MasterCategoryID with the respective SubCategoryDesc + ID.
CAN I SIMPLIFY THIS QUESTION BY REMOVING THE SubCategoryDesc and SubCategoryID columns. How can I achieve this now just with the 2 columns MasterCategoryID and MasterCategoryDesc
INSERT into Table1
([MasterCategoryID], [MasterCategoryDesc], [SubCategoryDesc], [SubCategoryID])
select TOP 1
case when 'Town' not in (select [MasterCategoryDesc] from Table1)
then (select max([MasterCategoryID])+1 from Table1)
else (select [MasterCategoryID] from Table1 where [MasterCategoryDesc]='Town')
end as [MasterCategoryID]
,'Town' as [MasterCategoryDesc]
,'owner' as [SubCategoryDesc]
,case when 'owner' not in (select [SubCategoryDesc] from Table1)
then (select max([SubCategoryID])+1 from Table1)
else (select [SubCategoryID] from Table1 where [SubCategoryDesc]='owner')
end as [SubCategoryID]
from Table1
SQL FIDDLE
If you want i can create a SP too. But you said you want an T-SQL
This will take three steps, preferably in a single Stored Procedure. Make sure it's within a transaction.
a) Check if the MasterCategoryDesc you are trying to insert already exists. If so, take its ID. If not, find the highest MasterCategoryID, increase by one, and save it to a variable.
b) The same with SubCategoryDesc and SubCategoryID.
c) Insert the new record with the two variables you created in steps a and b.
Create a table for the MasterCategory and a table for the SubCategory. Make an ___ID column for each one that is identity (1,1). When loading, insert new rows for nonexistent values and then look up existing values for the INSERT.
Messing around with finding the Max and looking up data in the existing table is, in my opinion, a recipe for failure.
I have a table with 4 columns - ID, ClubID, FitnessTestNameID and DisplayName
I have another table called Club and it has ID and Name
I want to add two rows of data to the 1st table for each club
I can write a statement like this, but can someone tell me how to create a loop so that I can insert the two rows, set the #clubid + 1 and then loop back again?
declare #clubid int
set #clubid = 1
insert FitnessTestsByClub (ClubID,FitnessTestNameID,DisplayName)
values (#clubid,'1','Height (cm)')
insert FitnessTestsByClub (ClubID,FitnessTestNameID,DisplayName)
values (#clubid,'2','Weight (kg)')
You can probably do this with one statement only. No need for loops:
INSERT INTO FitnessTestsByClub
(ClubID, FitnessTestNameID, DisplayName)
SELECT
c.ID, v.FitnessTestNameID, v.DisplayName
FROM
Club AS c
CROSS JOIN
( VALUES
(1, 'Height (cm)'),
(2, 'Weight (kg)')
) AS v (FitnessTestNameID, DisplayName)
WHERE
NOT EXISTS -- a condition so no duplicates
( SELECT * -- are inserted
FROM FitnessTestsByClub AS f -- and the statement can be run again
WHERE f.ClubID = c.ID -- in the future, when more clubs
) -- have been added.
;
The Table Value Constructor syntax above (the (VALUES ...) construction) is valid from version 2008 and later.
There is a nice article with lots of useful examples of how to use them, by Robert Sheldon: Table Value Constructors in SQL Server 2008