I want to use the merge command to update all rows in the target which are not in the source set.
I only use two operations with an update command, but I always get "Can not insert duplicate key errors" - as I read the technet article there should not be any insert statement executed...
declare #stagingid as decimal = 2
MERGE SDH.dbo.SDH_ProduktPreis AS T
USING (SELECT *
FROM Staging.mtc.SDH_Produktpreis
WHERE Staging_ID = #stagingid) AS S
ON S.PRPR_CPC = T.PRPR_CPC
AND S.PRPR_Preistyp = T.PRPR_Preistyp
AND T.PRPR_Origin = 'FromSource'
WHEN MATCHED
AND S.PRPR_CPC = T.PRPR_CPC
AND S.PRPR_Preistyp = T.PRPR_Preistyp
AND S.PRPR_Preis <> T.PRPR_Preis
AND S.PRPR_Gueltig_bis = T.PRPR_Gueltig_bis
AND S.Staging_ID = #stagingid
THEN UPDATE SET T.PRPR_sys_status = 6,
T.PRPR_Gueltig_bis = (SELECT CAST(GETDATE() as datetime)),
T.PRPR_sys_change_timestamp = GETDATE()
WHEN NOT MATCHED BY SOURCE
AND T.PRPR_Origin= 'FromSource'
THEN UPDATE SET T.PRPR_Gueltig_bis = (SELECT CAST(GETDATE() as datetime)),
T.PRPR_sys_status = 6,
T.PRPR_sys_change_timestamp = GETDATE();
If I add the "WHEN NOT MATCHED BY SOURCE" command SSMS always complain about inserting a duplicate key is not possible. (All Rows in the target has the field PRPR_Origin set, so only those get minded in the command.
From Technet:
WHEN NOT MATCHED BY SOURCE THEN
Specifies that all rows of target_table that do not match the rows returned by ON , and that satisfy any additional search condition, are either updated or deleted according to the clause.
There does not stand anything from an insert command.
Does anybody has an advice for me? Where does my brain mislead me?
EDIT:
Source and target table are named alike but not the same. They are not even in the same database..
The source table is an extended version of the target with only one key: Staging_ID.
The target table has the same schema but not the Staging_ID column. In the target the PK is a combination of PRPR_CPC, PRPR_Vertriebsweg, RPR_Preistyp and PRPR_Gueltig_bis.
Some example data with PK marked yellow:
Source
Target
But what I really don't get is, why my statement creates an insert with the "WHEN NOT MATCHED BY SOURCE", that behaviour does not fit to the MS Technet article.
EDIT:
For anyone to stumble upon this thread, experiencing something similar.
The Problem here was, that the merge command executes as a bulk operation. In my case this lead temporary to primary key violations, while executing the query.
Related
I am working on a task where my source is AWS RDS - SQL Server and my target is Azure SQL Server.
There's a table with 80M records in my source that needs to be merged with my target table.
This merging will happen every 15 mins and based on the business key, I need to -
Update the target table if the key is updated in the source table.
Insert a new key into the target table.
Mark isDeleted as true in the target if the key is no more present in the source.
IMP Note - The source row is hard-deleted and no history is maintained.
Since this merging happens every 15 mins and the source table is pretty big, I use lastUpdated column to select only limited records in the source query of my merge query.
With this, I am able to perfectly handle the "upsert" scenario, but on delete, it is deleting all the records from the target which is not desirable.
I have tried the below option -
Read the entire source table in a temp_table every 15 mins and then perform merge from temp_table to the target table. But this is very costly in terms of processing and time.
Is there any better way to handle this scenario? I am happy to share more information as needed.
I think you can solve the problem by adding new column called SyncStamp, the idea is, we update or insert the same value for SyncStamp, So the other rows that have not this value should be updated as IsDeleted.
I prefer to get the actual timestamp for SyncStamp but you can choose random numbers.
--get timestamp
Declare #SyncStamp bigint = (SELECT DATEDIFF_BIG(Second, '1970-01-01 00:00:00', GETUTCDATE()))
MERGE TargetTable AS Target
USING SourceTable AS Source
ON Source.BusinessKey = Target.BusinessKey
-- For Inserts
WHEN NOT MATCHED BY Target THEN
INSERT (ProductID,ProductName, SyncStamp)
VALUES (Source.ProductID,Source.ProductName, #SyncStamp)
-- For Updates
WHEN MATCHED THEN UPDATE SET
Target.ProductName = Source.ProductName,
Target.SyncStamp = #SyncStamp;
--Update isDeleted
UPDATE TargetTable
SET IsDeleted= 1
Where IsDeleted=0 and SyncStamp <> #SyncStamp
I have a query that has as the output the following table (2 columns). What it means, is that I'm going to use the result to wrap it inside a merge into statement.
INSERT_COLUMNS UPDATE_COLUMNS
BANK_NAME target.BANK_NAME = source.BANKNAME
What I'm talking about is this:
with sql_prepare_merge as (
SELECT *
FROM another_table
),
MERGE INTO bank_raw AS target
USING bank AS source
ON source.id = target.id
WHEN MATCHED THEN
UPDATE SET (select update_columns from sql_prepare_merge)
WHEN NOT MATCHED THEN
INSERT (select insert_columns from sql_prepare_merge)
VALUES (source.id, (select insert_columns from sql_prepare_merge));
Keep in mind that the "sql_prepare_merge" is the name from the CTE where I'm getting the table I shared with you - and it has much more code in it, but they don't help here. So, I'm planning to take the text resulting from the subquery and insert it inside the merge statement.
So far, the error I'm getting is: syntax error line 19 at position 4 unexpected 'WHEN'.. By the way, this is inside Snowflake.
Unfortunately this is not possible as per my understanding of the docs: https://docs.snowflake.com/en/sql-reference/sql/merge.html#notmatchedclause-for-inserts
In the non matched clause for inserts you can specify only such values/expressions, refer to the source relations. This means you somehow need to adjust your source part itself (e.g. by joining the initial source with another_table). If this is not possible, you would need to go for separate INSERT and UPDATE statements.
I'm trying to merge a temporary table, populated with data from a c# application with a table in my SQL database.
Having read through some articles it seems that SQL Merge should be the most appropriate option and it works great when I'm updating or deleting groups of entries from the database.
My problem is that I'm unable to add a new row of data where the foreign keys don't match what is already in the database table, but instead it is just removing the rows in the database and replacing the new ones in the temporary table.
I have two foreign keys referenced in the table FirstDBTableID and SecondDBTableID.
This is my SQL so far:
ALTER PROCEDURE [dbo].[SP_UpdateTable]
#TempTable as TempTableType READONLY
AS
BEGIN
MERGE dbo.Table AS target
USING #TempTable AS source ON target.FirstDBTableID = source.FirstDBTableID
AND target.SecondDBTableID = source.SecondDBTableID
WHEN MATCHED THEN
UPDATE SET target.Provider = source.Provider, target.Value = source.Value, target.Quantity = source.Quantity
-- value doesn't exist in the target table, so add it
WHEN NOT MATCHED BY TARGET THEN
INSERT (FirstDBTableID, SecondDBTableID, Provider, Value, Quantity)
VALUES (source.FirstDBTableID, source.SecondDBTableID, source.Provider, source.Value, source.Quantity)
-- value doesn't exist in the source table, so delete from target
WHEN NOT MATCHED BY SOURCE THEN
DELETE;
END
Have a missed statement? Maybe SQL Merge isn't going to work here? If I haven't made it clear please ask me questions.
Thanks in advance.
My table has two columns: Amount and Date
When I am adding new values, I want to update the amount if there is already a record with that date. If there is not, I want to just insert my new values.
something like:
if exists(select * from table where date = #date)
update table set amount = #amount where date = #date
else
insert table (amount, date) select #amount, date
Assuming you have at least SQL Server 2008, you could use the MERGE keyword. This keyword was added specifically for cases like yours. You specify a MERGE clause (basically a join) and then statements telling it how to handle when it does or doesn't find a match.
There's a decent example at the bottom that shows merging between a target table and a source "table" created out of a single row of parameters. To boil it down a bit, you could use something like:
MERGE [TargetTable] AS target -- This is the table we'll be updating
USING (SELECT #Date) AS source (Date) -- These are the new values
ON (target.Date = source.Date) -- Here we define how we find a "match"
WHEN MATCHED THEN -- If the date already exists:
UPDATE SET Amount = #Amount
WHEN NOT MATCHED THEN -- If the date does not already exist:
INSERT (Date, Amount) VALUES (#Date, Amount)
Note that the nested UPDATE and INSERT clauses do not specify a table. That is because we already told SQL Server which table to perform those actions on when we defined our target table (the table specified right after the MERGE keyword).
Update: This functionality is apparently not supported by SQL Server CE, so it will not work in your specific case. I am leaving this answer here as it may help others attempting to do something similar in the full version of SQL Server. For the SQL Server CE solution, check out BJury's answer.
As an alternative to IF .. NOT EXISTS, you could assert an UPDATE, and fall back on an insert. This is usually quite a performant UPSERT pattern if data generally does exist already.
UPDATE MyTable
SET Amount = #NewAmount
WHERE Date = #Date;
IF ##ROWCOUNT = 0
INSERT INTO MyTable(Amount, Date) VALUES (#NewAmount, #Date);
If you are using nhibernate ORM, then you can use SaveOrUpdate session method.
I’ve been tasked to synchronize 2 tables (both are identical). They have 60 columns each. Table A is the primary table that will be initially filled. I need to create a stored procedure (done) that will merge these 2 tables and populate both with the same exact data (Update, insert, delete) when called. How would I use the MERGE function in SQL to achieve this? I’ve looked at both the MSDN documentation and similar that’s on technet, but I’m pretty confused on getting started. Do I need to specify each field I need merged? Or is it a simple call I’m missing that will perform this action?
Here is a link to a simple example of the MERGE statement:
http://www.simple-talk.com/sql/learn-sql-server/the-merge-statement-in-sql-server-2008/
The basic syntax reads as:
MERGE table1
USING table2
ON table1.id = table2.id
WHEN MATCHED THEN
--Do an update here
WHEN NOT MATCHED BY TARGET THEN
--Do an insert here (or a delete)
;
You can also use WHEN NOT MATCHED BY SOURCE
Over 60 columns is a great number! When I need to sync 2 identical table I do:
;WITH tbl_to_synch as (
-- Prepare table to update,
Select *,chk = CHECKSUM(*) from [dbo].[tableA]
)
MERGE tbl_to_synch as [Target]
USING (Select *,chk = CHECKSUM(*) from [dbo].[tableB]) as [source]
ON [Target].key = [source].key
WHEN MATCHED AND [Target].chk <> [source].chk THEN
-- UPDATE ONLY row that is changed
UPDATE
SET
column01 = [source].[column01]
,column02 = [source].[column01]
-- ....
,column59 = [source].[column59]
,column60 = [source].[column59]
WHEN NOT MATCHED BY TARGET THEN
insert (column01, column02, ...,column59,column60)
values (column01, column02, ...,column59,column60)
WHEN NOT MATCHED BY SOURCE THEN DELETE
-- Show what is changed
OUTPUT $action, ISNULL(INSERTED.key,DELETED.key);