I'm trying to use MERGE to insert new values to a table only if they don't already exists in the same table.
This is the query I am using:
MERGE [dbo].[TARGET_TABLE] AS Target
USING
(SELECT [NAME]
FROM [dbo].[TARGET_TABLE]
WHERE [NAME]='ThisValuesDoesntExists' AND [STATUS] IS NULL) AS Source
ON Target.[NAME]= Source.[NAME]
WHEN NOT MATCHED
THEN INSERT ([NAME],[file_first_upload],[upload_date])
VALUES('ThisValuesDoesntExists',1,DEFAULT);
But when I execute it, I get a (0 rows affected) message.
If I execute the "Source" query, I get 0 rows.
SELECT [NAME]
FROM [dbo].[TARGET_TABLE]
WHERE [NAME] = 'ThisValuesDoesntExists' AND [STATUS] IS NULL
What am I doing wrong?
Thanks
If you look at the MERGE documentation, you will see that the source data must exist in order to match (or not match) against existing rows in the target table:
WHEN NOT MATCHED [ BY TARGET ] THEN <merge_not_matched>
Specifies that a row is inserted into target_table for every row
returned by <table_source> ON <merge_search_condition> that doesn't
match a row in target_table, but satisfies an additional search
condition, if present. The values to insert are specified by the
<merge_not_matched> clause. The MERGE statement can have only one WHEN
NOT MATCHED [ BY TARGET ] clause.
The problem you're facing is that your "source" data is not returning anything and so the MERGE query has nothing to match against or insert.
Sample code below to demo:
IF OBJECT_ID('dbo.TARGET_TABLE', 'U') IS NOT NULL DROP TABLE dbo.TARGET_TABLE
GO
CREATE TABLE TARGET_TABLE ([Name] VARCHAR(100), file_first_upload BIT, upload_date DATETIME, [STATUS] VARCHAR(100))
MERGE [dbo].[TARGET_TABLE] AS Target
USING
(SELECT [NAME]
FROM [dbo].[TARGET_TABLE]
WHERE [NAME]='ThisValuesDoesntExists' AND [STATUS] IS NULL) AS Source
ON Target.[NAME]= Source.[NAME]
WHEN NOT MATCHED
THEN INSERT ([NAME],[file_first_upload],[upload_date])
VALUES('ThisValuesDoesntExists',1,DEFAULT);
SELECT *
FROM TARGET_TABLE
MERGE [dbo].[TARGET_TABLE] AS Target
USING (VALUES ('ThisValuesDoesntExistss',1,GETDATE())) AS Source ([Name], [file_first_upload],[upload_date])
ON Target.[NAME] = Source.[Name]
WHEN NOT MATCHED
THEN INSERT ([NAME],[file_first_upload],[upload_date]) VALUES (Source.[Name], Source.file_First_upload, Source.upload_date);
SELECT *
FROM TARGET_TABLE
Related
Can I use Merge to insert a record when it's doesn't exist like below,
MERGE INTO [dbo].[Test] AS [Target]
USING (SELECT DISTINCT [Name] FROM [dbo].[Test]) AS [Source]
ON [Target].[Name] = [Source].[Name]
WHEN NOT MATCHED THEN
INSERT ([Id], [Name])
VALUES (NEWID(), 'Hello');
If the record with value Hello does not exists in table Test, insert it otherwise don't do anything. With above code record is not inserted even I don't have this record in table. And there are no errors.
I know how to accomplish this using insert ... where not exists (...) but am specifically wanting to know how to do it using a merge statement.
The reason your merge statement wasn't working is that you were merging the same table, dbo.Test, back onto itself, so of course there is no missing record.
You can insert a single missing record as follows, where you create a source query to contain the record(s) you wish to insert:
declare #Test table (id uniqueidentifier, [Name] nvarchar(64))
select * from #Test
-- Returns
-- id | Name
-- ----------------------------------------------
MERGE INTO #Test AS [Target]
USING (select 'Hello' [Name]) AS [Source]
ON [Target].[Name] = [Source].[Name]
WHEN NOT MATCHED THEN
INSERT ([Id], [Name])
VALUES (NEWID(), [Name]);
select * from #Test
-- Returns
-- id | Name
-- ----------------------------------------------
-- C1C87CD5-F745-436D-BD8D-55B2AF431BED | Hello
I agree with the answer from Dale K. Its correct.
If I suppose you might have a source_table from where the data needs to get inserted and not to get inserted if the record already exists then you can do the following.
Instead of the MERGE you can
insert
into dbo.Test
(id
,name
)
select top 1
newID()
,'Hello'
from dbo.Test a
where not exists(select 1
from dbo.Test b
where b.name='Hello')
Have a question about the MERGE syntax for which I cannot find the answer.
I have the following case:
Step1:
create temp table #TempTbl
Step2: MERGE:
MERGE INTO T1 target
USING T2 AS source ON (bunch of columns)
WHEN MATCHED
UPDATE
SET some columns from target equal some columns from source
WHEN NOT MATCHED BY TARGET
THEN INSERT (bunch of columns)
VALUES (bunch of columns from SOURCE)
OUTPUT $action, deleted.* into #TempTbl
What I need to know is for my above steps wouldn't I find only empty data in my temporary table #TempTbl, as I only stated WHEN NOT MATCHED ... THEN INSERT, not DELETE?
Second question, what type of column should $action be, as I'm having the error message:
Column name or supplied values do not match table definition
Although I've tried to define the first column from my table both varchar(100), nvarchar(100), but with no luck. But, If I omit the $action field, then my statement works.
So, the column that will hold the $action should be nvarchar(10).
The following statement would add rows to the temp table for both insert and update (as the update is really a delete followed by an insert) but with different actions:
-- sample test data
create table t1 (col1 int, col2 int)
create table t2 (col1 int, col2 int)
insert t1 values (1,1),(2,1)
insert t2 values (2,2),(3,3)
create table #temptbl (dml_action nvarchar(10), col1 int, col2 int)
-- merge statement
merge into t1 target
using t2 as source
on target.col1 = source.col1
when matched
then update set target.col2 = source.col2
when not matched by target
then insert (col1, col2) values (source.col2, source.col2)
output $action, inserted.col1, inserted.col2 into #temptbl ;
-- sample result
select * from #temptbl
dml_action col1 col2
---------- ----------- -----------
INSERT 3 3
UPDATE 2 2
If you don't want the update rows you could wrap the entire batch into another statement like so:
insert #temptbl (dml_action, col1, col2)
select dml_action, col1, col2
from
(
merge into t1 target
using t2 as source
on target.col1 = source.col1
when matched
then update set target.col2 = source.col2
when not matched by target
then insert (col1, col2) values (source.col2, source.col2)
output $action as dml_action, inserted.col1, inserted.col2
) a
where a.dml_action = 'INSERT'
Let say I have,
MERGE INTO SHARE_AD_GROUP A
USING (
SELECT SHARE_AD_GROUP_ID,
SHARE_ID,
AD_GROUP,
SHARE_PERMISSIONS
FROM SHARE_AD_GROUP
WHERE SHARE_ID = #shareID AND AD_GROUP = #ownerId
) B ON (A.SHARE_AD_GROUP_ID = B.SHARE_AD_GROUP_ID)
WHEN MATCHED THEN
UPDATE SET A.SHARE_PERMISSIONS = B.SHARE_PERMISSIONS
WHEN NOT MATCHED THEN
INSERT (SHARE_PERMISSIONS) VALUES(#sharePermissions);
-- In Here how do I know that it is insert or update
How to know that MERGE operation was INSERT or UPDATE after INSERT OR UPDATE?
Please refer here
DECLARE #SummaryOfChanges TABLE(Change VARCHAR(20));
MERGE tblTarget AS Target
USING (SELECT Col1,Col2 FROM tblSource)
AS Source
ON (Target.Col1 = Source.Col1)
WHEN MATCHED THEN
UPDATE SET target.Col2 = source.Col2 -- Need to get affected rows here
WHEN NOT MATCHED BY TARGET THEN
INSERT (Col1,Col2) VALUES (Col1,Col2); -- Need to get affected rows here
OUTPUT $action INTO #SummaryOfChanges;
SELECT Change, COUNT(*) AS CountPerChange
FROM #SummaryOfChanges
GROUP BY Change;
Given is a simple MERGE statement. Where I Insert/Update records into traget table. Question: is it possible to also Insert those values in another table with a flag beeing 0 for insert and 1 for update? Eg. when not match do insert into target and another table, when matched do update target and insert into another table.
MERGE dbo.FactBuyingHabits AS Target
USING (SELECT CustomerID, ProductID, PurchaseDate FROM dbo.Purchases) AS Source
ON (Target.ProductID = Source.ProductID AND Target.CustomerID = Source.CustomerID)
WHEN MATCHED THEN
UPDATE SET Target.LastPurchaseDate = Source.PurchaseDate
--and insert into test_tbl values (1, Source.ProductID, Source.CustomerID) --?
WHEN NOT MATCHED BY TARGET THEN
INSERT (CustomerID, ProductID, LastPurchaseDate)
VALUES (Source.CustomerID, Source.ProductID, Source.PurchaseDate)
--and insert into test_tbl values (0, Source.ProductID, Source.CustomerID) --?
you should read about OUTPUT
ex (source);
DECLARE #MergeOutput1 table
(
ActionType nvarchar(10),
BookID int,
OldBookTitle nvarchar(50),
NewBookTitle nvarchar(50),
ModifiedDate datetime
);
-- use MERGE statement to perform update on Book2
MERGE Books2 AS b2
USING Books AS b1
ON (b2.BookID = b1.BookID)
WHEN MATCHED
THEN UPDATE
SET b2.BookTitle = b1.BookTitle
OUTPUT
$action,
INSERTED.BookID,
DELETED.BookTitle,
INSERTED.BookTitle,
INSERTED.ModifiedDate
INTO #MergeOutput1;
I am attempting to insert many records using T-SQL's MERGE statement, but my query fails to INSERT when there are duplicate records in the source table. The failure is caused by:
The target table has a Primary Key based on two columns
The source table may contain duplicate records that violate the target table's Primary Key constraint ("Violation of PRIMARY KEY constraint" is thrown)
I'm looking for a way to change my MERGE statement so that it either ignores duplicate records within the source table and/or will try/catch the INSERT statement to catch exceptions that may occur (i.e. all other INSERT statements will run regardless of the few bad eggs that may occur) - or, maybe, there's a better way to go about this problem?
Here's a query example of what I'm trying to explain. The example below will add 100k records to a temp table and then will attempt to insert those records in the target table -
EDIT
In my original post I only included two fields in the example tables which gave way to SO friends to give a DISTINCT solution to avoid duplicates in the MERGE statement. I should have mentioned that in my real-world problem the tables have 15 fields and of those 15, two of the fields are a CLUSTERED PRIMARY KEY. So the DISTINCT keyword doesn't work because I need to SELECT all 15 fields and ignore duplicates based on two of the fields.
I have updated the query below to include one more field, col4. I need to include col4 in the MERGE, but I only need to make sure that ONLY col2 and col3 are unique.
-- Create the source table
CREATE TABLE #tmp (
col2 datetime NOT NULL,
col3 int NOT NULL,
col4 int
)
GO
-- Add a bunch of test data to the source table
-- For testing purposes, allow duplicate records to be added to this table
DECLARE #loopCount int = 100000
DECLARE #loopCounter int = 0
DECLARE #randDateOffset int
DECLARE #col2 datetime
DECLARE #col3 int
DECLARE #col4 int
WHILE (#loopCounter) < #loopCount
BEGIN
SET #randDateOffset = RAND() * 100000
SET #col2 = DATEADD(MI,#randDateOffset,GETDATE())
SET #col3 = RAND() * 1000
SET #col4 = RAND() * 10
INSERT INTO #tmp
(col2,col3,col4)
VALUES
(#col2,#col3,#col4);
SET #loopCounter = #loopCounter + 1
END
-- Insert the source data into the target table
-- How do we make sure we don't attempt to INSERT a duplicate record? Or how can we
-- catch exceptions? Or?
MERGE INTO dbo.tbl1 AS tbl
USING (SELECT * FROM #tmp) AS src
ON (tbl.col2 = src.col2 AND tbl.col3 = src.col3)
WHEN NOT MATCHED THEN
INSERT (col2,col3,col4)
VALUES (src.col2,src.col3,src.col4);
GO
Solved to your new specification. Only inserting the highest value of col4: This time I used a group by to prevent duplicate rows.
MERGE INTO dbo.tbl1 AS tbl
USING (SELECT col2,col3, max(col4) col4 FROM #tmp group by col2,col3) AS src
ON (tbl.col2 = src.col2 AND tbl.col3 = src.col3)
WHEN NOT MATCHED THEN
INSERT (col2,col3,col4)
VALUES (src.col2,src.col3,src.col4);
Given the source has duplicates and you aren't using MERGE fully, I'd use an INSERT.
INSERT dbo.tbl1 (col2,col3)
SELECT DISTINCT col2,col3
FROM #tmp src
WHERE NOT EXISTS (
SELECT *
FROM dbo.tbl1 tbl
WHERE tbl.col2 = src.col2 AND tbl.col3 = src.col3)
The reason MERGE fails is that it isn't checked row by row. All non-matches are found, then it tries to INSERT all these. It doesn't check for rows in the same batch that already match.
This reminds me a bit of the "Halloween problem" where early data changes of an atomic operation affect later data changes: it isn't correct
Instead of GROUP BY you can use an analytic function, allowing you to select a specific record in the set of duplicate records to merge.
MERGE INTO dbo.tbl1 AS tbl
USING (
SELECT *
FROM (
SELECT *, ROW_NUMBER() OVER (PARTITION BY col2, col3 ORDER BY ModifiedDate DESC) AS Rn
FROM #tmp
) t
WHERE Rn = 1 --choose the most recently modified record
) AS src
ON (tbl.col2 = src.col2 AND tbl.col3 = src.col3)