Filtering duplicate records while importing data from source table to target table - sql

This may be a simple query to some of you. But I am not strong in Sql, so expecting some solution for my problem.
I have 2 tables, ProductVenueImport and SupplierVenueImport.
We are dumping all the records from SupplierVenueImport to ProductVenueImport using MERGE clause and a Temp table. Temp will have valid records from SupplerVenuImport and from Temp table we are importing records to ProductVenueImport.
But before importing data to ProductVenueImport from Temp table I need to check for the duplicate records in my target (ProductVenueImport).
For example if I am importing a record with name as 'A', I need to look into ProductVenueImport whether 'A' already existing or not. If it is not existing then only I need to insert 'A' otherwise not.
Could somebody tell me how to do this?
Is using Cursors only the option?
Thanks,
Naresh

Assuming the Temp table itself doesn't have duplicates, you could use MERGE like this:
Insert non-existing products.
Do a NO-OP in case of an existing product.
Use $action in the OUTPUT clause to mark which rows were considered for insertion (and inserted) and which for update (but not really updated).
This is what I mean:
DECLARE #noop int; -- needed for the NO-OP below
MERGE INTO ProductVenueImport AS tgt
USING Temp AS src
ON src.ProductID = tgt.ProdutID
WHEN NOT MATCHED THEN
INSERT ( column1, column2, ...)
VALUES (src.column1, src.column2, ...)
WHEN MATCHED THEN
UPDATE SET #noop = #noop -- the NO-OP instead of update
OUTPUT $action, src.column1, src.column2, ...
INTO anotherTempTable
;

I think this would do this :
INSERT INTO PRODUCTTBL(FEILD1, FIELD2, FIELD3, FIELD4, FIELD5)
SELECT (FIELD1,FIELD2,FIELD3,FIELD4,FIELD5) FROM TEMP WHERE CRITERIAFIELD NOT IN(SELECT DISTINCT CRITERIAFIELD FROM PRODUCTTBL)

This should allow you to check for duplicates in a table
select columnname from tablename
group by columnname
having count(columnname) >1

sorry if I am not getting the question right, can't you use the merge statement on the source table with "When not matched Insert" to insert the new records alone
so in your case it should be like this
merge into ProductVenueImport using temp on (<condition for duplicate>)
when not matched then insert <clause>;
the merge clause will make sure that no duplicate records are inserted into your source table.

Related

Insert into select, target data unaffected

We have a simple query
INSERT INTO table2
SELECT *
FROM table1
WHERE condition;
I can read somewhere that to use INSERT INTO SELECT statement, the following condition must be fulfilled:
The existing records in the target table are unaffected
What does it mean?
INSERT is a SQL operations that add some new rows into your table, with not affect on the others. This is happening instead of UPDATE operations, that cand affect multiple rows from your table if you use a wrong WHERE Clause.

SQL Merge - Persist data

I use SQL Server 2014.
In my procedure, I have a MERGE statement and I have a question about it.
My MERGE statement has simple following structure:
MERGE dbo.T1 AS tgt
USING (SELECT ...) AS src ON ...
WHEN MATCHED THEN
UPDATE ...
WHEN NOT MATCHED THEN
INSERT ...
OUTPUT inserted.MyColumn
INTO #NewTable (MyColumnValue);
Just like how it populates a table for all inserts, I also need it to populate another table for all updates too.
Is is possible, and if yes then would you please let me know how?
No, it's not possible to direct the results to two tables. See this question.
You can make the table wider and output both the inserted and deleted columns on the same row:
MERGE dbo.T1 AS tgt
USING (SELECT ...) AS src ON ...
WHEN MATCHED THEN
UPDATE ...
WHEN NOT MATCHED THEN
INSERT ...
OUTPUT $action, inserted.col1, inserted.col2, deleted.col1, deleted.col2
INTO #NewTable (action, inserted_col1, inserted_col2, deleted_col1, deleted_col2);
Then you can split #NewTable however you want.

Push deleted items into temporary table after MERGE

A simple query on the INTO clause. When i try the below statement, the items get pushed into CustomersBackup2013 whether the table exists or not.
SELECT *
INTO CustomersBackup2013
FROM Customers;
However, when i try using the into clause in a MERGE like
MERGE TargetTable tt
USING SyncTable st on <condition>
.
.
WHEN Not MATCHED BY SOURCE
DELETE
OUTPUT deleted.* INTO #Sometemptable;
I get an error saying invalid object name '#Sometemptable'
Isnt it supposed to create the table if it does not exist? Is there something I am doing wrong.
Is there any way i can modify the clause to push items into #Sometemptable?
No, the table has to exists for the output clause to work.
First create the #temp table and then you can output deleted values in it. Columns in the temp table must match columns from output by ordinal and type.
select into table creates a table if required. it is much like oracle's create table as select. see here select into table
merge only inserts, updates or deletes

How to select data and insert those data using single sql?

I want to select some data using simple sql and insert those data into another table. Both table are same. Data types and column names all are same. Simply those are temporary table of masters table. Using single sql I want to insert those data into another table and in the where condition I check E_ID=? checking part. My another problem is sometime there may be any matching rows in the table. In that time is it may be out sql exception? Another problem is it may be multiple matching rows. That means one E_ID may have multiple rows. As a example in my attachment_master and attachments_temp table has multiple rows for one single ID. How do I solve those problems? I have another problem. My master table data can insert temp table using following code. But I want to change only one column and others are same data. Because I want to change temp table status column.
insert into dates_temp_table SELECT * FROM master_dates_table where e_id=?;
In here all data insert into my dates_temp_table. But I want to add all column data and change only dates_temp_table status column as "Modified". How should I change this code?
You could try this:
insert into table1 ( col1, col2, col3,.... )
SELECT col1, col2, col3, ....
FROM table2 where (you can check any condition here on table1 or table2 or mixed)
For more info have a look here and this similar question
Hope it may help you.
EDit : If I understand your requirement properly then this may be a helpful solution for you:
insert into table1 ( col-1, col-2, col-3,...., col-n, <Your modification col name here> )
SELECT col-1, col-2, col-3,...., col-n, 'modified'
FROM table2 where table1.e_id=<your id value here>
As per your comment in above other answer:
"I send my E_ID. I don't want to matching and get. I send my E_ID and
if that ID available I insert those data into my temp table and change
temp table status as 'Modified' and otherwise don't do anything."
As according to your above statements, If given e_id is there it will copy all the columns values to your table1 and will place a value 'modified' in the 'status' column of your table1
For more info look here
You can use merge statement if I understand your requirement correctly.
Documentation
As I do not have your table structure below is based on assumption, see whether this cater your requirement. I am assuming that e_id is primary key or change as per your table design.
MERGE INTO dates_temp_table trgt
USING (SELECT * FROM master_dates_table WHERE e_id=100) src
ON (trgt.prm_key = src.prm_key)
WHEN NOT MATCHED
THEN
INSERT (trgt.col, trgt.col2, trgt.status)
VALUES (src.col, src.col2, 'Modified');
More information and examples here
insert into tablename( column1, column2, column3,column4 ) SELECT column1,
column2, column3,column4 from anothertablename where tablename.ID=anothertablename.ID
IF multiple values are there then it will return the last result..If not you have narrow your search..

Merge SQL With Condition

I have a Scenario where i need to User Merge SQL statement to Synchronize two Tables. Let's suppose i have two tables Table A and Table B. Schema is same with the exception of one Extra Column in Table A. That Extra column is a flag that tells me which records are ready to be inserted/updated in Table B. Lets say that flag column is IsReady. It will be either true or False.
Can i use Isready=True in Merge Statement or I need a Temp table to move all records from Table A to Temp Table where IsReady=True and then use Merge SQL on TempTable and Table B???
Yes, you can use that column in the merge condition.
merge tableB targetTable
using tableA sourceTable
on sourceTable.IsReady = 1 and [any other condition]
when not matched then
insert ...
when matched and [...] then
update ...
This may help you,
merge into tableB
using tableA
on tableB.IsReady=true
when not matched then
insert (tableB.field1,tableB.field2..)
values (tableA.field1,tableA.field2..);
commit;