I have a table than over time can get bigger and I want to insert some of its rows in another table but I also want to make sure I am not duplicating the rows that I had inserted before.
So here is the type of condition for my insert:
INSERT INTO SecondTable(Col1,Col2)
SELECT Col5,Col6
FROM
FirstTable ft
WHERE ft.RecType = 'ABC'
So if I keep running this it will keep inserting the same rows again and again. How can I tell it only insert if it is not already there?
You can use not exists:
INSERT INTO SecondTable(Col1,Col2)
SELECT Col5,Col6
FROM FirstTable ft
WHERE ft.RecType = 'ABC' AND
NOT EXISTS (SELECT 1 FROM SecondTable t2 WHERE t2.col1 = ft.col5 AND t2.col2 = ft.colt6);
Generate unique constraint on table with proper columns which identifies unicity. This will also help you to preserve integrity of your table. when you try to insert records into the RDBMS will give you an error.
ALTER TABLE SecondTable
ADD UNIQUE (col1, col2, col3);
INSERT INTO SecondTable(Col1,Col2)
SELECT Col5,Col6
FROM FirstTable ft
LEFT JOIN SecondTable st ON st.Col1 = ft.Col1
WHERE st.Col1 IS NULL AND ft.RecType = 'ABC'
Related
Using SQL Server, I Need to return the entire row from whatever table contains 'value' in the Filename column (A column each of the tables contain), but the tables do not have the same number of columns, and each table has unique columns with their own specific data types (The only column Name/Type they have in common is the Filename column that I need to check for 'value').
Ideally, I would be able to do something along the lines of:
SELECT * FROM Table1, Table2, Table3, Table4, Table5
WHERE Filename = 'someValue'
Since all tables share the same column name for the Filename.
I have tried using Union but have issues since the number of columns and datatypes of the tables do not align.
I have also tried every combination of JOIN I could find.
I'm sure this could be accomplished with IF EXISTS, but that would be many, many lines of what seems like unnecessary code. Hoping there is a more elegant solution.
Thanks in advance!
You can try to join the tables together. First create temporary table where you store the input. And then join the tables with this temporary to get all records you want. When there is no record for that filename in the table, then you will get NULL values.
create table Table1 (id int,value int);
insert into Table1 values (1,10)
create table Table2 (id int,value int);
insert into Table2 values (1,20)
create table Table3 (id int,value int);
insert into Table3 values (2,30)
Here is the query itself
create table #tmp (id int)
insert into #tmp
values (1)
select t.id, t1.value, t2.value, t3.value from #tmp as t
left join Table1 as t1
on t.id = t1.id
left join Table2 as t2
on t.id = t2.id
left join Table3 as t3
on t.id = t3.id
And this is what you get
id value value value
1 10 20 NULL
this should work too:
EXEC sp_MSforeachtable
#command1='SELECT * FROM ? where filename = ''someValue''',
#whereand='AND o.id in (select object_id from sys.tables where name in (''Table1'',''Table2'',''Table3''))'
I did a mistake insert in such way
insert into tab1 ( col1 , col2 , col3) select col1 , col2,col3 from tab2
i got 7600 row effects
i want to delete those 7600 rows , if I made below delete would it be safe ?
delete from tab1 where exists select col1 , col2,col3 from tab2
I am asking this to know if i will lost data in tab1 . would this query delete more then 7600 rows ?. if there was common data i would got unique constraint in isert right ? ( yes both have the same PK)
Your DELETE statement isn't even valid syntax, so I don't think that it will do what you want. Even if you fixed the syntax though, you have nothing relating tab1 to tab2.
Further, you haven't provided your table structures, so anything from here on is mostly a guess on my part. Please read: https://stackoverflow.com/help/how-to-ask
Try running:
SELECT COUNT(*)
FROM
tab1
INNER JOIN tab2 ON tab2.col1 = tab1.col1 -- Is this the primary key? If not then you'll need to join on the whole primary key
If you get the same number as you got when you did the INSERT then you can do:
DELETE T1
FROM
Tab1 T1
INNER JOIN Tab2 T2 ON T2.col1 = T1.col1
I would put the DELETE into a transaction and roll it back once just to verify the count, then if it matches, run it and commit.
I need to copy a lot of data from one table to another. If the data already exists, I need to update it, otherwise I need to insert it. The data to be copied is selecting using a WHERE condition. The data has a primary key (a string of up to 12 characters).
If I was just inserting the data, I would do
INSERT INTO T2 SELECT COL1, COL2 FROM T1 WHERE T1.ID ='I'
but I cannot figure out how to do the INSERT / UPDATE. I keep seeing references to upserts and MERGE, but MERGE appears to have issues,and I cannot figure ut how to do the upsert for multiple records.
What is the best solution for this?
If you want to avoid merges (though you should not be afraid of it) you can do something like
update t2
set col1 = t1.col1
,col2 = t1.col2
from t2
join t1
on t2.[joinkey] = t1.[joinkey]
where [where clause]
And after for the ones that you do not have
insert into t2(col1,col2)
select col1,col2 from t1
where not exists (select * from t2 where t1.[joinkey] = t2.[joinkey])
in such way you first update the ones that match and then insert the ones that do not. Also if you want it in one go you can wrap it in a transaction.
It is commonly known as UPSERT operation. Yes you are correct in saying merge has some issues with it so stay away from it.
A simple approach assuming there is a Primary Key column in Both tables called PK_Col would be something like this...
BEGIN TRANSACTION;
-- Update already existing records
UPDATE T2
SET T2.Col1 = T1.Col1
,T2.Col2 = T1.Col2
FROM T2 INNER JOIN T1 ON T2.PK_COl = T1.PK_Col
-- Insert missing records
INSERT INTO T2 (COL1, COL2 )
SELECT COL1, COL2
FROM T1
WHERE T1.ID ='I'
AND NOT EXISTS (SELECT 1
FROM T2
WHERE T2.PK_COl = T1.PK_Col )
COMMIT TRANSACTION;
Wrap the whole UPSERT operation in one transaction.
You can use IF EXISTS something like:
if exists (select * from table with (updlock,serializable) where key = #key)
begin
update table set ...
where key = #key
end
else
begin
insert table (key, ...)
values (#key, ...)
end
Another solution is to check ##ROWCOUNT
UPDATE MyTable SET FieldA=#FieldA WHERE Key=#Key
IF ##ROWCOUNT = 0
INSERT INTO MyTable (FieldA) VALUES (#FieldA)
Hi I am stuck here need your advice. I have a server with multiple DB's.
Now I want to map if the data in one table of 1 db is equal to the data in another db with same table name
can anyone suggest how to do that?? thanks in advancve
select * from db1.table1 t1
full outer join db2.table2 t2 on t1.id = t2.id
where t1.id <> t2.id
or t1.col2 <> t2.col2 or ...
depends on what you need to map.
If you just want to know the differences by primary key, I would try a full join on the PK, so it will tell you records that exist on A but not on B and records that exists on B but not on A. Like this:
create table DB_A(id int)
create table DB_B(id int)
insert into DB_A values (1)
insert into DB_A values (2)
insert into DB_B values (2)
insert into DB_B values (3)
select DB_A.ID as 'Exists on A but not on B', DB_B.id as 'Exists on B but not on A'
from DB_A full join DB_B on DB_A.id=DB_B.id
where DB_A.id is null or DB_B.id is null
if you need more than that like compare the values of all columns, I suggest you to use a data compare tool. It would not be so strait forward to do it using just SQL
I have the following two tables:
Table1
----------
ID Name
1 A
2 B
3 C
Table2
----------
ID Name
1 Z
I need to insert data from Table1 to Table2. I can use the following syntax:
INSERT INTO Table2(Id, Name) SELECT Id, Name FROM Table1
However, in my case, duplicate IDs might exist in Table2 (in my case, it's just "1") and I don't want to copy that again as that would throw an error.
I can write something like this:
IF NOT EXISTS(SELECT 1 FROM Table2 WHERE Id=1)
INSERT INTO Table2 (Id, name) SELECT Id, name FROM Table1
ELSE
INSERT INTO Table2 (Id, name) SELECT Id, name FROM Table1 WHERE Table1.Id<>1
Is there a better way to do this without using IF - ELSE? I want to avoid two INSERT INTO-SELECT statements based on some condition.
Using NOT EXISTS:
INSERT INTO TABLE_2
(id, name)
SELECT t1.id,
t1.name
FROM TABLE_1 t1
WHERE NOT EXISTS(SELECT id
FROM TABLE_2 t2
WHERE t2.id = t1.id)
Using NOT IN:
INSERT INTO TABLE_2
(id, name)
SELECT t1.id,
t1.name
FROM TABLE_1 t1
WHERE t1.id NOT IN (SELECT id
FROM TABLE_2)
Using LEFT JOIN/IS NULL:
INSERT INTO TABLE_2
(id, name)
SELECT t1.id,
t1.name
FROM TABLE_1 t1
LEFT JOIN TABLE_2 t2 ON t2.id = t1.id
WHERE t2.id IS NULL
Of the three options, the LEFT JOIN/IS NULL is less efficient. See this link for more details.
In MySQL you can do this:
INSERT IGNORE INTO Table2(Id, Name) SELECT Id, Name FROM Table1
Does SQL Server have anything similar?
I just had a similar problem, the DISTINCT keyword works magic:
INSERT INTO Table2(Id, Name) SELECT DISTINCT Id, Name FROM Table1
I was facing the same problem recently...
Heres what worked for me in MS SQL server 2017...
The primary key should be set on ID in table 2...
The columns and column properties should be the same of course between both tables. This will work the first time you run the below script. The duplicate ID in table 1, will not insert...
If you run it the second time, you will get a
Violation of PRIMARY KEY constraint error
This is the code:
Insert into Table_2
Select distinct *
from Table_1
where table_1.ID >1
Using ignore Duplicates on the unique index as suggested by IanC here was my solution for a similar issue, creating the index with the Option WITH IGNORE_DUP_KEY
In backward compatible syntax
, WITH IGNORE_DUP_KEY is equivalent to WITH IGNORE_DUP_KEY = ON.
Ref.: index_option
From SQL Server you can set a Unique key index on the table for (Columns that needs to be unique)
A little off topic, but if you want to migrate the data to a new table, and the possible duplicates are in the original table, and the column possibly duplicated is not an id, a GROUP BY will do:
INSERT INTO TABLE_2
(name)
SELECT t1.name
FROM TABLE_1 t1
GROUP BY t1.name
In my case, I had duplicate IDs in the source table, so none of the proposals worked. I don't care about performance, it's just done once.
To solve this I took the records one by one with a cursor to ignore the duplicates.
So here's the code example:
DECLARE #c1 AS VARCHAR(12);
DECLARE #c2 AS VARCHAR(250);
DECLARE #c3 AS VARCHAR(250);
DECLARE MY_cursor CURSOR STATIC FOR
Select
c1,
c2,
c3
from T2
where ....;
OPEN MY_cursor
FETCH NEXT FROM MY_cursor INTO #c1, #c2, #c3
WHILE ##FETCH_STATUS = 0
BEGIN
if (select count(1)
from T1
where a1 = #c1
and a2 = #c2
) = 0
INSERT INTO T1
values (#c1, #c2, #c3)
FETCH NEXT FROM MY_cursor INTO #c1, #c2, #c3
END
CLOSE MY_cursor
DEALLOCATE MY_cursor
I used a MERGE query to fill a table without duplications.
The problem I had was a double key in the tables ( Code , Value ) ,
and the exists query was very slow
The MERGE executed very fast ( more then X100 )
examples for MERGE query
For one table it works perfectly when creating one unique index from multiple field. Then simple "INSERT IGNORE" will ignore duplicates if ALL of 7 fields (in this case) will have SAME values.
Select fields in PMA Structure View and click Unique, new combined index will be created.
A simple DELETE before the INSERT would suffice:
DELETE FROM Table2 WHERE Id = (SELECT Id FROM Table1)
INSERT INTO Table2 (Id, name) SELECT Id, name FROM Table1
Switching Table1 for Table2 depending on which table's Id and name pairing you want to preserve.