ORACLE insertion from other table using TOAD does not align properly (Showing NULL) - sql

I have created new table name NEW_TABLE like
Create table NEW_TABLE
(
Col_1 VARCHAR(50),
Col_2_ VARCHAR(50),
Col_3_ VARCHAR(50)
)
I am inserting value from OLD_TABLE like this way
INSERT INTO NEW_TABLE (Col_1)
SELECT Col_1
FROM OLD_TABLE_A
WHERE Col_1 IS NOT NULL;
INSERT INTO NEW_TABLE (Col_2)
SELECT Col_1
FROM OLD_TABLE_B
WHERE Col_1 IS NOT NULL;
When I want to see the NEW_TABLE it show the data like this
Col_1 Col_2
----- -----
AA
BB
CC
XX
MM
ZZ
PP
CC
I am getting NULL value at the start of Col_2.
I want this:
Col_1 Col_2
----- -----
AA XX
BB MM
CC ZZ
PP
CC
I have to insert different column in different time separately.while inserting a column I do not want to consider other

insert creates new row. If you want to fill column2 values where column1 is already filled you need to use update or merge. But as mentioned in comments you need to know how to match column2 with column1. You haven't provided any join condition for the data so people are guessing what you need. Please post some sample data from tableA and tableB and how it should look in new_table.
I think you need something like:
step1:
INSERT INTO NEW_TABLE (Col_1)
SELECT Col_1
FROM OLD_TABLE_A
WHERE Col_1 IS NOT NULL;
step2:
merge into NEW_TABLE n
using OLD_TABLE_B b
on (/*HERE PUT JOIN CONDITION*/)
when matched then update set n.col_2_ = b.col_1;
step3:
merge into NEW_TABLE n
using OLD_TABLE_C c
on (/*HERE PUT JOIN CONDITION*/)
when matched then update set n.col_3_ = c.col_1;

Since you stated in a comment that there is no relation between the columns, and that there are the same number of columns in old_table_a and old_table_b this will work. I broke it into steps to make following it easier.
First establish the original table with a WITH clause. Then with another WITH clause, add an ID column which is the row number. Finally SELECT, joining on the ID (uncomment the INSERT line at the top when you are satisfied with the results).
Note the "ID" is meaningless as a true ID and serves only to match rows one for one in each table. If these tables have different numbers of rows you will get unexpected results but it meets your requirements.
SQL> --insert into new_table(col_1, col_2)
SQL> -- Set up the original old table A
SQL> with old_table_a(col_1) as (
select 'AA' from dual union
select 'BB' from dual union
select 'CC' from dual
),
-- Add the id, which is the row_number
ota_rn(id, col_1) as (
select row_number() over (order by col_1) as id, col_1
from old_table_a
),
-- Set up the original old table B
old_table_b(col_1) as (
select 'XX' from dual union
select 'YY' from dual union
select 'ZZ' from dual
),
-- Add the id, which is the row_number
otb_rn(id, col_1) as (
select row_number() over (order by col_1) as id, col_1
from old_table_b
)
-- Now join on the ID (which is really meaningless)
select a.col_1, b.col_1
from ota_rn a
join otb_rn b
on (a.id = b.id);
COL_1 COL_1
---------- ----------
AA XX
BB YY
CC ZZ
SQL>
Update before I even post the answer: I see from subsequent comments as I was about to post that you want to allow for adding additional columns with perhaps differing numbers of rows, etc. That will call for UPDATING, not INSERTING and unless you use the fake row_number ID method I use above really makes no sense in a true relational table. In that case this answer will not meet your needs but I will leave it here in case you want to adapt it for your needs.
I suggest you reconsider your approach to your original problem as this path will take you down a dark hole. You will have unrelated attributes in a table which violates basic database design and makes selecting this data in the future problematic at best (how will you query results? I'm curious how you will use this table). Maybe you should take a step back and reconsider your approach and at least start with some properly normalized tables. What's the real issue your are trying to solve? I bet there is a better way.

The second INSERT should be UPDATE, something like:
UPDATE NEW_TABLE
SET Col_2 = (SELECT Col_2
FROM OLD_TABLE
WHERE Col_1 = <selection value>
)
WHERE Col_1 = <selection value> ;

The basic answer is that you should
insert into NEW_TABLE (Col_1, Col_2)
select OLD_TABLE_A.Col_1, OLD_TABLE_B.Col_2
from OLD_TABLE_A, OLD_TABLE_B
where OLD_TABLE_A.Col_1 is not null
and OLD_TABLE_B.Col_2 is not null;
the problem is that you will then get
Col_1 Col_2
----- -----
AA XX
AA YY
AA ZZ
BB XX
BB YY
BB ZZ
CC XX
CC YY
CC ZZ
now the question you need to answer (that's what Dimitry asked in his comment) is how do you decide that you do not want the AA,YY, AA,ZZ, BB,XX, BB,ZZ, CC,XX and CC,YY ? Once you have an answer to this you can augment the where condition to remove them.

select min (case tab when 'A' then Col_1 end) as Col_1
,min (case tab when 'B' then Col_1 end) as Col_2
from ( SELECT 'A' as tab ,rownum as rn ,Col_1 FROM OLD_TABLE_A
union all SELECT 'B' ,rownum ,Col_1 FROM OLD_TABLE_B
)
group by rn
order by rn
;
OR
select min (Col_1) as Col_1
,min (Col_2) as Col_2
from ( SELECT 'A' as tab,rownum as rn,Col_1 ,null as Col_2 FROM OLD_TABLE_A
union all SELECT 'B' ,rownum ,null ,Col_1 FROM OLD_TABLE_B
)
group by rn
order by rn
;
OR
select a.Col_1
,b.Col_1 as Col_2
from (SELECT rownum as rn,Col_1 FROM OLD_TABLE_A) a
full join (SELECT rownum as rn,Col_1 FROM OLD_TABLE_B) b
on b.rn = a.rn
order by coalesce (a.rn,b.rn)
;
Results
+-------+-------+
| COL_1 | COL_2 |
+-------+-------+
| AA | XX |
+-------+-------+
| BB | MM |
+-------+-------+
| CC | ZZ |
+-------+-------+
| | PP |
+-------+-------+
| | CC |
+-------+-------+

The problem as I see it is:
Fill any holes in Col_2 with one of each of the values from
OLD_TABLE_B, when you've run out of holes then add new rows.
Exactly the same technique should to fill Col_3 from OLD_TABLE_C, as so on. Ideally the initial Col_1 from OLD_TABLE_A should also be able to use the technique although it's a simple insert.
If you end up with an OLD_TABLE_B_PART_2 this should be able to be run against Col_2 later with the same technique.
The solution needs the following parts:
A MERGE statement as you need to do updates otherwise inserts.
To use a single MERGE for each pass to update multiple rows, each row with different values, you need a unique way of identifying the row for the ON clause. With no unique column(s) / primary key you need to use the ROWID pseudo-column. This will be very efficient at targeting the row in the table when we get to the UPDATE clause as ROWID encodes the physical location of the row.
You need all the rows from OLD_TABLE and as many matching rows from NEW_TABLE you can find with holes, so it's a LEFT OUTER JOIN. You could do some sort of UNION then aggregate the rows but this would need an often expensive GROUP BY and you many need to discard an unknown number of surplus rows from NEW_TABLE.
To match a (potentially non-unique) row in the OLD_TABLE with a unique hole in the NEW_TABLE, both will need a temporary matching IDs. The ROWNUM pseudo-column does this and is cheap.
Given the above, the following statement should work:
MERGE INTO NEW_TABLE
USING
( SELECT Col_1, ntid
FROM
( SELECT ROWNUM num, Col_1
FROM OLD_TABLE_B
WHERE Col_1 IS NOT NULL
) ot
LEFT OUTER JOIN
( SELECT ROWNUM num, ROWID ntid
FROM NEW_TABLE
WHERE Col_2 IS NULL
) nt ON nt.num=ot.num
) sel
ON (NEW_TABLE.ROWID=ntid)
WHEN MATCHED THEN
UPDATE SET Col_2=sel.Col_1
WHEN NOT MATCHED THEN
INSERT (Col_2) VALUES (sel.Col_1);
Check the execution plan before using on big data tables. I've seen the optimiser (in Oracle 12c) use a MERGE or HASH join against the target (NEW_TABLE) for the update rather than a plain USER-ROWID access. In this case the workaround I have used was to force NESTED-LOOP joins i.e. add an optimisation hint to the MERGE at the start of the query, so MERGE /*+ use_nl(NEW_TABLE) */. You may also need to check how it does LEFT JOIN depending on your data.

Create table NEW_TABLE
(
Col_1 VARCHAR(5),
Col_2_ VARCHAR(5),
Col_3_ VARCHAR(5)
);
Create table OLD_TABLE
(
Col_1 VARCHAR(5),
Col_2_ VARCHAR(5),
Col_3_ VARCHAR(5)
);
insert into old_table values ('AA','XX', null);
insert into old_table values ('BB','MM', null);
insert into old_table values ('CC','ZZ', null);
insert into old_table values (null,'PP', 'YYY');
insert into old_table values (null,'CC', 'XXX');
select * from old_table;
COL_1 COL_2 COL_3
----- ----- -----
AA XX
BB MM
CC ZZ
PP YYY
CC XXX
alter table new_table add (position number);
.
MERGE INTO new_table D
USING (select rownum position, old_table.* from old_table where col_1 is not null) S
ON (d.position = s.position)
WHEN MATCHED THEN UPDATE SET D.Col_1 = S.Col_1
WHEN NOT MATCHED THEN INSERT (d.position, D.Col_1)
VALUES (s.position, S.Col_1);
MERGE INTO new_table D
USING (select rownum position, old_table.* from old_table where col_2_ is not null) S
ON (d.position = s.position)
WHEN MATCHED THEN UPDATE SET D.Col_2_ = S.Col_2_
WHEN NOT MATCHED THEN INSERT (d.position, D.Col_2_)
VALUES (s.position,S.Col_2_);
MERGE INTO new_table D
USING (select rownum position, old_table.* from old_table where col_3_ is not null) S
ON (d.position = s.position)
WHEN MATCHED THEN UPDATE SET D.Col_3_ = S.Col_3_
WHEN NOT MATCHED THEN INSERT (d.position, D.Col_3_)
VALUES (s.position, S.Col_3_);
select * from new_table order by position;
COL_1 COL_2 COL_3 POSITION
----- ----- ----- ----------
AA XX YYY 1
BB MM XXX 2
CC ZZ 3
PP 4
CC 5
You can drop POSITION column from new_table after the operation if you wish.

run below query
INSERT INTO NEW_TABLE (Col_1, Col_2)
( SELECT Col_1, Col_2
FROM OLD_TABLE_A
WHERE not (Col_1 IS NULL and Col_2 IS NULL))

You can't do that like your way.
TRY THIS
INSERT INTO NEW_TABLE (Col_1,COL_2)
SELECT A.Col_1,B.COL_1
FROM OLD_TABLE_A A FULL OUTER JOIN OLD_TABLE_B B ON 1=1
AND A.Col_1 IS NOT NULL
AND B.Col_1 IS NOT NULL;

Related

Oracle -- Update the exact column referenced in the ON clause

I think this requirement is rarely encountered so I couldn't search for similar questions.
I have a table that needs to update the ID. For example ID 123 in table1 is actually supposed to be 456. I have a separate reference table built that stores the mapping (e.g. old 123 maps to new id 456).
I used the below query but apparently it returned error 38104, columns referenced in the ON clause cannot be updated.
MERGE INTO table1
USING ref_table ON (table1.ID = ref_table.ID_Old)
WHEN MATCHED THEN UPDATE SET table.ID = ref_table.ID_New;
Is there other way to achieve my purpose?
Thanks and much appreciated for your answer!
Use the ROWID pseudocolumn:
SQL Fiddle
Oracle 11g R2 Schema Setup:
CREATE TABLE TABLE1( ID ) AS
SELECT 1 FROM DUAL UNION ALL
SELECT 2 FROM DUAL UNION ALL
SELECT 3 FROM DUAL;
CREATE TABLE REF_TABLE( ID_OLD, ID_NEW ) AS
SELECT 1, 4 FROM DUAL UNION ALL
SELECT 2, 5 FROM DUAL;
MERGE INTO TABLE1 dst
USING ( SELECT t.ROWID AS rid,
r.id_new
FROM TABLE1 t
INNER JOIN REF_TABLE r
ON ( t.id = r.id_old ) ) src
ON ( dst.ROWID = src.RID )
WHEN MATCHED THEN
UPDATE SET id = src.id_new;
Query 1:
SELECT * FROM table1
Results:
| ID |
|----|
| 4 |
| 5 |
| 3 |
You can't update a column used in the ON clause in a MERGE. But if you don't need to make other changes that MERGE allows like WHEN NOT MATCHED or deleting, etc. you can just use a UPDATE to achieve this.
You mentioned this is an ID that needs an update. Here's an example using a scalar subquery. As it is an ID, this presumes UNIQUE ID_OLD values in REF_TABLE. I wasn't sure if Every row needs an update or only a sub-set, so set the update here to only update rows that have a value in REF_TABLE.
CREATE TABLE TABLE1(
ID NUMBER
);
CREATE TABLE REF_TABLE(
ID_OLD NUMBER,
ID_NEW NUMBER
);
INSERT INTO TABLE1 VALUES (1);
INSERT INTO TABLE1 VALUES (2);
INSERT INTO TABLE1 VALUES (100);
INSERT INTO REF_TABLE VALUES (1,10);
INSERT INTO REF_TABLE VALUES (2,20);
Initial State:
SELECT * FROM TABLE1;
ID
1
2
100
Then make the UPDATE
UPDATE TABLE1
SET TABLE1.ID = (SELECT REF_TABLE.ID_NEW
FROM REF_TABLE
WHERE REF_TABLE.ID_OLD = ID)
WHERE TABLE1.ID IN (SELECT REF_TABLE.ID_OLD
FROM REF_TABLE);
2 rows updated.
And check the change:
SELECT * FROM TABLE1;
ID
10
20
100

SQL group data (find data family)

Please help me, I need to find out a SQL solution for grouping data using SQL Server database.
I'm pretty sure that it could be done in one SQL request but I can't see the trick.
Let' see the problem :
I have a two columns table (please see below an example). I just want to add a new column containing a number or a string which indicates the group
BEFORE :
Col1 | Col2
-----+-----
A | B
B | C
D | E
F | G
G | H
I | I
J | U
AFTER TRANSFORMATION :
Col1 | Col2 | Group
-----+------+------
A | B | 1
B | C | 1
D | E | 2
F | G | 3
G | H | 3
I | I | 4
J | U | 5
In other words: A, B, C are in the same group; D and E too; F, G, H in group 3 ....
Do you have any lookup table to get this group mapping?
Or if you just have a logic defined to decide a group, i would recommend to add a UDF which will return group for supplied values.
SELECT Col1,Col2,GetGroupID(Col1,Col2) AS Group
FROM Table
Your UDF will be something like following
CREATE FUNCTION GetGroupID
(
-- Add the parameters for the function here
#Col1 varchar(10),
#Col2 varchar(10)
)
RETURNS int
AS
BEGIN
DECLARE #groupID int
IF (#Col1="A" AND #Co2 = "B") OR (#Col1="B" AND #Co2 = "C")
BEGIN
SET #groupID = 1
END
IF #Col1="D" AND #Co2 = "E"
BEGIN
SET #groupID = 2
END
-- You can write saveral conditions in the same manner.
return #groupID
END
However, in case you have this mapping defined somewhere in another table, let us know the structure of the table and we can then update the query to join with that table instead of using UDF.
Considering the performance of the query, if the amount of data is huge in your table , it is recommended to have these mappings to one fix table and Join that table in query. Using UDF may harm performance if data amount is huge.
There is absolutely no need for a UDF here. Regardless of whether you are looking to update the table with a new column or simply pull out the data with the grouping applied, you will be best off using a set based solution, ie: create and join to a table.
I am assuming here that you don't have messy data, such as a row with Col1 = 'A' and Col2 = 'F'.
If you are able to add new tables permanently you can use the following to create your lookup table:
create table Col1Groups(Col1 nvarchar(10), GroupNum int);
insert into Col1Groups(Col1,GroupNum) values ('A',1),('B',1),('C',1),('D',2),('E',2),('F',3),('G',3),('H',3);
and then join to it:
select t.Col1
,t.Col2
,g.GroupNum
from Table t
inner join Col1Groups g
on t.Col1 = g.Col1
If you can't, you can just create a derived table via a CTE:
with Col1Groups as
(
select Col1
,GroupNum
from (values('A',1),('B',1),('C',1),('D',2),('E',2),('F',3),('G',3),('H',3)) as x(Col1,GroupNum)
)
select t.Col1
,t.Col2
,g.GroupNum
from Table t
inner join Col1Groups g
on t.Col1 = g.Col1
You get the first rows per group with
select col1, col2 from mytable where col1 not in (select col2 from mytable) or col1 = col2;
We can give these rows numbers with
rank() over (order by col1) as grp
Now we must iterate through the rows to find the ones belonging to those first ones, then those belonging to these, etc. A recursive query.
with cte(col1, col2, grp) as
(
select col1, col2, rank() over (order by col1) as grp
from mytable where col1 not in (select col2 from mytable) or col1 = col2
union all
select mytable.col1, mytable.col2, cte.grp
from cte
join mytable on mytable.col1 = cte.col2
where mytable.col1 <> mytable.col2
)
select * from cte
order by grp, col1;
Additional answer for a more flexible approach
Originally you asked for chains A|B -> B|C, F|G -> G|H etc., but in your comment to my other answer you introduced forks like A|B -> B|C, B|D and I've adjusted my answer.
If you want to go one step further and introduce net-like relations such as A|B -> B|C, D|C, we can no longer follow chains forward only (in the example D belongs to the A group, because though A doesn't lead to D directly, it leads to C and D also leads to C. Here is a way to solve this:
Get all letters from the table (no matter whether in col1 or col2). Then for each of them find related letters (again no matter whether in col1 or col2). And for these again find related letters and so on. That will give you complete groups. But duplicates (as D is in the A group, A is in the D group also), which you can get rid of by simply taking the smallest (or greatest) group key per letter. Then join the Groups to the table.
The query:
with cte(col, grp) as
(
select col, rownum as grp from
(select col1 as col from mytable union select col2 from mytable)
union all
select case when mytable.col1 = cte.col then mytable.col2 else mytable.col1 end, cte.grp
from cte
join mytable on cte.col in (mytable.col1, mytable.col2)
where mytable.col1 <> mytable.col2
)
cycle col set is_cycle to 'y' default 'n'
select mytable.col1, mytable.col2, x.grp
from mytable
join (select col, min(grp) as grp from cte group by col) x on x.col = mytable.col1
order by grp, col;

In SQL, how to get MERGE to update relevant row(s) with a single row from a grouped set of returned results

I am using MERGE (Oracle) to do updates to records that match on criteria specified in the ON clause joined on a virtual table create by a subquery. Form of the statement is:
MERGE INTO table1 t1 USING SELECT (t2.f21, MAX(t2.f22), t3.f31, t3.f32
from
table2 t2, table3 t3
where
{... various join/filter criteria ...}
group by t2.f21, t3.f31, t3.f32) MATCHDATA
ON (t1.f11 = MATCHDATA.f21)
where t1.f12 = 'something';
Now the rub: MATCHDATA will return multiple rows because the "... criteria ..." will by nature return multiple groups of matching records. So the 'group by' along with the use of 'MAX()' is not buying me any guarantees; on the contrary, if I added:
where rownum = 1
after MATCHDATA after wrapping the MATCHDATA result in a another SELECT statement that simply repeated the returned field names, I would then be limiting myself to being able to update only the one record in the one group of records that needs updating that has the highest value as determined by MAX(). Instead, I need to have the records in table1 that match on the join field in each MAX() record for their group of records updated. I started on Fri. down the PARTITION BY path and am new to that one, so didn't make much headway. But it looks promising and maybe tomorrow will yield better results. As it is, when I try to use it without for example limiting the returned recordset in MATCHDATA to one record via use of "rownum = 1", I get that familiar "could not return a stable set of records" error that MERGE proponents (like myself) must smile sheepishly at when their colleagues come to them for advice on this "better-than-correlated-subqueries"-evangelized nouveau SQL command as they face this same error.
As you can see, I am treating MERGE as the more successful brother of the correlated subquery. But is this a case where I should be looking back to the lesser of two weevils (i.e., use a correlated subquery instead) to get the job done? Or is the fix to be found in PARTITION BY or another modification to the above?
Thanks to all who take the time to offer their advice, I appreciate it.
I get that familiar "could not return a stable set of records" error
Because the join key you have used in the ON clause is not enough to make the row unique to perform the WHEN MATCHED THEN UPDATE statement.
You must include more keys in the ON clause until the matched rows are unique and thus returning a stable set of records.
Let's see a test case:
Set up
SQL> CREATE TABLE source_table (
2 col1 NUMBER,
3 col2 VARCHAR2(10),
4 col3 VARCHAR2(10)
5 );
Table created.
SQL>
SQL> INSERT INTO source_table (col1, col2, col3) VALUES (1, 'a', 'p');
1 row created.
SQL> INSERT INTO source_table (col1, col2, col3) VALUES (1, 'b', 'q');
1 row created.
SQL> INSERT INTO source_table (col1, col2, col3) VALUES (2, 'c', 'r');
1 row created.
SQL> INSERT INTO source_table (col1, col2, col3) VALUES (3, 'c', 's');
1 row created.
SQL>
SQL> COMMIT;
Commit complete.
SQL>
SQL> CREATE TABLE target_table (
2 col1 NUMBER,
3 col2 VARCHAR2(10),
4 col3 VARCHAR2(10)
5 );
Table created.
SQL>
SQL> INSERT INTO target_table (col1, col2, col3) VALUES (1, 'b', 'p');
1 row created.
SQL> INSERT INTO target_table (col1, col2, col3) VALUES (3, 'd', 'q');
1 row created.
SQL>
SQL> COMMIT;
Commit complete.
SQL>
SQL> SELECT * FROM source_table;
COL1 COL2 COL3
---------- ---------- ----------
1 a p
1 b q
2 c r
3 c s
SQL> SELECT * FROM target_table;
COL1 COL2 COL3
---------- ---------- ----------
1 b p
3 d q
SQL>
Error reproduce
SQL> MERGE INTO target_table trg
2 USING source_table src
3 ON (trg.col1 = src.col1) -- Not Unique
4 WHEN MATCHED THEN UPDATE SET
5 trg.col2 = src.col2,
6 trg.col3 = src.col3
7 WHEN NOT MATCHED THEN INSERT
8 (
9 col1,
10 col2,
11 col3
12 )
13 VALUES
14 (
15 src.col1,
16 src.col2,
17 src.col3
18 );
USING source_table src
*
ERROR at line 2:
ORA-30926: unable to get a stable set of rows in the source tables
SQL>
So, as expected we get the error ORA-30926: unable to get a stable set of rows in the source tables
Let's make the ON clause unique.
SQL> MERGE INTO target_table trg
2 USING source_table src
3 ON (trg.col1 = src.col1
4 AND
5 trg.col2 = src.col2) -- Unique
6 WHEN MATCHED THEN UPDATE SET
7 trg.col3 = src.col3
8 WHEN NOT MATCHED THEN INSERT
9 (
10 col1,
11 col2,
12 col3
13 )
14 VALUES
15 (
16 src.col1,
17 src.col2,
18 src.col3
19 );
4 rows merged.
SQL> SELECT * FROM target_table;
COL1 COL2 COL3
---------- ---------- ----------
1 b q
3 d q
2 c r
3 c s
1 a p
SQL>
Problem solved!
Remember, you cannot update the columns which are referenced in the ON clause.
Let's say we have this table T2:
C1 C2 AMOUNT UF
-- -- ---------- ----------
A X 12 101
A Y 3 102
A Y 12 103
B X 7 104
B Y 9 105
I need to have the records in table1 that match on the join field in
each MAX() record for their group of records updated. I started on
Fri. down the PARTITION BY path and am new to that one, so didnt make
much headway.
This is good path and you can do this using function rank():
select * from (
select t2.*, rank() over (partition by c1 order by amount desc) rn from t2 )
where rn=1
C1 C2 AMOUNT UF RN
-- -- ---------- ---------- --
A X 12 101 1
A Y 12 103 1
B Y 9 105 1
But if your joining field for merge is only 'C1' then this set of records is not stable, because for C1='A'
we have two rows and Oracle looks sheepishly, it does not know which one interests you.
To resolve this you can use row_number()
instead of rank() - if it's all the same. But if this matters you need something more in order clause, for instance:
select * from (
select t2.*, rank() over (partition by c1 order by amount desc, c2) rn from t2 )
where rn = 1
C1 C2 AMOUNT UF RN
-- -- ---------- ---------- --
A X 12 101 1
B Y 9 105 1
This set of rows is stable, because for C1 there are no duplicates and you can use it in your merge.
merge into t1
using (
select * from (
select t2.*, rank() over (partition by c1 order by amount desc, c2) rn from t2 )
where rn=1) md
on (md.c1 = t1.c1)
when matched then update set t1.uf = md.uf
when not matched then insert (t1.c1, t1.uf)
values (md.c1, md.uf)

Detect differences between two versions of the same table

I am looking for a method to detect differences between two versions of the same table.
Let's say I create copies of a live table at two different days:
Day 1:
CREATE TABLE table_1 AS SELECT * FROM table
Day 2:
CREATE TABLE table_2 AS SELECT * FROM table
The method should identify all rows added, deleted or updated between day 1 and day 2;
if possible the method should not use a RDBMS-specific feature;
Note: Exporting the content of the table to text files and comparing text files is fine, but I would like a SQL specific method.
Example:
create table table_1
(
col1 integer,
col2 char(10)
);
create table table_2
(
col1 integer,
col2 char(10)
);
insert into table_1 values ( 1, 'One' );
insert into table_1 values ( 2, 'Two' );
insert into table_1 values ( 3, 'Three' );
insert into table_2 values ( 1, 'One' );
insert into table_2 values ( 2, 'TWO' );
insert into table_2 values ( 4, 'Four' );
Differences between table_1 and table_2:
Added: Row ( 4, 'Four' )
Deleted: Row ( 3, 'Three' )
Updated: Row ( 2, 'Two' ) updated to ( 2, 'TWO' )
If you want differences in both directions. I am assuming you have an id, because you mention "updates" and you need a way to identify the same row. Here is a union all approach:
select t.id,
(case when sum(case when which = 't2' then 1 else 0 end) = 0
then 'InTable1-only'
when sum(case when which = 't1' then 1 else 0 end) = 0
then 'InTable2-only'
when max(col1) <> min(col1) or max(col2) = min(col2) or . . .
then 'Different'
else 'Same'
end)
from ((select 'table1' as which, t1.*
from table_1 t1
) union all
(select 'table2', t2.*
from table_2 t2
)
) t;
This is standard SQL. You can filter out the "same" records if you want to.
This assumes that all the columns have non-NULL values and that rows with a given id appear at most once in each table.
I think I found the answer - one can use this SQL statement to build a list of differences:
Note: "col1, col2" list must include all columns in the table
SELECT
MIN(table_name) as table_name, col1, col2
FROM
(
SELECT
'Table_1' as table_name, col1, col2
FROM Table_1 A
UNION ALL
SELECT
'Table_2' as table_name, col1, col2
FROM Table_2 B
)
tmp
GROUP BY col1, col2
HAVING COUNT(*) = 1
+------------+------+------------+
| table_name | col1 | col2 |
+------------+------+------------+
| Table_2 | 2 | TWO |
| Table_1 | 2 | Two |
| Table_1 | 3 | Three |
| Table_2 | 4 | Four |
+------------+------+------------+
In the example quoted in the question,
Row ( 4, 'Four' ) present in table_2 ; verdict row "Added"
Row ( 3, 'Three' ) present in table_1; verdict row "Deleted"
Row ( 2, 'Two' ) present in table_1 only; Row ( 2, 'TWO' ) present in table_2 only; if col1 is primary key then verdict "Updated"
If you can asasume that the table has a unique primary key then the following SQL statements will at the end have created three views which contain the changed, new and deleted IDs.
In the following "tbl1" is the old version and "tbl2" is the new version (same schema). The primary key in the table is assumed to be named "_id".
The intermediate view "_NewChanged" will contain both the new and changed IDs in the new version of the table and the view "_RemovedChanged" will contain both the removed and changed IDs in the new version.
To generate delta SQL statements is just a matter of programatically looping the respective views to create a delta set and do the delete, update, insert statemens that transforms the old version to the new version.
The solution have three caveats:
It will not pin-point exactly what columns have changed
The schema is assumed to be the same in the old and new version of the table
It assumes that the SQL statments INTERSECT and EXCEPT are available
To keep the code brief no comments have been inserted.
drop view if exists _NewChanged;
drop view if exists _RemovedChanged;
create view _NewChanged as
select * from tbl2
except
select * from tbl1;
create view _RemovedChanged as
select * from tbl1
except
select * from tbl2;
drop view if exists changed;
drop view if exists new;
drop view if exists deleted;
create view changed as
select _id from _NewChanged
intersect
select _id from _RemovedChanged;
create view new as
select _id from _NewChanged
except
select _id from _RemovedChanged;
create view deleted as
select _id from _RemovedChanged
except
select _id from _NewChanged;

Is it possible to use WHERE IN along with LIKE?

If I have to search for some data I can use wildcards and use a simple query -
SELECT * FROM TABLE WHERE COL1 LIKE '%test_string%'
And, if I have to look through many values I can use -
SELECT * FROM TABLE WHERE COL1 IN (Select col from AnotherTable)
But, is it possible to use both together. That is, the query doesn't just perform a WHERE IN but also perform something similar to WHERE LIKE? A query that just doesn't look through a set of values but search using wildcards through a set of values.
If this isn't clear I can give an example. Let me know. Thanks.
Example -
lets consider -
AnotherTable -
id | Col
------|------
1 | one
2 | two
3 | three
Table -
Col | Col1
------|------
aa | one
bb | two
cc | three
dd | four
ee | one_two
bb | three_two
Now, if I can use
SELECT * FROM TABLE WHERE COL1 IN (Select col from AnotherTable)
This gives me -
Col | Col1
------|------
aa | one
bb | two
cc | three
But what if I need -
Col | Col1
------|------
aa | one
bb | two
cc | three
ee | one_two
bb | three_two
I guess this should help you understand what I mean by using WHERE IN and LIKE together
SELECT *
FROM TABLE A
INNER JOIN AnotherTable B on
A.COL1 = B.col
WHERE COL1 LIKE '%test_string%'
Based on the example code provided, give this a try. The final select statement presents the data as you have requested.
create table #AnotherTable
(
ID int IDENTITY(1,1) not null primary key,
Col varchar(100)
);
INSERT INTO #AnotherTable(col) values('one')
INSERT INTO #AnotherTable(col) values('two')
INSERT INTO #AnotherTable(col) values('three')
create table #Table
(
Col varchar(100),
Col1 varchar(100)
);
INSERT INTO #Table(Col,Col1) values('aa','one')
INSERT INTO #Table(Col,Col1) values('bb','two')
INSERT INTO #Table(Col,Col1) values('cc','three')
INSERT INTO #Table(Col,Col1) values('dd','four')
INSERT INTO #Table(Col,Col1) values('ee','one_two')
INSERT INTO #Table(Col,Col1) values('ff','three_two')
SELECT * FROM #AnotherTable
SELECT * FROM #Table
SELECT * FROM #Table WHERE COL1 IN(Select col from #AnotherTable)
SELECT distinct A.*
FROM #Table A
INNER JOIN #AnotherTable B on
A.col1 LIKE '%'+B.Col+'%'
DROP TABLE #Table
DROP TABLE #AnotherTable
Yes. Use the keyword AND:
SELECT * FROM TABLE WHERE COL1 IN (Select col from AnotherTable) AND COL1 LIKE '%test_string%'
But in this case, you are probably better off using JOIN syntax:
SELECT TABLE.* FROM TABLE JOIN AnotherTable on TABLE.COL1 = AnotherTable.col WHERE TABLE.COL1 LIKE '%test_string'
no because each element in the LIKE clause needs the wildcard and there's not a way to do that with the IN clause
The pattern matching operators are:
IN, against a list of values,
LIKE, against a pattern,
REGEXP/RLIKE against a regular expression (which includes both wildcards and alternatives, and is thus closest to "using wildcards through a set of valuws", e.g. (ab)+a|(ba)+b will match all strings aba...ba or bab...ab),
FIND_IN_SET to get the index of a string in a set (which is represented as a comma separated string),
SOUNDS LIKE to compare strings based on how they're pronounced and
MATCH ... AGAINST for full-text matching.
That's about it for string matching, though there are other string functions.
For the example, you could try joining on Table.Col1 LIKE CONCAT(AnotherTable.Col, '%'), though performance will probably be dreadful (assuming it works).
Try a cross join, so that you can compare every row in AnotherTable to every row in Table:
SELECT DISTINCT t.Col, t.Col1
FROM AnotherTable at
CROSS JOIN Table t
WHERE t.col1 LIKE ('%' + at.col + '%')
To make it safe, you'll need to escape wildcards in at.col. Try this answer for that.
If I understand the question correctly you want the rows from "Table" when "Table.Col1" is IN "AnotherTable.Col" and you also want the rows when Col1 IS LIKE '%some_string%'.
If so you want something like:
SELECT
t.*
FROM
[Table] t
LEFT JOIN
[AnotherTable] at ON t.Col1 = at.Col
WHERE (at.Col IS NOT NULL
OR t.Col1 LIKE '%some_string%')
Something like this?
SELECT * FROM TABLE
WHERE
COL1 IN (Select col from AnotherTable)
AND COL1 LIKE '%test_string%'
Are you thinking about something like EXISTS?
SELECT * FROM TABLE t WHERE EXISTS (Select col from AnotherTable t2 where t2.col = t.col like '%test_string%' )