I have a temp table with data that needs to be split into 3 other tables. Each of those tables has a primary key that is not shared with each other or with the temp table. Here is a small sampling:
Table 1
RSN AGENT STATUS STATUS DECRIPTION
0 280151 51 Terminated
1 86 57 C/O Comp Agent Dele
2 94 57 C/O Comp Agent Dele
3 108 51 Terminated
Table 2
RSN AGENT CITY
1 10 Englewood
2 123 Jackson
3 35 Eatontown
4 86 Trenton
Table 3
RSN AGT SIGN NO_EMP START_DATE
0 241008 Y 1 2002-10-31 00:00:00.000
1 86 Y 0 2002-10-24 09:51:10.247
2 94 Y 0 2002-10-24 09:51:10.247
3 108 Y 0 2002-10-24 09:51:10.247
I need to check each table to see if the data in the temp table exists and if it does not I want to insert those rows with a RSN# starting with the max number in that table. So if I have 5000 records in the first table and I am adding 5000 new rows they will be numbered 5001 through 10000.
I then need to check to see if any columns have changed for matching rows and update them.
Thanks in advance for your assistance.
Scott
You have to repeat the code bellow for T1, 2 and 3 and update matching and not matching columns.
Insert new value:
Insert Into Table1(col1, col2, ...)
Select t.col1, t.col2
From temp as t
Left Join table1 as t1 On t.matchcol1 = t1.matchcol1 and t.matchcol2 = t1.matchcol2
Where t.col1 is null
replace matchcol1 by a list of matching columns between T and T1
update:
Update t1 set col1 = t.col1, t.col2 = t1.col2, ...
From table1 as t1
Inner Join temp as t On t.matchcol1 = t1.matchcol1 and t.matchcol2 = t1.matchcol2 and ...
Where col1 <> t.col1 or t.col2 <> t1.col2 or ...
This may work as well:
I am not sure you really need to update something or just insert and how you link temp and table1 in order to know if it has been changed.
Insert Into Table1(RSN, AGENT, STATUS, STATUS, DECRIPTION)
Select (Select max(RSN) From table1) + Row_number() over(order by agent)
, AGENT, STATUS, STATUS, DECRIPTION
From (
Select AGENT, STATUS, STATUS, DECRIPTION From TempTable
Except
Select AGENT, STATUS, STATUS, DECRIPTION From Table1
) as t1
Or you can upgrade to SQL Server 2008 and use Merge. It would be a lot easier
I ended up adding 4 new columns to my staging table; a temp rsn#, which is an identity column starting with 1, and an rsn# for each of my 3 destination tables. I created a variable getting the max value from each table and then added that to my temp rsn#.
Related
Source Table
Id, Name, hits
1 A 10
1 A 20
1 A 30
2 A 10
Target Table
Id, Name, hits
1 A NULL
After Merge
Id, Name, hits
1 A 60
2 A 10
is the above possible ? using Merge statement ?
Try below
MERGE targetTable AS [pi]
USING (
SELECT id,name,sum(hits) as hits from sourcetable
GROUP BY id,name
) AS src (id,name,hits) ON src.id= [pi].id and scr.name=pi.name
WHEN MATCHED
THEN UPDATE SET [pi].hits= src.hits
WHEN NOT MATCHED
THEN INSERT values (src.id, src.name,hits)
I have two tables and i need to compare data and update/insert one table records. What iam trying to do is I need to take each record from Table1,
use a split function then for each text in split, compare dataelement field between both these tables. We are syncing data in Table2 to similar to Table1.
Please let me know how this can be done. I am ok using cursor or merge. This is the scenario
DataTable:
dataId dataelement
1 Check
2 System
3 Balances
4 City
5 State
6 Zip
7 Other
Table1:
Id reqId dataelementValues
1 52 Check
2 52 City;State;System
3 52 Other
Table2:
elId dataId dataelement reqId Active
1 6 Zip 52 1
2 1 Check 52 1
3 4 city 52 1
4 5 State 52 1
Outcome Should be similar to after compare in table2
Table2:
elId dataId dataelement reqId Active
1 6 Zip 52 0 (Should be set to inactive as it exists in table2 but not in table1)
2 1 Check 52 1 (NO Updates as it exists in both the tables)
3 4 city 52 1 (NO Updates as it exists in both the tables)
4 5 State 52 1 (NO Updates as it exists in both the tables)
5 2 System 52 1 (Get the dataid for system from datatable and insert in table2 as it exists in table1 but not in table2)
6 7 Other 52 1 (Get the dataid for other from datatable and insert in table2 as it exists in table1 but not in table2)
This is where iam at, not sure how to set inactive on table2.
WHILE Exists(Select * from #Table1)
BEGIN
Select #currentId = Id, #dataValue = dataelementValues FROM #Table1 where rowID=(SELECT top 1 rowID from #Table1 order by rowID asc)
SET #pos = 0
SET #len = 0
WHILE CHARINDEX(';', #dataValue, #pos+1)>0
BEGIN
SET #dataValueValue = SUBSTRING(#dataValue, #pos, CHARINDEX('|', #dataValue, #pos+1) - #pos)
SET #glbaDEId = (Select DataTable.dataId from datatable where dataelement = #dataValue)
IF NOT Exists (Select * from #Table2 Where DataElement=#dataValue)
BEGIN
--Insert into table2
END
SET #pos = CHARINDEX('|', #dataValue, #pos+#len) +1
END
DELETE from #Table1 where rowID=(SELECT top 1 rowID from #Table1 order by rowID asc )
END
You can try using a MERGE statement with a few other tricks.
Merge Guide
-- Create a CTE that will split out the combined column and join to DataTable
-- to get the dataId
;WITH cteTable1Split AS
(
SELECT reqId, dt.* FROM
(
SELECT
[dataelement] = y.i.value('(./text())[1]', 'nvarchar(4000)'),
reqId
FROM
(
-- use xml to split column
-- http://sqlperformance.com/2012/07/t-sql-queries/split-strings
SELECT x = CONVERT(XML, '<i>'
+ REPLACE([dataelementValues], ';', '</i><i>')
+ '</i>').query('.'),
reqId
FROM Table1
) AS a CROSS APPLY x.nodes('i') AS y(i)
) a
JOIN DataTable dt ON dt.[dataelement] = a.[dataelement]
)
-- Merge Table2 with the CTE
MERGE INTO Table2 AS Target
USING cteTable1Split AS Source
ON Target.[dataelement] = Source.[dataelement]
-- If exists in Target (Table2) but not Source (CTE) then UPDATE Active flag
WHEN NOT MATCHED BY Source THEN
UPDATE SET ACTIVE = 0
-- If exists in Source (CTE) but not Target (Table2) then INSERT new record
WHEN NOT MATCHED BY TARGET THEN
INSERT ([dataId], [dataelement], [reqId], [Active])
VALUES (SOURCE.[dataId], SOURCE.[dataelement], SOURCE.[reqId], 1);
SQL Fiddle
You've not mentioned whether you have control over the structure of these tables, so I'm going to go ahead and suggest you redesign Table1 to normalise the dataelementValues column.
That is, instead of this:
Table1:
Id reqId dataelementValues
1 52 Check
2 52 City;State;System
3 52 Other
You should be storing this:
Table1_New:
Id reqId dataelementValues
1 52 Check
2 52 City
2 52 State
2 52 System
3 52 Other
You may also need a new, surrogate, primary key column on the table, using an IDENTITY(1,1) specification.
Storing your data like this is how relational databases are intended to be used/designed. As well as simplifying the problem at hand right now, you might find it removes potential problems in the future as well.
The main challenge here is creating a rowset with the dataelementValues correctly split into separate rows. The accepted answer shows clearly how this can be used as the source for a merge statement to achieve the update and insert operation.
However, there are alternative ways to create the split or normalised rowset.
One way, which I personally prefer for clarity over resorting to xml in this particular case, is a like join. This takes advantage of the fact that you already have the separate rows you need in DataTable, just not with all the columns you need.
Select DT.dataId, DT.dataelement, T1.reqid
From Table1 T1
Inner join DataTable DT
On T1.dataelementValues like '%' + DT.dataelement + '%'
I've not been able to test this right now but should give the rowset you need, because the like operator causes one T1 row to match three times to the corresponding DT rows.
I am using MERGE (Oracle) to do updates to records that match on criteria specified in the ON clause joined on a virtual table create by a subquery. Form of the statement is:
MERGE INTO table1 t1 USING SELECT (t2.f21, MAX(t2.f22), t3.f31, t3.f32
from
table2 t2, table3 t3
where
{... various join/filter criteria ...}
group by t2.f21, t3.f31, t3.f32) MATCHDATA
ON (t1.f11 = MATCHDATA.f21)
where t1.f12 = 'something';
Now the rub: MATCHDATA will return multiple rows because the "... criteria ..." will by nature return multiple groups of matching records. So the 'group by' along with the use of 'MAX()' is not buying me any guarantees; on the contrary, if I added:
where rownum = 1
after MATCHDATA after wrapping the MATCHDATA result in a another SELECT statement that simply repeated the returned field names, I would then be limiting myself to being able to update only the one record in the one group of records that needs updating that has the highest value as determined by MAX(). Instead, I need to have the records in table1 that match on the join field in each MAX() record for their group of records updated. I started on Fri. down the PARTITION BY path and am new to that one, so didn't make much headway. But it looks promising and maybe tomorrow will yield better results. As it is, when I try to use it without for example limiting the returned recordset in MATCHDATA to one record via use of "rownum = 1", I get that familiar "could not return a stable set of records" error that MERGE proponents (like myself) must smile sheepishly at when their colleagues come to them for advice on this "better-than-correlated-subqueries"-evangelized nouveau SQL command as they face this same error.
As you can see, I am treating MERGE as the more successful brother of the correlated subquery. But is this a case where I should be looking back to the lesser of two weevils (i.e., use a correlated subquery instead) to get the job done? Or is the fix to be found in PARTITION BY or another modification to the above?
Thanks to all who take the time to offer their advice, I appreciate it.
I get that familiar "could not return a stable set of records" error
Because the join key you have used in the ON clause is not enough to make the row unique to perform the WHEN MATCHED THEN UPDATE statement.
You must include more keys in the ON clause until the matched rows are unique and thus returning a stable set of records.
Let's see a test case:
Set up
SQL> CREATE TABLE source_table (
2 col1 NUMBER,
3 col2 VARCHAR2(10),
4 col3 VARCHAR2(10)
5 );
Table created.
SQL>
SQL> INSERT INTO source_table (col1, col2, col3) VALUES (1, 'a', 'p');
1 row created.
SQL> INSERT INTO source_table (col1, col2, col3) VALUES (1, 'b', 'q');
1 row created.
SQL> INSERT INTO source_table (col1, col2, col3) VALUES (2, 'c', 'r');
1 row created.
SQL> INSERT INTO source_table (col1, col2, col3) VALUES (3, 'c', 's');
1 row created.
SQL>
SQL> COMMIT;
Commit complete.
SQL>
SQL> CREATE TABLE target_table (
2 col1 NUMBER,
3 col2 VARCHAR2(10),
4 col3 VARCHAR2(10)
5 );
Table created.
SQL>
SQL> INSERT INTO target_table (col1, col2, col3) VALUES (1, 'b', 'p');
1 row created.
SQL> INSERT INTO target_table (col1, col2, col3) VALUES (3, 'd', 'q');
1 row created.
SQL>
SQL> COMMIT;
Commit complete.
SQL>
SQL> SELECT * FROM source_table;
COL1 COL2 COL3
---------- ---------- ----------
1 a p
1 b q
2 c r
3 c s
SQL> SELECT * FROM target_table;
COL1 COL2 COL3
---------- ---------- ----------
1 b p
3 d q
SQL>
Error reproduce
SQL> MERGE INTO target_table trg
2 USING source_table src
3 ON (trg.col1 = src.col1) -- Not Unique
4 WHEN MATCHED THEN UPDATE SET
5 trg.col2 = src.col2,
6 trg.col3 = src.col3
7 WHEN NOT MATCHED THEN INSERT
8 (
9 col1,
10 col2,
11 col3
12 )
13 VALUES
14 (
15 src.col1,
16 src.col2,
17 src.col3
18 );
USING source_table src
*
ERROR at line 2:
ORA-30926: unable to get a stable set of rows in the source tables
SQL>
So, as expected we get the error ORA-30926: unable to get a stable set of rows in the source tables
Let's make the ON clause unique.
SQL> MERGE INTO target_table trg
2 USING source_table src
3 ON (trg.col1 = src.col1
4 AND
5 trg.col2 = src.col2) -- Unique
6 WHEN MATCHED THEN UPDATE SET
7 trg.col3 = src.col3
8 WHEN NOT MATCHED THEN INSERT
9 (
10 col1,
11 col2,
12 col3
13 )
14 VALUES
15 (
16 src.col1,
17 src.col2,
18 src.col3
19 );
4 rows merged.
SQL> SELECT * FROM target_table;
COL1 COL2 COL3
---------- ---------- ----------
1 b q
3 d q
2 c r
3 c s
1 a p
SQL>
Problem solved!
Remember, you cannot update the columns which are referenced in the ON clause.
Let's say we have this table T2:
C1 C2 AMOUNT UF
-- -- ---------- ----------
A X 12 101
A Y 3 102
A Y 12 103
B X 7 104
B Y 9 105
I need to have the records in table1 that match on the join field in
each MAX() record for their group of records updated. I started on
Fri. down the PARTITION BY path and am new to that one, so didnt make
much headway.
This is good path and you can do this using function rank():
select * from (
select t2.*, rank() over (partition by c1 order by amount desc) rn from t2 )
where rn=1
C1 C2 AMOUNT UF RN
-- -- ---------- ---------- --
A X 12 101 1
A Y 12 103 1
B Y 9 105 1
But if your joining field for merge is only 'C1' then this set of records is not stable, because for C1='A'
we have two rows and Oracle looks sheepishly, it does not know which one interests you.
To resolve this you can use row_number()
instead of rank() - if it's all the same. But if this matters you need something more in order clause, for instance:
select * from (
select t2.*, rank() over (partition by c1 order by amount desc, c2) rn from t2 )
where rn = 1
C1 C2 AMOUNT UF RN
-- -- ---------- ---------- --
A X 12 101 1
B Y 9 105 1
This set of rows is stable, because for C1 there are no duplicates and you can use it in your merge.
merge into t1
using (
select * from (
select t2.*, rank() over (partition by c1 order by amount desc, c2) rn from t2 )
where rn=1) md
on (md.c1 = t1.c1)
when matched then update set t1.uf = md.uf
when not matched then insert (t1.c1, t1.uf)
values (md.c1, md.uf)
Say I have a table that looks something like this
COL1
1
1
1
2
2
3
3
3
3
4
5
5
6
6
7
7
With some other columns that are unimportant for this question. If I want to return all but the first two values from 4 how would I do this with derby?
Here is the expected output to clear up what I'm wanting
COL1
3
3
3
3
4
5
5
6
6
Thanks for the help, I'm not the best with SQL but I'm trying :)
try this...
SELECT t.*
FROM mytab t
INNER JOIN (SELECT MIN(COL1) AS VAL2
FROM mytab
WHERE COL1 NOT IN (SELECT MIN(COL1) FROM mytab)) x
ON t.COL1 > x.VAL2
working example at
http://www.datagloop.com/?USERNAME=DATAGLOOP/SO_DERBY&ACTION=LOGIN
Here is a solution that uses a temporary table that allows more flexibility, readability and also allows you to tune the parameters better.
The ROWID part of the queries is your position reference.
CREATE TEMP TABLE mycol_order
(COL1 INTEGER NOT NULL,
TOTAL_COLS INTEGER NOT NULL,
PRIMARY KEY (COL1));
INSERT INTO mycol_order
(COL1,TOTAL_COLS)
SELECT DISTINCT t1.COL1,
t2.total
FROM mytab t1,
(SELECT COUNT(DISTINCT COL1) AS total FROM mytab) t2
ORDER BY 1;
SELECT t.*
FROM mytab t
INNER JOIN mycol_order co
ON co.col1 = t.col1
AND co.ROWID > 2
AND co.ROWID < co.total_cols;
Also updated the working example at
working example at http://www.datagloop.com/?USERNAME=DATAGLOOP/SO_DERBY&ACTION=LOGIN
I have 2 tables that are not related at all and I need to put them together - one column per table. When I try the cartesian join, I end up with every combination:
SELECT Field1, Field2 FROM Table1, Table2
Result:
Table1.Field1 Table2.Field2
---------------------------
1 1
1 2
1 3
2 1
2 2
2 3
3 1
3 2
3 3
I need it to return side-by-side:
Table1.Field1 Table2.Field2
---------------------------
1 1
2 2
3 3
is this possible? Thanks in advance
EDIT
Table1.Table1IDs
----------------
1
2
3
4
5
Table2.Table2IDs
----------------
6
7
8
9
10
Desired output (into a temp table/select statement)
Table1.Table1IDs Table2.Table2IDs
------------------------------------
1 6
2 7
3 8
4 9
5 10
So that I can then do my insert into the actual table I need to do an insert:
INSERT INTO dbo.MTMObjects
SELECT Table1IDs, Table2IDs
FROM [temp table or solution]
ANSWER
Bluefeet gave me the idea to use temp tables with an identity column that i can then use to join. His is 'safer' because you aren't relying on SQL's good humor to sort both recordsets the same, but this might help the next guy:
DECLARE #tmp_Table1 TABLE(ID int IDENTITY(1,1) NOT NULL, TableID1 int NOT NULL)
DECLARE #tmp_Table2 TABLE(ID int IDENTITY(1,1) NOT NULL, TableID2 int NOT NULL)
INSERT INTO #tmp_Table1
OUTPUT INSERTED.Field1
SELECT * FROM Table1
INSERT INTO #tmp_Table2
OUTPUT INSERTED.Field2
SELECT * FROM Table2
OUTPUT
SELECT tmp1.Field1, tmp2.Field2
FROM #tmp_Table1 tmp1 INNER JOIN #tmp_Table2 tmp2 ON tmp2.ID = tmp1.ID
CHEERS!
You can try something like this using row_number(). This will force a relationship between the two tables based on the row_number:
select t1.col1, t2.col2
from
(
select col1, row_number() over(order by col1) rn
from table1
) t1
inner join
(
select col2, row_number() over(order by col2) rn
from table2
) t2
on t1.rn = t2.rn
See SQL Fiddle with Demo
if i understood ...the only you need is to add a filter in where clause
SELECT Field1, Field2 FROM Table1, Table2 WHERE Table1.Field1=Table2.Field2
You should probably change your final solution to use an outer join instead of inner join, in case the tables don't have exactly the same number of rows.