What am I doing wrong with this MERGE / INSERT query? - sql

I have a database table that I want to update with SQL. Basically, it carries a set of description information for parts of a timetable booklet, but that isn't important. Some of the data is entered already via an application, but it is time consuming and blocks of it are "boiler plate" that want to be updated each week. In some cases, I may have missed entering the data via the application, and so also want to create data automatically if it doesn't exist.
That leads me to use a MERGE query, as follows:
MERGE INTO TTP_LINE_DESCRIPTION o
USING
(SELECT DISTINCT lv.lv_nv_id TLDE_NV_ID,
lv.lv_id TLDE_LV_ID,
dir.dir_id TLDE_DIR_ID,
8 TLDE_MED_FLAG,
1 TLDE_TYPE,
0 TLDE_SORT_NO,
'Timetable valid from ' || to_char(lv.lv_valid_from,'DD/MM/YYYY') || ' until ' || nvl2(lv.lv_valid_until,to_char(lv.lv_valid_until,'DD/MM/YYYY'),'further notice') TLDE_TEXT,
0 TLDE_ALIGNMENT,
null TLDE_FONT_SIZE,
null TLDE_FONT_STYLE
FROM LINE_VERSION lv
JOIN line_point_sequence lps ON (lv.lv_id = lps.lps_lv_id)
JOIN direction dir ON (dir.dir_id = lps.lps_dir_id)
where lv.lv_nv_id=3799 and lv.lv_id=10455244) n
ON (o.TLDE_NV_ID=n.TLDE_NV_ID
and o.TLDE_LV_ID=n.TLDE_LV_ID
and o.TLDE_DIR_ID=n.TLDE_DIR_ID
and o.TLDE_TYPE=n.TLDE_TYPE
and o.TLDE_SORT_NO=n.TLDE_SORT_NO)
WHEN MATCHED THEN
UPDATE SET o.TLDE_TEXT=n.TLDE_TEXT,
o.TLDE_ALIGNMENT=n.TLDE_ALIGNMENT,
o.TLDE_FONT_SIZE=n.TLDE_FONT_SIZE,
o.TLDE_FONT_STYLE=n.TLDE_FONT_STYLE
WHEN NOT MATCHED THEN
INSERT (o.tlde_id, o.tlde_nv_id, o.tlde_lv_id, o.tlde_dir_id,
o.tlde_med_flag, o.tlde_type, o.tlde_sort_no, o.tlde_text,
o.tlde_alignment, o.tlde_font_size, o.tlde_font_style,
o.updated_by, o.updated_on, o.updated_prog)
VALUES ((select max(tld.tlde_id)+1 from TTP_LINE_DESCRIPTION tld),
n.tlde_nv_id, n.tlde_lv_id, n.tlde_dir_id, n.tlde_med_flag,
n.tlde_type, n.tlde_sort_no, n.tlde_text, n.tlde_alignment,
n.tlde_font_size, n.tlde_font_style, 'STUARTR',
SYSDATE, 'PL/SQL Developer');
There is one thing to note here, which is the WHERE clause in the SELECT DISTINCT. The lv.lv_id=10455244 is a reference that I know will force the select to return only a single pair of rows in that SELECT, so that I can limit my testing. For this purpose, 10455244 is a valid value that is NOT currently in the TTP_LINE_DESCRIPTION table.
When I use a value that is in the table, the WHEN MATCHED code executes correctly, and updates a pair of rows in 0.016s.
Running the SELECT statement on its own using the value I have shown above returns the two rows that need adding in 0.109s.
Getting the max id and adding one to it (this is the primary key) as per the first item in the VALUES line at the end takes 0s.
Finally, if I write an INSERT INTO and explicitly write all of the values that I want to write for one of the rows, I can do the INSERT of one row in 0.016s.
But put it all together and ... nothing. The execution just sits there, executing, and doesn't appear to end. Or I get nervous waiting to see if it will end. I've left it a reasonable time, and nothing seems to go in.
So what's going on, and why won't it do what I think that it should?

You are doing this:
create table t (id, nv_id, val) as (select 1, 101, 'A' from dual);
merge into t o
using (
select 102 nv_id, 'P' val from dual union all
select 103 nv_id, 'Q' val from dual ) n
on (o.nv_id = n.nv_id)
when matched then update set val = n.val
when not matched then
insert (o.id, o.nv_id, o.val)
values ((select max(id) + 1 from t), n.nv_id, n.val);
In my case it worked, but id for inserted rows was the same: 2. When I did rollback and added primary key constraint:
alter table t add constraint t_pk primary key(id);
merge caused error - unique constraint violated. I suspect that this is something connected to your tlde_id, maybe not constraint but something else. Generating values this way is a big no-no. If your Oracle version is 12c or higher you can change this column to auto generated
generated by default on null as identity
or in older versions use sequence and trigger (find max id, add 1 and set it as starting value for sequence)
create sequence seq_t_id start with 2;
create or replace trigger t_on_insert
before insert on t for each row
begin
select seq_t_id.nextval into :new.id from dual;
end;
Then modify your merge, either by removing id from insert clause or just insert null and trigger will set proper values:
merge into t o
using (
select 102 nv_id, 'P' val from dual union all
select 103 nv_id, 'Q' val from dual ) n
on (o.nv_id = n.nv_id)
when matched then update set val = n.val
when not matched then
insert (o.nv_id, o.val)
values (n.nv_id, n.val);
Even if this does not solve your problem you should not generate ids as max() + 1, this causes problems with concurent sessions, commits, multi inserts etc.

Related

How can I auto increment the primary key in SQL Server 2016 merge insert without sequences?

I am writing a query to import data from one table to a new table. I need to insert records that do not exist in the new table, and update records that do exist. I am trying to use a MERGE "upsert" method.
I have some unique problems due to the client's database and application structure. The table I am inserting into has a Unique ID field that increments by 1 for each new row, but the table does not do the auto incrementating; the insert statement needs to pull the highest ID in the target table and add 1 for the new record.
From my research, I can't figure out how to do that with MERGE. I do not database permissions to create a sequence. I have tried a lot of things, but currently my query looks like:
MERGE
dbo.targetTable as target
USING
dbo.sourceTable AS source
ON
target.account_no = source.account_ID
WHEN NOT MATCHED THEN
INSERT (
ID,
FIELD1,
FIELD2,
FIELD3
) VALUES (
(SELECT MAX(ID) + 1 FROM dbo.targetTable),
'field1',
'field2',
'field3'
)
The problem I am then running into with this code is that it appears to only run the select statement for the new ID once. That is, if the highest ID in the target table was 10, it would insert every new record with ID 11. That won't work as I'm getting a
Violation of PRIMARY KEY constraint. Cannot insert duplicate key in object error. I've been doing a ton of googling and trying different things but haven't been able to figure this one out. Any help is appreciated, thank you.
EDIT: For clarification, the unique ID column does not auto-populate. If I do not insert a value for the ID column, I get
Cannot insert the value NULL into column 'ID', table 'dbo.targetTable'; column does not allow nulls. UPDATE fails.
And again, as I mentioned originally I do not have permissions to create sequences. It just throws an error and says I do not have permission to do that.
I agree that changing the ID column to auto-increment automatically would be perfect, but I do not have the capability to modify the table like that either.
If you don't need the IDs to be consecutive, you can add the last available ID to a ROW_NUMBER() to generate new, non-repeated IDs.
BEGIN TRANSACTION
DECLARE #NextAvailableID INT = (SELECT ISNULL(MAX(ID), 0) FROM dbo.targetTable WITH (TABLOCKX))
;WITH SourceWithNewIDs AS
(
SELECT
S.*,
NewID = #NextAvailableID + ROW_NUMBER() OVER (ORDER BY S.account_ID)
FROM
dbo.sourceTable AS S
)
MERGE
dbo.targetTable as target
USING
SourceWithNewIDs AS source
ON
target.account_no = source.account_ID
WHEN NOT MATCHED THEN
INSERT (
ID,
FIELD1,
FIELD2,
FIELD3
) VALUES (
NewID,
'field1',
'field2',
'field3'
)
COMMIT
Keep in mind that this example is missing the proper error handling with rollback and the lock used to retrieve the max ID will block all other operations until commited or rollbacked.
If you need the new rows to have consecutive IDs then you can use this same approach with a regular INSERT (with WHERE NOT EXISTS...) instead of a MERGE (will have to write the UPDATE separately).
This is just a different way without using a Merge. Permissions aren't required for temp tables, so I would use one to hold the account numbers that need to be inserted, with an identity field to help with traversal. A while loop can traverse the identity, inserting the values with respect to the source table's account_no- into the target table. Since the insert is done in a loop, the MAX function should grab the target table's MAX(account_no) correctly on each loop.
DECLARE #tempTable TABLE (pkindex int IDENTITY(1,1) PRIMARY KEY, account_no int)
DECLARE #current int = 1
,#endcount int = 0
--account_no's that should be inserted
INSERT INTO #tempTable(account_no)
SELECT account_no
FROM sourceTable
WHERE account_no NOT IN (SELECT account_no FROM targetTable)
SET #endcount = (SELECT COUNT(*) FROM #tempTable)
--looping condition, should select the MAX(ID) with each subsequent loop
WHILE (#endcount > 0) AND (#current <= #endcount)
BEGIN
INSERT INTO dbo.targetTable(ID, FIELD1, FIELD2, FIELD3)
SELECT (SELECT MAX(T2.ID) + 1 FROM dbo.targetTable T2) AS MAXID
,S.field1
,S.field2
,S.field3
FROM #tempTable T INNER JOIN sourceTable S ON T.account_no = S.account_no
WHERE T.pkindex = #current --traversing temp table by its identity
SET #current += 1
END

Oracle Merge statement not inserting data

I have came up with below sql statement on oracle 11g for merge the data.
MERGE INTO myTable tgt
USING ( SELECT myTable.ROWID AS rid
FROM myTable
WHERE myTable.myRef = 'uuuu' or
myTable.name = 'sam'
--union
--select rowId,null from dual
) src
ON (tgt.rowid = src.rid)
WHEN MATCHED THEN
update set tgt.myRef = 'uuuu',tgt.name='name_worked'
when not matched then
insert (
tgt.myRef,tgt.name) values ('RRRR','HHH');
I need to insert data except ID column here which will manage from the trigger for primary key insertion. due to error ORA-38104: Columns referenced in the ON Clause cannot be updated i used RowId approach here.
now my issue is merge statement works fine when it comes to update but fail in inserting data in this situation.
I went through this post and added the union statement. but still it fails when goes to insert duplicate record due to the constraint in my table instead of running it smoothly.
Can anyone please help me out here? Appreciate it a lot. thank you in advanced.
==========Edited===========
Please find the complete code sample and the error messages below.
MERGE INTO myTable tgt
USING ( SELECT myTable.ROWID AS rid
FROM myTable
WHERE myTable.myRef = 'RRRR' or
myTable.mytablename = 'sam'
union
select rowId from dual
) src
ON (tgt.rowid = src.rid)
WHEN MATCHED THEN
update set tgt.myRef = 'uuuu',tgt.mytablename='myt_name', tgt.name='nameA'
when not matched then
insert (
tgt.mytableid,tgt.mytablename,tgt.name,tgt.myRef) values (11,'RRRR','HHH','myref1');
and my table is -
CREATE TABLE "sa"."MYTABLE"
(
"MYTABLEID" NUMBER NOT NULL ENABLE,
"MYTABLENAME" VARCHAR2(20 BYTE) NOT NULL ENABLE,
"NAME" VARCHAR2(20 BYTE),
"MYREF" VARCHAR2(20 BYTE),
CONSTRAINT "MYTABLE_PK" PRIMARY KEY ("MYTABLEID", "MYTABLENAME")
)
if i run this first time it will insert the record as expected.
when i run it the second time my expectation is it should match the myRef = 'RRRR' and do the update. i intentionally put 'or' between condition because if i find any value exist in the table it should go and update the existing record.
but instead of doing that update it will throw this error because merge statement try to insert again.
SQL Error: ORA-00001: unique constraint (sa.MYTABLE_PK) violated
00001. 00000 - "unique constraint (%s.%s) violated"
my requirement is when it run the first time it should insert and when i run the same again it should pick that record and update it. Please let me know what to adjust in my merge statement in order to work as expected here. Thanks in advance.
The below MERGE query is what you are looking for.
MERGE INTO myTable tgt
USING ( select x.rid from
(SELECT myTable.ROWID AS rid
FROM myTable
WHERE myTable.myRef IN ('myref1','uuuu')) x
right outer join dual
on x.rowid <> dual.rowid
) src
ON (tgt.rowid = src.rid)
WHEN MATCHED THEN
update set tgt.myRef = 'uuuu',tgt.mytablename='myt_name', tgt.name='nameA'
when not matched then
insert (tgt.mytableid,tgt.mytablename,tgt.name,tgt.myRef) values (mytable_seq.nextval,'RRRR','HHH','myref1');
There have been 3 changes made from the query that you provided.
Inside the 'using' subquery 'RRRR' was being checked in myTable.myRef column, but while inserting 'myref1' was being inserted in the same. Hence this has been changed in the using subquery to check 'myref1'.
'uuuu' has been introduced in the same check.
DUAL tas been introduced in right outer join.
The query will process as below -
1.During first run, there would be no row in mytable. Hence the right outer join with dual will prevail. This will enable the insert to happen.
During 2nd run, there will be a row with myRef column of 'myref1'. Its Rowid will be picked up and the Update will happen. After update, myRef column will be updated to 'uuuu'.
3.During all subsequent runs, 1 row will always be returned in the inside using subquery, this time because of the column value of 'uuuu'. This will enable the update to happen , which will again update the columns with the same existing values.
Thus in effect, 1st time INSERT will happen and in all subsequent times UPDATE will take place.
It appears you want something like the following:
MERGE INTO MYTABLE t
USING (SELECT newID, newTable_name from DUAL) d
ON (t.MYTABLEID = d.newID AND
t.MYTABLENAME = d.newTable_name)
WHEN MATCHED THEN
UPDATE SET NAME = newName,
MYREF = newRef
WHEN NOT MATCHED THEN
INSERT (MYTABLEID, MYTABLENAME, NAME, MYREF)
VALUES (newID, newTable_name, newName, newRef)
where the variables newID, newTable_name, newName, and newRef are populated from whatever source of data you're using.
Also - do you really want the primary key to be (MYTABLEID, MYTABLENAME)? Can you actually have multiple rows with the same MYTABLEID value? And do you really want to allow multiple rows having the same MYTABLENAME value? My thought is that the primary key should be MYTABLEID, with a UNIQUE constraint on MYTABLENAME. If this is the case then MYTABLEID is redundant, and MYTABLENAME could be used as the primary key. ????
Best of luck.
I was able to make this work on below query.
MERGE INTO myTable tgt
USING ( SELECT (SELECT myTable.ROWID AS rid
FROM myTable
WHERE myTable.myRef = '2' or
myTable.mytablename = 'sam'
) as rid from dual) src
ON (tgt.rowid = src.rid)
WHEN MATCHED THEN
update set tgt.myRef = 'uuuu',tgt.mytablename='test1', tgt.name='nameA'
when not matched then
insert (
tgt.mytableid,tgt.mytablename,tgt.name,tgt.myRef) values (11,'RRRR1','1','2');
first time it will insert the row and for subsequent runs it will update the row.

MERGE - to replace ON Duplicate with sql server

I've predominantly used mySQL so moving over to azure and sql server I realise that on duplicate does not work.
I'm trying to do this:
INSERT INTO records (jid, pair, interval, entry) VALUES (1, 'alpha', 3, 'unlimited') ON DUPLICATE KEY UPDATE entry = "limited";
But of course on duplicate key isn't allowed here. So MERGE is the right form.
I've looked at:
https://technet.microsoft.com/en-gb/library/bb522522(v=sql.105).aspx
But honestly the example is a bit excessive and eye watering. Could someone dumb it down for me to fit my example so I can understand it better?
In order to do the merge you need some form of source table/table var for the merge statement. Then you can do the merging. So something along the lines of this maybe (note: not completely syntax checked, apologies in advance):
WITH src AS (
-- This should be your source
SELECT 1 AS Id, 2 AS Val
)
-- The above is not neccessary if you have a source table
MERGE Target -- the detination table, so in your case records
USING src -- as defined above
ON (Target.Id = src.Id) -- how do we join the tables
WHEN NOT MATCHED BY TARGET
-- if we dont match, what do to the destination table. This case insert it.
THEN INSERT(Id, Val) VALUES(src.Id, src.Val)
WHEN MATCHED
-- what do we do if we match. This case update Val
THEN UPDATE SET Target.Val = src.Val;
Don't forget to read the proper syntax page: https://msdn.microsoft.com/en-us/library/bb510625.aspx
I think this translates to your example (tm):
WITH src AS (
-- This should be your source
SELECT 1 AS jid, 'alpha' AS pair, 3 as 'interval'
)
MERGE records -- the detination table, so in your case records
USING src -- as defined above
ON (records.Id = src.Id) -- how do we join the tables
WHEN NOT MATCHED BY TARGET
-- if we dont match, what do to the destination table. This case insert it.
THEN INSERT(jid, pair, interval, entry) VALUES(src.jid, src.pair, src.interval, 'unlimited')
WHEN MATCHED
-- what do we do if we match. This case update Val
THEN UPDATE SET records.entry = 'limited';

SQL With... Update

Is there any way to do some kind of "WITH...UPDATE" action on SQL?
For example:
WITH changes AS
(...)
UPDATE table
SET id = changes.target
FROM table INNER JOIN changes ON table.id = changes.base
WHERE table.id = changes.base;
Some context information: What I'm trying to do is to generate a base/target list from a table and then use it to change values in another table (changing values equal to base into target)
Thanks!
You can use merge, with the equivalent of your with clause as the using clause, but because you're updating the field you're joining on you need to do a bit more work; this:
merge into t42
using (
select 1 as base, 10 as target
from dual
) changes
on (t42.id = changes.base)
when matched then
update set t42.id = changes.target;
.. gives error:
ORA-38104: Columns referenced in the ON Clause cannot be updated: "T42"."ID"
Of course, it depends a bit what you're doing in the CTE, but as long as you can join to your table withint that to get the rowid you can use that for the on clause instead:
merge into t42
using (
select t42.id as base, t42.id * 10 as target, t42.rowid as r_id
from t42
where id in (1, 2)
) changes
on (t42.rowid = changes.r_id)
when matched then
update set t42.id = changes.target;
If I create my t42 table with an id column and have rows with values 1, 2 and 3, this will update the first two to 10 and 20, and leave the third one alone.
SQL Fiddle demo.
It doesn't have to be rowid, it can be a real column if it uniquely identifies the row; normally that would be an id, which would normally never change (as a primary key), you just can't use it and update it at the same time.

SQL pivoted table is read-only and cells can't be edited?

If I create a VIEW using this pivot table query, it isn't editable. The cells are read-only and give me the SQL2005 error: "No row was updated. The data in row 2 was not committed. Update or insert of view or function 'VIEWNAME' failed because it contains a derived or constant field."
Any ideas on how this could be solved OR is a pivot like this just never editable?
SELECT n_id,
MAX(CASE field WHEN 'fId' THEN c_metadata_value ELSE ' ' END) AS fId,
MAX(CASE field WHEN 'sID' THEN c_metadata_value ELSE ' ' END) AS sID,
MAX(CASE field WHEN 'NUMBER' THEN c_metadata_value ELSE ' ' END) AS NUMBER
FROM metadata
GROUP BY n_id
Assuming you have a unique constraint on n_id, field which means that at most one row can match you can (in theory at least) use an INSTEAD OF trigger.
This would be easier with MERGE (but that is not available until SQL Server 2008) as you need to cover UPDATES of existing data, INSERTS (Where a NULL value is set to a NON NULL one) and DELETES where a NON NULL value is set to NULL.
One thing you would need to consider here is how to cope with UPDATES that set all of the columns in a row to NULL I did this during testing the code below and was quite confused for a minute or two until I realised that this had deleted all the rows in the base table for an n_id (which meant the operation was not reversible via another UPDATE statement). This issue could be avoided by having the VIEW definition OUTER JOIN onto what ever table n_id is the PK of.
An example of the type of thing is below. You would also need to consider potential race conditions in the INSERT/DELETE code indicated and whether you need some additional locking hints in there.
CREATE TRIGGER trig
ON pivoted
INSTEAD OF UPDATE
AS
BEGIN
SET nocount ON;
DECLARE #unpivoted TABLE (
n_id INT,
field VARCHAR(10),
c_metadata_value VARCHAR(10))
INSERT INTO #unpivoted
SELECT *
FROM inserted UNPIVOT (data FOR col IN (fid, sid, NUMBER) ) AS unpvt
WHERE data IS NOT NULL
UPDATE m
SET m.c_metadata_value = u.c_metadata_value
FROM metadata m
JOIN #unpivoted u
ON u.n_id = m.n_id
AND u.c_metadata_value = m.field;
/*You need to consider race conditions below*/
DELETE FROM metadata
WHERE NOT EXISTS(SELECT *
FROM #unpivoted u
WHERE metadata.n_id = u.n_id
AND u.field = metadata.field)
INSERT INTO metadata
SELECT u.n_id,
u.field,
u.c_metadata_value
FROM #unpivoted u
WHERE NOT EXISTS (SELECT *
FROM metadata m
WHERE m.n_id = u.n_id
AND u.field = m.field)
END
You'll have to create trigger on view, because direct update is not possible:
CREATE TRIGGER TrMyViewUpdate on MyView
INSTEAD OF UPDATE
AS
BEGIN
SET NOCOUNT ON;
UPDATE MyTable
SET ...
FROM INSERTED...
END