I just observed a strange behaviour (of course Oracle is probably supposed to behave this way, but it didn't fit in my world view yet):
I try to insert two rows into a parent and a child table, both within the same transaction:
INSERT INTO V_Parent (ID, Name) VALUES (777, 'Hello World');
INSERT INTO T_Child (ParentID, Name) VALUES (777, 'Foo Bar');
The Child table has a (ParentID) references Parent.ID foreign key constraint.
On the second statement Oracle fails with the error message "Parent key not found."
If I disable the FK constraint, it works. I have asserted that the ParentID and the Parent.ID match, and I am 100% sure that the first line is executed successfully before the second one. Further, I have tried to commit each statement, which worked fine.
However, as the prefixes in my code example suggest, the first INSERT is actually done on a view of the parent table. The reason is that I use NHibernate and the mapping uses the view in background (which didn't cause any problems until today).
Q1: Could it be that inserting on a view is deferred by Oracle so that the second statement fails?
Q2: How can I remedy this problem best?
Do I need to define INSTEAD OF triggers on the views?
Can I change a setting on the VIEW definition?
Can I change a setting on the FOREIGN KEY definition?
(I must not bend the hibernate mapping to the original table: It's a demand to use the views so changes and/or security issues can be hidden behind the views)
Details: C# WinForms Application - NHibernate - Oracle 10.2 - T_Child: Sooner or later I will use a view for that table, too, it's simply not defined yet.
Edit: More Details according to the comments:
The ID is assigned by NHibernate using an Oracle sequence (<generator class="sequence">), and is part of the INSERT statement as in my example. I also verified that the resulting ID in the table row matches the one NHibernate saved in the mapped object.
The view is defined as a SELECT that JOINS some fields of other tables. However, on insert/update I only change the fields belonging to the main table ("T_PARENT"), and that normally works fine.
The current foreign key constraint is not deferrable, but that shouldn't have any effect because the parent statement is executed before the child statement. *)
*) Hmm... let me think: Since I use an NHibernate session for submitting the SQL queries, could it be that NHibernate executes them in a different order than I told it to?
I'll investigate on that. => It seems so, see my own answer.
This is how the actual code looks like:
ISession session = this.DAOFactory.NHibernateHelper.SessionFactory.OpenSession();
ITransaction tx = session.BeginTransaction();
try
{
// parent.ID == 0
session.SaveOrUpdate(parent);
// parent.ID == 777 (for example)
ISQLQuery query = session.CreateSQLQuery(
"INSERT INTO T_CHILD (PARENT_ID, NAME) VALUES (:parentId, :name)");
query.SetDecimal("parentId", parent.ID);
query.SetDecimal("name", "Foo Bar");
query.ExecuteUpdate(); // Fails with ORA-Exception
tx.Commit();
}
catch (Exception)
{
tx.Rollback();
throw;
}
finally
{
session.Close();
}
You don't need to define an INSTEAD OF trigger if the view is already updateable. Inserting in a view that has a simple relation with its base table will behave as inserting in the base table -- consider:
SQL> CREATE TABLE t (a NUMBER, b NUMBER, c NUMBER);
Table created
SQL> CREATE VIEW v AS SELECT a, b FROM t;
View created
SQL> INSERT INTO v VALUES (1,2);
1 row inserted
SQL> SELECT * FROM t;
A B C
---------- ---------- ----------
1 2
For your insert problem, you probably have a BEFORE INSERT trigger on the base table (with the id colomn filled by a sequence).
I've got it.
As stated in the update to my question, it seems that the NHibernate session mixes the order of the SQL statements. To remedy this, I added the following line of code:
session.SaveOrUpdate(parent);
session.Flush();
// (...)
query.ExecuteUpdate();
Related
Does a SELECT query following an INSERT … ON CONFLICT DO NOTHING statement always find a row, given the default transaction isolation (read committed)?
I want to INSERT-or-SELECT a row in one table, then reference that when inserting rows in a second table. Since RETURNING doesn't work well with ON CONFLICT, I have so far used a simple CTE that should always give me the identity column value even if the row already exists:
$id = query(
`WITH ins AS (
INSERT INTO object (scope, name)
VALUES ($1, $2)
ON CONFLICT (scope, name) DO NOTHING
RETURNING id
)
SELECT id FROM ins
UNION ALL
SELECT id FROM object WHERE scope = $1 AND name = $2
LIMIT 1;`,
[$scope, $name]
)
query(
`INSERT INTO object_member (object_id, key, value)
SELECT $1, UNNEST($2::text[]), UNNEST($3::int[]);`
[$id, $keys, $values]
)
However, I learned that this CTE is not entirely safe under concurrent write load, where it can happen that both the upsert and the select come up empty when a different transaction does insert the same row.
In the answers there (and also here) it is suggested to use another query to do the SELECT:
start a new command (in the same transaction), which then can see these conflicting rows from the previous query.
If I understand correctly, it would mean to do
$id = query(
`INSERT INTO object (scope, name)
VALUES ($1, $2)
ON CONFLICT (scope, name) DO NOTHING
RETURNING id;`,
[$scope, $name]
)
if not $id:
$id = query(
`SELECT id FROM object WHERE scope = $1 AND name = $2;`
[$scope, $name]
)
query(
`INSERT INTO object_member (object_id, key, value)
SELECT $1, UNNEST($2::text[]), UNNEST($3::int[]);`
[$id, $keys, $values]
)
or even shortened to
query(
`INSERT INTO object (scope, name)
VALUES ($1, $2)
ON CONFLICT (scope, name) DO NOTHING;`,
[$scope, $name]
)
query(
`INSERT INTO object_member (object_id, key, value)
SELECT (SELECT id FROM object WHERE scope = $1 AND name = $2), UNNEST($3::text[]), UNNEST($3::int[]);`
[$scope, $name, $keys, $values]
)
I believe this would be enough to prevent that particular race condition (dubbed "concurrency issue 1" in this answer) - but I'm not 100% certain not to have missed anything.
Also what about "concurrency issue 2"? If I understand correctly, this is about another transaction deleting or updating the existing row, inbetween the INSERT and SELECT statements - and it's more likely to happen when using multiple queries instead of the CTE approach. How exactly should I deal with that? I assume locking the SELECT with FOR KEY SHARE is necessary in the second code snippet - but would I also need that in the third snippet where the id is used within the same query? If it helps to simplify the answer, let's assume an object can only be inserted or deleted, but is never updated.
To make absolutely sure that the single row in the first table is there, and it's ID returned, you could create a function like outlined here:
Is SELECT or INSERT in a function prone to race conditions?
To make sure the row also stays there for the duration of the transaction, just make sure it's locked. If you INSERT the row, it's locked anyway. If you SELECT an existing id, you have to lock it explicitly - just like you suggested. FOR KEY SHARE is strong enough for our purpose as long as there is a (non-partial, non-functional) UNIQUE index on (scope, name), which is safe to assume given your ON CONFLICT clause.
CREATE OR REPLACE FUNCTION f_object_id(_scope text, _name text, OUT _object_id int)
LANGUAGE plpgsql AS
$func$
BEGIN
LOOP
SELECT id FROM object
WHERE scope = $1
AND name = $2
-- lock to prevent deletion in the tiny time frame before the next INSERT
FOR KEY SHARE
INTO _object_id;
EXIT WHEN FOUND;
INSERT INTO object AS o (scope, name)
VALUES ($1, $2)
ON CONFLICT (scope, name) DO NOTHING
RETURNING o.id
INTO _object_id;
EXIT WHEN FOUND;
END LOOP;
END
$func$;
You really only need to lock the row if it's conceivable that a concurrent transaction might DELETE it (you don't UPDATE) in the tiny time frame between the SELECT and the next INSERT statement.
Also, if you have a FOREIGN KEY constraint from object_member.object_id to object.id (which seems likely), referential integrity is guaranteed anyway. If you don't add the explicit lock, and the row is deleted in between, you get a foreign key violation, and the INSERT to object_member is cancelled, along with the whole transaction. Else, the other transaction with the DELETE has to wait until your transaction is done, and is then cancelled by the same FK constraint since depending rows are now there (unless it's defined to CASCADE ...) So by locking (or not) you can decide whether to prevent the DELETE or the INSERT in this scenario.
Then your call burns down to just:
query(
`WITH o(id) AS (SELECT f_object_id($1, $2))
INSERT INTO object_member (object_id, key, value)
SELECT o.id, UNNEST($3::text[]), UNNEST($4::int[])
FROM o;`
[$scope, $name, $keys, $values]
)
Since you obviously insert multiple rows into object_member, I moved f_object_id($1, $2) to a CTE to avoid repeated execution - which would work, but pointlessly expensive.
In Postgres 12 or later I would make that explicit by adding MATERIALIZED (since the INSERT is hidden in a function):
WITH o(id) AS MATERIALIZED (SELECT f_object_id($1, $2)) ...
Aside: For the multiple unnest() in the SELECT list, make sure you are on Postgres 10 or later. See:
What is the expected behaviour for multiple set-returning functions in SELECT clause?
Matters of detail
Will it make any difference (apart from execution time) to do this in the application logic with multiple queries in the same transaction?
Basically no. The only difference is performance. Well, and short code and reliability. It's objectively more error prone to go back and forth between db and client for each loop. But unless you have extremely competitive transactions, you would hardly ever be looping anyway.
The other consideration is this: the matter is tricky, and most developers do not understand it. Encapsulated in a server-side function, it's less likely to be broken by the next application programmer (or yourself). You have to make sure that it's actually used, too. Either way, properly document the reasons you are doing it one way or another ...
I really wonder whether my second snippet is safe, or why not (given the quote about visibility in the SELECT after the INSERT).
Mostly safe, but not absolutely. While the next separate SELECT will see (now committed) rows of a transactions competing with the previous UPSERT, there is nothing to keep a third transaction from deleting it again in the meantime. The row has not been locked, and you have no way to do that while it's not visible, and there is no generic predicate locking available in Postgres.
Consider this (T1, T2, T3 are concurrent transactions):
T2: BEGIN transaction
T1: BEGIN transaction
T2: INSERT object 666
T1: UPSERT object 666
unique violation?
-> wait for T2
T2: COMMIT
T1: unique violation -> NO ACTION
finish statement
can't return invisible object 666
T3: DELETE object 666 & COMMIT
T1: SELECT object 666 -> no row!
BOOM!
Typically it's extremely unlikely that it ever happens.
But it's possible. Hence the loop.
The other option is SERIALIZABLE transaction isolation. Typically more expensive, and you need to prepare for serialization failures. Catch 22.
Is there any way in Firebird to execute an UPDATE instead a DELETE through a trigger?
This is possible in Microsoft SQL Server by declaring the triggers as "INSTEAD".
The problem is that we have an application that uses a Firebird database and we want to prevent the deletes of records and mark then as "deleted" (a new field), but without showing any error to the user, and "cheating" the app.
You cannot do this with tables, but you can do it with views. Views can have triggers on insert, update and delete that modify the underlying table(s). See also Updatable Views in the Firebird 2.5 Language Reference.
In short, create a table for the data, add a view, add triggers that insert/update/delete through the view to the underlying table. Users can then use the view as if it is a table.
An example
I'm using Firebird 3, but this will work with minor modifications in Firebird 2.5 and earlier.
A table example_base:
create table example_base (
id bigint generated by default as identity constraint pk_example_base primary key,
value1 varchar(100),
deleted boolean not null default false
)
A view example:
create view example (id, value1)
as
select id, value1
from example_base
where not deleted;
Do not create the view with with check option, as this will disallow inserts as the absence of the deleted column in the view will prevent Firebird from checking the invariant.
Then add an insert trigger:
create trigger tr_example_insert before insert on example
as
begin
if (new.id is not null) then
-- Don't use identity
insert into example_base(id, value1) values (new.id, new.value1);
else
-- Use identity
-- mapping generated id to new context
-- this way it is available for clients using insert .. returning
insert into example_base(value1) values (new.value1)
returning id into :new.id;
end
The above trigger ensures the 'by default as identity' primary key of the underlying table is preserved, and allows insert into example .. returning to report on the generated id.
An update trigger
create trigger tr_example_update before update on example
as
begin
-- Consider ignoring modification of the id (or raise an exception)
update example_base
set id = new.id, value1 = new.value1
where id = old.id;
end
The above trigger allows modification of the primary key; you may want to consider just ignoring such a modification or even raising an exception.
And finally a delete trigger:
create trigger tr_example_delete before delete on example
as
begin
update example_base
set deleted = true
where id = old.id;
end
This trigger will mark the record in the base table as deleted.
To use this, just grant your users select, insert and update privileges to the view (and not the table).
The only caveat I'm aware of is that defining foreign keys will need to point to example_base, not to example, and the behavior of foreign keys will be slightly off. The record in the base table will continue to exist, so the foreign key will not block deletion. If that is something that is necessary, you will need to emulate constraint behavior (which could be tricky).
YES! It can be made on VIEWs.
That's the way I solved it.
If a View has a trigger, then the trigger is the responsible of making the real update or delete on the underlying table.... So... a DELETE trigger that makes an UPDATE to the table solved my problem.
I would like to the the old & new rows as XML to an exceptions table when a trigger cannot succeed. I am used to using a generic EXCEPTION WHEN OTHERS THEN clause to log out failures, what I cannot figure out is have to capture (so I can log) the OLD and NEW pseudorows into XML.
It seems like
old_x := dbms_xmlgen.getxml('select * from OLD');
ought to work, but perhaps I am missing something simple.
You can't select from old, and there is no way to access the old values generically, I'm afraid - you have to specify old.empno, old.ename etc. one by one.
Tom Kyte has shown how to generate triggers to overcome this here on asktom.oracle.com.
Reading the fine document:
A trigger that fires at row level can access the data in the row that it is processing by using correlation names. The default correlation names are OLD, NEW, and PARENT.
OLD, NEW, and PARENT are also called pseudorecords, because they have record structure, but are allowed in fewer contexts than records are. The structure of a pseudorecord is table_name%ROWTYPE, where table_name is the name of the table on which the trigger is created (for OLD and NEW) or the name of the parent table (for PARENT).
(Pseudorecords have also other restrictions explicitly mentioned in the documentation but not quoted here.)
Among all the other things the restrictions mean one can't use pseudorecords e.g. in insert statement extension.
So very much the only thing you can do with a pseudorecord is to refer it's fields one-by-one.
Example
create table a (a number, b date);
create table b (a number, b date);
create or replace trigger a_trg
before insert on a
for each row
begin
-- NEW pseudorecord is not a real record and therefore compilation will fail:
-- PLS-00049: bad bind variable 'NEW'
-- insert into b values :new;
-- NEW pseudorecord fields have to be referenced one-by-one
insert into b(a, b) values(:new.a, :new.b);
end;
/
show errors
insert into a values(1, sysdate);
select * from a;
select * from b;
I'm working on Oracle SQL database, quite big database. One of (among 150 tables) this table has to be changed because it's redundant (it can be generated through a join). I have been asked to delete a column from this table, to get rid of the redundancy. The problem is that now I have to change code everywhere someone made a insert/update/etc on this table (and don't forget the constraint!). I thought "I can make a view that do the right join" so the problem it's solved for all the select, but it's not working for the insert, because I'm updating 2 tables... Is there a way to solve this problem?
My goal is to rename my original table original_table in original_table_smaller (with one less column) and create a view (or something like a view) called original_table that work like the original table.
Is this possible?
As your view will contain one column that is not present in the real table, you will need to use an instead of trigger to make the view updateable.
Something like this:
create table smaller_table
(
id integer not null primary key,
some_column varchar(20)
);
create view real_table
as
select id,
some_column,
null as old_column
from smaller_table;
Now your old code would run something like this:
insert into real_table
(id, some_column, old_column)
values
(1, 'foo', 'bar');
which results in:
ORA-01733: virtual column not allowed here
To get around this, you need an INSTEAD OF trigger:
create or replace trigger comp_trigger
instead of insert on smaller_table
begin
insert into old_table
(id, some_column)
values
(:new.id, :new.some_column);
end;
/
Now the value for the "old_column" will be ignored. You need something similar for updates as well.
If your view contains a join, then you can handle that situation as well in the trigger. Simply do an update/insert according to the data to two different tables
For more details and examples, see the manual
http://docs.oracle.com/cd/E11882_01/appdev.112/e25519/triggers.htm#i1006376
It is possible to insert/update on views.
You might want to check with USER_UPDATABLE_COLUMNS which columns you can insert to the view.
Check with this query:
select * from user_updatable_columns where table_name = 'VIEW_NAME';
Oracle has two different ways of making views updatable:-
The view is "key preserved" with respect to what you are trying to update. This means the primary key of the underlying table is in the view and the row appears only once in the view. This means Oracle can figure out exactly which underlying table row to update OR
You write an instead of trigger.
Assuming that all foreign keys have the appropriate constraint, is there a simple SQL statement to delete rows not referenced anywhere in the DB?
Something as simple as delete from the_table that simply skip any rows with child record?
I'm trying to avoid manually looping through the table or adding something like where the_SK not in (a,b,c,d).
You might be able to use the extended DELETE statement in 10g that includes error logging.
First use DBMS_ERRLOG to create a logging table (which is just a copy of the original table with some additional prefixing columns: ORA_ERR_MESG$, ..., ORA_ERR_TAG$)
execute dbms_errlog.create_error_log('parent', 'parent_errlog');
Now, you can use the LOG ERRORS clause of the delete statement to capture all rows that have existing integrity constraints:
delete from parent
log errors into parent_errlog ('holding-breath')
reject limit unlimited;
In this case the "holding-breath" comment will go into the ORA_ERR_TAG$ column.
You can read the full documentation here.
If the parent table is huge and you're only looking to delete a few stray rows, you'll end up with a parent_errlog table that is essentially a duplicate of your parent table. If this isn't ok, you'll have to do it the long way:
Directly reference the child tables (following Tony's solution), or,
Loop through the table in PL/SQL and catch any exceptions (following Confusion's and Bob's solutions).
The easiest way may be to write an application or stored procedure that attempts to delete the rows in the table one-by-one and simply ignores the failures due to foreign key constraints. Afterwards, all rows not under a foreign key constraint should be removed. Depending on the required/possible performance, this may be an option.
No. Obviously you can do this (but I realise you would rather not):
delete parent
where not exists (select null from child1 where child1.parent_id = parent.parent_id)
and not exists (select null from child2 where child2.parent_id = parent.parent_id)
...
and not exists (select null from childn where childn.parent_id = parent.parent_id);
One way to do this is to write something like the following:
eForeign_key_violation EXCEPTION;
PRAGMA EXCEPTION_INIT(eForeign_key_violation, -2292);
FOR aRow IN (SELECT primary_key_field FROM A_TABLE) LOOP
BEGIN
DELETE FROM A_TABLE A
WHERE A.PRIMARY_KEY_FIELD = aRow.PRIMARY_KEY_FIELD;
EXCEPTION
WHEN eForeign_key_violation THEN
NULL; -- ignore the error
END;
END LOOP;
If a child row exists the DELETE will fail and no rows will be deleted, and you can proceed to your next key.
Note that if your table is large this may take quite a while.