Rolling back inner transaction when outer transaction fails - sql

I am using ORMLite to access a h2 database in Java.
To perform transactions i'm using the static method TransactionManager.callInTransaction.
In case of single independent transactions this works fine. However in case the transactions are nested within each other the inner transactions get committed even if the outer transaction fails.
As in this piece of pseudo-code:
OuterDatabaseTransaction
{
Loop
{
InnerDatabaseTransaction
{
//Multiple update or create statements
//One of the InnerDatabaseTransactions throws a random exception
}
//Alternatively the OuterDatabaseTransaction throws a random
//exception but all commited InnerDatabaseTransactions should rollback still
}
}
So what I would expect is that if any of the inner transactions fails the outer transaction fails too. And if the outer transaction fails none of the inner transactions get committed.
At the moment it seems that each of the inner transactions are committed individually and do not for example share the same Savepoint as the outer transaction.
UPDATE (Thanks dened)
Looking at the trace reveals the following
[TRACE] JdbcDatabaseConnection connection is closed returned false
[TRACE] JdbcDatabaseConnection connection autoCommit is true
[TRACE] JdbcDatabaseConnection connection set autoCommit to false
[DEBUG] TransactionManager had to set auto-commit to false
[TRACE] JdbcDatabaseConnection save-point sp14: id=0 name=ORMLITE15 set with name ORMLITE15
[DEBUG] TransactionManager started savePoint transaction ORMLITE15
[TRACE] JdbcDatabaseConnection connection is closed returned false
[TRACE] JdbcDatabaseConnection connection autoCommit is false
[TRACE] JdbcDatabaseConnection save-point sp15: id=0 name=ORMLITE16 set with name ORMLITE16
[DEBUG] TransactionManager started savePoint transaction ORMLITE16
[TRACE] JdbcDatabaseConnection connection is closed returned false
[TRACE] JdbcDatabaseConnection update statement is prepared and executed returning 1: <...>
[DEBUG] BaseMappedStatement update data with statement <...> changed 1 rows
[TRACE] BaseMappedStatement update arguments: <...>
[TRACE] JdbcDatabaseConnection connection is committed for save-point ORMLITE16
[DEBUG] TransactionManager committed savePoint transaction ORMLITE16
-> [ERROR] TransactionManager after commit exception, rolling back to save-point also threw exception
[TRACE] JdbcDatabaseConnection connection set autoCommit to true
[DEBUG] TransactionManager restored auto-commit to true
[TRACE] JdbcDatabaseConnection connection is closed returned false
Stepping into the source reveals that during rollback of OuterDatabaseTransaction the exception is thrown in the h2 source in org.h2.engine.Session.java within the function below. The reason behind that however I don't understand yet.
private void checkCommitRollback() {
if (commitOrRollbackDisabled && locks.size() > 0) {
throw DbException.get(ErrorCode.COMMIT_ROLLBACK_NOT_ALLOWED);
}
}
UPDATE 2
Posted ORMLite Bug Report

To find the cause you may turn trace level logging on. Working nested transactions should produce the log like this:
TRACE: connection supports save points is true
TRACE: save-point <...> set with name ORMLITE1
DEBUG: started savePoint transaction ORMLITE1
...
TRACE: save-point <...> set with name ORMLITE2
DEBUG: started savePoint transaction ORMLITE2
...
TRACE: connection is committed for save-point ORMLITE2
...
TRACE: save-point <...> set with name ORMLITE3
DEBUG: started savePoint transaction ORMLITE3
...
TRACE: save-point ORMLITE3 is rolled back
...
TRACE: save-point ORMLITE1 is rolled back
in this example ORMLITE1 is save point for outer transaction, ORMLITE2 and ORMLITE3 are for inner transactions. The first inner transaction is initially commited, the second transaction is rolled back to ORMLITE3 and caused rollback of the outer transaction to ORMLITE1, it in turn implicitly cancelled the first inner transaction.
But if you see this in the log:
TRACE: connection supports save points is false
then save points are not supported by the JDBC driver and nested transaction wouldn't work. Theoretically this shouldn't happen as H2 states support of save points.
If you see this:
ERROR: after commit exception, rolling back to save-point also threw exception
then rollback to a save point is failed for some reason. Check you use same ConnectionSource for outer and inner transactions.
Or you my find in the log something else that caused the problem. Also it may be helpful to attach the log to your question. And I suggest to replace the pseudo-code with the real Java code.
Update
Here is a official description of the error you got:
COMMIT_ROLLBACK_NOT_ALLOWED = 90058
The error with code 90058 is thrown when trying to call commit or rollback inside a trigger, or when trying to call a method inside a trigger that implicitly commits the current transaction, if an object is locked. This is not because it would release the lock too early.
Link: http://www.h2database.com/javadoc/org/h2/constant/ErrorCode.html#c90058
May be this will help you further to find the cause of the problem. For me, it is hard to say more not seeing your code. Good luck!

You are correct,
TransactionManager commits each transaction at the end (same behaviour for inner and ounter transaction) as Gray wrote in comments.
The problem is in TransactionManager.java:170, which commits transaction even after inner transaction.
But H2, Mysql, Oracle databases do not support sql COMMIT with savepoint parameter.
From Mysql documentation:
"All savepoints of the current transaction are deleted if you execute a COMMIT, or a ROLLBACK that does not name a savepoint."
From Oracle documentation:
"A simple rollback or commit erases all savepoints. When you roll back to a savepoint, any savepoints marked after that savepoint are erased. The savepoint to which you roll back remains."
There is also comment on this in OrmLite code : JdbcDatabaseConnection.java:94

Related

Rollback not working properly while using sqlalchemy on SQL Server

I am facing an issue with SQLalchemy's sessions in python. Underlying database - SQL Server.
What I am trying to do?
Insert into parent and child tables(both have an auto-increment PK) as part of one transaction and rollback changes on both if any error occurs else move on with commiting changes.
This is how the pseudo-code looks -
session = Session() # get the session from somewhere. (Auto-Commit=false)
try:
parent = Parent(...)
session.add(parent)
session.flush()
# By now I expect the new ID updated in parent object.
child = Child(...)
session.add(child)
session.flush()
# raise Exception('abc') ->
# In case of this exception I expect no changes to be written to database,
# but surprisingly after the execution of mere flush I am able see changes written in SQL server.
session.commit()
except:
session.rollback()
Now when I check out the logger, in case of exception I see these set of queries going in -
[20210612 03:43:23:530 base.py:727 INFO] BEGIN (implicit)
[20210612 03:43:23:531 base.py:1234 INFO] INSERT INTO PARENT TABLE
[20210612 03:43:23:534 base.py:1234 INFO] INSERT INTO CHILD TABLE
[20210612 03:43:23:574 base.py:747 INFO] ROLLBACK
Any ideas how is this happening and what might be the corrective actions for it?
Please do let me know in case any other detail is required.

Could not restore Flink keyed state backend from savepoint when upgrading from 1.10 to 1.11

We tried to migrate to Flink 1.11 recovering the job from a savepoint taken in 1.10. The job code was not changed, only updated the Flink version of the dependencies to 1.11 (in SBT, we use Scala) and re-built the jar.
All operators have uids and the job correctly recovers from that savepoint if run on a 1.10 cluster, we are getting the following exception and have no clue:
java.lang.Exception: Exception while creating StreamOperatorStateContext.
at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:204)
at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:247)
at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:290)
at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:473)
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47)
at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:469)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:522)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flink.util.FlinkException: Could not restore keyed state backend for CoStreamFlatMap_8a6da66867c6cf8469bae55e9f834297_(1/1) from any of the 1 provided restore options.
at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:135)
at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.keyedStatedBackend(StreamTaskStateInitializerImpl.java:317)
at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.streamOperatorStateContext(StreamTaskStateInitializerImpl.java:144)
... 9 more
Caused by: org.apache.flink.runtime.state.BackendBuildingException: Failed when trying to restore heap backend
at org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder.build(HeapKeyedStateBackendBuilder.java:116)
at org.apache.flink.runtime.state.filesystem.FsStateBackend.createKeyedStateBackend(FsStateBackend.java:540)
at org.apache.flink.streaming.api.operators.StreamTaskStateInitializerImpl.lambda$keyedStatedBackend$1(StreamTaskStateInitializerImpl.java:301)
at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.attemptCreateAndRestore(BackendRestorerProcedure.java:142)
at org.apache.flink.streaming.api.operators.BackendRestorerProcedure.createAndRestore(BackendRestorerProcedure.java:121)
... 11 more
Caused by: java.lang.IllegalStateException: Missing value for the key 'org.apache.flink.runtime.checkpoint.savepoint.Savepoint'
at org.apache.flink.util.LinkedOptionalMap.unwrapOptionals(LinkedOptionalMap.java:190)
at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializerSnapshot.restoreSerializer(KryoSerializerSnapshot.java:86)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:546)
at java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260)
at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:505)
at org.apache.flink.api.common.typeutils.NestedSerializersSnapshotDelegate.snapshotsToRestoreSerializers(NestedSerializersSnapshotDelegate.java:225)
at org.apache.flink.api.common.typeutils.NestedSerializersSnapshotDelegate.getRestoredNestedSerializers(NestedSerializersSnapshotDelegate.java:83)
at org.apache.flink.api.common.typeutils.CompositeTypeSerializerSnapshot.restoreSerializer(CompositeTypeSerializerSnapshot.java:204)
at org.apache.flink.runtime.state.StateSerializerProvider.previousSchemaSerializer(StateSerializerProvider.java:189)
at org.apache.flink.runtime.state.StateSerializerProvider.currentSchemaSerializer(StateSerializerProvider.java:164)
at org.apache.flink.runtime.state.RegisteredKeyValueStateBackendMetaInfo.getStateSerializer(RegisteredKeyValueStateBackendMetaInfo.java:136)
at org.apache.flink.runtime.state.heap.StateTable.getStateSerializer(StateTable.java:315)
at org.apache.flink.runtime.state.heap.CopyOnWriteStateTable.createStateMap(CopyOnWriteStateTable.java:54)
at org.apache.flink.runtime.state.heap.CopyOnWriteStateTable.createStateMap(CopyOnWriteStateTable.java:36)
at org.apache.flink.runtime.state.heap.StateTable.<init>(StateTable.java:98)
at org.apache.flink.runtime.state.heap.CopyOnWriteStateTable.<init>(CopyOnWriteStateTable.java:49)
at org.apache.flink.runtime.state.heap.AsyncSnapshotStrategySynchronicityBehavior.newStateTable(AsyncSnapshotStrategySynchronicityBehavior.java:41)
at org.apache.flink.runtime.state.heap.HeapSnapshotStrategy.newStateTable(HeapSnapshotStrategy.java:243)
at org.apache.flink.runtime.state.heap.HeapRestoreOperation.createOrCheckStateForMetaInfo(HeapRestoreOperation.java:185)
at org.apache.flink.runtime.state.heap.HeapRestoreOperation.restore(HeapRestoreOperation.java:152)
at org.apache.flink.runtime.state.heap.HeapKeyedStateBackendBuilder.build(HeapKeyedStateBackendBuilder.java:114)
Can anyone help?
Thanks
UPDATE
The savepoint comes from a savepoint processed with the stateprocessor API and the state in the KeyedStateBootstrapFunction is made of:
var mapToDetector: MapState[String, Map[String, Detector]] = null
var detectorsConfigs: MapState[String,AnomalyStepConfiguration] = null
var outputTopic : ValueState[String]= null
var pipeStatus: MapState[String, String] = null
var debounceMap: MapState[String, Map[String, DebounceStats]] = null
org.apache.flink.runtime.checkpoint.savepoint.Savepoint is renamed in FLINK-16247. However, this class is used in savepoint metadata and should not exist in keyed state serializer on task side. In other words, did you use something related to checkpoint or savepoint on task side in state access?
I also try to use StateMachineExample to create savepoint in Flink-1.10.2 and it resumes successfully within Flink-1.11.1 cluster. The program also used CopyOnWriteStateTable by default which is what you see in your exception stack trace.

'SavePoint' do not work in NPoco(Oracle) for separate execute statements

I'm trying to use NPoco + Oracle + SavePoint. But when I'm trying to rollback savepoint, it throws exception.
Example:
NPoko.IDatabase db = new ...
db.Execute("SAVEPOINT ABC");
db.Execute("ROLLBACK TO SAVEPOINT ABC");
Exception:
"ORA-01086: savepoint 'ABC' never established in this session or is invalid"
I need to call 'savepoint' then call some other methods and then rollback or commit.
Thank you.

Solution for java.sql.SQLException: could not use local transaction commit in a global transaction

I'm running a J2EE MDB in a Weblogic 10.3 clustered environment.
Within one of the methods in the code I create a JDBC connection and turn off auto commit. The connection to the database is made through a XA enabled Data source. The reason for turning the auto-commit off is because I'm using a temp table.
When I'm done with the temp table & other SQL queries I commit the Transaction and turn auto-commit back on. For the most part this has not been a problem but I received the following exception when executing the method, the exception occurred on the line in the code where I issue the commit.
Not sure why I received as in the method I only make calls to a single database, I wonder if its because the data source is XA enabled? Any insight as to why this exception occurs and why it only occurs sometimes would be greatly appreciated
java.sql.SQLException: could not use local transaction commit in a global transaction
at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:70)
at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:112)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:173)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:229)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:403)
at oracle.jdbc.driver.PhysicalConnection.disallowGlobalTxnMode(PhysicalConnection.java:6139)
at oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:3320)
at oracle.jdbc.driver.PhysicalConnection.commit(PhysicalConnection.java:3357)
at oracle.jdbc.OracleConnectionWrapper.commit(OracleConnectionWrapper.java:130)
at weblogic.jdbc.wrapper.XAConnection.commit(XAConnection.java:844)
at weblogic.jdbc.wrapper.JTAConnection.commit(JTAConnection.java:326)
at com.myPackage.myApp.myDBCaller.tempTableMethod(myDBCaller.java:390)
at com.myPackage.myApp.myDBCaller.start(myDBCaller.java:153)
at com.myPackage.myApp.myDBCaller.onMessage(myDBCaller.java:117)
at sun.reflect.GeneratedMethodAccessor1683.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:599)
at com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:281)
at com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:187)
at com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:154)
at com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:126)
at com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:114)
at com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176)
at com.bea.core.repackaged.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
at com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176)
at com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:126)
at com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:114)
at com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:176)
at com.bea.core.repackaged.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:210)
at $Proxy228.onMessage(Unknown Source)
at weblogic.ejb.container.internal.MDListener.execute(MDListener.java:518)
at weblogic.ejb.container.internal.MDListener.transactionalOnMessage(MDListener.java:423)
at weblogic.ejb.container.internal.MDListener.onMessage(MDListener.java:325)
at weblogic.jms.client.JMSSession.onMessage(JMSSession.java:4547)
at weblogic.jms.client.JMSSession.execute(JMSSession.java:4233)
at weblogic.jms.client.JMSSession.executeMessage(JMSSession.java:3709)
at weblogic.jms.client.JMSSession.access$000(JMSSession.java:114)
at weblogic.jms.client.JMSSession$UseForRunnable.run(JMSSession.java:5058)
at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:516)
at weblogic.work.ExecuteThread.execute(ExecuteThread.java:201)
at weblogic.work.ExecuteThread.run(ExecuteThread.java:173)
Try set property
<property name="AutomaticEnlistingEnabled" value="false"/>
of xaDatasource.

How to throw message and commit transaction using RAISERROR?

I need to use RAISERROR to throw a message(pop up message) and commit that transaction.Is there any option?
For severity 18 the transaction got rollback.I have changed the severity to 10 and tried like
RAISERROR('Your Reference Number is %s',10,0,#param);
this but it commits the transaction but doesnt show message.
What i need here is Message has to be thrown and transaction has to be commited
Any suggestion?
Don't use exceptions to pass back "OK" messages. You wouldn't in a c# or java program. Exception means "I've aborted because SHTF"
You'd use thsi to return meaningful data
SELECT 'Your Reference Number is ' + #param
In a typical template (from my answer Nested stored procedures containing TRY CATCH ROLLBACK pattern?)
SET XACT_ABORT, NOCOUNT ON
BEGIN TRY
BEGIN TRANSACTION
[...Perform work, call nested procedures...]
COMMIT TRANSACTION
SELECT 'Your Reference Number is ' + #param
END TRY
BEGIN CATCH
IF XACT_STATE() <> 0
ROLLBACK TRANSACTION
RAISERROR [rethrow caught error using #ErrorNumber, #ErrorMessage, etc]
END CATCH
RAISERROR with severity above 10 will be treated like an exception by the ADO.Net client. Depending on how your call context looks like, this may or may not rollback the transaction. If you use a SqlTransaction or a TransactionScope in the client, or a BEGIN TRY/BEGIN CATCH block on the server side, this will likely roll back the transaction. Point is that RAISERROR neither commits nor rolls back the transaction, is your own code that rolls back or commits and we cannot know what you're doing from your post.
RAISERROR with severity bellow 10 will be considered a informational message and not cause an exception. See Database Engine Error Severities. This is probably why you say that the 'it doesn't show the message' (whatever that means). Client side frameworks treat the informational messages differently, for instance ADO.Net will raise an SqlConnection.InfoMessage event on the connection but will not raise an exception. You probably don't have anything set up in your application for this event and your code is simply ignoring the info messages. For example how to use the InfoMessage event see Connection Events (ADO.NET)
It sounds like you need to use the WITH NOWAIT parameter for RAISERROR - this will output it to the message window immediately:
RAISERROR('Your Reference Number is %s',10,0,#param) WITH NOWAIT