I'm using jberet implementation of JSR 352 java batch specs.
Actually I need a separate transaction for doing a singular update, something like this:
class MyItemWriter implements ItemWriter
#Inject
UserTransaction transaction
void resetLastProductsUpdateDate(String uidCli) throws BusinessException {
try {
if (transaction.getStatus() != Status.STATUS_ACTIVE) {
transaction.begin();
}
final Customer customer = dao.findById(id);
customer.setLastUpdate(null);
customer.persist(cliente);
transaction.commit();
} catch (RollbackException | HeuristicMixedException | HeuristicRollbackException | SystemException | NotSupportedException e) {
logger.error("error while updating user products last update");
throw new BusinessException();
}
}
I first tried marking resetLastProductsUpdateDate methoad as #Transactional(REQUIRES_NEW), however it didn't worked.
My question is:
Is there any more elegant way to achieve this singular transaction without manually handle of transaction?
While does UserTransation works, EntityManager.transaction doesn't. I don't get it why.
Class below, which is injected from a Batchlet, works properly; Why I can't get to make work the #Transactional annotation on resetLastProductsUpdateDate method instead?
public class DynamicQueryDAO {
#Inject
EntityManager entityManager;
#Inject
private Logger logger;
#Transactional(Transactional.TxType.REQUIRED)
public void executeQuery(String query) {
logger.info("executing query: {}", query);
final int output = entityManager.createNativeQuery(query).executeUpdate();
logger.info("rows updated: {}", output);
}
}
EDIT
Actually I guess neither usertransaction is a good solution, because it affects entire itemwriter transaction management. Still Don't know how to deal with transaction isolation :(
In general the batch application should avoid directly handling transaction. You can have your batch component to throw some business exceptions upon certain conditions, and configure your job.xml to trigger retry upon this business exception. During retry, each individual data will be processed and committed in its own chunk.
Related
https://docs.particular.net/nservicebus/testing/ has a very neat example of how to test a simple Saga. Unfortunately it does not explain, how to do the same with a SqlSaga - ie. a Saga with persistence of state to a database.
Given this saga:
public class SomeFancySaga : SqlSaga<MySagaData>,
IAmStartedByMessages<ImportantCommand>
{
protected override string CorrelationPropertyName => nameof(Data.MyPrimaryKey);
protected override void ConfigureMapping(IMessagePropertyMapper mapper)
{
mapper.ConfigureMapping<ImportantCommand>(x => x.CommandId);
}
public async Task Handle(ImportantCommand command, IMessageHandlerContext context)
{
if (Data.State == MyState.ReadyForUse)
<do some stuff>
}
...
}
If I try to write test-code like the example in the link, I would do something like this:
// arrange
var context = new NServiceBus.Testing.TestableMessageHandlerContext();
var command = ImportantCommand.Create();
var sut = new CreateSomeFancySaga();
// act
sut.Handle(command, context);
// assert
...
The call to sut.Handle() will throw a NullReferenceException, because the Saga property Data has not been initialized.
How do I correctly wireup the saga for testing so that:
Data is initialized
A real database connection is not really needed
We have a related code sample showing unit testing in more detail: https://docs.particular.net/samples/unit-testing/. That includes a couple of tests for saga (SagaTests.cs).
You can take this as a starting point and modify the saga tests in the following way:
Add NServiceBus.Persistence.Sql package.
Modify the DiscountPolicy policy saga to inherit from SqlSaga instead of a Saga.
Resolve compilation errors (add missing method and property, you can keep them empty, plus remove the ConfigureHowToFindSaga method).
I hope that helps, but let me know if there's anything missing or your scenario can't be tested this way.
We are currently using Cassandra as NoSQL Database and GemFire as In memory Database. We have been using the GemFire CacheWriter to insert the records in Cassandra. I would like your feedback on whether it’s a good engineering practice to use Concurrent threads in CacheWriter to insert/Update records. Your feedback on this would be appreciated.
public class GenericWriter<K, V> extends CacheWriterAdapter<K, V> implements Declarable {
private static Logger log = LoggerFactory.getLogger(GenericWriter.class);
#Autowired
private CassandraOperations cassandraOperations;
ExecutorService executor = null;
#Override
public void beforeCreate(EntryEvent<K, V> e) {
executor = Executors.newSingleThreadExecutor();
executor.submit(() -> {
if (eventOperation.equals("CREATE") || eventOperation.equalsIgnoreCase("PUTALL_CREATE")) {
try {
cassandraOperations.insert(e.getNewValue());
} catch (CassandraConnectionFailureException | CassandraWriteTimeoutException
| CassandraInternalException cassException) {
} catch (Exception ex) {
log.error("Exception in GenericCacheWriter->" + ExceptionUtils.getStackTrace(ex));
throw ex;
}
}
});
executor.shutdown();
}
#Override
public void init(Properties arg0) {
// TODO Auto-generated method stub
}
}
The CacheWriter handler is called synchronously, so the application does not continue until the handler returns. Therefore, is not recommended to execute long-running operations inside this listener. If a long-running operation is needed, consider processing the operation asynchronously through an AsyncEventListener instead.
Using an ExecutorService to delegate the execution to a different thread is possible but it is an anti-pattern, as it no longer implements the fail-fast property, and the handling of the event is no longer synchronous, so its timing would not be guaranteed relative to the application's completion of the event.
You can read more about this topic in the Geode Wiki, specifically in CacheWrite and CacheListener Best Practices.
Hope this helps.
Best regards.
Yes, it's a fine pattern but remove the Executor and partition your data such that all updates into GemFire go to one and only one node. Partition Cassandra the same way. Put a write lock around the Cassandra update. Use this only when your throughput is low.
If you need high throughput, use the AsyncEventListener and guarantee eventual consistency to your users. If you must use Executors in the AEL, use them in a way so as to throw an exception in the main thread. If the update fails after a number of tries, you write the failed entry to a different region with an expiration of a few seconds or a minute. When that expires, retry the operation. Keep doing this until the succeeds and then and only then, delete the expired entry.
You will need to track version numbers and what you are updating watching old values/ new values if order of updates is important to you or not.
I use nHibernate for ORM and Ninject for IoC.
I create nHibernate sessions per some custom scope (which you can assume is per request).
I begin the transaction onActivation.
I commit the transaction onDeactivation.
The problem is that if an exception happens during the request I want to rollback the transaction rather than committing it. Any idea how to detect (in a clean way, most probably using Ninject Context) that an exception has happened?
Note: I am not concerned about the exceptions that can happen on commit which I can catch in the following code and role back easily.
protected void BindWithSessionWrapper<T>(Func<IContext, T> creationFunc) where T : ISessionWrapper
{
Bind<T>().ToMethod(creationFunc)
.InScope(x => new NinjectCustomScope()) // work in progress !!!
.OnActivation(t => t.Session.BeginTransaction(IsolationLevel.ReadCommitted))
.OnDeactivation((c, t) =>
{
t.Session.Transaction.Commit();
t.Session.Dispose();
});
}
Update:
I followed the suggestion by #BatteryBackupUnit.
So I added the following to the Error EventHandler:
Error += (s, e) =>
{
HttpContext.Current.Items["ErrorRaised"] = true;
};
And I modified the OnDeactivation to look like this:
OnDeactivation(t =>
{
if ((bool?)HttpContext.Current.Items["ErrorRaised"] == true)
t.Session.Transaction.Rollback();
else
t.Session.Transaction.Commit();
t.Session.Dispose();
});
It works fine, but that would be better if Ninject would take care of this by setting a flag in the Context if an exception happened :)
How about implementing an IHTTPModule and subscribing to the Error event?
Like described here
In the Error event handler, use System.Web.Mvc.DependencyResolver.Current.GetService(typeof (ISession)) to retrieve the current session and rollback the transaction.
Note, however, that in case the request did not use a session, this will create one, which is quite superfluous.
You might do something like checking whether a transaction was started and only then rolling it back. But you'd still create a session unnecessarily.
You could further improve that by using the Error event handler to set a flag on HttpContext.Current.Items, like
HttpContext.Current.Items["RollbackTransaction"] = true;
and then use it in the OnDeactivation of the session like:
.OnDeactivation((c, t) =>
{
if(HttpContext.Current.Items.Contains("RollbackTransaction"])
{
t.Session.Transaction.Rollback();
}
else
{
t.Session.Transaction.Commit();
}
t.Session.Dispose();
});
Please note that HttpContext is thread local, that means when you switch threads it may be null or -worst case - it might even be another HttpContext.
Please also note that i was unable to try it out so it may not work. Feedback appreciated.
Passing the state through HttpContext is not acceptable to me for 2 reasons.
HttpContext issue: https://stackoverflow.com/a/12219078/656430)
Passing state seems like passing a global state (https://softwareengineering.stackexchange.com/questions/148108/why-is-global-state-so-evil)
After a lot of trial and error, I think this should be one solution:
Assuming we are working on WebApi project, having rollback transaction for all actions once hit exception, with Ninject:
install Ninject.Extension.Factory (https://www.nuget.org/packages/Ninject.Extensions.Factory/), this is very important step as to inject ISession in request scope into filters.
use the following configuration for binding ISessionFactory and ISession (I made use of this example: Need a simple example of using nhibernate + unit of work + repository pattern + service layer + ninject), plus ISessionInRequestScopeFactory
Bind<ISessionFactory>().ToProvider<NhibernateSessionFactoryProvider>().InSingletonScope();
Bind<ISession>()
.ToMethod(context => context.Kernel.Get<ISessionFactory>().OpenSession())
.InRequestScope(); // notice that we don't need to call `BeginTransaction` at this moment
Bind<ISessionInRequestScopeFactory>().ToFactory(); // you don't need to make your implementation, the Ninject.Extension.Factory extension will help you so.
the code for interface ISessionInRequestScopeFactory:
public interface ISessionInRequestScopeFactory
{
ISession CreateSessionInRequestScope(); // return ISession in the request scope
}
Make use of ninject filter injection to add Transaction behaviour to every action (https://github.com/ninject/Ninject.Web.WebApi/wiki/Dependency-injection-for-filters):
Kernel.BindHttpFilter<ApiTransactionFilter>(System.Web.Http.Filters.FilterScope.Action)
.WhenControllerHas<ApiTransactionAttribute>();
add [ApiTransaction] attribute into controller:
[ApiTransaction]
public class YourApiController{ /* ... */}
So we are now binding the ApiTransactionFilter into YourApiController which are having [ApiTransaction] Attribute
Inside ApiTransactionFilter, you should extends AbstractActionFilter and inject the factory ISessionInRequestScopeFactory for getting the correct request scope session:
public class ApiTransactionFilter : AbstractActionFilter{
private readonly ISessionInRequestScopeFactory factory;
public ApiTransactionFilter(ISessionInRequestScopeFactory factory){
this.factory = factory;
}
public override void OnActionExecuting(HttpActionContext actionContext)
{
ISession session = factory.CreateSessionInRequestScope(); // get the request scope session through factory
session.BeginTransaction(); // session can begin transaction here ...
base.OnActionExecuting(actionContext);
}
public override void OnActionExecuted(HttpActionExecutedContext actionExecutedContext)
{
ISession session = factory.CreateSessionInRequestScope(); // get the request scope session through factory
if (actionExecutedContext.Exception == null) // NO EXCEPTION!
{
session.Transaction.Commit();// session commit here ... may be you like to have try catch here
}
else
{
session.Transaction.Rollback(); // session rollback here ...
}
base.OnActionExecuted(actionExecutedContext);
}
}
ok, so Ayende recommends always using a transaction, even for read operations.
but supposing I have the following scenario:
public Employee GetEmployeeByName(string name)
{
using (ITransaction tx = CurrentSession.BeginTransaction())
{
return dao.GetEmployeeByName(name);
}
}
public void SaveNewEmployee(Employee employee)
{
using (ITransaction tx = CurrentSession.BeginTransaction())
{
if (GetEmployeeByName(employee.Name) != null)
{
throw new ArgumentException("employee with same name found");
}
CurrentSession.Save(employee);
}
}
this would actually throw an exception, since nhibernate doesn't support nested transactions.
how can I get around this?
EDIT
this is even a better solution than the one I accepted...
Typically you would get around it by using a unit of work pattern in which you can start your transaction at the same time you open your session. That is to say at the beginning of the unit of work. And you would commit it at the end of the unit of work.
First, some background: I'm new to ASP.NET MVC 2 and NHibernate. I'm starting my first application and I want to use NHibernate, because I come from JSP + Struts 1 + Hibernate web applications.
No one seems to be talking about this, so I guess it must be pretty obvious. Still I scratch my head because I can't find a solution that accomplish the following things:
1) I want to use the "session per request" strategy. So, everytime a user makes a request, he gets an Nhibernate session, starts a transaction, and when the request is over, the transaction commits, and the NHibernate session closes (and returns to the pool if there is one). This guarantees that my transactions are atomic.
2) When a database exception occurs (PK violation, unique violation, whatever) I want to capture that exception, rollback my transaction and give the user a explicit message: if it was PK violation, then that message, and the same with all integrity errors.
So, what is my problem? I come from the Java World, where I used a Filter to open the session, start the transaction, process the request, then commit the transaction and close the session. This works, except when an DB exception occurs, and by the time you are in the filter there's no way to change the destination page because the response is already committed.
So the user sees the success page when in reality the transaction was rollbacked. To avoid this I have to write a lot of data integrity checks in Java in order to prevent all integrity exceptions, because I could not handle them correctly. This is bad because I'm doing the work instead of leaving it to the database (or maybe I'm wrong and I always have to write all this data integrity code in my app?).
So I've found the IHttpModule interface which I'm guessing is pretty much the same concept as a javax.servlet.Filter (correct me if I'm wrong), so I'm guessing I could have the same problem again.
Where should I put my commits in order to make sure that my transactions are atomic, and when they throw exceptions I can capture them and change my destination page and give the user a comprehensive message?
So far the only possible solution I've come up with is to keep my IHttpModule to start and close the transaction, and put the commit calls in the last line of my controllers methods, thus being able to capture exceptions there and then return an appropiate view with the message. Now I would have to copy those commit and exception handling lines into all my controller methods that require commits. And there is the separation of concerns issue, that my controllers have to know about DB, which I don't like at all.
Is there a better way?
If you're using ASP.NET MVC, you could use an ActionFilter to achieve the same effect.
Something like (this is hacked together from difference pieces of my architecture):
public class TransactionalAttribute : ActionFilterAttribute, IAuthorizationFilter, IExceptionFilter
{
ITransaction transaction = NullTransaction.Instance;
public IsolationLevel IsolationLevel { get; set; }
public TransactionalAttribute()
{
IsolationLevel = IsolationLevel.ReadCommitted;
}
public override void OnResultExecuted(ResultExecutedContext filterContext)
{
try
{
transaction.Commit();
transaction = NullTransaction.Instance;
}
catch (Exception exception)
{
Log.For(this).FatalFormat("Problem trying to commit transaction {0}", exception);
}
}
public override void OnActionExecuting(ActionExecutingContext filterContext)
{
if (transaction == NullTransaction.Instance) transaction = UnitOfWork.Current.BeginTransaction(IsolationLevel);
}
public override void OnActionExecuted(ActionExecutedContext filterContext)
{
if (filterContext.Result != null) return;
transaction.Commit();
transaction = NullTransaction.Instance;
}
public void OnAuthorization(AuthorizationContext filterContext)
{
transaction = UnitOfWork.Current.BeginTransaction(IsolationLevel);
}
public void OnException(ExceptionContext filterContext)
{
try
{
transaction.Rollback();
transaction = NullTransaction.Instance;
}
catch (Exception exception)
{
Log.For(this).FatalFormat("Problem trying to rollback transaction {0}", exception);
}
}
private class NullTransaction : ITransaction
{
public static ITransaction Instance { get { return Singleton<NullTransaction>.Instance; } }
public void Dispose()
{
}
public void Commit()
{
}
public void Rollback()
{
}
}
}
Well after thinking about it and discussed it with coworkers I've come up with a solution that meets almost all my requirements.
I implemented the solution with my Java projects and it worked great. I'll just pust the idea so everybody can use it within any framework.
The solution consist in putting the commit call in the last line of the controller method, inside a try-catch block. If a constraint exception occurs you can get the name of the violated constraint. With the name you can tell the user exactly what went wrong. I used a properties file to store the message to show to the user wich constraint was violated. The keys of the properties file are the constraints names and the values are the constraint violation messages.
Yo can refactor the commit-handle_exception-find_constraint_message to a method, that's what I did.
For now it solves my problem of writing code to check database integrity and I believe it's pretty elegant with the constraint violation messages in a properties file. Now, I still don't like the idea that my controllers need to call the commit, but that's way better than writing integrity checks that the database already does.
I will continue to use a filter just like David Kemp said, just that the filter will only open the (n)hibernate session and the transaction, and then, at the end of the request, close the session.
Comments are more than welcome. Thanks.