I am doing some development inside a long running screen transaction and it is timing out with:
--- 1761034 [-task-scheduler] WARN bitronix.tm.BitronixTransaction
transaction timed out: a Bitronix Transaction with GTRID [3132372E302E302E31000000000EE89002000002A5], status=MARKED_ROLLBACK, 1 resource(s) enlisted
The process keeps running, but it seems to be rolling back a database create action though I don't see a rollback message anywhere. I am not using a service call anywhere.
I thought I would try to lengthen the transition time out value, but I don't see how to do it. It is a screen transaction that is timing out, right?
I will be moving the code to a service, but won't I still have the same timeout problem?
By default the screen.#begin-transaction attribute is set to false and no transaction is begun when rendering the screen. On a screen transition element the begin-transaction attribute defaults to true, as those are normally used to process input.
If the logic is in a service you'll have more control over timeouts. Services are where the heavy-lifting is meant to be done. The transactions in screens are just meant for basic transaction needs.
Related
I have a program where I start several process instances using a cron. For each process instance I have a maximum time, and if the execution time exceeds it, I have to consider it as failure and use some specific methods.
For now what I did was simply to check, once my process instance has finished, if the elapsed time exceeds or not the given maximum time.
But what if my process instance gets blocked for some reason (e.g. server not responding)? I need to catch this event and perform failure operations as soon as the process gets blocked and timeout is exceeded.
How can I catch these two conditions?
I had a look at the FlowableEngineEventType, but there isn’t a PROCESS_BLOCKED/SUSPENDED type of event. But, even if it were, how do I fire it only if a certain amount of time has passed?
I assume that this is the same question as this from the Flowable Forum.
If you are using the Flowable HTTP Task then have a look at the documentation to see how you can set the timeouts on it and how you can react on errors there. If you are firing GET requests from your own code you would need to write your own business logic that would throw some kind of BpmnError and you would then handle that in your process.
The Flowable Process instance does not have the concept of being blocked, and you have to manually to that in your modelling.
I have an NServiceBus handler that creates a new sql connection and new sql command.
However, the command that is executed is not being committed to the database until after the whole process is finished.
It's like there is a hidden sql transaction in the handler itself.
I moved my code into a custom console application without nservicebus and the sql command executed and saved immediately. Unlike in nservicebus where it doesn't save until the end of the handler.
Indeed every handler is wrapped in a transaction, the default transaction guarantee is relying on DTC. That is intentional :)
If you disable it then you might get duplicate messages or lose some data, so that must be done carefully. You can disable transactions using endpoint configuration API instead of using options in connection string.
Here you can find more information about configuration and available guarantees http://docs.particular.net/nservicebus/transports/transactions.
Unit of work
Messages should be processed as a single unit of work. Either everything succeeds or fails.
If you want to have multiple units of work executed then
create multiple endpoints
or send multiple messages
This also has the benefit that these can potentially be processed in parallel.
Please note, that creating multiple handlers WILL NOT have this effect. All handlers on the same endpoint will be part of the same unit of work thus transaction.
Immediate dispatch
If you really want to send a specific message when the sending of the message must not be part of the unit of work then you can immediately send it like this:
using (new TransactionScope(TransactionScopeOption.Suppress))
{
var myMessage = new MyMessage();
bus.Send(myMessage);
}
This is valid for V5, for other versions its best to look at the documentation:
http://docs.particular.net/nservicebus/messaging/send-a-message#dispatching-a-message-immediately
Enlist=false
This is a workaround that MUST NOT be used to circumvent a specific transactional configuration as is explained very well by Tomasz.
This can result in data corruption because the same messsage can be processed multiple times in case of error recovery while then the same database action will be performed again.
Found the solution.
In my connection string I had to add Enlist=False
As mentioned by #wlabaj Setting Enlist=False will indeed make sure that a transaction opened in the handler will be different from transaction used by the transport to receive/send messages.
It is however important to note that it changes the message processing semantics. By default, when DTC is used, receive/send and any transactional operations inside a handler will be commited/rolled-back atomically. With Enlist=False it's not the case so it's possible that there will be more than one handler transaction being committed for the same message. Consider following scenario as a sample case when it can happen:
message is received (transport transaction gets started)
message is successfully processed inside the handler (handler transaction committed successfully)
transport transaction fails and message is moved back to the input queue
message is received second time
message is successfully processed inside the handler
...
The behavior with Enlist-False setting is something that might a be desirable behavior in your case. That being said I think it's worth clarifying what are the consequences in terms of message processing semantics.
I've got a JMS messaging system implemented with two queues. One is used as a standard queue second is an error queue.
This system was implemented to handle database concurrency in my application. Basically, there are users and users have assets. One user can interact with another user and as a result of this interaction their assets can change. One user can interact with single user at once, so they cannot start another interaction before the first one finishes. However, one user can be in interaction with other users multiple times [as long as they started the interaction].
What I did was: crated an "interaction registry" in redis, where I store the ID of users who begin an interaction. During interaction I gather all changes that should be made to the second user's assets, and after interaction is finished I send those changes to the queue [user who has started the interaction is saved within the original transaction]. After the interaction is finished I clear the ID from registry in redis.
Listener of my queue will receive a message with information about changes to the user that need to be done. Listener will get all objects which require a change from the database and update it. Listener will check before each update if there is an interaction started by the user being updated. If there is - listener will rollback the transaction and put the message back on the queue. However, if there's something else wrong, message will be put on to the error queue and will be retried several times before it is logged and marked as failed. Phew.
Now I'm at the point where I need to create a proper integration test, so that I make sure no future changes will screw this up.
Positive testing is easy, unfortunately I have to test scenarios, where during updates there's an OptimisticLockFailureException, my own UserInteractingException & some other exceptions [catch (Exception e) that is].
I can simulate my UserInteractingException by creating a payload with hundreds of objects to be updated by the listener and changing one of it in the test. Same thing with OptimisticLockFailureException. But I have no idea how to simulate something else [I can't even think of what could it be].
Also, this testing scenario based on a fluke [well, chance that presented scenario will not trigger an error is very low] is not something I like. I would like to have something more concrete.
Is there any other, good, way to test this scenarios?
Thanks
I did as I described in the original question and it seems to work fine.
Any delays I can test with camel.
I'm making an API and have a function which takes a task and runs it. When the task is finished successfully, it's status is set to 'Completed'. Now, lets say the user of the API accidentally (or for whatever reason) sends that same task (or any other already completed task) back into the same function. What should the API do?
Throw an exception
Pretend as if I've rerun the task and tell the user (through events or whatever) that it is done/completed (again).
Do nothing and just ignore it.
Is there a standard or best practice for something like this?
Pretending to rerun hides what's probably a user error - this can lead to deadlocks or other logic bugs (i.e. - I create an event, wait on it and run a task that should reset it at some point - it never happens, deadlock). Also done handlers may fail if invoked twice per one successful task run.
Doing nothing is more or less the same - done handlers can't fail now :), but they are not invoked at all - a bug is more probable if done handler performed necessary communication with the spawning thread.
The worst thing is - these may happen or not happen, depending on the timing. I.e. the task may still be running by the time the user calls the function the second time (what do you do then, by the way?)
So, do throw an exception unless task status is "not started". The user can always check the status and perform the necessary processing in the unlikely case she needs it.
I have an endpoint that has a message handler which does some FTP work.
Since the ftp process can take some time, I encapsulated the ftp method within a TransactionScope with TransactionScopeOption.Suppress to prevent the transaction timeout exceptions.
Doing this got rid of the timeout exceptions, however the handler was fired 5 times
(retries is set to 5 in my config file)
The files were ftp'd ok, but they were just ftp'd 5 times.
the handler look like it is re-fired after 10 or 11 minutes.
some test code looks as follows:
public void Handle(FtpMessage msg)
{
using (TransactionScope t = new TransactionScope(TransactionScopeOption.Suppress))
{
FtpFile(msg);
}
}
Any help would be greatly appreciated.
thanks.
If this truly is an FTP communication that cannot be completed within the transaction timeout, another approach would be to turn this into a Saga.
The Saga would be started by FtpMessage, and in the handler it would start the FTP work asynchronously, whether in another thread, or via another process, or whatever, and store enough information in saga data to be able to look up the progress later.
Then it would request a timeout from the TimeoutManager for however long makes sense. Upon receiving that timeout, it would look up the state in saga data, and check on the FTP work in progress. If it's completed, mark the saga as complete, if not, request another timeout.
Alternatively, you could have a process wrapping the FTP communication that hosts its own Bus but does not have any message handlers of its own. It could receive its FTP information via the command line (including requesting endpoint), do its work, and then send a message back to the requesting endpoint saying it is complete. Then you would not have to wait for a timeout to move on with the process.
I'd recommend configuring that endpoint as non-transactional rather than trying to suppress the transaction. Do that by including .IsTransactional(false) in your initialization code if self-hosting or by implementing IConfigureThisEndpoint, AsA_Client when using the generic host.
My guess is that by not completing the inner scope you're causing the outer scope, created by NSB, to rollback. This will cause NSB to retry your FtpMessage.
Try to add: t.Complete(); after your call to FtpFile and see if that does it for you.
Edit: After rereading your question I realized that this won't solve your timeout issue. Have you tried to increase the timeout? (10 min is the default maxValue in machine.config so you can't set it to higher without modifying machine.config)