I'm making an API and have a function which takes a task and runs it. When the task is finished successfully, it's status is set to 'Completed'. Now, lets say the user of the API accidentally (or for whatever reason) sends that same task (or any other already completed task) back into the same function. What should the API do?
Throw an exception
Pretend as if I've rerun the task and tell the user (through events or whatever) that it is done/completed (again).
Do nothing and just ignore it.
Is there a standard or best practice for something like this?
Pretending to rerun hides what's probably a user error - this can lead to deadlocks or other logic bugs (i.e. - I create an event, wait on it and run a task that should reset it at some point - it never happens, deadlock). Also done handlers may fail if invoked twice per one successful task run.
Doing nothing is more or less the same - done handlers can't fail now :), but they are not invoked at all - a bug is more probable if done handler performed necessary communication with the spawning thread.
The worst thing is - these may happen or not happen, depending on the timing. I.e. the task may still be running by the time the user calls the function the second time (what do you do then, by the way?)
So, do throw an exception unless task status is "not started". The user can always check the status and perform the necessary processing in the unlikely case she needs it.
Related
I’m currently modeling a process with 2 exception statuses (a patient dies & No Neurologist found).
If no Neurologist is found (this can only happen once in my process), the process stops.
Another exception status is triggered when a patient dies at any point during the process. If this exception status occurs, the process stops.
I have difficulties modeling these exception statuses. Attached you can find my current attempt. I’m not 100% sure it is correct.
Example of my attempt
Terminating events are rarely needed. There are usually more elegant, clearer solution than this 'kill all switch'. Their purpose is to terminate any parallel activities / consume any tokes which exist in the same scope. The same can usually be achieved with interrupting (e.g. conditional) boundary events, which get triggered e.g. by a data change. A boundary event makes it clearly visible in the process where a cancellation can occur, under which circumstances, and allows ending a process in more controlled manner.
In your particular use case (diagram you attached) you don't need to use the terminating events at all. You are using two interrupting boundary events (escalation and error) on a scope created by the embedded sub process. The scope of the embedded sub process is already terminated when these events interrupting occur. A subsequent terminating event in the parent process' scope would cancel everything in this scope. In your case the parent scope is the root process instance, but since there is no token flow parallel to the embedded sub process, there is nothing to cancel.
Also see:
https://docs.camunda.org/manual/latest/reference/bpmn20/events/terminate-event/
https://docs.camunda.org/manual/latest/reference/bpmn20/events/error-events/#error-boundary-event
Scenario: I have a web service with a "Delete" ASP.NET Core controller action. The implementation consists of two steps: one cheap after which other operations can already no longer see the deleted data, and a long-running second step which performs the actual deletion.
Is it okay to use Task.Run for the second operation and never await it and so return to the caller very soon? I know that this task could be lost due to application shutdown, but is it even allowed to offload work to the background using Task.Run in ASP.NET Core and never await the resulting task?
I know that this task could be lost due to application shutdown, but is it even allowed to offload work to the background using Task.Run in ASP.NET Core and never await the resulting task?
Oh, sure, nothing will prevent this. .NET and ASP.NET allow you to use Task.Run, and the response will be returned to the user without delay (assuming you ignore the task returned from Task.Run).
Is it okay to use Task.Run for the second operation and never await it and so return to the caller very soon?
You say that you're aware that the task could be lost in an app shutdown. And that's true; so it is entirely possible that the second deletion may never happen, and there will be no error or log message informing you of the fact. Furthermore, since the task is ignored, there will be no notification if that task fails, so if there's something wrong with the deletion code, no logs or anything will notify you of regular failures. If you're OK with that, then using fire-and-forget is fine.
On ASP.NET Core, you don't even need Task.Run; you can just call the (asynchronous) method that does the second delete operation and then ignore the task instead of awaiting it.
Simply ignoring tasks will work (with or without Task.Run), but this does have the negative side effect that ASP.NET isn't aware of the offloaded work at all. So I would recommend registering the task and having something delay the app shutdown until the tasks have a chance to complete. With registration, it is still possible to lose tasks, but registering them reduces the chance of losing tasks.
I have a program where I start several process instances using a cron. For each process instance I have a maximum time, and if the execution time exceeds it, I have to consider it as failure and use some specific methods.
For now what I did was simply to check, once my process instance has finished, if the elapsed time exceeds or not the given maximum time.
But what if my process instance gets blocked for some reason (e.g. server not responding)? I need to catch this event and perform failure operations as soon as the process gets blocked and timeout is exceeded.
How can I catch these two conditions?
I had a look at the FlowableEngineEventType, but there isn’t a PROCESS_BLOCKED/SUSPENDED type of event. But, even if it were, how do I fire it only if a certain amount of time has passed?
I assume that this is the same question as this from the Flowable Forum.
If you are using the Flowable HTTP Task then have a look at the documentation to see how you can set the timeouts on it and how you can react on errors there. If you are firing GET requests from your own code you would need to write your own business logic that would throw some kind of BpmnError and you would then handle that in your process.
The Flowable Process instance does not have the concept of being blocked, and you have to manually to that in your modelling.
I have a task that can be started by the user, that could take hours to run, and where there's a reasonable chance that the user will start the task multiple times during a run.
I've broken the processing of the task up into smaller batches, but the way the data looks it's very difficult to tell what's still to be processed. I batch it using messages that each process a bite sized chunk of the data.
I have thought of using a Saga to control access to starting this process, with a Saga property called Processing that I set at the start of the handler and then unset at the end of the handler. The handler does some work and sends the messages to process the data. I check the value at the start of the handler, and if it's set, then just return.
I'm using Azure storage for Saga storage, if it makes a difference for the next bit. I'm also using NSB 6
I have a few questions though:
Is this the correct approach to re-entrancy with NSB?
When is a change to Saga data persisted? (and is it different depending on the transport?)
Following on from the above, if I set a Saga value in a handler, wait a while and then reset it to its original value will it change the persistent storage at all?
Seem to be cross posted in the Particular Software google group:
https://groups.google.com/forum/#!topic/particularsoftware/p-qD5merxZQ
Sagas are very often used for such patterns. The saga instance would track progress and guard that the (sub)tasks aren't invoked multiple times but could also take actions if the expected task(s) didn't complete or is/are over time.
The saga instance data is stored after processing the message and not when updating any of the saga data properties. The logic you described would not work.
The correct way would be having a saga that orchestrates your process and having regular handlers that do the actual work.
In the saga handle method that creates the saga check if the saga was already created or already the 'busy' status and if it does not have this status send a message to do some work. This will guard that the task is only initiated once and after that the saga is stored.
The handler can now do the actual task, when it completes it can do a 'Reply' back to the saga
When the saga receives the reply it can now start any other follow up task or raise an event and it can also 'complete'.
Optimistic concurrency control and batched sends
If two message are received that create/update the same saga instance only the first writer wins. The other will fail because of optimistic concurrency control.
However, if these messages are not processed in parallel but sequential both fail unless the saga checks if the saga instance is already initialized.
The following sample demonstrates this: https://github.com/ramonsmits/docs.particular.net/tree/azure-storage-saga-optimistic-concurrency-control/samples/azure/storage-persistence/ASP_1
The client sends two identical message bodies. The saga is launched and only 1 message succeeds due to optimistic concurrency control.
Due to retries eventually the second copy will be processed to but the saga checks the saga data for a field that it knows would normally be initialized by by a message that 'starts' the saga. If that field is already initialized it assumes the message is already processed and just returns:
It also demonstrates batches sends. Messages are not immediately send until the all handlers/sagas are completed.
Saga design
The following video might help you with designing your sagas and understand the various patterns:
Integration Patterns with NServiceBus: https://www.youtube.com/watch?v=BK8JPp8prXc
Keep in mind that Azure Storage isn't transactional and does not provide locking, it is only atomic. Any work you do within a handler or saga can potentially be invoked more than once and if you use non-transactional resources then make sure that logic is idempotent.
So after a lot of testing
I don't believe that this is the right approach.
As Archer says, you can manipulate the saga data properties as much as you like, they are only saved at the end of the handler.
So if the saga receives two simultaneous messages the check for Processing will pass both times and I'll have two processes running (and in my case processing the same data twice).
The saga within a saga faces a similar problem too.
What I believe will work (and has done during my PoC testing) is using a database unique index to help out. I'm using entity framework and azure sql, so database access is not contained within the handler's transaction (this is the important difference between the database and the saga data). The database will also operate across all instances of the endpoint and generally seems like a good solution.
The table that I'm using has each of the columns that make up the saga 'id', and there is a unique index on them.
At the beginning of the handler I retrieve a row from the database. If there is a row, the handler returns (in my case this is okay, in others you could throw an exception to get the handler to run again). The first thing that the handler does (before any work, although I'm not 100% sure that it matters) is to write a row to the table. If the write fails (probably because of the unique constraint being violated) the exception puts the message back on the queue. It doesn't really matter why the database write fails, as NSB will handle it.
Then the handler does the work.
Then remove the row.
Of course there is a chance that something happens during processing of the work, so I'm also using a timestamp and another process to reset it if it's busy for too long. (still need to define 'too long' though :) )
Maybe this can help someone with a similar problem.
So I am running into a race condition and I have a few solutions on how to fix the issue. I am new to threading so obviously, my opinion and research is limited. I have a large amount of asynchronization calls that can happen if a user receives certain messages from server. Thus, my design is poor due to the dependent nature of my objects.
Lets say I have a function called
adduser:(NSString s){
does some asynchronize activity
}
Messageuser:(NSString s)
{
Does some more asychronize activity
}
if a user were to recieve a message telling it to addUser "Ryan". he would than create a thread and proceed with looking up Ryan and storing him. However, if the user has the application in suspended mode, and in the buffered of messages waiting to be recieved there is a addUser request and a MessageUser request, a race condition occures because it takes longer to complete Adduser than it does to complete MessageUser. Thus, If messageUser is called , and (in our example) "Ryan" has not been fully added, it throws an error.
What would be a possible solution to this issue. I looked into locks and semaphores, and what I am trying to do is, when MessageUser recieves a call, check to make sure there is no thread currently proccessing addUser. If there is none, proceed. Else wait, than proceed after it has finished.
Well it depends on how the messages are being issued in the first place and what the async response events are.
If the operations have dependencies (ordering requirements) then perhaps a background serial queue would be appropriate? That is a simple way to ensure the messages are processed in order.
If the async operations take completion blocks, then you could have the completion block issue the request for the next operation to be performed, though you may not know about that ahead of time.
If you need to solve this in a more general way then you need some kind of system for tracking prerequisites so you can skip work items that don't have their prerequisites met yet. That probably means your own background thread that monitors a list of waiting tasks and receives notification of all task completions so it can scan for items waiting on that completion and issue them.
It seems really complicated though... I suspect you don't really have such strong async parallel processing requirements and a much simpler design would be just as effective. Given your situation where you are receiving messages from a server, I think a serial queue would be the best option. Then you can process messages in the order the server sent them and keep things simple.
//do this once at app startup
dispatch_queue_t queue = dispatch_queue_create("com.example.myapp", NULL);
//handle server responses
dispatch_async(queue, ^{
//handle server message here, one at a time
});
In reality, depending on how you connect to your server you might be able to just move the entire connection handling to the background queue and communicate with it via messages from the UI, and update the UI by dispatching to the dispatch_get_main_queue() which will be the UI thread.