I am creating tasks by inheriting from Greenlet. I have a single parent task that calls start() on two children in its _run(). Elsewhere (it happens to be a systemd service) start() and join() are called.
The behavior seems correct. For example the use of a Queue with timeouts achieves the desired effect but I haven't found a good way to shutdown the children from say KeyboardInterrupt or by registering a callback to the parent task for SIGTERM. In the handler I would call child1.kill() and 'child2.kill()but only the first called seemed to raiseGreenletExit`.
I never call join() on the children and I'm not sure how I would do this properly. Am I misusing the library?
My error was that I was handling gevent.greenlet.GreenletExit in the child tasks. If you need to handle the exit you can catch and reraise this exception.
Related
From the description in the pika documentation, I can't quite get what add_callback_threadsafe() method does. It says, "Requests a call to the given function as soon as possible in the context of this connection’s thread". Specifically, which event does this callback get associated to? Since, add_callback_threadsafe() doesn't receive any "event" argument, how does pika know when to invoke that callback?
Also, in the official example, why do we even need to build the partial function and register the callback in the do_work() method? Could we not just call the ack_message() method after the time.sleep() is over?
The reason why the method exists:
How to add multiprocessing to consumer with pika (RabbitMQ) in python
If you use the same rabbit connection/channel with multiple threads/processes then you'll be prone to crashes. If you want to avoid that, you need to use that method.
Could we not just call the ack_message() method after the
time.sleep() is over?
No, you would be calling ack_message from a different thread than the main one, that's not thread safe and prone to crashes. You need to call ack_message from a thread-safe context, i.e., the add_callback_threadsafe().
I'm writing a scheduler. It has a single form frmMain, which shows jobs that are currently running, and a history of job steps that have run. It has an object of class Scheduler that manages running new jobs. Scheduler keeps a collection class, List which contains objects of class RunningJob. RunningJob executes each step in turn through a series of sub-classes.
When a job is started, the Scheduler creates a new BackgroundWorker with the DoWork, ProgressChanged and RunWorkerCompleted methods setup with handlers that point back into the instance of RunningJob.
Each time a job/step starts/ends, one of these handers in RunningJob raises an appropriate event into Scheduler and Scheduler raises an appropriate event into frmMain. i.e.:
frmMain (1 instance) <---- Scheduler (1 instance) <---- RunningJob.WorkerProgressChanged (many instances)
The RunningJob executes correctly, but the reporting going up to the interface is not working correctly. Also any logging to files I do is suspect (I'm using a single function: LogInfo to do this). I have a number of questions:
When I use InvokeRequired() and Invoke() within frmMain, do I have to do this with every single control I want to update (there are several). Can I just check InvokeRequired() on one control and use Invoke on all of them based on that result.
Why bother checking InvokeRequired() at all and just use Invoke() every single time? It will make for simpler code.
There is only one instance of Scheduler and I am raising events to get execution back into it from each Job. I think this is part of the problem. How is multithreading handled doing this? Is there some sort of InvokeRequired/Invoke check I can do on the events before raising them? Can I raise events at all in this situation? I like events, rather than calling methods on the owner class, because it improves encapsulation. What is best practice here?
In general, if I'm calling a piece of code from many different threads, not necessarily to update a form, but just to perform some function (e.g. add a line of text to a file for logging purposes), how do I block one thread until the other has completed?
This query is with respect to the example provided in hiredis
Can event_base_dispatch(base) be called from a different thread by creating pthread_create()?
It is a fact that event_base_dispatch() is a loop and it is a blocking call. My idea here is to send all my redis command from the parent thread by invoking redisAsyncCommand(), event base will be run in the other thread.
you need to use add enable_thread_safe_windows function before you create a event base.
During org.mule.component.BindingInvocationHandler invoke method Mule is trying to get the Current Event from RequestContext.getEvent();
But the value is coming as Null and we are getting NullPointerException.
What could be the reason that sets CurrentEvent of RequestContext to null?
Update: We are using java.util.concurrent.ExecutorService to invoke a method bound by BindingInvocationHandler.
RequestContext.getEvent() uses ThreadLocal to find out the in-flight event so maybe you're calling it from within a thread that is not the one that processes the MuleEvent?
If that's the case, you can try cloning the event then passing it to your thread and re-establish it as the current event with RequestContext.setEvent(xxx).
Expect turbulence as this is not small feat, though Mule does this internally.
Use the newThreadCopy() on the event to get a copy that can be processed by another Mule thread without throwing an exception.
I've been busy updating my brain with the TPL because I intend on using it for my new application (which uses .Net Framework 4.0). However I have some doubts that someone might clarify for me. Previously, I had a progress form which I would launch from the main (GUI) thread after I started the thread which needed to display its' progress. It looked something like this:
sortThread = New Thread(AddressOf _Sorter.Sort())
_ProgressForm = New FrmProgress()
_Sorter.ProgressForm = _ProgressForm
sortThread.Start()
progressForm.ShowDialog()
Basically it would initialize the thread, initialize a FrmProgress form object and assign it to the Sorter object which would then update the progress form (which contained a progress bar and some labels) from its Sort() sub on the separate thread. Updating these control properties was achieved by checking the InvokeRequired property of the FrmProgress form and if needed it would then use the Invoke() method of the control that was to be updated... ex:
Public Sub IncrementProgressBar(x As Integer)
If Me.InvokeRequired Then
pb_MainProgressBar.Invoke(Sub() IncrementProgressBar(x))
Else
pb_MainProgressBar.Increment(x)
End If
End Sub
Now I am interested in using TPL to launch separate worker threads (multiple) that may want to update the progress bar. Should I use the same pattern or should I consider accessing a public TaskScheduler.FromCurrentSynchronizationContext context that was obtained in the main GUI thread? In both cases I suppose I should provide some kind of locking mechanism on the form (SyncLock?)
Invoke should be sufficient, as you are doing. If two different threads try to invoke in parallel the first one will execute first, then the second when the UI thread becomes free. The UI thread cannot service two invokes simultaneously - they are naturally handled in FIFO sequence so there is no issue with thread safety. Any number of threads can invoke on the main thread without worrying about each other or using any additional locking mechanism.
Note, however, that any thread calling Invoke will block until the main thread can service the call. If you, for example, had many threads invoking heavy code at the same time then your various threads would block on the invoke calls until they got their kick at the can, so to speak. If you use BeginInvoke then the calling thread will simply continue executing and the invoked method will be placed in the UI thread's queue (which it will service as soon as it can).