I'm implementing small service using asyncio using a loop that is structured as follows:
pending = {...}
while True:
done, pending = yield from asyncio.wait(
pending,
return_when=asyncio.FIRST_COMPLETED,
)
for future in done:
if future is x:
# ...
if future is y:
# ...
This loop is currently controlling a sub-process, but having written a bunch of ZMQ-based services, this style feels very natural, so I'll likely be writing more of these in the near future.
I have something that runs just the way I like, but I'm kind of at a loss as to how I would write automated tests for this.
I would like to have my tests start this loop until it blocks on asyncio.wait() and inject one particular event to test the loop's handling of that particular event. That way, I can test each possible event handling and know I cover all cases as expected.
However, I can't find anything in asyncio that provides for this. If I simply yield from this coroutine, the test does not unblock until the coroutine completes.
Any ideas on how to test this particular kind of loop?
Edit: OK, so I managed to get something running using a socketpair() by patching sys.stdout with the write end and wrapping the other end in a StreamReader.
This works when I run pytest without capture, but as soon as remove the -s argument, the test seems to deadlock.
Any ideas?
Related
I've got a long-running loop which involves a fair amount of UI functions. This loop therefore must be run on the main thread. However, I also want to display progress of this task, so this must also run on the main thread as displaying the current progress would involve updating the UI. I am really struggling to find a way of allowing the UI to update with current progress on the main thread when the main loop is also running on the main thread. What happens is that the UI is frozen during the loop and then updates to show that the process is finished when it's done.
This is not for a production app, it's for a personal project that will never be release. So it is of no concern that the UI is frozen from a UX perspective. If the solution involves putting the processing in the background then this refactoring is fine, but I'm not sure how to do it when a lot of the heavy lifting during this loop involves UI stuff too.
Isn't it funny how you sometimes come up with a solution just after posting the question?! The key seemed to be rather than using a for loop for the processing, instead putting the processing function inside a separate method and repeatedly calling it, passing the array of objects to process to it. Doing this, you can call the function using [self performSelector:withObject:afterDelay:]. Even if you provide a value of zero for the delay, it causes the method to be called on the next run loop. This means you can update the UI, process the next item, and repeat this process until the array of items is empty. Here's my completed solution. If anybody knows a better way I'd still love to hear it, but for now this is at least working!
Edit - I packaged this solution up into a class of its own to make it easier to manage, and put it on my Github. Maybe it will help somebody else out :)
Edit 2 - made the processing class more flexible by making it run loops instead of iterating through arrays. You can of course use it to iterate through an array yourself, as per the example in the readme. But if you're not working with an array, you can just run the loop runCount times and do whatever you need to do in the processingBlock.
https://github.com/mashers/BackgroundLoopProcessor
I would like to create a wxPython app such that:
If I run a second instance of that app (e.g., call the Python script from the shell a second time), no new instance should be created.
Instead, the toplevel frame of the already running instance should be raised and focussed.
The first point can be easily implemented by wx.SingleInstanceChecker (see the example code there), but at least the example code only gives a way for making the second instance of the app abort, but not raise the existing app's main frame.
I am using wxPython-Phoenix with Python 3.
Claritication: I would much prefer an out-of-the-box solution like wx.SingleInstanceChecker (that is, not implement my own locking and IPC solution).
You can use any kind of IPC to send a message asking the other program to do whatever needs to be done (just raise its top level window or maybe handle the command line options passed to the second instance). In C++ there are wxConnection and the related wxServer and wxClient classes that can be used for this, but I'm not sure if they're wrapped by wxPython -- however you could use any Python IPC module instead, if they aren't.
As has been pointed out, the "correct" way to do this is IPC because you have a new process that is supposed to affect a change (raise and focus) in another process.
What you seem to want is to take advantage of the IPC channel that wx.SingleInstanceChecker is already using to do your work. Unfortunately, you can't. That class is implemented in the wxWidgets c++ code and therefore there are no Python bindings to the internal workings of the class.
However, you can probably abuse wx.SingleInstanceChecker to do what you want. In your program, you can set up a timer at some rapid interval (say, 250ms) that will constantly check IsAnotherRunning() from your main process. Therefore, when your second process starts up, the first will notice and can raise itself to the front. You would just have to wait for a little bit in the secondary process before it exits, to give the first time to notice.
I needed to provide a delay between the two http calls, so i have created a java program and invoked the script between the two http calls in the scenario outline, But the java program is being executed in the background. Could some help in this.
I strongly discourage you from depending on Java code like this - because you will not be able to get the benefits of Karate's unique approach.
Here's how you can use Java seamlessly from within Karate itself:
* def sleep = function(millis){ java.lang.Thread.sleep(millis) }
* print 'sleeping...'
* call sleep 5000
* print 'resumed'
So once you have the function defined in say the Background, you can re-use it easily, that too with different values.
It also sounds to me that you might be better off using polling, so do look at this also as a reference: polling.feature
I have a custom MSBuild task for xUnit.net. When the task is running, if I hit Ctrl+C, it 'tries' to cancel the task, but of course it fails (since my task doesn't support cancelation). No amount of MSDN doc searchs or Google-fu have landed on a solution. Since I can't find an obvious interface to implement, I'm guessing maybe cancelation is supported by way of some convention.
Has anybody done this before, and knows what's required to get cancelation to work?
Your task needs to implement ICancelableTask. It's a very simple interface added in 4.0.
Basically you just add a Cancel() method. It must be ready to be called on a different thread, at any time, and return promptly. Your task must then return from Execute promptly. Typically you'd set a boolean flag inside Cancel(). Then inside your task you'd typically have a loop processing each input in turn -- for example, copying one file after another -- and in each iteration, check the flag; if it's true, break out. It doesn't matter whether you return true or false from Execute in this context.
If you're deriving from ToolTask -- if your task spawns a tool, it's very strongly recommended that you do this, as it saves a great deal of code, handles async logging, and other things -- then it already handles Cancel automatically. When Cancel happens, it kills the tool it spawned and all its children. The C++ team's tasks in some cases override this default behavior, so that their compiler/linker has a few seconds to clean up their half-written outputs before returning.
(Trivia: when I first implemented this in MSBuild, I accidentally made VS bluescreen the box occasionally. This nearly shipped in VS10 beta but was discovered just in time. The bluescreen was because the logic for figuring out the process tree was wrong, and would sometimes kill a system process. Oops.)
Dan
I know you're well aware of the Task hierarchy, but on the offchance this is what you're looking for and it's just the fact that you're not implementing a ToolTask...
Inside MSBuild 2nd ed says (p118) of ToolTask.Cancel
This method is called to cancel the task execution. Once this method is called by MSBuild, if the task does not complete, it will be forcefully terminated
There are no other references to cancellation in it.
I have written a task script using vb.net that have thread used in the code, the problem is how i can know when will be finished all the threads so i can return the success result.
Thanks alot.
i think you need to use a waitHandle object and the waitAll method
more info here: http://msdn.microsoft.com/en-us/library/system.threading.waithandle.aspx
That being said, I suspect you can refactor the design of your package to let the script task handle the execution, and let SSIS handle the execution scheduling. this gives you the parallelism you want without any of the hassle of multi threaded programming in .net.
a simple setup would be n foreach loops (which execute in serial) each running a partitioned chunk of the work load.
Another simpler option is have the package driven by variables and spawn multiple executions of the package. This could occur across 1-N servers to scale out.