I have a program I'm writing in vb.net that has ballooned into the most complicated thing I've ever written. Because of some complex math and image rendering that's happening constantly I've been delving into multithreading for the first time to improve overall performance. Things have honestly been running really smoothly, but we've just added more functionality that's causing me some trouble.
The new functionality comes from a pair of DLLs that are each processing a video stream from a USB camera and looking for moving objects. When I start my program I initiate the DLLs and they start viewing the cameras and processing the videos. I then periodically ping them to see if they have detected anything. This is how I start and stop them:
Declare Function StartLeftCameraDetection Lib "DetectorLibLeft.dll" Alias "StartCameraDetection" () As Integer
Declare Function StopLeftCameraDetection Lib "DetectorLibLeft.dll" Alias "StopCameraDetection" () As Integer
When I need to check if they've found any objects I use several functions like this:
Declare Function LeftDetectedObjectLeft Lib "DetectorLibLeft.dll" Alias "DetectedObjectLeft" () As Integer
All of that works really well. The problem is, I've started to notice some significant lag in my UI and I'm thinking it may be coming from the DLLs. Forgive my ignorance on this, but as I said I'm new to using multiple threads (and incorporating DLLs too if I'm honest). It seems to me that when I start a DLL it running it's background tasks on my main thread and just waiting for me to ping it for information. Is that the case? If so, is it possible to have the DLL running on a sperate thread so it doesn't affect my UI?
I've tried a few different things but I can't seem to address the lag. I moved the code that pings the DLL and processes whatever information it gets into a sperate thread, but that hasn't made any difference. I also tried calling StartLeftCameraDetection from a separate thread but that didn't seem to help either. Again, I'm guessing that's because the real culprit is the DLL itself running these constant background tasks on my main thread no what thread I actually call it's functions from.
Thanks in advance for any help you might be able to offer!
There's a lot to grok when it comes to threading, but I'll try to write a concise summary that hits the high points with enough details to cover what you need to know.
Multi-threaded synchronization is hard, so you should try to avoid it as much as possible. That doesn't mean avoiding multi-threading at all, it just means avoiding doing much more than sending a self-contained task off to a thread to run to completion and getting the results back when it's done.
Recognizing that multi-threaded synchronization is hard, it's even worse when it involves UI elements. So in .NET, the design is that any access to UI elements will only occur through one thread, typically referred to as the UI thread. If you are not explicitly writing multi-threaded code, then all of your code runs on the UI thread. And, while your code is running, the UI is blocked.
This also extends to external routines that you run through Declare Function. It's not really accurate to say that they are doing anything with "background tasks on the main thread", if they are doing anything with "background tasks" they are almost certainly implementing their own threading. More likely, they aren't doing any task breakdown at all, and all of their work is being done on whichever thread you use to call them---the UI thread if you're not doing anything else.
If the work being done in these routines is CPU-bound, then it would definitely make sense to push it off onto a worker thread. Based on your comments on what you already tried:
I moved the code that pings the DLL and processes whatever information it gets into a sperate thread, but that hasn't made any difference. I also tried calling StartLeftCameraDetection from a separate thread but that didn't seem to help either.
I think the most likely problem is that you're blocking in the UI thread waiting for a result from the background thread.
The best way to avoid this depends on exactly what the routines are doing and how they produce results. If they do some sort of extended process and return everything in function results, then I would suggest that using Await would work well. This will basically return control to the UI until the operation finishes, then resume whatever the rest of the calling routine was going to do.
Note that if you do this, the user will have full interaction with the UI, and you should react accordingly. You might need to disable some (or all) operations until it's done.
There are a lot of resources on Async and Await. I'd particularly recommend reading Stephen Cleary's blog articles to get a better understanding of how they work and potential pitfalls that you might encounter.
Related
Sorry I cannot post proprietary codes here. Basically, it's a Mac GUI application. The codes were not properly designed to make use of the asynchronousity concept. Everything is processed on the main thread, and it's impossible to change the design overnight. Therefore, I'd like not to go with the dispatch_async(…) solution.
The context of the problem is: I have a time-consuming task that runs on the main thread. While the task is being processed, I try to update/redraw a progress bar (NSProgressIndicator) based on the task's completion percentage (from 0% to 100%). However, because the task runs on the main thread, the main thread is blocked, and any update/redraw event in the event queue has to wait until the main thread has a chance to look at it, so the progress bar is not updated/redrawn at all during the task execution.
The solution I'm thinking about is to create another app (with an .exe file) that handles the progress bar drawing. From the main app, I'll create another process and have that process execute the other app. The task's completion percentage can be sent from the main app to the other app by using the Boost inter-process message queue.
I'm hoping to hear about both advantages and disadvantages of this solution, so any thoughts will be much appreciated!
You can do that from a thread in the same process as well. Interprocess message queues still work, though any threadsafe solution would suffice.
In general, it can be worth running some non-trivial tasks out-of-process. The kernel-level process-isolation has benefits that threads can never have:
memory space separation (security)
privilege separation (the other process can potentially run in a different security context)
Therefore when dealing with untrusted inputs or unreliable third-party library code you can gain stability guarantees for the main process.
However for your purposes it sounds like severe overkill.
I have been experimenting with porting the underlying 'story engine' of my Objective-C iPhone adventure Scarlett and the Spark of Life to HTML5 using CoffeeScript (and I am looking into IcedCoffeeScript).
The graphical part can just use DIVs on the DOM — the requirements there are fairly simple. The problematic part is the 'command and control' story-type commands. The ideal is to be able to express high-level story commands — including conditionals — and have them executed sequentially. So, for example, in faux-CoffeeScript:
scarlett.walkTo(200,300)
scarlett.turnTo(0)
story.wait(0.8)
if interesting
scarlett.think('Looks interesting.')
else
scarlett.think('Looks boring.')
In Objective-C (this was back when scripting languages like Lua were banned on the App Store), we achieved this by having two threads. The main thread ran cocos2d-phone which handled all the OpenGL calls, animation and other cocos niceties. The 'story' thread handled the command-and-control of the story, and if necessary the thread would sleep, awaiting an NSCondition before returning from a function and proceeding to the next call.
It sounds awkward, but it allowed us to express story commands and conditionals in a sequential, natural way, just using normal-looking code. Note that in the example above, the if check for the variable interesting would be evaluated right before Scarlett says something, not at the start of the function. Also, the walkTo(), turnTo(), wait() and think() calls will not return until their associated animation, delay or text box is finished back on the main thread.
What I'm struggling with is how to achieve this expressiveness using web technologies. As I see it, my options are:
Using a Web Worker as the story 'thread'. However, as far as I'm aware, workers can't sleep, and state isn't shared so they can't even perform a busy wait.
Using a callback chain, probably utilising IcedCoffeeScript's await and defer keywords to keep the code tidier. Even with those, though, that's a lot of extra line noise.
Somehow evaluate lines from the story script one-by-one as strings. I can't help feeling that it would be highly problematic.
(Similar in some ways to 3.) Write the story commands in a specially-designed interpreted language, where the program counter could be stopped and started as needed. It seems like this is unnecessarily re-inventing the wheel.
I can't help feeling like I'm overlooking some really obvious solution, though. Am I looking at this back-to-front, somehow? Is there an acknowledged pattern for scripting sequential actions and conditionals over time using actual code, without a mountain of callbacks?
I'm learning VB.NET coming from a VB6 and Java background.
In my app, I've got a function that validates the fields on a form. All it is doing is reading them, not updating. I've searched and see info about the backgroundWorker class, but all the examples are about updating the fields.
I understand the idea of threading and how it works, but have never written code that spawned threads myself. I've always let the language handle it. It seems like a lot of work that I would have to write a sub using the backgroundWorker for every time I wanted to read or update each field. The couple of books I've got that introduce you to the language show you reading or updating the field directly.
How do I know what threads are running other than writing the code like I'm used to then running through debugger to figure out what variables are on which thread?
Thanks.
Here and here is some reading on the BackgroundWorkerProcess. My advice, don't use this unless you have to i.e. only when you have a long running process and want to
Have the user switch between screens while that task is running.
Use a progress indicator on the form
That being said, I find it useful in cases like processing invoices. When I have to generate say, 4k invoices, while that task is running I can put an indacator on the form.
I find the following book helpful "Visual Basic 2008 Recipes" in explaining several use of threading, including the BackGroundWorker
The background worker does a lot of work for you. Certainly easier than managing threads and marshalling callbacks yourself. However, I agree with Saif... no point in doing any work unless there is some benefit to be had. Use it only for processes that may potentially take a lot of time.
Hopefully you're using VS2010, as it added some threading features. For example, use the Debug Location toolbar to select the thread of interest.
I have a loop of several hundred items which need to be processed.
Each item is processed by conditionally setting a global SQLConnection where upon the item is processed using this SQLConnection as part of the processing.
For this reason it is vital that none of these items is allowed to be processed in parallel.
I appreciate that this is not good design and I hope to rectify it as soon as is practical.
However it would seem that despite my best efforts, this code is experiencing some form of multi-threading. Somehow one of these tasks has thrown an exception.
This exception is the violation of a foreign key constraint, but indicates that it was operating against a SQLConnection which it has no business connecting to.
Naturally I have concerns about this, however to my knowledge there is no multi threading code in this app.
I wonder Is it possible to introduce multi threading without explicitly creating new threads
EDIT:
VB.Net 3.5SP1
Console App + Class Libraries
Occasionally Calls out to web services
Makes SQL calls
not much of anything else. No Winforms, no WPF.
Yes - using System.Timers.Timer and/or System.Threading.Timer can cause the effect your describing. Whenever a timer ticks a new work item is queued in the ThreadPool - so essentially you have a multi threading program without explicitly creating new threads.
If the timer is AutoReset (remains enabled after elapsed has been called) you might cause another call to the same handler concurrently.
In addition to the others that have been mentioned: parallel extensions (PLINQ and task parallel library).
Alternatively tasks (ie Task objects) are not called threads, but are. Tasks are commonly found near lambda expressions, check if you have any.
Oh, and async sockets too and all the other async IOs.
BUT:
Instead of trying to avoid multithreading at all cost, wouldn't it be easier to lock ? Sorry if the question is naive, I may miss something.
Could it be that your code is called from a 3rd party library. By using events another library can call your code - from as many threads as it like.
I suggest you check the code that invoke the code that changes and make sure that there's no suspicious calls to your code.
Or, equivalently, how would you design such an API. Expected/example usage would be illustrative as well.
My curiosity comes directly from the comments (and subsequent editting on my part) of this answer. Similar questions/discussions in the past provide a bit of inspiration to actually asking it.
Executive summary:
I don't feel a multithreaded UI api is possible in a meaningful way, nor particularly desirable. This view seems somewhat contentious and being a (relatively) humble man I'd like to see the error of my ways, if they actually are erroneous.
*Multithreaded is defined pretty loosely in this context, treat** it however makes sense to you.
Since this is pretty free-form, I'll be accepting whichever answer has the most coherent and well supported answer in my opinion; regardless of whether I agree with it.
Answer Accepted
**Ok, perhaps more clarification is necessary.
Pretty much every serious application has more than one thread. At the very least, they'll spin up an additional thread to do some background task in response to a UI event.
I do not consider this a multithreaded UI.
All the UI work is being done on single thread still. I'd say, at a basic level, a multithreaded UI api would have to do away with (in some way) thread based ownership of UI objects or dispatching events to a single thread.
Remeber, this is about the UI api itself; not the applications that makes use of it.
I don't see how a multithreaded UI API would differ much from existing ones. The major differences would be:
(If using a non-GC'd language like C++) Object lifetimes are tracked by reference-counted pointer wrappers such as std::tr1::shared_ptr. This ensures you don't race with a thread trying to delete an object.
All methods are reentrant, thread-safe, and guaranteed not to block on event callbacks (therefore, event callbacks shall not be invoked while holding locks)
A total order on locks would need to be specified; for example, the implementation of a method on a control would only be allowed to invoke methods on child controls, except by scheduling an asynchronous callback to run later or on another thread.
With those two changes, you can apply this to almost any GUI framework you like. There's not really a need for massive changes; however, the additional locking overhead will slow it down, and the restrictions on lock ordering will make designing custom controls somewhat more complex.
Since this usually is a lot more trouble than it's worth, most GUI frameworks strike a middle ground; UI objects can generally only be manipulated from the UI thread (some systems, such as win32, allow there to be multiple UI threads with seperate UI objects), and to communicate between threads there is a threadsafe method to schedule a callback to be invoked on the UI thread.
Most GUI's are multithreaded, at least in the sense that the GUI is running in a separate thread from the rest of the application, and often one more thread for an event handler. This has the obvious benefit of complicated backend work and synchronous IO not bringing the GUI to a screeching halt, and vice versa.
Adding more threads tends to be a proposition of diminishing returns, unless you're handling things like multi-touch or multi-user. However, most multi-touch input seems to be handled threaded at the driver level, so there's usually no need for it at the GUI level. For the most part you only need 1:1 thread to user ratio plus some constant number depending on what exactly you're doing.
For example, pre-caching threads are popular. The thread can burn any extra CPU cycles doing predictive caching, to make things run faster in general. Animation threads... If you have intensive animations, but you want to maintain responsiveness you can put the animation in a lower priority thread than the rest of the UI. Event handler threads are also popular, as mentioned above, but are usually provided transparently to the users of the framework.
So there are definitely uses for threads, but there's no point in spawning large numbers of threads for a GUI. However, if you were writing your own GUI framework you would definitely have to implement it using a threaded model.
There is nothing wrong with, nor particularly special about multithreaded ui apps. All you need is some sort of synchronization between threads and a way to update the ui across thread boundaries (BeginInvoke in C#, SendMessage in a plain Win32 app, etc).
As for uses, pretty much everything you see is multithreaded, from Internet Browsers (they have background threads downloading files while a main thread is taking care of displaying the parts downloaded - again, making use of heavy synchronization) to Office apps (the save function in Microsoft Office comes to mind) to games (good luck finding a single threaded big name game). In fact the C# WinForms UI spawns a new thread for the UI out of the box!
What specifically do you think is not desirable or hard to implement about it?
I don't see any benifit really. Let's say the average app has 3 primary goals:
Rendering
User input / event handlers
Number crunching / Network / Disk / Etc
Dividing these into one thread each(several for #3) would be pretty logical and I would call #1 and #2 UI.
You could say that #1 is already multithreaded and divided on tons of shader-processors on the GPU. I don't know if adding more threads on the CPU would help really. (at least if you are using standard shaders, IIRC some software ray tracers and other CGI renderers use several threads - but i would put such applications under #3)
The user input metods, #2, should only be really really short, and invoke stuff from #3 if more time is needed, that adding more threads here wouldn't be of any use.