NOTE: Updated below...
I have a cocoa desktop application which consists of a series of controls around a custom NSView. I am using displayLink to drive the updates.
When a user clicks on an NSControl (a slider, a button, a checkbox, a radio button) the application appears to freeze until the mouse is released. I can confirm in fact that the displayLink callback (getFrameForTime) is NOT firing during the time. If I create a timer, that also does not fire, both remain paused until the user releases the mouse, at which point the application resumes updating.
The control is bound, and if I update that value from another thread (for example, via a callback from a MIDI interface) the slider behaves as expected: it moves, the value updates and the application does not pause.
I feel like this should be a fairly obvious fix, but I'm stumped.
Checking "continuous" in IB does as advertised: sends the values continuously, but still exhibits this behavior (preventing the UI update) until the mouse is released.
This seems to be related specifically to mouseDown on NSControl? Why would this block, and do I really need to subclass all my UI elements to change this behavior (seems extreme)
DisplayLink is in its own thread, so why mouseDown on the main thread block it? If this is the case, given the injunction on updating the Cocoa UI from other than the main thread, how do I deal with it?
Any help much appreciated.
Update
Per #Nikolai's comments below, I can confirm that using an NSTimer and adding it to NSEventTrackingRunLoopMode does NOT block. However, I would really like to use CVDisplayLink which (according to the documentation) runs in it's own thread and should not be blocked in this way. Unlike CADisplayLink, I cannot find a way to explicitly assign a runloop to CVDisplayLink (it seems it doesn't work that way), so perhaps the new question should be:
Why does CVDisplayLink block on NSEventTrackingRunLoopMode?
When clicking on an NSControl the runloop mode goes from NSDefaultRunLoopMode to NSEventTrackingRunLoopMode, as long as the mouse is down. That means that only run loop sources (display link) and timers fire that have been added to this mode.
You can add timers to any mode by using -[NSRunLoop addTimer:forMode:]. For a display link the equivalent method is -[CADisplayLink addToRunLoop:forMode:].
To make your animation continue during event tracking you would do something like:
[myDisplayLink addToRunLoop:[NSRunLoop currentRunLoop]
forMode:NSEventTrackingRunLoopMode];
Your test project shows that you are calling a view's display method from within the display link's callback.
When commenting the display message out, the display link is called continuously even while moving the slider.
So what goes wrong is that when the runloop goes into event tracking mode, the call to display on the display link's thread blocks until the mouse is released and the run loop goes back to default mode. You can easily confirm this by putting a log statement before the call and one after it.
Why exactly that happens is not clear to me. What is clear is that it's illegal to call a view's methods from a background thread. You have to trigger the view's display by dispatching a setNeedsDisplay: on the main thread:
static CVReturn MyDisplayLinkCallback(CVDisplayLinkRef displayLink, const CVTimeStamp* now, const CVTimeStamp* outputTime, CVOptionFlags flagsIn, CVOptionFlags* flagsOut, void* displayLinkContext)
{
dispatch_async(dispatch_get_main_queue(), ^{
[(__bridge MyCustomView*)displayLinkContext setNeedsDisplay:YES];
});
return kCVReturnSuccess;
}
Related
Restarting app development after a 15 year hiatus. Current project is conversion of old windows-type command line utility into interactive OS X windowed app.
I created a view delegate inside main window and can draw and update NSTable view.
The updates are generated in the App's main loop which takes a UDP/TCP stream, parses and updates view via appropriate delegation.
Here's the problem: When I run the app, the main window does not apparently get
focus (window control buttons on upper left are grey), the Menu created from my .xib is inert, and the window itself does not respond to resizing or to mouse hits inside the table view scroll bars. Also, the mouse pointer is the spinning beachball when over the App's window.
My guess is that I am not providing time to the Objective C run loop for event processing. I do send a "display" to my window on every iteration of my app loop, but I guess it is not sufficient (Apple is not very clear about what objects get what messages when sending this kind of update message). Am I on the right track?
Is there a way to let the system Event loop run an iteration each time through my app main loop?
Thanks!
Update: I tried explicitly providing event loop time in my app's main loop with:
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:0.1]];
There was no apparent change in app behavior.
My guess is that I am not providing time to the Objective C run loop for event processing.
In this case, I recommend you read about Grand Central Dispatch, which provides concurrency, allowing the GUI to remain responsive.
There's a good explanation of GCD here and whilst it looks like a large and complicated subject, you'll probably only need a few lines of code to make use of GCD. For example: -
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
// do some work
});
Despite comparing the nibs and connections between my code and a freshly created .xib prototype, i could find no reason for my code to not get focus.
The fix was to move my code guts into the project with a new .xib.
This works but is unsatisfying
I have been working on an opencv project for iOS. I was given a simple project to start developing with that captured and displayed frames. I never payed much attention to how it worked until I started having memory issues and traced them back to the original project setup. I now plan to re-write the capturing/displaying code but I don't understand why it worked in the first place. There was a play/pause button which called the method
- (IBAction)play_pause:(id)sender
{
play = !play;
while(play)
{
if (_videoCapture && _videoCapture->grab())
{
(*_videoCapture) >> _display_frame;
//process frame
self.imageView.image = [UIImage imageWithCVMat:_display_frame];
}
}
}
play is just a global bool that signifies whether the application is playing or paused. The strange thing is that the processing should be taking place inside an infinite loop, there is no way out. play is never modified within the loop. In spite of that, when the application is running the play/pause button remains responsive and is capable of flipping the play bool and pausing the execution. Not only that, other bools (use_greyscale for instance) can be flipped by other buttons and their values change inside the loop. I would have expected the application to freeze and never even draw new frames to the screen. The application should stay trapped inside that function for most of its lifetime, unable to perform other tasks such as drawing and UIControl. It seems as though the only way this is possible is if the IBAction call is running on its own thread. I cannot find any evidence of threading in the source code. Could someone explain how apple handles threading in its UI? I was under the impression that there was one main runloop thread and that extra threads were not created automatically. If that is true, how can this behavior be explained?
side note-
What finally made me investigate this was that [UIImage imageWithCVMat:_display_frame] returns an auto-released object. Since all this takes place inside a loop, the objects could not be released without the execution being paused which was causing crashes.
The reason is worked is because the implementation of the cv::VideoCapture::grab() method runs the current run loop to pause the thread until it gets a frame.
When you launch your application, the main function executes a function named UIApplicationMain, which executes CFRunLoopRun. When CFRunLoopRun is executed on the main thread, it runs the main run loop, which is the run loop that processes all the UI events received from the system and refresh the user interface. For information on run loops, you may read Apple Threading Programming Guide.
So, when you execute an infinite loop, your code never returns to the run loop and the waiting events cannot be processed. But in your case, the grab method runs the run loop again with an expiration delay. So the run loop may process incoming events (which may invoke your code again) until the delay expires, then return to your code that will run the run loop again.
If you look at the callstack when you touch the button to pause, you will see this:
main function → run loop → event handling → your code → OpenCV → run loop → event handling → your code
The run loop is running inside itself, which is perfectly fine because run loops are reentrant. Scroll views actually use that behavior: When you scroll a UIScrollView, it runs the run loop again in a different mode in order to ignore some events until you end scrolling.
But I'm not sure the developers of OpenCV had this in mind when they wrote their code. So I think it would be better to load your frames in a background thread/queue.
You are correct, there is no "automatic threading" in iOS. Grand Central Dispatch (GCD) certainly makes threading much easier, but it does not happen automatically.
You can write some debug code and test for [NSThread isMainThread] in the while loop to see if play_pause is indeed being run on the main UI thread, which I suspect it is not.
I have an app that imports a potentially large amount of data from the web after the user explicitly presses a Sync button, and stores that data using Core Data. Since I want to show feedback and I don't want the user interacting with the rest of the app while this happens, pressing the Sync button brings up a Modal dialog. Since I want the operation to happen immediately, the operation executes in the viewDidAppear method. I'm sure this is frowned upon.
There are a bunch of problems with the approach right now:
Everything happens in the main thread. The user kind of gets feedback because there is an activity indicator that continues to animate, but there's no way to indicate progress or show intermediate messages. This is not the right way to do things.
But, I am told that when using Core Data, everything has to use the main thread, so breaking off the work into another thread does not seem like it will be straightforward.
If the app enters the background state (user hits Home button or iPad falls sleep), it's game over - the operation dies. It's clear to me from the documentation why this is the case.
I know there are "I'm about to enter the background" events that you can handle, but it's not as though I can move execution of code from one place to another in the middle of a file download. Whatever solution I use has to be a continuous action that executes in the same way both before and after the transitions to/from the background.
I want the operation to execute in the foreground as far as the user is concerned. It does not make sense for them to interact with other parts of the app while this operation is taking place.
I am reading the Apple documentation on this, but I'm asking this in hopes of finding more concise guidance on this particular combination of needs. Thanks.
You really should not freeze the main thread. You can still "prohibit" certain UI actions.
Create a separate context, as a child, and do all your work in there. When done (or at certain intervals), save the context to the main context, and notify the main thread to do some UI update interaction... maybe a progress bar or something...
NSManagedContext *backgroundContext = [NSManagedContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
backgroudContext.parentContext = [self mainManagedObjectContext];
[backgroundContext performBlock:^{
// This block is running in a background thread.
// Go get your data from the web
// Call this to push data to the main MOC (either at end, or at intervals)
[backgroundContext save:&error];
// When you want to do something on the main thread...
dispatch_async(dispatch_get_main_queue(), ^{
// This block is running on the main queue... I can do anything with the UI...
}];
}];
Couple of things to note... your mainMOC needs to be private or main queue concurrency type. If you are using the Core Data template, where it is in the app delegate, just change the alloc/init to initWithConcurrencyType:NSMainQueueConcurrencyType.
I would, however, suggest using the canonical main/parent relationship. Create a private MOC, assign it to the persistent store, then create a main MOC, set its parent to be that private MOC. Now you are ready to handle any I/O with background operations, without blocking your UI.
Still, when loading from the web, use the pattern above: create a child MOC, then load objects into the main MOC.
Note, that the data is not saved to disk until the "root" MOC calls save.
I have a class containing a method with a loop. I need to be able to break the loop if a certain event (e.g. button press) occurs.
I am using the NSNotificationCenter to notify the class containing the loop when the button is pressed.
However, If I press the button while the loop is being executed, the notification occurs after the loop is complete instead of interrupting the loop.
I'm guessing this is because it is operating in the same thread.
So how do I get the NSNotificationCenter operating in a background / different thread? Is this possible? Or is there a better way to do it?
It's not just the notification center.
I have a class containing a method with a loop. I need to be able to break the loop if a certain event (e.g. button press) occurs.
The events for that button press come in on the main thread. If your loop is running on the main thread, then the button press itself does not get processed until your loop is finished. The notification is posted immediately, relative to the button press actually getting processed by your application.
Or, in list form:
The user presses the button.
Your loop runs out of things to do and returns.
The button press arrives in your application and is turned by the button into an action message.
You post the notification.
You receive the notification.
The delay that you're seeing is between steps 1 and 2; step 4 happens immediately after step 3.
Notifications on a local (not distributed) NSNotificationCenter are dispatched on the thread you post them from, so posting it from your action method means that it will be dispatched on the main thread. This is normal and OK.
Move the loop, not the notification, to a background thread, dispatch queue, or operation queue. If you use an operation queue, you may not need the notification at all, as you can tell an operation queue to cancel all pending operations. (Your operations will need to check at any appropriate time(s) whether they have been canceled; for reasons previously discussed, killing a thread/operation at a random time is a Bad Idea.)
Background threads, blocks, and operations can communicate back to the main thread when needed (e.g., to update the UI). To send a message through the main thread's run loop, use performSelectorOnMainThread:withObject:waitUntilDone:. To dispatch a block on the main thread, use dispatch_async and dispatch_get_main_queue. To schedule an operation on the main thread, add it to [NSOperationQueue mainQueue].
For more info, read the Concurrency Programming Guide and the Notification Programming Topics.
I would run your loop in a separate thread, and have an instance variable BOOL abort;, when your button press notification comes in, set abort = TRUE; then in the loop check this value and exit if it is true.
I would run the loop in a separate thread. Even better, make it an NSOperation so that you can call [.. cancel]. Just make sure to use performSelectorOnMainThread when updating the UI from the NSOperation object. It's not a good idea to have a long running loop on the main thread.
You can't put the notification center on another thread. That object is out of your control. The problem isn't so much that they are on the same thread as that you are not allowing the run loop, which is responsible for handling the button press, to do anything. (There's one and only one run loop per thread.) As has been stated by both edsko and Peter Hosey, the button press itself, and in fact your entire UI, is stopped while your loop is running. It is generally a good idea to put long-running operations onto a background thread, then call back to the main thread to update the UI, performSelectorOnMainThread:withObject:waitUntilDone: being an easy way to do such a call back.
That said, if you were to keep the loop on the main thread, you need to let control return to the run loop periodically so that the button press will be registered. There are two ways I can think of to do this. First, you can explicitly give the run loop control briefly during each iteration of your loop:
while( !buttonWasPressed ){
// Do work...
// Let the run loop do some processing before the next iteration.
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:0.01]];
}
Or, you can make a single method that consists only of the code from your loop, and use performSelector:withObject:afterDelay: to have the method repeatedly called while still allowing the run loop to work:
- (void) loopTheLoop {
if( buttonWasPressed ) return;
// Do work...
// Run this method again as soon as possible.
[self performSelector:#selector(loopTheLoop)
withObject:nil
afterDelay:0.0];
}
Newbie Objective C/Cocoa question: I have an application with some data entry fields and a "do it" button. When the button is pressed, some computation takes place and output data is displayed in a table view and some text fields in the same window. What I'd like is that when the button is pressed that the text fields and the table view are both cleared while the computation takes place.
I've tried making the appropriate calls as the first few statements of the action routine for the button press, but that doesn't work. I would imagine that the runtimes don't get called to do the screen update until after my action routine is finished.
Is there a simple way to do what I want to do? Thanks.
You imagine correctly.
The usual way to do this sort of thing is to use NSObject's performSelectorInBackground:withObject: to start the heavy calculation in the background. Then once the background code finishes doing its work, use performSelectorOnMainThread:withObject:waitUntilDone: to call another selector on the main thread to update the UI (remember, UI calls may only be done from the main thread).
You're correct about the screen updates not taking place until after your routine finishes. Most drawing to the screen is queued to improve performance.
When you change the value in an NSTextField, it knows to call [self setNeedsDisplay:YES] in order to queue its need for redrawing. If you want to force it to display, you can call [textField display]. (Note that calling [textField setNeedsDisplay:YES] will not cause immediate display). Things get a bit more difficult with an NSTableView, as this -display method is unlikely to work for it.
While you could create a secondary thread to do your processing, that would create a lot of complexity that may not be worth it. You might consider using -performSelector:withObject:afterDelay: to begin your processing routine rather than calling it directly.
- (IBAction)buttonClicked:(id)sender {
[textField setStringValue:#""];
[tableView reloadData];
// instead of doing the following:
// [self processData:nil];
// do
[self performSelector:#selector(processData:) withObject:nil afterDelay:0.0];
}
- (void)processData:(id)sender {
// process the data
[textField setStringValue:#"the results"];
[tableView reloadData];
}
Using -performSelector:withObject:afterDelay: is different than calling the method directly, as it causes the method to be called not immediately, but scheduled to be called "ASAP". In many cases, your app will be able to squeeze in the updates to the UI before it can get to performing that computation method. If testing reveals this to be the case, then you can avoid having to go to the trouble of creating a secondary thread to do the processing.
If you want to force updating screen then call setNeedsDisplay from your UIView.
I would imagine that the runtimes
don't get called to do the screen
update until after my action routine
is finished.
Bingo. Your button's action method is called on the main thread, which is the same thread that is responsible for updating the user-interface. So the interface will not update until after your action method returns.
To get around this, you can split your action method into two parts. The first part makes the calls to clear your previous view and set whatever new state you want to use for rendering. The second part does the new calculations, and is moved to its own method. Then, at the end of the first part, add something roughly like:
[self performSelectorInBackground:#selector(myActionSecondPart) withObject:nil];
...to run the computation part in the background. Then your UI will update while the computation runs.