I receive events from clicks with a global monitor in macOS.
The NSEvent give me coordinates but i need to get the UIElement.
Using the accessibility API, i can get the focused element but i need the clicked element which is totaly different. For example, if i click on a textarea, i get a AXTextArea element. If i click on the application taskbar, the focused element remains AXTextArea which make sense. I want the element associated to the mouse event.
I found on the web a method called accessibilityHitTest which is supposed to retrieve the UIElement under a given NSPoint. I always get null so i am not sure this is supposed to work that way.
Maybe there is another function which retrieve the UIElement from a given point in the screen?
Here is what ive tried :
NSEvent *event; // My event received from global monitoring
id clickedObject = [self accessibilityHitTest:event.locationInWindow];
//This give me an error because self is not a NSWindow.
NSEvent *event; // My event received from global monitoring
id clickedObject =
[[NSApp keyWindow] accessibilityHitTest:event.locationInWindow];
//This give me null at every click.
Maybe this is not possible but i just which i could get the UIElement under my mouse click and not the focused UIElement.
You need to use AXUIElementCopyElementAtPosition() for this. Note that you have to be careful to provide the coordinates in the proper coordinate system. Since the event has no window associated with (because it's for a different app), NSEvent's locationInWindow uses the Cocoa coordinate system with the origin in the lower-left of the primary display, with y increasing in the up direction. AXUIElementCopyElementAtPosition() takes coordinates in the Core Graphics coordinate system with the origin in the upper-left of the primary display, with y increasing down. Basically, you do:
point.y = NSMaxY(NSScreen.screens[0].frame) - point.y;
Related
I want to be able to get the global caret position inside any application in Mac High Sierra using cocoa or appleScript. I already use NSEvent to get the keyboard and mouse hook but is there a way to get the caret position hook?
The caret is different from the mouse position. It moves on key event or mouse click. In windows, you can get the caret position almost anywhere. I want to know if there is the equivalent for macos.
I want to show a popup over the text caret, if i type on the keyboard or line return, it moves with the text.I tried getting the position of the key event, (locationInWindow) but it give me back the mouse position. I am not sandbox so i can even call applescripts
UPDATE : It is possible doing this by getting the bounds of the letter before the caret with the use of accessibility API.
thanks
I don't have the opportunity to try it for myself just yet, so you might beat me to the punch of confirm/reject this.
UIEvent has addGlobalMonitorForEventsMatchingMask:handler: where the mask can have a value of NSEventMaskCursorUpdate and presumably the returned NSEvent object will contain a coordinate that can be acted upon (i.e. converted to screen-space).
Caveat here is the docs explicitly say
Key-related events may only be monitored if accessibility is enabled
or if your application is trusted for accessibility access (see
AXIsProcessTrusted).
Your post seems to suggest that you do not wish to use Accessibility API ("but if not using accessibility API") so that may mean you're out of luck in the specific combination of requirements you seek to fulfill.
I must achievement somenthing tricky in my application for MAC OS, and because it's not easy to explain I will put an image: custom window
I created the NSWindow from image but the problema is how can create the NSView (the blue one) which is over and which have the purpose to block other action from user and just left small portion to be active. Any suggestion?
You can create a Custom NSView and draw clear color for enbaled rect area and disabled color for rest of the area. Capture the mouseDown event for the custom view and discard all the mouse events outside enabled rect area and if the mouse down in enabled area then call the mouse down event to the control behind the enabled area.
Wouldn't it be better to put up a sheet with a dialog to ask the user for the required information (and prevent interaction with the rest of the window) instead of mimicking a very un-Mac-like UI?
For example, consider the UI when creating a new project or adding a file to a project in Xcode.
If you really must attempt this, don't use a view to cover the window. Use another window. Make it borderless. Set its backgroundColor to [NSColor clearColor] and its opaque property to NO. Set its hasShadow property to NO. Set its frame to match the content rect of the window you want to block (or maybe its frame if you need to prevent interaction with its title bar).
Attach your overlay window to the main window as a child, using -addChildWindow:ordered:.
For the contentView of the overlay window, create a view that will draw the semi-transparent color everywhere except for over the control you want to leave accessible. To get that rect in the main window's coordinate system, you would use something like [specialView convertRect:specialView.bounds toView:nil].
I know how to retrieve the mouse coordinate in a PictureBox.Click event though e
In a PictureBox.MouseHover, e does not return such information.
How do I get the mouse coordinates in a MouseHover event ? Is there a way ?
Thanks in advance.
Control.MouseHover "occurs when the mouse pointer rests on the control."
A typical use of MouseHover is to display a tool tip when the mouse pauses on a control within a specified area around the control (the "hover rectangle"). The pause required for this event to be raised is specified in milliseconds by the MouseHoverTime property.
So this event is not raised only whenever the mouse is over the control - there is a delay associated. So the position is somewhat irrelevant, as the mouse could have moved somewhat during that delay.
Do you really need to be using this event? As Dan-o mentioned, MouseMove passes a MouseEventArgs which does provide the coordinates, as you request. It may be the right option, depending on what exactly you're trying to do.
To get the mouse position at any time though, you can use the Cursor.Position property. This will give you the screen coordinates of the cursor. From here, you can call the Control.PointToClient method, to get the coordinates relative to a particular Control.
The problem: I have to follow the app's NSStatusItem popup/window right beneath it.
The general solution for this (like it's suggested here or here) is to add a custom view to the NSStatusItem and get the view's window's frame (statusItem.view.window.frame), and connect to move event through NSWindowDidMoveNotification notification of the statusItem.view.window.
This solution works 99% unless the user connects an external display which is taller than the previous display (for example a Mac Book user connects an external monitor), in this case the statusItem.view.window.frame will be incorrect, X coordinate will be actually correct but the Y coordinate will be the same as it was in the smaller screen!
I've checked and most of the menubar apps which have a popup window when you click the status item are misplaced just like I've described.
My solution was to don't use the Y coordinate from this frame but use the corresponding NSScreen's visibleFrame's height like this:
NSScreen *screenWithMenuBar = [[NSScreen screens] objectAtIndex:0];
if( screenWithMenuBar ) {
// correct the top position by the 'screen with the MenuBar's height
// workaround: without this the window will be off-placed when the user plugs in an external display,
// a display with more height
NSRect visibleRect = [screenWithMenuBar visibleFrame];
originPoint.y = visibleRect.size.height + visibleRect.origin.y;
}
As stated in the official Apple documentation the first screen should be always the screen which contains the menubar so it should work, but it seems too much of a hack for me.
Have you guys experienced this bug? Have you found a better solution?
What's the best practice for setting zoom factor of an image within IKImageView via NSSlider?
I was able to bind a slider either to zoom in OR zoom out action of an IKImageView.
Obviously, what I'd rather see is a single slider controlling both those actions.
Best, if image is refreshed after each change of the slider (continuously, even if a mouse button is not released yet).
This demo explains a lot: ImageKitDemo
In particular, this snippet is what I was looking for:
- (IBAction) zoomSliderDidChange:(id)sender
{
[addProductPhotoImageView setZoomFactor:[sender floatValue]];
}
The Bindings way would be to bind both the Zoom Factor of the IK image view and the Value of the slider to the same property of your controller. When the slider changes the value of the property, the image view will be notified, and will go get the new value from your controller.
One advantage of this way is that you can add more ways of zooming in and out and the value in the slider won't go stale. For one example, if IKImageView adds pinch-zooming (or if it has it already—I don't have multi-touch on my Mac), the user can zoom that way and the slider will update automatically. That won't happen with the IBAction solution.
Another example would be Zoom In and Zoom Out menu commands (perhaps with ⌘+ and ⌘- keyboard shortcuts) that send action messages to your controller. Your controller would respond by increasing or decreasing the value of the property (using a setter method it implements). With Bindings, both the image view and the slider will update for free. Without Bindings, you would have to explicitly talk to both the image view and the slider, telling one to update its zoom factor and the other to update its slider.
A third example would be a “Zoom factor: X%” display in a corner of your window. With Bindings, this can update for free no matter how the user zooms the image: moving the slider thumb, pinching/unpinching the image, or pressing a menu item. Without Bindings, this would be yet another thing you have to talk to in your (at least three) change-the-zoom-value action methods.