I'm trying to move other applications in OSX using the accessibility API. I actually have this working. But I ran into a major issue.
AXUIElementRef appRef = AXUIElementCreateApplication(self.pid);
This uses the PID of my application to to later move the window. I found that multiple windows have the same PID, and when i try and move the second window of application my app crashes.
Can I move my application using just WindowID?
The way to uniquely identify a particular window if you really need to is by going through the results of CGWindowListCopyWindowInfo to select the ones matching the pid that you're using for appRef there; then the kCGWindowNumber CFNumber is Quartz's unique CGWindowID for that window.
However, sounds more likely that you're just indexing through the windows incorrectly. This should work for your appRef:
CFArrayRef windows;
AXUIElementCopyAttributeValue(appRef, kAXWindowsAttribute, (CFTypeRef*)&windows);
int windowCount = windows ? CFArrayGetCount(windows) : 0;
for (int windowIndex = 0; windowIndex < windowCount; windowIndex++)
{
AXUIElementRef windowRef = (AXUIElementRef)CFArrayGetValueAtIndex(windows, windowIndex);
... move windowRef here ...
}
If that doesn't make the problem obvious, post the code which crashes and how it does so.
Related
I am creating a window switcher for macOS.
I am using CGWindowListCopyWindowInfo to get the list of open windows.
let windows = CGWindowListCopyWindowInfo([.excludeDesktopElements], kCGNullWindowID)
I am not using the optionOnScreenOnly option along with CGWindowListCopyWindowInfo because that excludes off-space windows.
However, this leads to certain dummy windows also getting included in the list of windows. For example, there is a window present for each open tab on Xcode. There is also an additional invisible window for VLC player apart from the actual window. These windows aren't present if I use optionOnScreenOnly.
Is there a way to separate these dummy windows from actual windows?
I have explored the kCGWindowLayer and kCGWindowStoreType keys in the dictionaries returned by CGWindowListCopyWindowInfo but couldn't tell the windows apart.
Edit:
I am open to both Swift and Objective-C solutions, as well as private API calls.
Edit 2:
I know this is possible to do because the app Witch handles these cases correctly.
I'm not sure if this captures all the cases you're interested in, but if we take the array of dictionaries returned by CGWindowListCopyWindowInfo() and group the entries by kCGWindowNumber (the window ID), it looks like we'll end up with one of three cases:
Only one dictionary entry has that window number (it's a simple widow, which may be on or off screen)
Several dictionaries have that window id, and only one of them in the group has the key kCGWindowIsOnscreen set to 1: the others either lack the key or have it set to 0. This means we have a window with several tabs, and the entry with kCGWindowIsOnscreen set is the visible tab.
Several dictionaries have that window id, and none have the key kCGWindowIsOnscreen set to 1 (it is missing or set to 0 in all cases). This means we have a window with several tabs that is currently off-screen.
I'm assuming you're probably already filtering out anything that isn't in layer 0 (kCGWindowLayer = 0). It seems that most of the things that we visually perceive as a 'window' are in layer 0 (status menu items seem to be in layers with single or low double digits; widgets seem to be up around layer 100...).
I don't see any simple, direct way to get at this, but you should be able to build this logic into something workable.
How about using a different approach, by checking what apps are running and visible using the following:
let runningApps = NSWorkspace.shared.runningApplications.filter{ $0.activationPolicy == .regular }
If you print(runningApps) you will get a list of apps currently ran in your workspace.
Hope this helps.
I am working on an application with VTK windows. (my VTK version is 5.8.0)
Leaving "VTK render window", all other windows are hidden.
But it is necessary to render the hidden windows because the rendered images of the hidden windows are needed to be stored as images.
here is sample code.
std::vector<vtkRenderer*> renderers;
renderers.reserve(m_numberOfRenderer); // m_numberOfRenderer = 4
std::vector<vtkRenderWindow*> windows;
windows.reserve(m_numberOfRenderer);
for(int i=0; i<m_numberOfRenderer; i++)
{
vtkRenderer* renderer = vtkRenderer::New();
vtkRenderWindow* window = vtkRenderWindow::New();
renderer->TwoSidedLightingOff();
window->OffScreenRenderingOn();
window->SetSize(m_width, m_height);
window->AddRenderer(renderer);
renderer->SetActiveCamera(m_renderCamera.at(i));
renderer->AddActor(m_actor.at(i));
renderer->AddActor(m_outActor.at(i));
renderers.push_back(renderer);
windows.push_back(window);
}
My code is working fine but sometimes crashes on OffScreenRenderingOn method
THIS IS VTK CLASS REF --- http://www.vtk.org/doc/nightly/html/classvtkWindow.html#a25c2cd7f6dad61d91f148da42bc8e780)
virtual void vtkWindow::OffScreenRenderingOn()
Creates a window in memory instead of on the screen. This may not be supported for every type of window and on some windows you may need to invoke this prior to the first render.
i don't know that some windows you may need to invoke this prior to the first render.
i appreciate any help,
Thanks in advance
I would like to create a system tool / application which has the capacity to aid in window management. I'm trying to find documentation about the following topics, if they are indeed possible given the security sandboxing of OSX.
Show a list of running applications with the name & icon, and allow the user to choose one
Manipulate the frame(s) of said application's windows (eg, resize, reposition) from my app (with animations -- though I assume this will be trivial once I can perform the actual change)
Hide or show these applications from task managers, etc.
Be able to launch (or terminate) instances of the given application
It seems to me that Quicksilver accomplishes many of these things, but the lack of AppStore availability makes me wonder if it possible to do this while remaining in the OSX sandbox.
There are a lot of pieces of software out there that do window management. You can check out a tiling window manager I've been hacking on called Amethyst. The basic idea behind software like this relies on Accessibility (which you can find documentation for here). As a quick overview the APIs work by acquiring references to accessibility elements (applications, windows, buttons, text fields, etc.) which have properties (hidden, position, size, etc.), some of which are writable.
As an example let's say that you wanted to move all windows in every running application to the upper left corner of the screen. That code might look like
for (NSRunningApplication *runningApplication in [[NSWorkspace sharedWorkspace] runningApplications]) {
AXUIElementRef applicationRef = AXUIElementCreateApplication([runningApplication processIdentifier]);
CFArrayRef applicationWindows;
AXUIElementCopyAttributeValues(applicationRef, kAXWindowsAttribute, 0, 100, &applicationWindows);
if (!applicationWindows) continue;
for (CFIndex i = 0; i < CFArrayGetCount(applicationWindows); ++i) {
AXUIElementRef windowRef = CFArrayGetValueAtIndex(applicationWindows, i);
CGPoint upperLeft = { .x = 0, .y = 0 };
AXValueRef positionRef = AXValueCreate(kAXValueCGPointType, &upperLeft);
AXUIElementSetAttributeValue(windowRef, kAXPositionAttribute, positionRef);
}
}
Which illustrates how you get references to applications and their windows, how to copy attributes from an accessibility element, and how to set attributes of an accessibility element.
There are a variety of notifications documented in NSWorkspace for the launching and termination of applications, and the accessibility framework also has a sense of notifications for things like an application creating or destroying windows, or a window miniaturizing or deminiaturizing.
Animating the window changes is non-trivial and I haven't figured out how to do it yet, though it may be possible. It may not be possible at all without hitting private APIs. But the other things you have listed should be possible. Hiding an application, for example, could be done by setting the kAXHiddenAttribute on the application accessibility element. Launching an application can actually be done via -[NSWorkspace launchApplication:].
Note that the use of accessibility necessitates that the user have Enable access for assistive devices turned on in System Preferences > Accessibility.
When Mission Control runs, it prevents applications from receiving keyboard and mouse events. It also leaves the last application running thinking that it still has focus. This is a problem for me because I don't receive keyUp or mouseUp events if I start Mission Control with a mouse button or a key held down and my application will behave as if that mouse button or key is held down.
I would like a way to either read both keyboard and mouse events even when Mission Control is active, or a way of detecting that Mission Control is active. Ideally, I would like to be able to do the latter since I effectively can't use my application when Mission Control is running.
I've tried a couple of things with no luck:
Use addGlobalMonitorForEventsMatchingMask to register a global monitor for keyboard and mouse events. This captures mouse events (but not keyboard events, although the documentation says keyDown events should be sent to the global monitor) when I switch to another application, but Mission Control doesn't seem to let events propagate to global monitors.
Check [[NSRunningApplication currentApplication] {isActive, ownsMenuBar}].
Apparently, my application is active even though it's not receiving events!
Check [NSApp keyWindow] != nil.
Apparently, one of my windows should be receiving key events. None of them are.
Check if Mission Control is one of the running applications returned by [NSWorkspace runningApplications]. Mission Control does not show up in this list when it's running.
Edit:
I've finally worked around this problem (albeit not in a very satisfactory way). For the mouse, it turns out that you can query the state of the pressed buttons with [NSEvent pressedMouseButtons]. I simply keep track of what I think the mouse state should be from NSLeftMouseDown and NSLeftMouseUp events and compare that to [NSEvent pressedMouseButtons] every so often to make sure that they're consistent. If they're not, then I know that something has hijacked my NSLeftMouseUp event and act accordingly.
For the keyboard, I could not find a way to query the keyboard state, so I couldn't do a similar workaround. I ended up disabling application switching using presentation options when keys are pressed.
At least in OS X 10.10, you can use this code to check if Mission Control is active or not:
func missionControlIsActive() -> Bool
{
var result: Bool = false
let windowInfosRef = CGWindowListCopyWindowInfo(CGWindowListOption(kCGWindowListOptionOnScreenOnly), CGWindowID(0)) // CGWindowID(0) is equal to kCGNullWindowID
let windowList: NSArray = windowInfosRef.takeRetainedValue() // We own the returned CFArrayRef
for entry in windowList
{
if (entry.objectForKey("kCGWindowOwnerName") as! String) == "Dock"
{
var bounds: NSDictionary = entry.objectForKey("kCGWindowBounds") as! NSDictionary
if (bounds.objectForKey("Y") as! NSNumber) == -1
{
result = true
}
}
}
return result
}
In a nutshell, the code checks if a specific window owned by the OS X Dock process is visible on the screen and if it is in a specific position. If both conditions are met, Mission Control will be active right now. Code will work in a sandboxed app and no privileges for assistive devices are required.
Did you try on bash level using NSTask? Something like ps -faxU <username> should list all running processes and then you could parse the output, or indeed you could use ps -faxU <username> | grep -i "mission control" (At the top of my head I am not sure how the process may be called, but sth like "mission control" seems legit). Not the most elegant solution maybe, but if nothing else works it may be worth it.
May be i'm missing something, but have you tried to use event taps instead of global monitoring?
It does appear that DTrace has some ability to see Mission Control being activated. Try running:
sudo fs_usage -filesys | grep Mission
from the command line and then launching the Mission Control app from the /Application folder.
You should see a lot of output related to Mission Control starting up. Unfortunately, this same output did not appear by using the keyboard short cut or swiping. Of course, using DTrace in production code is not something I would actually recommend.
C++ and Qt implementation works in latest OS X.
bool Window::missionControlIsActive() {
bool result = false;
CFArrayRef windows = CGWindowListCopyWindowInfo(kCGWindowListOptionOnScreenOnly+kCGWindowListExcludeDesktopElements, kCGNullWindowID);
for (int i = 0; i < CFArrayGetCount(windows) ; i++) {
auto cfMutableDictionaryRef_dict = (CFMutableDictionaryRef)CFArrayGetValueAtIndex( windows, i );
auto cfStringRef_name = (CFStringRef)CFDictionaryGetValue(cfMutableDictionaryRef_dict, kCGWindowName);
if (QString::fromCFString(cfStringRef_name) != u"") continue;
auto cfStringRef_ownerName = (CFStringRef)CFDictionaryGetValue(cfMutableDictionaryRef_dict, kCGWindowOwnerName);
if (QString::fromCFString(cfStringRef_ownerName) != u"Dock") continue;
auto cfDictRef_bounds = (CFDictionaryRef)CFDictionaryGetValue(cfMutableDictionaryRef_dict, kCGWindowBounds);
auto cfNumRef_bounds_Y = (CFNumberRef)CFDictionaryGetValue(cfDictRef_bounds, QString("Y").toCFString());
double num;
CFNumberGetValue(cfNumRef_bounds_Y, kCFNumberFloat64Type, &num);
if (num > 1.0 and num < 1000000) continue;
result = true;
break;
}
CFRelease(windows);
return result;
}
I've needed to make a global hot key input box in my Cocoa App.
I know about Shortcut Recorder, but it is a very old solution. It has parts implemented using Carbon, which has been deprecated, and I can't publish my app to the Mac App Store if I use it.
Is there any ready-to-use modern solution? Can anybody give me the way to make this by myself (I don't know where to start from)?
There is a modern framework named MASShortcut for implementing Global Shortcuts in OS X 10.7+.
Not all of Carbon is deprecated. You can't make a pure-Carbon application anymore, but some APIs live on and some of them are still the easiest way to do certain things.
One of these is the Carbon Events hotkey API. You certainly can sift through all the events using NSEvent's event-monitor methods, but it's unnecessary work. The Carbon Events hotkey API is still supported and much simpler—you just tell it what key you want to match and what function to call when the key is pressed. And there are Cocoa wrappers such as DDHotKey that make it even simpler.
RegisterEventHotKey, the relevant Carbon Events function (see also UnregisterEventHotKey in the same doc)
DDHotKey
MASShortcut, yet another wrapper (suggested by TongG):
MASShortcut (with ARC)
MASShortcut (without ARC)
KeyboardShortcuts, written in Swift and includes a SwiftUI hotkey recorder view [added to this answer in edit by the project's author].
In Mac OS X 10.6 and higher, you can use the methods +addGlobalMonitorForEventsMatchingMask:handler: and +addLocalMonitorForEventsMatchingMask:handler: defined from the NSEvent class. Monitoring Events reports the following information:
Local and global event monitors are mutually exclusive. For example, the global monitor does not observe the event stream of the application in which it is installed. The local event monitor only observes the event stream of its application. To monitor events from all applications, including the "current" application, you must install both event monitors.
The code shown in that page is for a local event monitor, but the code for a global event monitor is similar; what changes is the invoked NSEvent's method.
_eventMonitor = [NSEvent addLocalMonitorForEventsMatchingMask:
(NSLeftMouseDownMask | NSRightMouseDownMask | NSOtherMouseDownMask | NSKeyDownMask)
handler:^(NSEvent *incomingEvent) {
NSEvent *result = incomingEvent;
NSWindow *targetWindowForEvent = [incomingEvent window];
if (targetWindowForEvent != _window) {
[self _closeAndSendAction:NO];
} else if ([incomingEvent type] == NSKeyDown) {
if ([incomingEvent keyCode] == 53) {
// Escape
[self _closeAndSendAction:NO];
result = nil; // Don't process the event
} else if ([incomingEvent keyCode] == 36) {
// Enter
[self _closeAndSendAction:YES];
result = nil;
}
}
return result;
}];
Once the monitor is not anymore necessary, you remove it using the following code:
[NSEvent removeMonitor:_eventMonitor];