OSX / Objective-C Window Management: manipulate the frames & visibility of other applications - objective-c

I would like to create a system tool / application which has the capacity to aid in window management. I'm trying to find documentation about the following topics, if they are indeed possible given the security sandboxing of OSX.
Show a list of running applications with the name & icon, and allow the user to choose one
Manipulate the frame(s) of said application's windows (eg, resize, reposition) from my app (with animations -- though I assume this will be trivial once I can perform the actual change)
Hide or show these applications from task managers, etc.
Be able to launch (or terminate) instances of the given application
It seems to me that Quicksilver accomplishes many of these things, but the lack of AppStore availability makes me wonder if it possible to do this while remaining in the OSX sandbox.

There are a lot of pieces of software out there that do window management. You can check out a tiling window manager I've been hacking on called Amethyst. The basic idea behind software like this relies on Accessibility (which you can find documentation for here). As a quick overview the APIs work by acquiring references to accessibility elements (applications, windows, buttons, text fields, etc.) which have properties (hidden, position, size, etc.), some of which are writable.
As an example let's say that you wanted to move all windows in every running application to the upper left corner of the screen. That code might look like
for (NSRunningApplication *runningApplication in [[NSWorkspace sharedWorkspace] runningApplications]) {
AXUIElementRef applicationRef = AXUIElementCreateApplication([runningApplication processIdentifier]);
CFArrayRef applicationWindows;
AXUIElementCopyAttributeValues(applicationRef, kAXWindowsAttribute, 0, 100, &applicationWindows);
if (!applicationWindows) continue;
for (CFIndex i = 0; i < CFArrayGetCount(applicationWindows); ++i) {
AXUIElementRef windowRef = CFArrayGetValueAtIndex(applicationWindows, i);
CGPoint upperLeft = { .x = 0, .y = 0 };
AXValueRef positionRef = AXValueCreate(kAXValueCGPointType, &upperLeft);
AXUIElementSetAttributeValue(windowRef, kAXPositionAttribute, positionRef);
}
}
Which illustrates how you get references to applications and their windows, how to copy attributes from an accessibility element, and how to set attributes of an accessibility element.
There are a variety of notifications documented in NSWorkspace for the launching and termination of applications, and the accessibility framework also has a sense of notifications for things like an application creating or destroying windows, or a window miniaturizing or deminiaturizing.
Animating the window changes is non-trivial and I haven't figured out how to do it yet, though it may be possible. It may not be possible at all without hitting private APIs. But the other things you have listed should be possible. Hiding an application, for example, could be done by setting the kAXHiddenAttribute on the application accessibility element. Launching an application can actually be done via -[NSWorkspace launchApplication:].
Note that the use of accessibility necessitates that the user have Enable access for assistive devices turned on in System Preferences > Accessibility.

Related

Controlling level and focus of windows other apps with CGPrivate functions

Question
How to use these private functions on other windows? It would be nice to have this knowledge back in the wild. I am specifically trying to get CGSOrderWindow and CGSSetWindowLevel to work.
I was trying in the direction of:
temporarily register as the dock and then register the dock as the dock again immediately afterwards
or
code injection into the Dock process per this comment:
Also, the author of the above project seems determined to make all core functionality available as a framework. It seems to be implemented as code injection into the Dock process.
Reason I know this is possible
I have been doing work on trying to setLevel on window of another app, and focus window of another app if focused. I am posting this again with the info I learned because from my searching online, I know this was done in the past, its just the knowledge is not publicly out there anymore. The sourceforge pages are no longer there. So I was wondering if you could help me make this information public again.
This is the topic I read that gave me this information - http://cocoadev.com/HowtoControlOtherAppsWindows
Here you see comments like:
You cannot control an another app's windows from a user-level process, unfortunately.
SlavaKarpenko
You can, Slava, you just need to register as the Dock. It might be possible to temporarily register as the dock and then register the dock as the dock again immediately afterwards, not sure. I think the call you'd be wanting to investigate as CoreDockRegisterDockOwner in HIServices.framework.
FinlayDobbie
You could also use APE or similar to do control the windows, or (as mentioned above) register as the Dock (look for the private APIs with Universal Connection in their name). Has anyone found a polite way of getting the Dock to give up its universal connection? The only way I can find is to force quit the Dock and grab the universal connection when it's not looking (which prevents the dock reloading).
SamTaylor
There's an open source project up on sourceforge.net that looks much more like the window managers I've used on Unix boxes than Space.app (or Space.dock): http://wsmanager.sourceforge.net/
SteveCook
Verifying things work
This is what I learned, from the sources at bottom of this post, we see all these functions work with CGWindowIds, so how do I get that, this is how:
Get all windows with CGWindowListCopyWindowInfo. Then access each element from that array with CFArrayGetValueAtIndex and then get the CGWindowId with objectForKey:, kCGWindowNumber, and then integerValue.
Now if I try to focus or set level of a window that is OWNED by the app running the code, it works fantastic. For instance:
MY_TARGET_CGWINDOW_ID = 179;
rez_CGError = CGSOrderWindow(_CGSDefaultConnection, MY_TARGET_CGWINDOW_ID, kCGSOrderAbove, 0);
Will focus it, rez_CGError is 0. Even if the window is minimized, it is unminimized, without animation, and shown.
Now however, if I try this on a window of a different app we get some errors:
MY_TARGET_CGWINDOW_ID_of_other_app = 40;
rez_CGError = CGSOrderWindow(_CGSDefaultConnection, MY_TARGET_CGWINDOW_ID_of_other_app, kCGSOrderAbove, 0);
This fails and rez_CGError is 1000, which I suspect means "cid (CGSConnection) used does not have permission to modify target window". The same happens if I first do [app activateWithOptions: (NSApplicationActivateIgnoringOtherApps | NSApplicationActivateAllWindows)] before making the call above.
So I first get the cid of that owning window like this:
var rez_CGError = CGSGetWindowOwner(_CGSDefaultConnection, MY_TARGET_CGWINDOW_ID_of_other_app, &ownerCid);
This works good and I get ownerCid is set to a value. Then I do the focus command with this new connection:
rez_CGError = CGSOrderWindow(ownerCid, MY_TARGET_CGWINDOW_ID_of_other_app, kCGSOrderAbove, 0);
However this gives rez_CGError of 268435459, which I suspect means "current app does not have permission to use this ConnectionId (cid)". (Same happens if I call activateWithOptions first.
My Sources for the Private Functions
Here is the sources for some private functions I found - https://code.google.com/p/undocumented-goodness/source/browse/trunk/CoreGraphics/CGSPrivate.h
This one source here contains a function that is not in the above link - CGSGetConnectionIDForPSN - i test it and it exists - from - https://github.com/mnutt/libqxt/blob/767498816dfa1742a6f3aee787281745afec11b8/src/gui/qxtwindowsystem_mac.h#L80

How to fake keyboard event to widget in Qt5

I have made a custom virtual keyboard widget for my kiosk application, and now comes the time when I want it to produce fake keyboard events and feed them to an QLineEdit of choice.
I do the following:
// target is the QWidget to receive the events
// k is the Qt::Key (keycode) I want to send (Testing with an 'A')
Qt::Key k=Qt::Key_A;
if(0!=target){
//According to docs this will be freed once posted
QKeyEvent * press=new QKeyEvent(QKeyEvent::KeyPress, (int )k,0);
QKeyEvent * release=new QKeyEvent(QKeyEvent::KeyRelease, (int )k,0);
//Give the target focus just to be sure it is available for input
target->setFocus();
//Post the events (queue up and let the target consume them when the eventloop gets around to the target)
QCoreApplication::postEvent ( target, press) ;
QCoreApplication::postEvent ( target, release) ;
}
I see the target widget receive focus, but there are no letters typed into the input field like I would expect. What am I doing wrong? Which assumptions are wrong?
PS: I know that this could be solved by using existing virtual keyboards or at least using the platform interface as is done in this post. In our approach we have decided to build the kayboard into the application to obtain full control over the UX and keyboard design.
Thanks!
Since no-one stepped up, I will try to provide some closure.
It turns out that Qt5 comes with a library of testing facilities called testlib. It has all sorts of goodies to facilitate easy creation, management and running of unit tests for Qt application. Among these facilities there is a set of functions for sending fake events such as fake typing of text, mouse clicks etc. It is quite comprehensive and covers many use-cases. Since this is used internally by Qt developers to test Qt itself it is also production proven code.
I simply copied what I needed from there.

Macos x Shortcuts of a Application

I need to read all Shortcuts of the frontmost Application in MAC OS. Is there a API or Class in Cocoa,Objective-c who provides this?
The Accessibility API is what you need. Unfortunately, it is rather convoluted, and it would be good to read through the UIElementInspector source code to see how it's used, as well as the sections relevant to assistive applications in the Accessibility Programming Guidelines for Mac.
What you want would take a good bit of code so I will just outline the steps.
Use [[NSWorkspace sharedWorkspace] runningApplications] to get a list of applications and get the application whose active property is YES.
Get the PID of that application using the NSRunningApplication's processIdentifier property.
Now we get to Accessibility, accessed through the Application Services API
Create an AXUIElement representing the active Application using AXUIElementCreateApplication, which takes pid as the argument.
Now you have an AXUIElement with the Application role, or AXApplication pseudo-class. You'll want to proceed down the hierarch of elements AXApplication -> AXMenuBar -> AXMenuBarItem -> AXMenuItem. Note that AXMenuItems can have other AXMenuItems nested under them.
To traverse the hierarchy, use AXUIElementCopyAttributeValues to get the values of kAXChildrenAttribute. This will return an array of children AXUIElements.
Finally, when you get to AXMenuItem elements, examine their Menu Item Cmd Char, Glyph, Modifiers, and Virtual Key attributes for the actual shortcuts. The constants for the attribute names, like kAXMenuItemCmdCharAttribute, are listed here.
Not trying to compete #woody :)
But I found this, and it might help too:
UI Browser

Register a global hotkey without support for assistive devices enabled

Using this code, I may register a global event handler:
[NSEvent addGlobalMonitorForEventsMatchingMask: NSKeyDownMask
handler: ^(NSEvent *incomingEvent) {
NSString *chars = [[incomingEvent characters] lowercaseString];
unichar character = [chars characterAtIndex:0];
// do something useful
NSLog(#"keydown globally! Which key? This key: %c", character);
}];
Unfortunately, events get passed along to this monitor, if support for assistive devices is enabled. Without assistive devices being enabled, no events get passed along.
Form the documentation:
Key-related events may only be monitored if accessibility is enabled or if your
application is trusted for accessibility access (see AXIsProcessTrusted).
I wonder, if another method exists, which passes along events without forcing the user to enable specific features of OS X.
While I didn't find a solution in Apple's docs, a solution must exist. E.g. the MAS-downloaded version of Alfred allows to define a hotkey.
Interestingly, Alfred's preferences only shows special keys and points out, that certain special key combinations may not work.
Since I basically want to show / hide a 'global' non-activating panel, I probably should simply prepare a system service. Should I?
You could try creating a Quartz Event Tap, with kCGSessionEventTap as the location.
Quartz Event Services Reference
Sample code from Mac OS X Internals

How to Write OS X Finder plugin

I'm looking for a guide or sample code for writing Mac OS X Finder plugins? It would like to know how to do some simple actions:
adding image overlayers to icons
adding context menu items
listen to file changes
I found the following two resources:
Writing Contextual Menu Plugins for OS X: An outdated document from 2002 that uses the COM API targeting Mac OS X 8/9.
SCPlugin: Open-source SVN Mac application that includes a Finder plug-in.
I am tempted to review the SCPlugin code, but was hoping to find an easier sample to digest.
The Finder Icon Overlay example project represents a small and very basic but actually working example of the answer below.
https://github.com/lesnie/Finder-Icon-Overlay
I know this is so old, but some may be still interested in topic (?)
Here is what I have it done under Leopard (10.6). At first proper Finder's headers are needed. Use class-dump tool to get it. Then write your code as a SIMBL plugin (refer to documentation how to do it), swizzling some methods. For instance to draw something over icon in ListView, drawIconWithFrame: method of TIconAndTextCell method must be overriden.
Here's the code for method swizzling:
+ (void) Plugin_load
{
Method old, new;
Class self_class = [self class];
Class finder_class = [objc_getClass("TIconAndTextCell") class];
class_addMethod(finder_class, #selector(FT_drawIconWithFrame:),
class_getMethodImplementation(self_class, #selector(FT_drawIconWithFrame:)),"v#:{CGRect={CGPoint=dd}{CGSize=dd}}");
old = class_getInstanceMethod(finder_class, #selector(drawIconWithFrame:));
new = class_getInstanceMethod(finder_class, #selector(FT_drawIconWithFrame:));
method_exchangeImplementations(old, new);
}
I am overriding "drawIconWithFrame:" method with my method "FT_drawIconWithFrame:". Below is sample implementation for this method.
- (void) FT_drawIconWithFrame:(struct CGRect)arg1
{
[self FT_drawIconWithFrame:arg1];
if ([self respondsToSelector:#selector(node)]) {
if ([[[[NSClassFromString(#"FINode") nodeWithFENode:[(TNodeIconAndNameCell *)self node]] fullPath] lastPathComponent] hasPrefix:#"A"])
[myPrettyIconOverlayImage drawInRect:NSMakeRect(arg1.origin.x, arg1.origin.y, arg1.size.height, arg1.size.height) fromRect:NSZeroRect operation:NSCompositeSourceOver fraction:1.0];
}
}
Essentially it draws "myPrettyIconOverlayImage" over every icon for file with filename starts with letter "A". This logic is up to you.
Pay attention to this line: [self FT_drawIconWithFrame:arg1]; this is how to call 'super' in order to get normal icon and name etc. I know, looks weird, like loop, but actually it isn't. Then wrap in into SIMBL plugin, install SIMBL and ...run.
Due to changes in Lion some work have to be done from scratch (make new "Finder.h" file with all declarations needed in it, find proper classess and methods to override), but this technique still works.
Happy hacking!
For Yosemite (MacOS 10.10 & newer), you can use Apple's FinderSync framework, which allows Finder extensions to:
Express interest in specific folder hierarchies
Provide "badges" to
indicate the status of items inside those hierarchies
Provide dynamic
menu items in Finder contextual menus, when the selected items (or
the window target) are in those hierarchies
Provide a Toolbar Item
that displays a menu with dynamic items (even if the selection is
unrelated)
Sadly, programming a Finder plugin actually does still require getting your hands dirty with COM. If you look at the SCFinderPlugin subproject of the SCPlugin project, you will find that it follows exactly the same techniques outlined in your first link, including setting up a vtable for COM, writing AddRef/ReleaseRef functions, and so on. Writing a plugin, where you're simultaneously managing old-school Carbon memory management, COM-style memory management, and Cocoa/new-style Carbon memory management, can be an incredible pain—and that totally ignores the fact that you'll be interacting in three or more radically different APIs, with different naming conventions and calling semantics. Calling the situation hysterically poor would be a vast understatement.
On the bright side, the Finder in Mac OS X 10.6 Snow Leopard has been fully rewritten in Cocoa--and with that come vastly superior plugin interfaces. If you are lucky enough to be in a situation where you can actually only target Snow Leopard, you probably should grab an ADC Premier or higher membership, download the prerelease builds, and code against that. Besides, your plugin may not work on 10.6 anyway without a Cocoa rewrite, so it might make good sense to take a look at Snow Leopard before it gets released, regardless.
There is no official or supported plugin system for the Finder. Starting with OS X 10.6, you will need to inject code into the Finder process and override objective C methods in the Finder process.
I've done this for a proprietary project. I can tell you that the reason that there are no examples or tutorials for this is because it is a significantly difficult and time consuming development task. For this reason, there's plenty of incentive for individuals or organizations who have accomplished this to guard the specifics of their process closely.
If there's any way at all that you can accomplish your goal using the Services API, do it. Writing a Finder plugin will take you 1-2 solid months of painstaking development and reasonably deep knowledge of C and Objective-C internals.
If you're still convinced that you want do to this, grab mach_star. Good luck.
As far as I know, there's no official plugin architecture for the Finder. You may be able to add image overlays to icons through an external application without having to hook into the Finder, although it wouldn't be on the fly. I don't think there is a way to add contextual menu items aside from Folder Actions and Automator. You can also look into writing an external application to monitor File System changes using the FSEvents API.
Here's a completed solution for Finder icon badges and contextual menus in Lion and Mountain Lion using the techniques described by Les Nie.
Liferay Nativity provides a scripting bundle that will swizzle the relevant Finder methods and a Java client for setting the icons and context menus. It also includes equivalent projects for Windows and Linux.
The project is open source under LGPL, so feel free to contribute any bug fixes or improvements!
The pickings are slim; it's never been really clear to me whether Finder Plugins are actually supported. A few more leads, though:
SampleCMPlugIn - Carbon-based of course, since so is Finder. Note that almost any Finder plugin is probably going to stop working with 10.6.
Automator can save things as a "Finder plugin." It's a more supported version of what you're discussing, but of course less flexible.
To add Finder/File browser icon overlays and context menus, in a cross-platform manner, from Java, take a look at the Liferay Nativity library.
I also make mention of this in another SO post, which also contains links to Apple's 'Finder Sync' docs and API.