Getting Window Number through OSX Accessibility API - objective-c

I am working on an application that moves windows of third party applications around on the screen.
To get an overview of all currently open windows, I use
CGWindowListCopyWindowInfo(kCGWindowListOptionOnScreenOnly | kCGWindowListExcludeDesktopElements, kCGNullWindowID);
This returns an array of dictionaries defining every open window.
Here's an exemplary dictionary returned:
{
kCGWindowAlpha = 1;
kCGWindowBounds = {
Height = 442;
Width = 475;
X = 3123;
Y = "-118";
};
kCGWindowIsOnscreen = 1;
kCGWindowLayer = 0;
kCGWindowMemoryUsage = 907184;
kCGWindowName = Untitled;
kCGWindowNumber = 7328;
kCGWindowOwnerName = TextEdit;
kCGWindowOwnerPID = 20706;
kCGWindowSharingState = 1;
kCGWindowStoreType = 2;
kCGWindowWorkspace = 3;
},
The dictionary is full of good information used elsewhere but lacks an accessibility object that could be used to modify the windows' positions. Windows are clearly identified by the Window Number.
I am now using the PID (kCGWindowOwnerPID) to create an accessibility object for the window's application:
AXUIElementRef app = AXUIElementCreateApplication(pid);
Followed by retrieving a list of all windows the application has opened using AXUIElementCopyAttributeValues:
NSArray *result;
AXUIElementCopyAttributeValues(
(AXUIElementRef) app,
kAXWindowsAttribute,
0,
99999,
(CFArrayRef *) &result
);
This works and returns an array of AXUIElements.
This is where I am stuck. There seems to be no API call to retrieve the Window Number of an accessibility object. Is there any way to either
a) Find the accessibility object's Window Number (to ultimately iterate over the array and find the right window)
or
b) Otherwise clearly match a window described in the array returned by CGWindowListCopyWindowInfo to the Accessibility Objects returned by AXUIElementCopyAttributeValues?

We ended up hiring a dedicated Accessibility Developer for this task.
It turns out there is no way to do this without using undocumented APIs (a no go in our case).
Luckily, there is a practical workaround:
Loop over all open windows of the app. Get their position, size and title:
AXUIElementCopyAttributeValue(target, kAXPositionAttribute, (CFTypeRef*)&posValue);
AXUIElementCopyAttributeValue(target, kAXSizeAttribute, (CFTypeRef*)&sizeValue);
AXUIElementCopyAttributeValue(target, kAXTitleAttribute, (CFTypeRef*)&titleValue);
Next, convert the position and size into actual CGPoint and CGSize values:
AXValueGetValue(posValue, kAXValueCGPointType, &point);
AXValueGetValue(sizeValue, kAXValueCGSizeType, &size);
Compare the size, position and title against the values returned by the object in CGWindowListCopyWindowInfo().
If they match, you can safely assume it's the window you were looking for and use the already open AXUIElement (target in our case) to work it.
The overhead for looping through all open windows turns out to be negligible on OSX. There is a pretty low cap on how many windows are open at the same time.
Also, while this is not 100% accurate (it is possible that 2 windows have the same position, size and title), we haven't encountered any situation in real usage where this happens so far.

There is a private function for obtaining CG window number for a given AX object for window: _AXUIElementGetWindow .
More details in SO discussion Uniquely identify active window on OS X
It looks like there is no public API to do the task with 100% probability. Identifying windows by title and frame (as described in answer above) will works in 99.9% of cases.

Clarifying other answers that suggest the undocumented call of _AXUIElementGetWindow.
In Swift, do this:
1. Create Bridged-Header.h with the following contents:
#import <AppKit/AppKit.h>
AXError _AXUIElementGetWindow(AXUIElementRef element, uint32_t *identifier);
2. Reference this file in your build settings: (your path may vary, it is relative to project root)
3. Call like this:
// variable 'window' is your AXUIElement window
var cgWindowId = CGWindowID()
if (_AXUIElementGetWindow(window, &gcWindowId) != .success) {.
print("cannot get CGWindow id (objc bridged call)")
}
Congrats, you succesfully called objective C functions via a bridge!

Related

windowId I get from appleScript and CGWindowInfo is different?

I want to use applescript to get the window of an application, but since the app can have several windows, I use objc to get windowId like follow:
CFArrayRef windowList = CGWindowListCopyWindowInfo(kCGWindowListOptionOnScreenOnly |
kCGWindowListExcludeDesktopElements,
kCGNullWindowID);
for (CFIndex i = 0; i < windowListCount; i++) {
CFDictionaryRef windowInfo = (CFDictionaryRef)CFArrayGetValueAtIndex(windowList, i);
CGWindowID windowId;
CFNumberGetValue((CFNumberRef)CFDictionaryGetValue(windowInfo, kCGWindowNumber), kCGWindowIDCFNumberType, &windowId);
}
but this windowId is different from the windowId I get from the appleScript like follow:
tell application "/Applications/QQBrowser.app"
repeat with theWindow in windows
if visible of theWindow is true then
get id of theWindow
end if
end repeat
end tell
I did not find sth useful, does anybody know why?
my destination is to distinguish the windows of an app, does anybody have some ideas?
Thanks very much~
tell application "Numbers"
get id of every window whose visible is true
end tell
Result: {1462, 1439}
These are unique id's for the open visible windows. You could use this unique id to distinguish windows in an app, and you can also get more information about each window with code like:
get properties of window id 1462

Getting Actual Placement of a Flyout

I have a flyout that appears near the cursor when a particular user input is triggered.
According to the docs, the Flyout.Placement property is only a preference and does not necessarily reflect the position of the flyout.
I expect the framework to call GetFlyoutPlacement to determine the actual placement. I want to know what the actual placement is so that I can add a little arrow pointing at the cursor position, something like shown in the following:
Is my only option to make the call to GetFlyoutPlacement myself (triggered on the Flyout.Opening event), or is there some more convenient way to do this?
My calls on GetFlyoutPlacement and GetFlyoutPlacementTargetInfo resulted in Error HRESULT E_FAIL has been returned from a call to a COM component. which didn't leave me very much to go on, so I just ended up implementing what I imagine is the logic behind those functions:
public static FlyoutPlacementMode GetActualFlyoutPlacement(
Rect placementTarget,
Size flyoutSize,
FlyoutPlacementMode preferredPlacement )
{
Rect ViewArea = Window.Current.Bounds;
ViewArea.X = 0; // may have non-zero offset for multi-monitor setups
ViewArea.Y = 0; // but we are only interested in offset relative to app view area
switch (preferredPlacement)
{
case FlyoutPlacementMode.Right:
if (desiredSize.Width < ViewArea.Width - placementTarget.Right) return FlyoutPlacementMode.Right;
if (desiredSize.Width < placementTarget.Left) return FlyoutPlacementMode.Left;
if (desiredSize.Height < placementTarget.Top) return FlyoutPlacementMode.Top;
return FlyoutPlacementMode.Bottom;
case FlyoutPlacementMode.Left:
...
}
}
Checkout the approach I suggested here. I believe it is better, because instead of guessing the internal placement algorithm it just compares the absolute coordinates of the flyout and the target element.

How do I dynamically change the height of a TableViewRow?

Application Type: mobile
Titanium SDK: 3.1.0.GA
Platform & version: iOS 6.1
Device: iOS Simulator
Host Operating System: OSX 10.8.3
Titanium Studio: 3.1.0.201304151600
I'd like to conditionally show/hide a textfield in a TableViewRow. In order to do this I need to expand the height of the row. The following code doesn't work, though. The TableViewRow is actually an Alloy controller. I first tried animating it before I realized that it can't be animated. Now I'm just trying to change the height and that isn't even working. I've tried using the setHeight method along with just setting the height property directly to no avail.
Any ideas?
var notesVisible = false;
function notes_click() {
if(notesVisible == false) {
Ti.API.info('expanding');
$.row.height = 200;
// $.notes_container.setHeight(124);
notesVisible = true;
} else {
Ti.API.info('contracting');
$.row.height = 75;
$.notes_container.setHeight(0);
notesVisible = false;
}
};
There are two good ways of doing this, both should be done from the click event listener.
Method 1) One way is to directly change the "height" variable of the row
Method 2) The second is to create a new row and replace the current row with the new row
Method 1 is more straightforward but I found it to be glitchy depending on what version of the SDK you are using, but with 3.1.0 it should work. Both methods should be called from the 'click' eventListener as its easier to tell Titanium which row to act on based on the click
So here is an example
currentTableview.addEventListener('click',function(e)
{
// DO whatever your row is supposed to do when clicked
// Now lets change the height of the row to a new height, 200 in this example
e.row.height = 200
}
With Method two, it involves creating a new row and then replacing the current row with this call
currentTableview.updateRow(e.index,newlyCreatedUpdatedRow);
I know its an old question asked by some one but the solution provided will not work and i think best solution for this is by making a recursive function and changes the height of your row and need to play with the visibility of views inside that row hopefully will help someone :)

Problems in my AS2 Game

Hey guys, I'm trying to make a 2D Platform style game similar to this game below:
http://www.gameshed.com/Puzzle-Games/Blockdude/play.html
I have finished making most of the graphic, and areas, and collision, but our character is still not able to carry things. I'm confused as to what code to use so that my character can carry the blocks. I need help as to how to make our character carry blocks that are in front of him, provided that the blocks that don't have anything on top of it. This has been confusing me for a week now, and any help would be highly appreciated. :D
I fondly remember my first AS2 game. The best approach is probably an object oriented approach, as I will explain.
In AS2, there is a hittest method automatically built into objects. There is a good tutorial on Kirupa here:
http://www.kirupa.com/developer/actionscript/hittest.htm
also
http://help.adobe.com/en_US/AS2LCR/Flash_10.0/help.html?content=00001314.html
First you'll want to generate your boxes using a Box class. Your class would need to look something like the following:
//Box.as pseudo-code
class Box {
var x_pos:Number;
var y_pos:Number;
var attachedToPlayer:Boolean;
function Box(_x:Number, _y:Number) {
this.x_pos = _x;
this.y_pos = _y;
}
//other code here
}
See this tutorial on how to attach a class to an object in the library:
http://www.articlesbase.com/videos/5min/86620312
To create a new Box, you'd then use something like
box1 = new Box(100,200);
// creates a box at position 100x,200y
However, you'll also want to store the blocks you want to pickup into some sort of array so you can loop through them. See http://www.tech-recipes.com/rx/1383/flash-actionscript-create-an-array-of-objects-from-a-unique-class/
Example:
//somewhere near the top of your main method, or whereever your main game loop is running from - note Box.as would need to be in the same folder
import Box;
//...then, somewhere before your game loop
//create an array to hold the objects
var boxArray:Array = new Array();
//create loop with i as the counter
for (var i=0; i<4; i++)
{
var _x:Number = 100 + i;
var _y:Number = 100 + i;
//create Box object
var box:Box = new Box();
//assign text to the first variable.
//push the object into the array
boxArray.push(box);
}
Similarly, you would need a class for your player, and to create a new Player object at the start of your game, e.g.
var player = new Player(0,0);
You could then run a hittest method for your player against the blocks in your array for the main game loop (i.e. the loop that updates your player's position and other game properties). There are probably more efficient ways of doing this, e.g. only looping for the blocks that are currently on the screen.
Once your array has been created, use a foreach loop to run a hittest against your player in your game's main loop, e.g.
//assuming you have an array called 'boxArray' and player object called 'player'
for(var box in boxArray){
if (player.hittest(box)) {
player.attachObjectMethod(box);
}
}
This is basically pseudo-code for "for every box that we have entered into the array, check if the player is touching the box. If the box is touching, use the box as the argument for a method in the player class (which I have arbitrarily called attachObjectMethod)".
In attachObjectMethod, you could then define some sort of behavior for attaching the box to the player. For example, you could create a get and set method(s) for the x and y position of your boxes inside the box class, along with a boolean called something useful like attachedToPlayer. When attachObjectMethod was called, it would set the box's boolean, e.g. in the Player class
//include Box.as at the top of the file
import Box;
//other methods, e.g. constructor
//somewhere is the Player.as class/file
public function attachObjectMethod (box:Box) {
box.setattachedToPlayer(true);
//you could also update fields on the player, but for now this is all we need
}
Now the attachedToPlayer boolean of the box the player has collided with would be true. Back in our game loop, we would then modify our loop to update the position of the boxes:
//assuming you have an array called 'boxArray' and player object called 'player'
for(var box in boxArray){
if (player.hittest(box)) {
player.attachObjectMethod(box);
}
box.updatePosition(player.get_Xpos, player.get_Ypos);
}
In our Box class, we now need to define 'updatePosition':
//Box.as pseudo-code
class Box {
var x_pos:Number;
var y_pos:Number;
var attachedToPlayer:Boolean;
function Box(box_x:Number, box_y:Number) {
this.x_pos = box_x;
this.y_pos = box_y;
}
public function updatePosition(_x:Number, _y:Number) {
if (this.attachedToPlayer) {
this.x_pos = _x;
this.y_pos = _y;
}
}
//other code here
}
As you can see we can pass the player's position, and update the box's position if the attachedToPlayer boolean has been set. Finally, we add a move method to the box:
public function move() {
if (this.attachedToPlayer) {
this._x = x_pos;
this._y = y_pos;
}
}
Examples of updating position:
http://www.swinburne.edu.au/design/tutorials/P-flash/T-How-to-smoothly-slide-objects-around-in-Flash/ID-17/
Finally, to make it all work we need to call the move method in the game loop:
//assuming you have an array called 'boxArray' and player object called 'player'
for(var box in boxArray){
if (player.hittest(box)) {
player.attachObjectMethod(box);
}
box.updatePosition(player.get_Xpos, player.get_Ypos);
box.move();
}
You have also specified that the blocks should only move with the player if they have nothing on top of them. When you call your attachedToPlayer method, you would also need to run a foreach loop inside the method between the box and the objects that might sit on top of the box. You should now have a fair idea from the above code how to do this.
I appreciate that this is quite a lengthy answer, and I haven't had an opportunity to test all the code (in fact I'm fairly positive I made a mistake somewhere) - don't hesitate to ask questions. My other advice is to understand the concepts thoroughly, and then write your own code one bit at a time.
Good luck!
The way I would do this is to design an individual hit test for each block he will be picking up, then code for the hit test to play a frame within the sprite's timeline of him carrying a block, and to play a frame within the block to be picked up's timeline of the block no longer at rest (disappeared?).
Good Luck if you're confused about what I've said just ask a little more about it and I'll try to help you if I can.

Placing a window near the system tray

I am writing a program that needs to set a window just above/below the traybar for gtk. I have tried using the 2 approaches that failed. One was using the gtk_status_icon_position_menu function and placing the window in the point where the user clicks (in the tray bar). The problem is that these solutions work in gnome(Linux) but not in Windows. In Linux they work because the window manager doesn't seem to allow placement of windows in the tray panel, honoring the closest possible. In Windows this doesn't happen and the window can go "out" of the screen which understandably is not desired.
With this said i went out for a work around. My idea was to set the window in the location of mouse click and get the x and y coordinates of a normal window placement and with it's size check if it would be within the screen boundaries. If it was not make the correction. I have came up with the functions needed but for some reason the gdk_drawable_get_size(window->window ,&WindowWidth, &WindowHeight) and other similar functions only give the correct size value after the second run of the signal function. The result of the first run is just 1 to both size and width. (I have read the issue of X11 not giving correct results, but i think this is not it)
event_button = (GdkEventButton *) event;
if (event_button->button == 1)
{
if (active == 0)
{
gboolean dummy;
gint WindowHeight, WindowWidth, WindowPosition[2];
GdkScreen *screen;
gint ScreenHeight, ScreenWidth;
dummy = FALSE;
gtk_widget_show_all(window);
gtk_window_present(GTK_WINDOW(window));
gtk_status_icon_position_menu(menu, &pos[X], &pos[Y], &dummy, statusicon);
gtk_window_move(GTK_WINDOW(window), pos[X],pos[Y]);
gdk_drawable_get_size(window->window ,&WindowWidth, &WindowHeight);
screen = gtk_status_icon_get_screen(statusicon);
ScreenWidth = gdk_screen_get_width(screen);
ScreenHeight = gdk_screen_get_height(screen);
g_print("Screen: %d, %d\nGeometry: %d, %d\n",ScreenWidth, ScreenHeight, WindowWidth, window->allocation.height);
gtk_entry_set_text(GTK_ENTRY(entry),"");
active = 1;
return TRUE;
}
How can i do what i want in a portable way?
I think gtk_status_icon_position_menu should work fine, as i used it for the same purpose.
Are you trying this stuff with X11, What is the actual result you are getting.