Capture screenshot of macOS window - objective-c

Note: this question is intentionally very general (e.g. both Objective-C and Swift code examples are requested), as it is intended to document how to capture a window screenshot on macOS as accessibly as possible.
I want to capture a screenshot of a macOS window in Objective-C/Swift code. I know this is possible because of the multitude of ways to take a screenshot on macOS (⇧⌘4, the Grab utility, screencapture on the command line, …), but I’m not sure how to do it in my own code. Ideally, I’d be able to specify a window of a particular application, and then capture it in an NSImage or CGImage that I could then process and display to the user or store in a file.

Screen capture on macOS is possible through Quartz Window Services, a facility of the Core Graphics framework. Our key function here is CGWindowListCreateImage, which “returns a composite image based on a dynamically generated list of windows,” or, in other words, finds windows based on specified criteria and creates an image with the contents of each. Perfect! Its declaration is as follows:
CGImageRef CGWindowListCreateImage(CGRect screenBounds,
CGWindowListOption listOption,
CGWindowID windowID,
CGWindowImageOption imageOption);
So, in order to capture one specific window on the screen, we’ll need its window ID (CGWindowID). To go about retrieving that, we’ll first need a list of all of the windows available on the system. We get that through CGWindowListCopyWindowInfo, which takes CGWindowListOptions and a corresponding CGWindowID that, together, select which windows to include in the resulting list. To get ALL the windows, we specify kCGWindowListOptionAll, and kCGNullWindowID, respectively. Also, if you haven’t figured it out already, this is a C API, so we’ll use a bridging cast to work with the friendlier Objective-C containers rather than the Core Foundation ones.
Objective-C:
NSArray<NSDictionary*> *windowInfoList = (__bridge_transfer id)
CGWindowListCopyWindowInfo(kCGWindowListOptionAll, kCGNullWindowID);
Swift:
let windowInfoList = CGWindowListCopyWindowInfo(.optionAll, kCGNullWindowID)!
as NSArray
From here, we need to filter our windowInfoList down to the specific window that we want. Chances are we want to filter first by application. To do that, we’ll need the process ID of our application of choice. We can use NSRunningApplication to accomplish this:
Objective-C:
NSArray<NSRunningApplication*> *apps =
[NSRunningApplication runningApplicationsWithBundleIdentifier:
/* Bundle ID of the application, e.g.: */ #"com.apple.Safari"];
if (apps.count == 0) {
// Application is not currently running
puts("The application is not running");
return; // Or whatever
}
pid_t appPID = apps[0].processIdentifier;
Swift:
let apps = NSRunningApplication.runningApplications(withBundleIdentifier:
/* Bundle ID of the application, e.g.: */ "com.apple.Safari")
if apps.isEmpty {
// Application is not currently running
print("The application is not running")
return // Or whatever
}
let appPID = apps[0].processIdentifier
With appPID in hand, we can now go ahead and filter down our window info list to only windows with a matching owner PID:
Objective-C:
NSMutableArray<NSDictionary*> *appWindowsInfoList = [NSMutableArray new];
for (NSDictionary *info in windowInfoList) {
if ([info[(__bridge NSString *)kCGWindowOwnerPID] integerValue] == appPID) {
[appWindowsInfoList addObject:info];
}
}
Swift:
var appWindowsInfoList = [NSDictionary]()
for info_ in windowInfoList {
let info = info_ as! NSDictionary
if (info[kCGWindowOwnerPID as NSString] as! NSNumber).intValue == appPID {
appWindowsInfoList.append(info)
}
}
We could have done additional filtering above by testing other keys of the info dictionary—for example, by name (kCGWindowName), or by whether the window is on-screen (kCGWindowIsOnscreen)—but for now, we’ll just take the first window in the list:
Objective-C:
NSDictionary *appWindowInfo = appWindowsInfoList[0];
CGWindowID windowID = [appWindowInfo[(__bridge NSString *)kCGWindowNumber] unsignedIntValue];
Swift:
let appWindowInfo: NSDictionary = appWindowsInfoList[0];
let windowID: CGWindowID = (appWindowInfo[kCGWindowNumber as NSString] as! NSNumber).uint32Value
And we have our window ID! Now, what else did we need for that call again?
CGImageRef CGWindowListCreateImage(CGRect screenBounds,
CGWindowListOption listOption,
CGWindowID windowID,
CGWindowImageOption imageOption);
First, we need a screenBounds to capture. According to the documentation, we can specify CGRectNull for this parameter to enclose all specified windows as tightly as possible. Works for me.
Second, we have to specify how we want to select our windows with listOption. We actually used one of these earlier, in our call to CGWindowListCopyWindowInfo, but there we wanted all the windows on the system; here, we only want one, so we’ll specify kCGWindowListOptionIncludingWindow, which, contrary to its documentation page, is meaningful on its own for CGWindowListCreateImage in that it specifies the window we pass, and only the window we pass.
Third, we pass our windowID as the window we want captured.
Fourth and finally, we can specify CGWindowImageOptions with the imageOption parameter. These affect the appearance of the resulting image; you can combine them through bitwise OR. The full list is here, but common ones include either kCGWindowImageDefault, which captures the window's contents along with its frame and shadow, or kCGWindowImageBoundsIgnoreFraming, which captures only the content, and kCGWindowImageBestResolution, which captures the window's content at the best resolution available, regardless of actual size (and, depending on the window, may be considerably large), or kCGWindowImageNominalResolution, which captures the window at its current size on the screen. Here, I’ve gone with kCGWindowImageBoundsIgnoreFraming and kCGWindowImageNominalResolution to capture only the content at the same size as on the screen.
Aaand, drumroll please:
Objective-C:
CGImageRef windowImage =
CGWindowListCreateImage(CGRectNull, kCGWindowListOptionIncludingWindow,
windowID, kCGWindowImageBoundsIgnoreFraming|
kCGWindowImageNominalResolution);
// NOTE: windowImage may be NULL if the capture failed
Swift:
let windowImage: CGImage? =
CGWindowListCreateImage(.null, .optionIncludingWindow, windowID,
[.boundsIgnoreFraming, .nominalResolution])

Here's the Objective C code without all the exposition, and no need to know your bundle ID ahead of time:
int processID = [[NSProcessInfo processInfo] processIdentifier];
NSArray<NSDictionary*>* windowInfoList = (__bridge_transfer id) CGWindowListCopyWindowInfo(kCGWindowListOptionOnScreenOnly, kCGNullWindowID);
int windowID = -1;
for (NSDictionary* info in windowInfoList) {
int thisProcess = [info[(__bridge NSString *)kCGWindowOwnerPID] integerValue];
if (thisProcess == processID) {
windowID = [info[(__bridge NSString *)kCGWindowNumber] integerValue];
break;
}
}
CGImageRef screenCG = nil;
if (windowID != -1)
screenCG = CGWindowListCreateImage(CGRectNull, kCGWindowListOptionIncludingWindow, windowID, kCGWindowImageBoundsIgnoreFraming);

Related

How to check CGWindowID still valid

If I am getting a CGWindowID (_windowID) as follows...
NSArray *windowList = (__bridge NSArray *)CGWindowListCopyWindowInfo(kCGWindowListOptionOnScreenOnly, kCGNullWindowID);
for (NSDictionary *info in windowList) {
if ([[info objectForKey:(NSString *)kCGWindowOwnerName] isEqualToString:#"window name"] && ![[info objectForKey:(NSString *)kCGWindowName] isEqualToString:#""]) {
_windowID = [[info objectForKey:(NSString *)kCGWindowNumber] unsignedIntValue];
}
}
How do I properly test the window id is still valid and the window has not been closed? Do i just run similar code just checking window id exists?
Thanks in advance
The documentation for the kCGWindowListOptionOnScreenOnly constant says:
List all windows that are currently onscreen. Windows are returned in
order from front to back. When retrieving a list with this option, the
relativeToWindow parameter should be set to kCGNullWindowID.
So the windows would certainly be on screen, since nothing appears to be happening between the call to CGWindowListCopyWindowInfo and your action on it.
Maybe you'd want to test to make sure they are not hidden or minimized?

CGWindowListCreate generates a hugely long list of windows

When I use CGWindowListCreate from quartz window services, it generates a very long array of window id's. I tried to turn on the option to exclude desktop elements, but I get a list of 30-40 windows even if there are only 3 or 4 of what I would call windows open.
Here is how I am doing it:
CGWindowListOption opt = 1 << 4;
CFArrayRef windowids =CGWindowListCreate(opt,kCGNullWindowID);
I am wondering what I am doing wrong that is causing this problem, and what I can do to fix it. I simply want the program to list windows created by applications, such as finder windows or browser windows, and not whatever else it is including. Thank you in advance for your help.
This will return every window whether it is on screen or off screen, you should combine it with the option kCGWindowListOptionOnScreenOnly (and also don't hardcode the one you are using). It will look like this:
CGWindowListOption opt = kCGWindowListOptionOnScreenOnly|kCGWindowListExcludeDesktopElements;
CFArrayRef windowids =CGWindowListCreate(opt,kCGNullWindowID);
That is what I gathered from the docs anyway.
I discovered a solution is to filter the window list to only those windows "below" the Dock (in terms of window layering).
The code below worked well for me. It fetches all on screen windows (excluding desktop elements). It extracts the window ID for the "Dock" window out of the list. Then fetches on screen windows again, filtering to only those windows "below" the Dock window.
// Fetch all on screen windows
CFArrayRef windowListArray = CGWindowListCreate(kCGWindowListOptionOnScreenOnly|kCGWindowListExcludeDesktopElements, kCGNullWindowID);
NSArray *windows = CFBridgingRelease(CGWindowListCreateDescriptionFromArray(windowListArray));
NSLog(#"All on screen windows: %#", windows);
// Find window ID of "Dock" window
NSNumber *dockWindowNumber = nil;
for (NSDictionary *window in windows) {
if ([(NSString *)window[(__bridge NSString *)kCGWindowName] isEqualToString:#"Dock"]) {
dockWindowNumber = window[(__bridge NSString *)kCGWindowNumber];
break;
}
}
NSLog(#"dockWindowNumber: %#", dockWindowNumber);
CFRelease(windowListArray);
if (dockWindowNumber) {
// Fetch on screen windows again, filtering to those "below" the Dock window
// This filters out all but the "standard" application windows
windowListArray = CGWindowListCreate(kCGWindowListOptionOnScreenBelowWindow|kCGWindowListExcludeDesktopElements, [dockWindowNumber unsignedIntValue]);
NSArray *windows = CFBridgingRelease(CGWindowListCreateDescriptionFromArray(windowListArray));
NSLog(#"On screen application windows: %#", windows);
}
else {
NSLog(#"Could not find Dock window description");
}

Play specific title in iTunes via ScriptingBridge

I'm trying to write an application that interacts with iTunes via ScriptingBridge. I works well so far, but the options of this method seem to be very limited.
I want to play song with a given name, but it looks like there's no way to do this. I haven't found anything similar in iTunes.h…
In AppleScript it's just three lines of code:
tell application "iTunes"
play (some file track whose name is "Yesterday")
end tell
And then iTunes starts to play a classic Beatles song.
Is there any was I can do this with ScriptingBridge or do I have to run this AppleScript from my app?
It's not as simple as the AppleScript version, but it's certainly possible.
Method one
Get a pointer to the iTunes library:
iTunesApplication *iTunesApp = [SBApplication applicationWithBundleIdentifier:#"com.apple.iTunes"];
SBElementArray *iTunesSources = [iTunesApp sources];
iTunesSource *library;
for (iTunesSource *thisSource in iTunesSources) {
if ([thisSource kind] == iTunesESrcLibrary) {
library = thisSource;
break;
}
}
Get an array containing all the audio file tracks in the library:
SBElementArray *libraryPlaylists = [library libraryPlaylists];
iTunesLibraryPlaylist *libraryPlaylist = [libraryPlaylists objectAtIndex:0];
SBElementArray *musicTracks = [self.libraryPlaylist fileTracks];
Then filter the array to find tracks with the title you're looking for.
NSArray *tracksWithOurTitle = [musicTracks filteredArrayUsingPredicate:[NSPredicate predicateWithFormat:#"%K == %#", #"name", #"Yesterday"]];
// Remember, there might be several tracks with that title; you need to figure out how to find the one you want.
iTunesTrack *rightTrack = [tracksWithOurTitle objectAtIndex:0];
[rightTrack playOnce:YES];
Method two
Get a pointer to the iTunes library as above. Then use the Scripting Bridge searchFor: only: method:
SBElementArray *tracksWithOurTitle = [library searchFor:#"Yesterday" only:kSrS];
// This returns every song whose title *contains* "Yesterday" ...
// You'll need a better way to than this to pick the one you want.
iTunesTrack *rightTrack = [tracksWithOurTitle objectAtIndex:0];
[rightTrack playOnce:YES];
Caveat to method two: The iTunes.h file incorrectly claims that the searchFor: only: method returns an iTunesTrack*, when in fact (for obvious reasons) it returns an SBElementArray*. You can edit the header file to get rid of the resulting compiler warning.

addGlobalMonitorForEventsMatchingMask only returning mouse position

I'm trying to learn to code for the Mac. I've been a Java guy for a while, so I hope the problem I'm running into is a simple misunderstanding of Cocoa.
I've got the following code:
-(IBAction)beginEventMonitor:(id)sender {
_eventMonitor = [NSEvent addGlobalMonitorForEventsMatchingMask:(NSLeftMouseUpMask)
handler:^(NSEvent *incomingEvent) {
//NSWindow *targetWindowForEvent = [incomingEvent window];
NSLog(#"Got a mouse click event at %#", NSStringFromPoint([incomingEvent locationInWindow]));
}];
}
-(IBAction)stopEventMonitor:(id)sender {
if (_eventMonitor) {
[NSEvent removeMonitor:_eventMonitor];
_eventMonitor = nil;
}
}
This is a simple hook to tell me when a mouse click happens at a global level. The handler is working, but the contents of the incomingEvent don't seem to be set to anything. The only useful information that I can find is the location of the mouse at the time of the click, and the windowId of the window that was clicked in.
Shouldn't I be able to get more information? Am I not setting up the monitor correctly? I'd really like to be able to know which window was clicked in, but I can't even find a way to turn the mouse location or windowId into something useful.
You can retrieve more information about the window using the CGWindow APIs (new in Leopard), for example:
CGWindowID windowID = (CGWindowID)[incomingEvent windowNumber];
CFArrayRef a = CFArrayCreate(NULL, (void *)&windowID, 1, NULL);
NSArray *windowInfos = (NSArray *)CGWindowListCreateDescriptionFromArray(a);
CFRelease(a);
if ([windowInfos count] > 0) {
NSDictionary *windowInfo = [windowInfos objectAtIndex:0];
NSLog(#"Name: %#", [windowInfo objectForKey:(NSString *)kCGWindowName]);
NSLog(#"Owner: %#", [windowInfo objectForKey:(NSString *)kCGWindowOwnerName]);
//etc.
}
[windowInfos release];
There's lots of information there (look in CGWindow.h or refer to the docs for available keys). There are also functions to create screenshots of just one window (which even works if it's partially covered by another window), cool stuff!

Displaying file copy progress using FSCopyObjectAsync

It appears after much searching that there seems to be a common problem when trying to do a file copy and show a progress indicator relative to the amount of the file that has been copied. After spending some considerable time trying to resolve this issue, I find myself at the mercy of the StackOverflow Gods once again :-) - Hopefully one day I'll be among those that can help out the rookies too!
I am trying to get a progress bar to show the status of a copy process and once the copy process has finished, call a Cocoa method. The challenge - I need to make use of File Manager Carbon calls because NSFileManager does not give me the full ability I need.
I started out by trying to utilize the code on Matt Long's site Cocoa Is My Girlfriend. The code got me some good distance. I managed to get the file copy progress working. The bar updates and (with some additional searching within Apple docs) I found out how to tell if the file copy process has finished...
if (stage == kFSOperationStageComplete)
However, I have one last hurdle that is a little larger than my leap right now. I don't know how to pass an object reference into the callback and I don't know how to call a Cocoa method from the callback once finished. This is a limit of my Carbon -> Cocoa -> Carbon understanding. One of the comments on the blog said
"Instead of accessing the progress indicator via a static pointer, you can just use the void *info field of the FSFileOperationClientContext struct, and passing either the AppDelegate or the progress indicator itself."
Sounds like a great idea. Not sure how to do this. For the sake of everyone else that appears to bump into this issue and is coming from a non-Carbon background, based mostly upon the code from Matt's example, here is some simplified code as an example of the problem...
In a normal cocoa method:
CFRunLoopRef runLoop = CFRunLoopGetCurrent();
FSFileOperationRef fileOp = FSFileOperationCreate(kCFAllocatorDefault);
OSStatus status = FSFileOperationScheduleWithRunLoop(fileOp,
runLoop, kCFRunLoopDefaultMode);
if (status) {
NSLog(#"Failed to schedule operation with run loop: %#", status);
return NO;
}
// Create a filesystem ref structure for the source and destination and
// populate them with their respective paths from our NSTextFields.
FSRef source;
FSRef destination;
// Used FSPathMakeRefWithOptions instead of FSPathMakeRef which is in the
// original example because I needed to use the kFSPathMakeRefDefaultOptions
// to deal with file paths to remote folders via a /Volume reference
FSPathMakeRefWithOptions((const UInt8 *)[aSource fileSystemRepresentation],
kFSPathMakeRefDefaultOptions,
&source,
NULL);
Boolean isDir = true;
FSPathMakeRefWithOptions((const UInt8 *)[aDestDir fileSystemRepresentation],
kFSPathMakeRefDefaultOptions,
&destination,
&isDir);
// Needed to change from the original to use CFStringRef so I could convert
// from an NSString (aDestFile) to a CFStringRef (targetFilename)
CFStringRef targetFilename = (CFStringRef)aDestFile;
// Start the async copy.
status = FSCopyObjectAsync (fileOp,
&source,
&destination, // Full path to destination dir
targetFilename,
kFSFileOperationDefaultOptions,
statusCallback,
1.0,
NULL);
CFRelease(fileOp);
if (status) {
NSString * errMsg = [NSString stringWithFormat:#"%# - %#",
[self class], status];
NSLog(#"Failed to begin asynchronous object copy: %#", status);
}
Then the callback (in the same file)
static void statusCallback (FSFileOperationRef fileOp,
const FSRef *currentItem,
FSFileOperationStage stage,
OSStatus error,
CFDictionaryRef statusDictionary,
void *info )
{
NSLog(#"Callback got called.");
// If the status dictionary is valid, we can grab the current values to
// display status changes, or in our case to update the progress indicator.
if (statusDictionary)
{
CFNumberRef bytesCompleted;
bytesCompleted = (CFNumberRef) CFDictionaryGetValue(statusDictionary,
kFSOperationBytesCompleteKey);
CGFloat floatBytesCompleted;
CFNumberGetValue (bytesCompleted, kCFNumberMaxType,
&floatBytesCompleted);
NSLog(#"Copied %d bytes so far.",
(unsigned long long)floatBytesCompleted);
// fileProgressIndicator is currently declared as a pointer to a
// static progress bar - but this needs to change so that it is a
// pointer passed in via the controller. Would like to have a
// pointer to an instance of a progress bar
[fileProgressIndicator setDoubleValue:(double)floatBytesCompleted];
[fileProgressIndicator displayIfNeeded];
}
if (stage == kFSOperationStageComplete) {
NSLog(#"Finished copying the file");
// Would like to call a Cocoa Method here...
}
}
So the bottom line is how can I:
Pass a pointer to an instance of a progress bar from the calling method to the callback
Upon completion, call back out to a normal Cocoa method
And as always, help is much appreciated (and hopefully the answer will solve many of the issues and complaints I have seen in many threads!!)
You can do this by using the last parameter to FSCopyObjectAsync(), which is a struct of type FSFileOperationClientContext. One of the fields of that struct is info, which is a void* parameter that you can basically use as you see fit. Whatever you assign to that field of the struct you pass into FSCopyObjectAsync() will be passed in turn to your callback function as the last info function parameter there. A void* can be anything, including a pointer to an object, so you can use that to pass the instance of your object that you want to handle the callback.
The setup code would look like this:
FSFileOperationClientContext clientContext = {0}; //zero out the struct to begin with
clientContext.info = myProgressIndicator;
//All the other setup code
status = FSCopyObjectAsync (fileOp,
&source,
&destination, // Full path to destination dir
targetFilename,
kFSFileOperationDefaultOptions,
statusCallback,
1.0,
&clientContext);
Then, in your callback function:
static void statusCallback (FSFileOperationRef fileOp,
const FSRef *currentItem,
FSFileOperationStage stage,
OSStatus error,
CFDictionaryRef statusDictionary,
void *info )
{
NSProgressIndicator* fileProgressIndicator = (NSProgressIndicator*)info;
[fileProgressIndicator setDoubleValue:(double)floatBytesCompleted];
[fileProgressIndicator displayIfNeeded];
}