OS X count pressed keys without accessibility access - objective-c

I found that upwork.app can count pressed keys without accessibility access but I can not understand how it is done.
I read a lot of themes like this one:
OSX: Detect system-wide keyDown events?
In all themes are saying that process should be trusted "Enable access for assistive devices" and I can not find how upwork.app can track keys without this.
This is official documentation for tracking events:
https://developer.apple.com/reference/appkit/nsevent/1535472-addglobalmonitorforeventsmatchin
Key-related events may only be monitored if accessibility is enabled or if your application is trusted for accessibility access (see AXIsProcessTrusted).
In apple mail list is saying same:
http://lists.apple.com/archives/carbon-dev/2010/Feb/msg00043.html
I think that upwork.app use some hacks.
How I can count pressed keys without accessibility access?

Still haven't received your answer in the comments, but since that might also help other people in the future I decided to answer anyway.
With the IOKit you can detect the keyboard has a device, and get the key press events like device events. I've used that to detect joystick events, but it should work with keyboards has well. I assume that the modifications that I made are enough and should work, however my Xcode is updating now, so I wasn't able to test it yet.
KeyboardWatcher.h File:
#import <Foundation/Foundation.h>
#import <IOKit/hid/IOHIDManager.h>
#import <IOKit/hid/IOHIDKeys.h>
#interface KeyboardWatcher : NSObject{
IOHIDManagerRef HIDManager;
}
#property (nonatomic) int keysPressedCount;
+(instancetype)sharedWatcher;
-(void)startWatching;
-(void)stopWatching;
#end
KeyboardWatcher.m File:
#import "KeyboardWatcher.h"
#implementation KeyboardWatcher
static KeyboardWatcher *_sharedWatcher;
+(instancetype)sharedWatcher {
#synchronized([self class]) {
if (!_sharedWatcher){
_sharedWatcher = [[KeyboardWatcher alloc] init];
}
return _sharedWatcher;
}
return nil;
}
-(instancetype)init {
self = [super init];
if (self){
self.keysPressedCount = 0;
}
return self;
}
-(void)startWatching {
[self watchDevicesOfType:kHIDUsage_GD_Keyboard];
}
-(void)watchDevicesOfType:(UInt32)deviceType {
// Create an HID Manager
HIDManager = IOHIDManagerCreate(kCFAllocatorDefault, kIOHIDOptionsTypeNone);
// Create a Matching Dictionary
CFMutableDictionaryRef matchDict = CFDictionaryCreateMutable(kCFAllocatorDefault, 2, &kCFTypeDictionaryKeyCallBacks,
&kCFTypeDictionaryValueCallBacks);
// That will make the app just return the computer keyboards
CFDictionarySetValue(matchDict, CFSTR(kIOHIDPrimaryUsageKey), CFNumberCreate(kCFAllocatorDefault, kCFNumberIntType, &deviceType));
// Register the Matching Dictionary to the HID Manager
IOHIDManagerSetDeviceMatching(HIDManager, matchDict);
// Register the HID Manager on our app’s run loop
IOHIDManagerScheduleWithRunLoop(HIDManager, CFRunLoopGetMain(), kCFRunLoopDefaultMode);
// Open the HID Manager
IOReturn IOReturn = IOHIDManagerOpen(HIDManager, kIOHIDOptionsTypeNone);
// Register input calls to Handle_DeviceEventCallback function
IOHIDManagerRegisterInputValueCallback(HIDManager, Handle_DeviceEventCallback, nil);
if (IOReturn) NSLog(#"IOHIDManagerOpen failed.");
}
-(void)stopWatching {
HIDManager = NULL;
}
static void Handle_DeviceEventCallback (void *inContext, IOReturn inResult, void *inSender, IOHIDValueRef value){
IOHIDElementRef element = IOHIDValueGetElement(value); // Keyboard pressed key
uint32_t uniqueIdentifier = IOHIDElementGetCookie(element); // Unique ID of key
int elementValue = (int)IOHIDValueGetIntegerValue(value); // Actual state of key (1=pressed)
NSLog(#"Unique ID = %u; Value = %d", uniqueIdentifier, elementValue);
if (elementValue == 1) KeyboardWatcher.sharedWatcher.keysPressedCount++;
}
#end
In case you want to identify which unique ID is which key, you can use these enums (instead of importing Carbon you can just create a CGKeyboardMapping.h file and paste them there):
https://stackoverflow.com/a/16125341/4370893
At last, in order to use it, you just need to do that to start watching for keyboard events:
[[KeyboardWatcher sharedWatcher] startWatching];
Get the key pressed count with that:
[[KeyboardWatcher sharedWatcher] keysPressedCount];
And that to stop:
[[KeyboardWatcher sharedWatcher] stopWatching];
These were my references to write my original joystick code:
http://ontrak.net/xcode.htm
http://ontrak.net/xcode2.htm
https://developer.apple.com/library/mac/documentation/DeviceDrivers/Conceptual/HID/new_api_10_5/tn2187.html
http://www.opensource.apple.com/source/IOHIDFamily/IOHIDFamily-315.7.16/IOHIDFamily/IOHIDUsageTables.h
As soon as the update finishes I will test the code and inform if it's working or not for sure.
EDIT: Just tested and it's working. Don't forget to add the IOKit framework to the project.

Related

Unsupported action method signature. Must return MPRemoteCommandHandlerStatus

I'm getting this error whenever I'm running simulator on iOS 13+. Everything works for iOS 12 and below so I'm not sure what to do here. Is there anything I can change/edit on my end to make react-native-music-control work for iOS 13?
Exception 'Unsupported action method signature. Must return MPRemoteCommandHandlerStatus or take a completion handler as the second argument.' was thrown while invoking enableControl on target MusicControlManager with params ( pause, 1, { } )
This react-native-music-control probably hasn't updated its iOS MediaPlayer methods.
One common method is MediaPlayer's addTarget. Starting iOS 13, this method must return a MPRemoteCommandHandlerStatus. It used to return nothing in previous iOS versions.
For example, if you have a play method that gets called when the Play button is tapped:
- (void) play {
/* start playing */
}
You are probably registering this play to be called when the Media Player's play command is trigged:
[[MPRemoteCommandCenter sharedCommandCenter].playCommand addTarget:self action:#selector(play)];
Then, all you need is to simply change your play method to return a MPRemoteCommandHandlerStatus like this:
- (MPRemoteCommandHandlerStatus) play
{
// if successfully played
return MPRemoteCommandHandlerStatusSuccess;
// else if there's an issue
// return MPRemoteCommandHandlerStatusCommandFailed;
}
This is in Objective-C. Changing the return value in Swift will be simple too.
Reference: https://developer.apple.com/documentation/mediaplayer/mpremotecommand/1622895-addtarget?language=objc
So this plugin is not being maintained anymore I don't know why, so for those using this plugin the solution is really Simple, based on what #CSawy. I was able to fix it.
1 - Change the method signature to MPRemoteCommandHandlerStatus
All the methods that interact with the control are defined as void so change them to MPRemoteCommandHandlerStatus. They are defined in the file MusicControls and I guess you would find it in your plugins folder. For me it was in myProject/plugins/MusiControls.h
These are the methods that require the change
playEvent
pauseEvent
nextTrackEvent
prevTrackEvent
skipForwardEvent
skipBackwardEvent
After changing all the methods signature u should have
(MPRemoteCommandHandlerStatus) playEvent:(MPRemoteCommandEvent *) event;
(MPRemoteCommandHandlerStatus) pauseEvent:(MPRemoteCommandEvent *) event;
(MPRemoteCommandHandlerStatus) nextTrackEvent:(MPRemoteCommandEvent *) event;
(MPRemoteCommandHandlerStatus) prevTrackEvent:(MPRemoteCommandEvent *) event;
(MPRemoteCommandHandlerStatus) skipForwardEvent: (MPSkipIntervalCommandEvent *) event;
(MPRemoteCommandHandlerStatus) skipBackwardEvent: (MPSkipIntervalCommandEvent *) event;
2-Add return to the implementations
Now you should change the implementation of these methods otherwise your app would still be crushing.
Simply open the file MusicControls.m. If you are using Xcode it will highlight the methods for you ... If it's no highlighted then just search for them
For all these functions you will need to add a return ;
For your play event do:
-(MPRemoteCommandHandlerStatus) playEvent:(MPRemoteCommandEvent *)event {
NSString * action = #"music-controls-play";
NSString * jsonAction = [NSString stringWithFormat:#"{\"message\":\"%#\"}", action];
CDVPluginResult * pluginResult = [CDVPluginResult resultWithStatus:CDVCommandStatus_OK messageAsString:jsonAction];
[self.commandDelegate sendPluginResult:pluginResult callbackId:[self latestEventCallbackId]];
return MPRemoteCommandHandlerStatusSuccess;
}
Do this for all the methods and I hope it works.

NSSpeechRecognizer and .delegate=self; Problems

I've run into an issue with this little Objective-C project I'm doing and it's proving to be a bit of a roadblock. I'm playing around with Apple's NSSpeechRecognizer software on El Capitan, and I'm trying to get this guy running properly so that when the riddle I give it is posed to the user, the user can respond with a word to "do something cool". As it stands right now, the delegate method:
-(void) speechRecognizer:(NSSpeechRecognizer *)sender didRecognizeCommand:(NSString *)command { ... }`
is never even called, even though it appears the recognition icon is correctly detecting the answer to the riddle.
The problem is that your main function has a loop that is continually checking whether the speech has been recognizing. You are not giving NSSpeechRecognizer a chance to actually deliver any messages to you.
Your app needs to let the main "run loop" run, so it can deliver messages. Normally, in an OS X app, your main would just call NSApplicationMain, which does this for you.
Your code is effectively this:
#interface RecognizerDelegate : NSObject <NSSpeechRecognizerDelegate>
#property (nonatomic) NSSpeechRecognizer *recognizer;
#property (nonatomic) BOOL didRecognize;
#end
#implementation RecognizerDelegate
- (id)init
{
if ((self = [super init])) {
self.didRecognize = NO;
self.recognizer = [[NSSpeechRecognizer alloc] init];
self.recognizer.listensInForegroundOnly = NO;
self.recognizer.blocksOtherRecognizers = YES;
self.recognizer.delegate = self;
self.recognizer.commands = #[ #"hello" ];
[self.recognizer startListening];
}
return self;
}
- (void)speechRecognizer:(NSSpeechRecognizer *)sender didRecognizeCommand:(NSString *)command
{
self.didRecognize = YES;
}
#end
int main(int argc, const char * argv[])
{
#autoreleasepool
{
RecognizerDelegate *recognizerDelegate = [[RecognizerDelegate alloc] init];
while (recognizerDelegate.didRecognize == NO) {
// do nothing
}
NSLog(#"Recognized!");
}
return 0;
}
That while loop is doing nothing useful, just running your CPU in a loop and wasting time and energy. You are not letting any other code in NSSpeechSynthesizer, or any of the system frameworks like Foundation or AppKit, get the chance to do anything. So, nothing happens.
To fix this in the short term: you can let the main run loop run for a little while in each pass through the loop. This code would let the system run for a second, then would return to your code, so you could check again:
while (recognizerDelegate.didRecognize == NO) {
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:1.0]];
}
The longer-term fix would be to move your code out of main and to structure it like a real OS X app. Instead of using a loop to poll a condition like recognizerDelegate.didRecognize, you would just trigger the "next thing" directly from delegate methods like -speechRecognizer:didRecognizeCommand:, or you would use things like NSTimer to run code periodically.
For more details, see the Apple doc Cocoa Application Competencies for OS X, specifically the "Main Event Loop" section.
I had the same problem using NSSpeechRecognizer. The callback function:
func speechRecognizer(_ sender: NSSpeechRecognizer,
didRecognizeCommand command: String) {}
...was never called, even though everything appeared to be working.
There were three things I changed to get the code working.
1) I had to enable the entitlement in my "sandboxed" mode application to allow for microphone use.
... I also did these other two things, as well.
2) I added the "Privacy - Microphone Usage Description" in the info.pList, and set the string value to "I want to listen to you speak"
3) I added the "Privacy - Speech Recognition Usage Description" in the info.pList, and set the string value to "I want to write down what you say"

Substitute keystroke

I am trying to intercept a keystroke, and substitute it with a different character. I have been able to intercept the key being pressed, as well as perform some extra operations. Now I need to hold the key being pressed if it matches one of the ones I am watching, and insert a different character. Here is the code I have right now:
#import "AppDelegate.h"
#import <AppKit/AppKit.h>
#import <CoreGraphics/CoreGraphics.h>
#include <ApplicationServices/ApplicationServices.h>
#interface AppDelegate ()
#property (weak) IBOutlet NSWindow *window;
#end
#implementation AppDelegate
- (void)applicationDidFinishLaunching:(NSNotification *)aNotification {
//Keys that are being watched to be switched out
NSArray *keysToWatch = [[NSArray alloc] initWithObjects:#"c",#".", nil];
// register for keys throughout the device...
[NSEvent addGlobalMonitorForEventsMatchingMask:NSKeyDownMask
handler:^(NSEvent *event){
//Get characters
NSString *chars = [[event characters] lowercaseString];
//Get the actual character being pressed
unichar character = [chars characterAtIndex:0];
//Transform it to a string
NSString *aString = [NSString stringWithCharacters:&character length:1];
//If it is in the list, start looking if Keynote is active
if ([keysToWatch containsObject:[NSString stringWithString:aString]]) {
//DEBUG: Print a message
NSLog(#"Key being watched has been pressed");
//Get a list of all running apps
for (NSRunningApplication *currApp in [[NSWorkspace sharedWorkspace] runningApplications]) {
//Get current active app
if ([currApp isActive]) {
//Check if it is Keynote, if yes perform remap
if ([[currApp localizedName] isEqualToString:#"Keynote"]){
//DEBUG: Print a small message
NSLog(#"Current app is Keynote");
if (character=='.') {
NSLog(#"Pressed a dot");
//I want to post a different character here
PostKeyWithModifiers((CGKeyCode)11, FALSE);
}
else if ([aString isEqualToString:#"c"]) {
NSLog(#"Pressed c");
}
}
else if ([[currApp localizedName] isEqualToString:#"Microsoft PowerPoint"]){
}
}
}
}
}
];
}
- (void)applicationWillTerminate:(NSNotification *)aNotification {
// Insert code here to tear down your application
}
- (BOOL)acceptsFirstResponder {
return YES;
}
void PostKeyWithModifiers(CGKeyCode key, CGEventFlags modifiers)
{
CGEventSourceRef source = CGEventSourceCreate(kCGEventSourceStateCombinedSessionState);
CGEventRef keyDown = CGEventCreateKeyboardEvent(source, key, TRUE);
CGEventSetFlags(keyDown, modifiers);
CGEventRef keyUp = CGEventCreateKeyboardEvent(source, key, FALSE);
CGEventPost(kCGAnnotatedSessionEventTap, keyDown);
CGEventPost(kCGAnnotatedSessionEventTap, keyUp);
CFRelease(keyUp);
CFRelease(keyDown);
CFRelease(source);
}
#end
My problem is that I am not able to stop the original keystroke. Please keep in mind that I am completely new at Obj C, so let me know if there is anything that I can do better. Thanks!
From the docs for +[NSEvent addGlobalMonitorForEventsMatchingMask:handler:]:
Events are delivered asynchronously to your app and you can only
observe the event; you cannot modify or otherwise prevent the event
from being delivered to its original target application.
You would have to use a Quartz Event Tap for that.
So, after a LOT of digging around, I found a way to do this using Quartz Event Taps here. Thanks to #Ken Thomases for pointing me in the correct direction. I then combined my code with the one explained in this article and voilà, it works.

Command-Key-Up Cocoa

I'm trying to mimic the functionality of the cmd-tab keyboard shortcut where the user can switch between applications hitting a certain key and then when they release command something happens.
I'm using this code right now but it can only detects keydown. I need this to fire on key up
- (void)flagsChanged:(NSEvent *)theEvent {
if ([theEvent modifierFlags] & NSCommandKeyMask) {
NSLog(#"Do my stuff here");
}
}
Thanks
According to the docs:
Informs the receiver that the user has pressed or released a modifier
key (Shift, Control, and so on).
What you need to do here is when you get the event in which the command key goes down, you need to set a flag somewhere, and in subsequent calls, check for the absence of the command key being down.
For instance, assuming you have an ivar called _cmdKeyDown:
- (void)flagsChanged:(NSEvent *)theEvent
{
[super flagsChanged:theEvent];
NSUInteger f = [theEvent modifierFlags];
BOOL isDown = !!(f & NSCommandKeyMask);
if (isDown != _cmdKeyDown)
{
NSLog(#"State changed. Cmd Key is: %#", isDown ? #"Down" : #"Up");
_cmdKeyDown = isDown;
}
}

Objective c - DRDevice.h

I'm trying to check my "tray" to see if it open or not, but i can't get it to work, it says:
"_DRDeviceIsTrayOpenKey", referenced from:
´ -[UntitledAppDelegate applicationDidFinishLaunching:] in UntitledAppDelegate.o
ld: symbol(s) not found
the code is:
#import <Foundation/Foundation.h>
#import <DiscRecording/DRCoreDevice.h>
#import <DiscRecording/DRMSF.h>
#import <AvailabilityMacros.h>
extern NSString* const DRDeviceIsTrayOpenKey;
if (!DRDeviceIsTrayOpenKey == NO ) {
[NSApp terminate:nil];
}
info:
DRDeviceIsTrayOpenKey
extern NSString* const DRDeviceIsTrayOpenKey;
Discussion
One of the keys in the dictionary returned by the status method. NSNumber containing a boolean value indicating whether the device's tray is open or not.
Availability
Introduced in Mac OS X v10.2
please help me fix that, i think my code is wrong.
Yes, your code is wrong. That DRDeviceIsTrayOpenKey is a key used to retrieve a value from a NSDictionary.
So if you can get your Disk Recording status, you'd do something like this:
// this code would depend on you passing in a valid DRDevice object
// which I've named myDiscRecordingDevice. A computer can have multiple
// DVD / CD readers attached, so you need to specify which one you care about
NSDictionary * status = [myDiscRecordingDevice status];
NSString* state = [status objectForKey: DRDeviceMediaStateKey];
if ([state isEqualTo: DRDeviceMediaStateNone])
{
if ([[status objectForKey: DRDeviceIsTrayOpenKey] boolValue])
return trayOpen;
return noDisc;
}