If the app is installed at the first time, need to allow notification, how can I confirm it? Is someone encountered?
you should typically be mocking notifications and other data requests in order to prevent the dialogs from coming up. You could also accept the notification manually and re-run your tests. We experimented with using the private UIAutomation framework for this and saw we could achieve this with it. For example, for pressing the left alert button.
#interface SystemAlert : NSObject
- (void)tapLeftButton;
#end
#interface SystemAlert (ForMethodCompletionOnly)
+ (id)localTarget;
- (id)frontMostApp;
- (id)alert;
- (id)buttons;
#end
#implementation SystemAlert
+ (void)load {
dlopen([#"/Developer/Library/PrivateFrameworks/UIAutomation.framework/UIAutomation" fileSystemRepresentation], RTLD_LOCAL);
}
- (void)tapLeftButton {
id localTarget = [NSClassFromString(#"UIATarget") localTarget];
id app = [localTarget frontMostApp];
id alert = [app alert];
id button = [[alert buttons] objectAtIndex:0];
[button tap];
}
#end
Related
I found that upwork.app can count pressed keys without accessibility access but I can not understand how it is done.
I read a lot of themes like this one:
OSX: Detect system-wide keyDown events?
In all themes are saying that process should be trusted "Enable access for assistive devices" and I can not find how upwork.app can track keys without this.
This is official documentation for tracking events:
https://developer.apple.com/reference/appkit/nsevent/1535472-addglobalmonitorforeventsmatchin
Key-related events may only be monitored if accessibility is enabled or if your application is trusted for accessibility access (see AXIsProcessTrusted).
In apple mail list is saying same:
http://lists.apple.com/archives/carbon-dev/2010/Feb/msg00043.html
I think that upwork.app use some hacks.
How I can count pressed keys without accessibility access?
Still haven't received your answer in the comments, but since that might also help other people in the future I decided to answer anyway.
With the IOKit you can detect the keyboard has a device, and get the key press events like device events. I've used that to detect joystick events, but it should work with keyboards has well. I assume that the modifications that I made are enough and should work, however my Xcode is updating now, so I wasn't able to test it yet.
KeyboardWatcher.h File:
#import <Foundation/Foundation.h>
#import <IOKit/hid/IOHIDManager.h>
#import <IOKit/hid/IOHIDKeys.h>
#interface KeyboardWatcher : NSObject{
IOHIDManagerRef HIDManager;
}
#property (nonatomic) int keysPressedCount;
+(instancetype)sharedWatcher;
-(void)startWatching;
-(void)stopWatching;
#end
KeyboardWatcher.m File:
#import "KeyboardWatcher.h"
#implementation KeyboardWatcher
static KeyboardWatcher *_sharedWatcher;
+(instancetype)sharedWatcher {
#synchronized([self class]) {
if (!_sharedWatcher){
_sharedWatcher = [[KeyboardWatcher alloc] init];
}
return _sharedWatcher;
}
return nil;
}
-(instancetype)init {
self = [super init];
if (self){
self.keysPressedCount = 0;
}
return self;
}
-(void)startWatching {
[self watchDevicesOfType:kHIDUsage_GD_Keyboard];
}
-(void)watchDevicesOfType:(UInt32)deviceType {
// Create an HID Manager
HIDManager = IOHIDManagerCreate(kCFAllocatorDefault, kIOHIDOptionsTypeNone);
// Create a Matching Dictionary
CFMutableDictionaryRef matchDict = CFDictionaryCreateMutable(kCFAllocatorDefault, 2, &kCFTypeDictionaryKeyCallBacks,
&kCFTypeDictionaryValueCallBacks);
// That will make the app just return the computer keyboards
CFDictionarySetValue(matchDict, CFSTR(kIOHIDPrimaryUsageKey), CFNumberCreate(kCFAllocatorDefault, kCFNumberIntType, &deviceType));
// Register the Matching Dictionary to the HID Manager
IOHIDManagerSetDeviceMatching(HIDManager, matchDict);
// Register the HID Manager on our app’s run loop
IOHIDManagerScheduleWithRunLoop(HIDManager, CFRunLoopGetMain(), kCFRunLoopDefaultMode);
// Open the HID Manager
IOReturn IOReturn = IOHIDManagerOpen(HIDManager, kIOHIDOptionsTypeNone);
// Register input calls to Handle_DeviceEventCallback function
IOHIDManagerRegisterInputValueCallback(HIDManager, Handle_DeviceEventCallback, nil);
if (IOReturn) NSLog(#"IOHIDManagerOpen failed.");
}
-(void)stopWatching {
HIDManager = NULL;
}
static void Handle_DeviceEventCallback (void *inContext, IOReturn inResult, void *inSender, IOHIDValueRef value){
IOHIDElementRef element = IOHIDValueGetElement(value); // Keyboard pressed key
uint32_t uniqueIdentifier = IOHIDElementGetCookie(element); // Unique ID of key
int elementValue = (int)IOHIDValueGetIntegerValue(value); // Actual state of key (1=pressed)
NSLog(#"Unique ID = %u; Value = %d", uniqueIdentifier, elementValue);
if (elementValue == 1) KeyboardWatcher.sharedWatcher.keysPressedCount++;
}
#end
In case you want to identify which unique ID is which key, you can use these enums (instead of importing Carbon you can just create a CGKeyboardMapping.h file and paste them there):
https://stackoverflow.com/a/16125341/4370893
At last, in order to use it, you just need to do that to start watching for keyboard events:
[[KeyboardWatcher sharedWatcher] startWatching];
Get the key pressed count with that:
[[KeyboardWatcher sharedWatcher] keysPressedCount];
And that to stop:
[[KeyboardWatcher sharedWatcher] stopWatching];
These were my references to write my original joystick code:
http://ontrak.net/xcode.htm
http://ontrak.net/xcode2.htm
https://developer.apple.com/library/mac/documentation/DeviceDrivers/Conceptual/HID/new_api_10_5/tn2187.html
http://www.opensource.apple.com/source/IOHIDFamily/IOHIDFamily-315.7.16/IOHIDFamily/IOHIDUsageTables.h
As soon as the update finishes I will test the code and inform if it's working or not for sure.
EDIT: Just tested and it's working. Don't forget to add the IOKit framework to the project.
I've run into an issue with this little Objective-C project I'm doing and it's proving to be a bit of a roadblock. I'm playing around with Apple's NSSpeechRecognizer software on El Capitan, and I'm trying to get this guy running properly so that when the riddle I give it is posed to the user, the user can respond with a word to "do something cool". As it stands right now, the delegate method:
-(void) speechRecognizer:(NSSpeechRecognizer *)sender didRecognizeCommand:(NSString *)command { ... }`
is never even called, even though it appears the recognition icon is correctly detecting the answer to the riddle.
The problem is that your main function has a loop that is continually checking whether the speech has been recognizing. You are not giving NSSpeechRecognizer a chance to actually deliver any messages to you.
Your app needs to let the main "run loop" run, so it can deliver messages. Normally, in an OS X app, your main would just call NSApplicationMain, which does this for you.
Your code is effectively this:
#interface RecognizerDelegate : NSObject <NSSpeechRecognizerDelegate>
#property (nonatomic) NSSpeechRecognizer *recognizer;
#property (nonatomic) BOOL didRecognize;
#end
#implementation RecognizerDelegate
- (id)init
{
if ((self = [super init])) {
self.didRecognize = NO;
self.recognizer = [[NSSpeechRecognizer alloc] init];
self.recognizer.listensInForegroundOnly = NO;
self.recognizer.blocksOtherRecognizers = YES;
self.recognizer.delegate = self;
self.recognizer.commands = #[ #"hello" ];
[self.recognizer startListening];
}
return self;
}
- (void)speechRecognizer:(NSSpeechRecognizer *)sender didRecognizeCommand:(NSString *)command
{
self.didRecognize = YES;
}
#end
int main(int argc, const char * argv[])
{
#autoreleasepool
{
RecognizerDelegate *recognizerDelegate = [[RecognizerDelegate alloc] init];
while (recognizerDelegate.didRecognize == NO) {
// do nothing
}
NSLog(#"Recognized!");
}
return 0;
}
That while loop is doing nothing useful, just running your CPU in a loop and wasting time and energy. You are not letting any other code in NSSpeechSynthesizer, or any of the system frameworks like Foundation or AppKit, get the chance to do anything. So, nothing happens.
To fix this in the short term: you can let the main run loop run for a little while in each pass through the loop. This code would let the system run for a second, then would return to your code, so you could check again:
while (recognizerDelegate.didRecognize == NO) {
[[NSRunLoop currentRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:1.0]];
}
The longer-term fix would be to move your code out of main and to structure it like a real OS X app. Instead of using a loop to poll a condition like recognizerDelegate.didRecognize, you would just trigger the "next thing" directly from delegate methods like -speechRecognizer:didRecognizeCommand:, or you would use things like NSTimer to run code periodically.
For more details, see the Apple doc Cocoa Application Competencies for OS X, specifically the "Main Event Loop" section.
I had the same problem using NSSpeechRecognizer. The callback function:
func speechRecognizer(_ sender: NSSpeechRecognizer,
didRecognizeCommand command: String) {}
...was never called, even though everything appeared to be working.
There were three things I changed to get the code working.
1) I had to enable the entitlement in my "sandboxed" mode application to allow for microphone use.
... I also did these other two things, as well.
2) I added the "Privacy - Microphone Usage Description" in the info.pList, and set the string value to "I want to listen to you speak"
3) I added the "Privacy - Speech Recognition Usage Description" in the info.pList, and set the string value to "I want to write down what you say"
The appDelegate instance is showing nil value, test case "testAppDelegate" is getting failed.
The same is working in the sample code provided by apple developer site but there SenTestCase is being used, please help me out, even the target is set as per the WWDC 2013 video "Testing in Xcode 5 session 409"
#interface Tests : XCTestCase
{
AppDelegate *appDelegate;
AppViewController *appVC;
UIView *appView;
}
#end
#implementation Tests
- (void)setUp
{
[super setUp];
appDelegate = [[UIApplication sharedApplication] delegate];
appVC = appDelegate.appViewController;
appView = appVC.view;
}
- (void)tearDown
{
[super tearDown];
}
- (void)testAppDelegate
{
XCTAssert(appDelegate, #"Cannot find the application delegate");
}
- (void)testCheckForViewInitializatio
{
XCTAssert(appVC, #"AppViewController initialized");
}
Try this:
go to Project Settings
select your test target
in General tab switch Target section to your main target
Run again
Anton gives right suggestion.
But if it doesn't works (as in my case) try to:
Remove your existing Test Target
Create new Test Target (5-th tab in left pane -> click on + in bottom left corner -> New Test Target) and
In appeared window don't forget to choose your application as target of your test target.
Add all your TestCases files to new target.
I don't know the reason, but after this action it start working correctly, when I run tests with new target.
I am building a simple step sequencer in IRTcmix (version of RTcmix for the iphone/ipad), using UI Buttons to control which notes are on or off in the sequence.
I have been working with a set of RTcmix examples of other apps, and trying to piece my own together, but i'm unsure how to change values of a variable within the .sco (score file) with a toggle-style UIButton.
Here are some snippets from a working RTcmix App:
From the RTcmix manager .h:
#interface RTcmixManager : RTcmixPlayer {
RTcmixScore *polyScore;
}
#property (nonatomic, retain) RTcmixScore *polyScore;
- (void)noteOn:(int)note;
- (void)noteOff:(int)note;
#end
and the RTcmix manager .m
- (void)noteOn:(int)note {
[polyScore.mainScoreParameters replaceObjectAtIndex:0 withObject:[NSNumber numberWithInt:note]];
[polyScore.mainScoreParameters replaceObjectAtIndex:1 withObject:[NSNumber numberWithInt:1]];
[self parseScoreWithRTcmixScore:polyScore];
polyScore.setupIsActive = YES;
}
- (void)noteOff:(int)note {
[polyScore.mainScoreParameters replaceObjectAtIndex:0 withObject:[NSNumber numberWithInt:note]];
[polyScore.mainScoreParameters replaceObjectAtIndex:1 withObject:[NSNumber numberWithInt:0]];
[self parseScoreWithRTcmixScore:polyScore];
}
the viewcontroller .h:
#interface RTPolyphonyViewController : UIViewController {
RTcmixManager *rtcmixManager;
}
- (IBAction)keyDown:(UIButton *)sender;
- (IBAction)keyUp:(UIButton *)sender;
#end
viewcontroller .m:
- (IBAction)keyDown:(UIButton *)sender {
[rtcmixManager noteOn:sender.tag];
}
- (IBAction)keyUp:(UIButton *)sender {
[rtcmixManager noteOff:sender.tag];
}
and a snippet from the .sco file (the "%#" is the variables changed by user input):
NoteNumber = %#
NoteVelocity = %#
alreadySounding = note_exists(oscNotes[NoteNumber])
MAXMESSAGE(0, alreadySounding)
I am very new to objective C and xcode, I was wondering how to make a UI button toggle between two values in the score file (acting like a switch) for example, a button when toggled making the value of NoteNumber(in the score file) set to 1, while the value is 0 when toggled off. Because I am a beginner with Xcode(and objective C), I'm not sure about the proper syntax for making a button able to be toggled on and off, and i am also not sure how to connect said button with my .sco file, as much as I look at the sample code, I cannot seem to figure out how they all connect.
I am building a really basic Cocoa application using WebKit, to display a Flash/Silverlight application within it. Very basic, no intentions for it to be a browser itself.
So far I have been able to get it to open basic html links (<a href="..." />) in a new instance of Safari using
[[NSWorkspace sharedWorkspace] openURL:[request URL]];
Now my difficulty is opening a link in a new instance of Safari when window.open() is used in JavaScript. I "think" (and by this, I have been hacking away at the code and am unsure if i actually did or not) I got this kind of working by setting the WebView's policyDelegate and implementing its
-webView:decidePolicyForNavigationAction:request:frame:decisionListener:
delegate method. However this led to some erratic behavior.
So the simple question, what do I need to do so that when window.open() is called, the link is opened in a new instance of Safari.
Thanks
Big point, I am normally a .NET developer, and have only been working with Cocoa/WebKit for a few days.
I made from progress last night and pinned down part of my problem.
I am already using webView:decidePolicyForNewWindowAction:request:newFrameName:decisionListener: and I have gotten it to work with anchor tags, however the method never seems to get called when JavaScript is invoked.
However when window.open() is called webView:createWebViewWithRequest:request is called, I have tried to force the window to open in Safari here, however request is always null. So I can never read the URL out.
I have done some searching around, and this seems to be a known "misfeature" however I have not been able to find a way to work around it.
From what I understand createWebViewWithRequest gives you the ability to create the new webview, the the requested url is then sent to the new webView to be loaded. This is the best explanation I have been able to find so far.
So while many people have pointed out this problem, I have yet to see any solution which fits my needs. I will try to delve a little deeper into the decidePolicyForNewWindowAction again.
Thanks!
Well, I'm handling it by creating a dummy webView, setting it's frameLoad delegate to a custom class that handles
- (void)webView:decidePolicyForNavigationAction:actionInformation :request:frame:decisionListener:
and opens a new window there.
code :
- (WebView *)webView:(WebView *)sender createWebViewWithRequest:(NSURLRequest *)request {
//this is a hack because request URL is null here due to a bug in webkit
return [newWindowHandler webView];
}
and NewWindowHandler :
#implementation NewWindowHandler
-(NewWindowHandler*)initWithWebView:(WebView*)newWebView {
webView = newWebView;
[webView setUIDelegate:self];
[webView setPolicyDelegate:self];
[webView setResourceLoadDelegate:self];
return self;
}
- (void)webView:(WebView *)sender decidePolicyForNavigationAction:(NSDictionary *)actionInformation request:(NSURLRequest *)request frame:(WebFrame *)frame decisionListener:(id<WebPolicyDecisionListener>)listener {
[[NSWorkspace sharedWorkspace] openURL:[actionInformation objectForKey:WebActionOriginalURLKey]];
}
-(WebView*)webView {
return webView;
}
There seems to be a bug with webView:decidePolicyForNewWindowAction:request:newFrameName:decisionListener: in that the request is always nil, but there is a robust solution that works with both normal target="_blank" links as well as javascript ones.
Basically I use another ephemeral WebView to handle the new page load in. Similar to Yoni Shalom but with a little more syntactic sugar.
To use it first set a delegate object for your WebView, in this case I'm setting myself as the delegate:
webView.UIDelegate = self;
Then just implement the webView:createWebViewWithRequest: delegate method and use my block based API to do something when a new page is loaded, in this case I'm opening the page in an external browser:
-(WebView *)webView:(WebView *)sender createWebViewWithRequest:(NSURLRequest *)request {
return [GBWebViewExternalLinkHandler riggedWebViewWithLoadHandler:^(NSURL *url) {
[[NSWorkspace sharedWorkspace] openURL:url];
}];
}
That's pretty much it. Here's the code for my class. Header:
// GBWebViewExternalLinkHandler.h
// TabApp2
//
// Created by Luka Mirosevic on 13/03/2013.
// Copyright (c) 2013 Goonbee. All rights reserved.
//
#import <Foundation/Foundation.h>
#class WebView;
typedef void(^NewWindowCallback)(NSURL *url);
#interface GBWebViewExternalLinkHandler : NSObject
+(WebView *)riggedWebViewWithLoadHandler:(NewWindowCallback)handler;
#end
Implemetation:
// GBWebViewExternalLinkHandler.m
// TabApp2
//
// Created by Luka Mirosevic on 13/03/2013.
// Copyright (c) 2013 Goonbee. All rights reserved.
//
#import "GBWebViewExternalLinkHandler.h"
#import <WebKit/WebKit.h>
#interface GBWebViewExternalLinkHandler ()
#property (strong, nonatomic) WebView *attachedWebView;
#property (strong, nonatomic) GBWebViewExternalLinkHandler *retainedSelf;
#property (copy, nonatomic) NewWindowCallback handler;
#end
#implementation GBWebViewExternalLinkHandler
-(id)init {
if (self = [super init]) {
//create a new webview with self as the policyDelegate, and keep a ref to it
self.attachedWebView = [WebView new];
self.attachedWebView.policyDelegate = self;
}
return self;
}
-(void)webView:(WebView *)sender decidePolicyForNavigationAction:(NSDictionary *)actionInformation request:(NSURLRequest *)request frame:(WebFrame *)frame decisionListener:(id<WebPolicyDecisionListener>)listener {
//execute handler
if (self.handler) {
self.handler(actionInformation[WebActionOriginalURLKey]);
}
//our job is done so safe to unretain yourself
self.retainedSelf = nil;
}
+(WebView *)riggedWebViewWithLoadHandler:(NewWindowCallback)handler {
//create a new handler
GBWebViewExternalLinkHandler *newWindowHandler = [GBWebViewExternalLinkHandler new];
//store the block
newWindowHandler.handler = handler;
//retain yourself so that we persist until the webView:decidePolicyForNavigationAction:request:frame:decisionListener: method has been called
newWindowHandler.retainedSelf = newWindowHandler;
//return the attached webview
return newWindowHandler.attachedWebView;
}
#end
Licensed as Apache 2.
You don't mention what kind of erratic behaviour you are seeing. A quick possibility, is that when implementing the delegate method you forgot to tell the webview you are ignoring the click by calling the ignore method of the WebPolicyDecisionListener that was passed to your delegate, which may have put things into a weird state.
If that is not the issue, then how much control do you have over the content you are displaying? The policy delegate gives you easy mechanisms to filter all resource loads (as you have discovered), and all new window opens via webView:decidePolicyForNewWindowAction:request:newFrameName:decisionListener:. All window.open calls should funnel through that, as will anything else that triggers a new window.
If there are other window opens you want to keep inside your app, you will to do a little more work. One of the arguments passed into the delegate is a dictionary containing information about the event. Insie that dictionary the WebActionElementKey will have a dictionary containing a number of details, including the original dom content of the link. If you want to poke around in there you can grab the actual DOM element, and check the text of the href to see if it starts with window.open. That is a bit heavy weight, but if you want fine grained control it will give it to you.
By reading all posts, i have come up with my simple solution, all funcs are in same class,here it is, opens a link with browser.
- (WebView *)webView:(WebView *)sender createWebViewWithRequest:(NSURLRequest *)request {
return [self externalWebView:sender];
}
- (void)webView:(WebView *)sender decidePolicyForNavigationAction:(NSDictionary *)actionInformation request:(NSURLRequest *)request frame:(WebFrame *)frame decisionListener:(id<WebPolicyDecisionListener>)listener
{
[[NSWorkspace sharedWorkspace] openURL:[actionInformation objectForKey:WebActionOriginalURLKey]];
}
-(WebView*)externalWebView:(WebView*)newWebView
{
WebView *webView = newWebView;
[webView setUIDelegate:self];
[webView setPolicyDelegate:self];
[webView setResourceLoadDelegate:self];
return webView;
}
Explanation:
Windows created from JavaScript via window.open go through createWebViewWithRequest.
All window.open calls result in a createWebViewWithRequest: with a null request, then later a location change on that WebView.
For further information, see this old post on the WebKit mailing list.
An alternative to returning a new WebView and waiting for its loadRequest: method to be called, I ended up overwriting the window.open function in the WebView's JSContext:
First, I set my controller to be the WebFrameLoadDelegate of the WebView:
myWebView.frameLoadDelegate = self;
Then, in the delegate method, I overwrote the window.open function, and I can process the URL there instead.
- (void)webView:(WebView *)webView didCreateJavaScriptContext:(JSContext *)context forFrame:(WebFrame *)frame{
context[#"window"][#"open"] = ^(id url){
NSLog(#"url to load: %#", url);
};
}
This let me handle the request however I needed to without the awkward need to create additional WebViews.