I want to run tests in Xcode 4 using OCUnit without launching the simulator. Please, don't try and convince me I am doing unit testing wrong or anything like that. I like to do TDD the traditional way: write the API for the class in the tests, then make the class pass the tests. I will write separate tests that are end-to-end that run in the simulator.
If there's no way to do this, then please can someone tell me how to have the test harness not instantiate the whole app? My app is event driven, and it sends a bunch of events through when it starts up that mess with my tests.
Please can someone tell me how to have the test harness not instantiate the whole app? My app is event driven, and it sends a bunch of events through when it starts up that mess with my tests.
I use Xcode 4's built-in testing. App instantiation may seem like a pain, but as I write on Xcode Unit Testing: The Good, the Bad, the Ugly, it makes it possible to write tests without distinguishing between logic tests and application tests. Specifically, it lets me write unit tests for view controllers.
Here's what I do to avoid my full startup sequence:
Edit the scheme
Select the Test action
In "Test" select the Arguments tab
Disable "Use the Run action's options"
Add an environment variable, setting runningTests to YES
Edit your app delegate
Add the following to -application:didFinishLaunchingWithOptions: as soon as it makes sense to:
#if DEBUG
if (getenv("runningTests"))
return YES;
#endif
Do the same for -applicationDidBecomeActive: but simply return.
Update: I have changed my approach. See How to Easily Switch Your App Delegate for Testing.
In the last xcode version (5.0.2) you can do this in very easy way. Choose your Test target, "General" tab. Set "None" in field "Target". Then tap on "Build phases" tab and remove your Main target from "Target dependencies".
In your situation, I am assuming that you have a separate Logic Tests and Application Tests target (if not - you need to). In your schemes configuration you define which targets are built for the 'Test' scheme. If your application tests are not running, the simulator will not launch.
I suspect that you might be trying to run 'logic tests' in an 'Application tests' target (such as the one created by default by Xcode). See more about this difference here (and how to set ut up).
It was pointed out in an earlier answer that logic tests are the right thing to do for this scenario. I had very tough time in getting the logic tests working with XCode Version 4.3.2 (4E2002). Looking at Apple's sample unit test project helped me to understand how to do this with a clear separation. In that example, logic tests test files from the library target, not the application target. The model was encapsulated into a library which was then linked with the main target and logic tests target. The application target contained only views and controllers.
Based on this model, this is what I did to get my logic tests work correctly. Create a new target (Cocoa Touch Static Library) and move all files to be logic tested (typically all your models) to this new target. Under "Build Phases" settings add this new library in "Link Binary With Libraries" of your application target and logic tests target.
I can imagine that these instructions are little confusing. If you dissect the sample project that is mentioned above you will get a better idea.
Note, untested on Xcode 5.
I used #jon-reid’s answer, but found that Xcode adds environment-variables to the xcuserstated part of XcodeProjects, and these are user specific and not typically committed to the repository. Thus I swizzle my AppDelegate to override its loading:
#implementation MyAppDelegate (Testing)
+ (void)initialize {
SEL new = #selector(application:didFinishLaunchingWithOptions:);
SEL orig = #selector(swizzled_application:didFinishLaunchingWithOptions:);
Class c = [self class];
Method origMethod = class_getInstanceMethod(c, orig);
Method newMethod = class_getInstanceMethod(c, new);
if (class_addMethod(c, orig, method_getImplementation(newMethod), method_getTypeEncoding(newMethod))) {
class_replaceMethod(c, new, method_getImplementation(origMethod), method_getTypeEncoding(origMethod));
} else {
method_exchangeImplementations(origMethod, newMethod);
}
}
- (BOOL)swizzled_application:(id)app didFinishLaunchingWithOptions:(id)opts {
return YES;
}
#end
Note, that the following is simpler and still works, though I'm not sure it is reliable:
#implementation MyAppDelegate (Testing)
- (BOOL)application:(id)app didFinishLaunchingWithOptions:(id)opts {
return YES;
}
#end
This works because categories of methods in dynamically loaded components (like the testing bundle) take precedence. Swizzling feels safer though.
Using xCode 7 and xctool
xctool is capable of executing unit tests without the simulator.
To get this working,
1 . Update the target settings run without a host app.
Select your project --> then test target --> Set the host application to none.
2. Install xctool , if you don't have it.
brew install xctool
3. Run the tests using terminal with xctool.
xctool -workspace yourWorkspace.xcworkspace -scheme yourScheme run-tests -sdk iphonesimulator
i've used GHUnit to create osx/ios compatible test suites. there are a few issues, but i found it was more reliable/compatible/straightforward than OCUnit.
GHUnit provides basic template projects for OS X and iOS, which makes initial setup simple.
Note: I generally just use my own kit for most of my testing.
Related
I am developing a framework that connects to a hard coded url. I'd like to be able to change that url in automated tests. I adding the following to my umbrella header:
#ifdef DEBUG
- (void)setServerUrl;
#endif
I thought that if I build the framework in -configuration Release mode, the preprocessor would delete the method declaration from the code, which is what I want.
So, maybe my approach was wrong to begin with, or it was right but the implementation wasn't. In any case, how do I expose a method in DEBUG mode only? Is it possible?
Don't put this method in public header.
In yours automated test add category, that makes this method visible for them.
I am playing around with the new UI testing introduced in Xcode 7 beta. In one of my UI testing scenarios, I need to add some code that does the same thing as clicking Simulator -> Reset Content and Settings in the setup() of my test file, which is a XCTestCase. Can the reset be done programmatically? Or, can we mimic the effect of a factory reset on an app in test code?
Not entirely programmatically, but you can always write a bash file to delete:
${user.home}/Library/Application Support/iPhone Simulator/${simulator.version}
That will clear the settings on the simulator.
My understanding is that you won't be able to that from within your app, as apps are sandboxed.
Usually people were using shell scripts or apple scripts. However, using hard reset is absolutely not necessary.
You shouldn't care about data in other apps, you should care only about the data in your app. You can always delete your app data (files, user defaults) in the beginning of your tests. So, why should you do a hard reset?
A better solution is mocking. If your test supposes that, for example, some variable in NSUserDefaults is not set, you don't have to care about the actual value stored there, just mock the method your implementation is using (e.g. objectForKey: and let it return nil.
I'm going to create a mail plugin for the OS X Mail.app application for some additional features.
I have no idea where to start as there is no official documentation for plugins.
Can anyone please help me, how can I start the project.
Is there any initial link or tutorial, please suggest?
As noted, writing Apple Mail plugins is not straightforward, since it only has a private plugin API, which is entirely undocumented and can change with any new version of Mail.app. The best code example is GPGMail, which is open source & still active (already working on Yosemite support). Here is what I successfully did to get started (will put it up on github once finished):
How to build a minimal Apple Mail plugin (as of Mavericks & Xcode 6.0.1)
you need to create an OSX "Bundle" project in XCode
wrapper extension is mailbundle (under Packaging in the project Build settings)
a bundle needs to be stored under ~/Library/Mail/Bundles (as Build Phase add a Copy Files action with that as absolute path destination and the *.mailbundle from your build/ folder as item to copy)
for development, I have set up /Applications/Mail.app as executable in my run scheme, so that Run in XCode will build it, copy the bundle and start mail; note that at this point you'll get an error from Mail that your plugin cannot be started and was disabled
you need to provide a list of SupportedPluginCompatibilityUUIDs in the Info.plist, I stole it from GPGMail, these change with new Mail/OSX versions
use class-dump to generate the header files from Mail.app's private API
starting point is MVMailBundle, which you have to inherit from and which has a registerBundle method to hook you in
I extracted that from the huge generated header file in a small MVMailBundle.h header to include where needed (as done by GPGMail)
create a new class MyMailBundle, inheriting from NSObject
it needs an initialize method
and set it as "Principle class" in the Info.plist so that it gets run when the bundle is loaded by Mail.app
#import <Cocoa/Cocoa.h>
#interface MyMailBundle : NSObject
+ (void)initialize;
#end
initialize implementation: previously, you could use the simple way and directly inherit as done in Letterbox, however, since 64-bit runtimes of Objective-C you have to use the dynamic way as done by GPGMail:
using NSClassFromString to dynamically get the MVMailBundle class
and class_setSuperclass from <objc/runtime.h> to have your own class inherit from it
and then call registerBundle on it casted as MVMailBundle (requires include of MVMailBundle.h)
#import <objc/runtime.h>
#import "MVMailBundle.h"
#import "MyMailBundle.h"
#implementation MyMailBundle
+ (void)initialize
{
NSLog(#"Loading MyMail plugin...");
// since 64-bit objective-c runtimes, you apparently can't load
// symbols directly (i.e. through class inheritance) and have to
// resort to NSClassFromString
Class mvMailBundleClass = NSClassFromString(#"MVMailBundle");
// If this class is not available that means Mail.app
// doesn't allow plugins anymore or has changed the API
if (!mvMailBundleClass)
return;
// dynamically change super class hierarchy
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wdeprecated"
class_setSuperclass([self class], mvMailBundleClass);
#pragma GCC diagnostic pop
// register our plugin bundle in mail
[[((MyMailBundle *)self) class] registerBundle];
NSLog(#"Done registering MyMail plugin.");
}
#end
add some NSLog logging calls to verify the right thing is happening, they'll be visible in XCode's console when running/debugging Mail.app from within XCode or alternatively in the system logs of Console.app
This should successfully run the plugin in Mail with no error!
The next steps involve crazy things like MethodSwizzling and ClassPosing to modify Mail's behavior, where GPGMail can be a helpful example. (Haven't been there myself yet)
For reference, here are some of the resources that helped me:
GPGMail
Adam Nash: Getting Ready to Write an Apple Mail.app Plug-in for Mac OS X - some good links, but apparently he never finished the project, so no code
James R. Eagan: Demystifying Mail.app Plugins on Leopard - using PyObjC to write a plugin in Python, explains the basic mechansims, very useful
Aaron Harnly: Mail Plugin Template - for XCode 2 I think, unfortunately the template (download a zip) doesn't work as template in Xcode anymore, but the code is still useful to look at
Aaron Harnly: Letterbox sources - from the same guy, but also from 2007, very outdated; contains a readme from the template, though it doesn't really help if you can't use the template.
There is no official supported way to build such a tool - you need to start trying to hook in to Mail.app without any official support.
If you want to persist on this sort of thing, then you'll need to understand how Mail.app internals work, which is a bunch of using the debugger and class dump to inspect libraries in other apps:
https://github.com/nygard/class-dump
You'll probably also want a way to inject code into other applications, for example:
https://github.com/rentzsch/mach_inject
And every time Apple update Mail.app you'll potentially need to redo everything :)
I'm creating an iPhone 5.0 project in Xcode 4.2 and would like to find the code coverage when the unit tests are executed. I'm quite new to the Xcode environment, and I've followed the steps provided here. I'm able to modify the Build Settings for the test target correctly, and link the "libprofile_rt.dylib" file fine.
At this point, when I execute the tests (using Command-U), the code compiles and the tests pass. I do not encounter the problem described here. In addition, I've installed CoverStory.
The author in the first link mentions "Just run your unit tests and view the code coverage data as usual"; however, I cannot find .../Objects-normal/i386.
Just to get things working, I created a new project with the following class:
#import "SomeClass.h"
#implementation SomeClass
#synthesize someValue;
-(void)performWork:(BOOL)now withValue:(int)value {
if (now) {
someValue = value;
}
else {
someValue = value - 1;
}
}
#end
and test class:
#import "CodeCoverageTests.h"
#import "SomeClass.h"
#implementation CodeCoverageTests
- (void)testExample {
SomeClass *obj = [[SomeClass alloc] init];
[obj performWork:YES withValue:3];
STAssertEquals(obj.someValue, 3, #"Value was not 3");
}
#end
Ideally, I'd like to be notified in some way that when the tests execute, the else clause in the performWork method is never fired.
I thus have the following questions:
Is the root problem that there's no support for what I'm trying to do with the new compiler?
Is the only solution the one described by user chown in response to the question I linked above?
Will I be able to use CoverStory (or something similar) if I follow the solution from 2) ?
Update:
After some struggle, I was finally able to find the location of the "SomeClass.gcno" and "SomeClass.gcda" files (thanks #bjhomer - see this link), and they depicted beautifully that the if part of the conditional statement in performWork was covered (and the else was not). To make sure, I modified the test as follows:
- (void)testExample
{
SomeClass *obj = [[SomeClass alloc] init];
[obj performWork:NO withValue:3];
STAssertEquals(obj.someValue, 2, #"Value was not 2");
}
After re-building and re-execution of the unit test, I reloaded the .gcno and .gcda files. CoverStory showed that the coverage changed to the else part of the performWork method. There was one small caveat however:
I needed to modify the build settings of the <TargetName> (not the <TargetNameTest> as shown here) in order for the "SomeClass.gcno" and "SomeClass.gcda" files to be created in ...<TargetName>.build/Objects-normal/i386/ directory.
Thanks again for your help!
It sounds like your main problem is that you cannot find the Build/Intermediates/<SchemeName>.build/<ConfigurationName>-<PlatformName>/<TargetName>.build/Objects-normal/<ArchitectureName> directory. That directory is always used when building, whether or not you're doing code coverage. That's where all the intermediate files generated by the compiler are stored, and is also where the final .gcno and .gcda files should exist if you're doing code coverage. If you cannot find that directory, the problem is not related to coverage, the problem is that you need to find that directory.
To answer your questions.
Code coverage is supported in the LLVM Compiler, as of Xcode 4.2. It will not work with LLVM-GCC, so make sure you're not using that.
No, it is not necessary to hack in support for GCC 4.2 in Xcode 4.2. I'm running code coverage with Xcode 4.2 right now, having done none of the above.
You should be able to use CoverStory either way; hacking in support for GCC 4.2 would not change where the built products go.
I am writing tests for my iPhone app using OCUnit.
Is there any way to debug the unit tests and have a break point to see what the heck is happening?
It's ridiculously hard to write unit tests without being able to use breakpoints.
The link posted by David Gelhar is correct for Xcode 3.
For Xcode 4, things are much simpler. Edit your current scheme and go to the "Test" action. Click the '+' at the bottom and add the test bundle that contains the tests you want to run. Now when you choose Product -> Test, it will run those tests. Any active breakpoints will be hit just like you'd expect.
Using XCode 4.2, (with SenTestKit unit tests as set up by checking the "Include Unit Tests" checkbox when setting up the project), Product->Test wasn't hitting my breakpoints, but Product->Perform Action->Test Without Building seems to do the trick. Hope this helps.
You may have also accidentally disabled "Debugger: Debug Executable" option in Scheme -> Test -> Info
Here's a blog post: Debugging Cocoa application unit tests with instructions for how to do this (for XCode 3 at least; not sure about XCode 4).
Another item to watch for in XCode 4 is that you haven't added the classes being tested to the Unit Test Target as well as the main project. It appears that it's not necessary and it will mess up your ability to have breakpoints hit.
A clue will be warning messages in the debug log when you run. The messages will look like this:
"Class XXX is implemented in both YYY and ZZZ. One of the two will be used. Which one is undefined."
Once I removed the classes noted in the warnings from the unit test target, Xcode started hitting the breakpoints.
You can remove classes from a target by clicking on the .M file, and turning off its membership in the unit test target in the inspector window under "Target Membership".