I'm using XCTest in an Objective C project.
In my unit test target function, I use NSAssert to exit when receive an exception.
Even this behavior is expected, the XCTest still treat it as a failure.
Is there a way for XCTest to mark that this unit test is expected to assert?
You use XCAssertThrow macro to test for an arbitrary exception of an expression:
- (void)method {
#throw [[NSException alloc] initWithName:#"My Exception" reason:nil userInfo:nil];
}
- (void)testThrows {
XCTAssertThrows([self method], #"The method must throw");
}
Related
I'm running fuzz testing on an application, and so am looking specifically for runtime errors that aren't handled. The application is written in both ObjC and Swift, but the unit tests are written in Swift.
I understand the basis of swift isn't to catch arbitrary runtime exceptions, but this is purely for unit tests. How do I catch runtime these exceptions (i.e. index out of bounds etc.)
To catch Obj-C exceptions in Swift, I am using a simple Obj-C class:
#import "ObjC2Swift.h"
#implementation ObjC
+ (id)catchException:(id(^)())tryBlock error:(__autoreleasing NSError **)error {
#try {
id result = tryBlock();
return result;
}
#catch (NSException *exception) {
if (error) {
*error = [[NSError alloc] initWithDomain:exception.name code:0 userInfo:exception.userInfo];
}
return nil;
}
}
#end
In Swift called as
let result = try? ObjC.catchException { ... dangerous code here ... }
You might need a different variant for blocks that don't return anything.
Not to be abused. Obj-C exception are evil and I am using this only because I need a library that uses them.
The part of a method that I am trying to test is as follows:
- (void)configureTableFooterView {
dispatch_async(dispatch_get_main_queue(), ^{
self.tableView.tableFooterView = nil;
if ([self.parser.resultSet isLastPage]) {
return;
}
});
}
I have written the unit test as follows:
- (void)testTableFooterViewConfigurationAfterLastPageLoaded {
id mockTableView = OCMClassMock([GMGFlatTableView class]);
OCMExpect([mockTableView setTableFooterView:[OCMArg isNil]]);
id resultSet = OCMClassMock([GMGResultSetInfo class]);
OCMStub([resultSet isLastPage]).andReturn(YES);
OCMStub([self.mockParser resultSet]).andReturn(resultSet);
id partialMockSUT = OCMPartialMock(self.sut);
OCMStub([partialMockSUT tableView]).andReturn(mockTableView);
[self.sut configureTableFooterView];
OCMVerifyAllWithDelay(mockTableView, 2.0);
//OCMVerifyAllWithDelay(partialMockSUT, 2.0);
}
I have another test in the same class which is testing the same things from with in the dispatch_async call on the main thread. The test expectations and verification setup in that test match this one. While that test passes, this one gets stuck in an infinite loop at the delayed verification step.
Interestingly, if I only run this 1 test, it passes with out any problems. Its only when this test is run with other tests that I see the problem.
UPDATE:
In unit test, execute the block passed in queue with dispatch_asyc
This is a much more relevant post. However, this fails almost in the exact same way as the original test method:
- (void)testTableFooterViewConfigurationAfterLastPageLoaded {
id mockTableView = OCMClassMock([GMGFlatTableView class]);
OCMExpect([mockTableView setTableFooterView:[OCMArg isNil]]);
id resultSet = OCMClassMock([GMGResultSetInfo class]);
OCMStub([resultSet isLastPage]).andReturn(YES);
OCMStub([self.mockParser resultSet]).andReturn(resultSet);
id partialMockSUT = OCMPartialMock(self.sut);
OCMStub([partialMockSUT tableView]).andReturn(mockTableView);
[self.sut configureTableFooterView];
[[NSRunLoop mainRunLoop] runUntilDate:[NSDate dateWithTimeIntervalSinceNow:0.01]];
OCMVerifyAll(mockTableView);
}
The line with NSRunLoop crashes with EXC_BAD_ACCESS when run as suite but runs fine alone!
You can make class wrapper around dispatch_async, and pass it as dependency. Also you can make fake wrapper, and pass it in tests. If you interested in, I can provide much more detailed explanation.
I'm newcomer to Objective-C and OCMock. I created this trivial test and expect it to pass, but it fails:
-(void)testMock
{
id dataMock = [OCMockObject niceMockForClass:[NSData class]];
[[[dataMock stub] andReturn:dataMock] initWithBase64EncodedString:[OCMArg any]
options:[OCMArg any]];
NSData *ret = [dataMock initWithBase64EncodedString:#"dGVzdA=="
options:NSDataBase64DecodingIgnoreUnknownCharacters];
XCTAssertEqualObjects(dataMock, ret);
}
As you see, I stub a method and command it to return dataMock, but it returns null:
Assertions: ((dataMock) equal to (ret)) failed:
("OCMockObject(NSData)") is not equal to ("(null)")
Note: Testing with init instead of initWithBase64EncodedString, works as expected and passes.
I'm using XCode 6.0.1 and programming for iOS 8.0.
I found out that this the famous problem of matching primitives in OCMock. options in the mentioned method, takes a primitive and [OCMArg any] does not match it.
So for now I'm going with the exact used value:
[[[dataMock stub] andReturn:dataMock] initWithBase64EncodedString:[OCMArg any]
options:NSDataBase64DecodingIgnoreUnknownCharacters];
When I'm running unit tests, I'd like to skip some code (e.g. I don't want [[UIApplication sharedApplication] openURL:..] to run). I'm looking for a runtime check if I'm currently running units tests or not.
I know I have seen code that checks the Objective-C runtime if unit tests are running but am not able to find it anymore.
You can use this method from google-toolbox-for-mac
// Returns YES if we are currently being unittested.
+ (BOOL)areWeBeingUnitTested {
BOOL answer = NO;
Class testProbeClass;
#if GTM_USING_XCTEST // you may need to change this to reflect which framework are you using
testProbeClass = NSClassFromString(#"XCTestProbe");
#else
testProbeClass = NSClassFromString(#"SenTestProbe");
#endif
if (testProbeClass != Nil) {
// Doing this little dance so we don't actually have to link
// SenTestingKit in
SEL selector = NSSelectorFromString(#"isTesting");
NSMethodSignature *sig = [testProbeClass methodSignatureForSelector:selector];
NSInvocation *invocation = [NSInvocation invocationWithMethodSignature:sig];
[invocation setSelector:selector];
[invocation invokeWithTarget:testProbeClass];
[invocation getReturnValue:&answer];
}
return answer;
}
The reason that NSClassFromString and NSInvocation are used is to allow code compile without linking to xctest or ocunit
Rather that sprinkling "am I testing?" conditionals throughout production code, I isolate the check to one place: main. There, I check for an alternate application delegate for testing. If it's available, I use it instead of the the regular application delegate. This completely bypasses the regular launch sequence:
int main(int argc, char *argv[])
{
#autoreleasepool {
Class appDelegateClass = NSClassFromString(#"TestingAppDelegate");
if (!appDelegateClass)
appDelegateClass = [AppDelegate class];
return UIApplicationMain(argc, argv, nil, NSStringFromClass(appDelegateClass));
}
}
You can read more about this technique here: How to Easily Switch Your iOS App Delegate for Testing
Select the project, and then the test target:
Select Build Settings and choose All and Combined. Type 'preproc' in the search box - you're after Preprocessor Macros.
Add a macro to the Debug configuration called TEST and set it equal to 1:
Then in your code, you can do this:
#ifndef TEST
[[UIApplication sharedApplication] doEvilThingForTesting];
#endif
Or if you have code that you want to only run in a test environment:
#ifdef TEST
[[UIApplication sharedApplication] doAwesomeTestOnlyThing];
#endif
It's not exactly runtime, but the unit tester compiles the code before it runs the tests IIRC, so it should be the same effect - you're essentially modifying the code right before running the tests.
I'm not sure how long this will continue to work, but it works for me right now with Version 9.0 beta 6 (9M214v).
let isTesting = { () -> Bool in
if let _ = ProcessInfo.processInfo.environment["XCTestConfigurationFilePath"] {
return true
} else if let testingEnv = ProcessInfo.processInfo.environment["DYLD_INSERT_LIBRARIES"] {
return testingEnv.contains("libXCTTargetBootstrapInject.dylib")
} else {
return false
}
}()
No build environment or scheme changes are necessary.
It appears that there are two different environment variables in play depending on whether you are running a single test case or the entire test suite. Also, the variable value also differs depending whether or not you are running in simulator or on a real device.
I think you can check like this for Xcode 7.3
-(BOOL) isRunningUnitTests
{
NSDictionary* environment = [ [ NSProcessInfo processInfo ] environment ];
NSString* theTestConfigPath = environment[ #"XCTestConfigurationFilePath" ];
return theTestConfigPath != nil;
}
The easiest (and working in Xcode 7 with XCTest!) way to check is to have a look at the process info for a matching xctest bundle:
static BOOL isRunningTests(void)
{
NSDictionary* environment = [[NSProcessInfo processInfo] environment];
NSString* injectBundle = environment[#"XCInjectBundle"];
return [[injectBundle pathExtension] isEqualToString:#"xctest"];
}
Source: https://www.objc.io/issues/1-view-controllers/testing-view-controllers/#integration-with-xcode
Just use this:
+ (BOOL)isUnitTestRunning
{
Class testProbeClass;
testProbeClass = NSClassFromString(#"XCTestProbe");
return (testProbeClass != nil);
}
I know how to resolve EXC_BAD_ACCESS issues, but I'm not sure how to unit test for it. Is there a way to capture EXC_BAD_ACCESS in code instead of simply crashing?
Here's why I ask: I have written a library that heavily uses blocks, like this:
- (void)doSomething:(void (^)())myBlock;
In my implementation of doSomething: I'm going to eventually run the block, like this:
myBlock();
If a caller passes nil for the block, then it will crash with EXC_BAD_ACCESS, so the solution is to check that the block exists, like this:
if (myBlock) {
myBlock();
}
This nil check is pretty easy to forget, so I'd like a way to write a unit test that fails when the crash occurs. I suppose a crash could be considered a test failure, but I think it would be nicer for others trying to run the tests to see a nice failure message rather than a crash. Any ideas?
I think you'll need to run the test in a subprocess; then you can let the subprocess crash, check for that crash, and fail the test neatly if it occurs.
Working from Peter Hosey's singleton test code.
- (void) runTestInSubprocess:(SEL)testCmd {
pid_t pid = fork();
// The return value of fork is 0 in the child process, and it is
// the id of the child process in the parent process.
if (pid == 0) {
// Child process: run test
// isInSubprocess is an ivar of your test case class
isInSubprocess = YES;
[self performSelector:testCmd];
exit(0);
} else {
// Parent process: wait for child process to end, check
// its status
int status;
waitpid(pid, &status, /*options*/ 0);
// This was a crash; fail the test
STAssertFalse(WIFSIGNALED(status), #"Test %# crashed due to signal %d", NSStringFromSelector(testCmd), WTERMSIG(status));
}
}
Each test will then run itself in a subprocess like so:
- (void) testSomething {
if (!isInSubprocess) {
// Hand off this test's selector to be run in a subprocess
[self runTestInSubprocess:_cmd];
return;
}
// Put actual test code here
STAssertEquals(1, 1, #"Something wrong with the universe.");
}
You may need to tweak this; I haven't tested it.
I would suggest using one of the assertion macros found in the Assertions and Logging Programming Guide
So you could do something like:
NSAssert(myBlock != nil, #"myBlock must not be nil")
This enforces the preconditions that must be met before the method continues executing. It also allows the app to crash and will give you a reason why other than EXEC_BAD_ACCESS.