Is there any non-Xcode-based command line unit testing tool for Objective-C? - objective-c

Question

The post Compiling Objective-C without GUI says the following:
To compile Objective-C on OSX the easy way you have to get XCode, which is free to obtain from the Application Store. Getting XCode will ensure you obtain the necessary frameworks (headers), like Foundation, Cocoa, etc. This will, however, not provide you with the necessary command line tools to compile Object-C from the command line. Open up XCode, go to Preference > Downloads > Components and Install Command Line Tools. This will install gcc, clang, make, etc.
I am looking for a non-Xcode-based tool to address this my question, I've just opened: Is it possible to make an Objective-C project to be tested on Travis?
This tool should meet the following requirements:
It should be not related to Xcode in any way.
The following level of simplicity is pretty enough: just some int main {} code collecting all test-cases files nearby and running test assertions on the code I want to test (like ST- in SetTestingKit or GH- in GHUnit)
I don't need UI, GUI, Xcode, simulator.
It will be great if it could work both on Mac and Ubuntu (yeah, Travis), possibly using
GNUstep like the quoted post describes.
UPDATE BASED ON ACCEPTED ANSWER:
The following simple setup, based on what Malte Tancred have said in the accepted answer, seems pretty enough for my current needs:
Three files, all in the tests directory of my project:
octest.m
#import <Foundation/Foundation.h>
#import <SenTestingKit/SenTestingKit.h>
int main() {
#autoreleasepool {
SenSelfTestMain();
}
return 0;
}
Makefile
CC=clang # or gcc
# Trick to get current dir: https://stackoverflow.com/questions/322936/common-gnu-makefile-directory-path
TESTS_DIR:= $(dir $(lastword $(MAKEFILE_LIST)))
PROJECT_DIR:= $(TESTS_DIR)/..
FRAMEWORKS_PATH:= -F/Applications/Xcode.app/Contents/Developer/Library/Frameworks
FRAMEWORKS:= -framework Foundation -framework SenTestingKit
LIBRARIES:= -lobjc
INCLUDE_PATHS:= -I$(PROJECT_DIR)/Projectfiles\
-I$(TESTS_DIR)/TestHelpers
SOURCE_FILES = $(wildcard $(PROJECT_DIR)/Projectfiles/*.m)
SOURCE_TEST_SUITE = $(wildcard $(TESTS_DIR)/Tests/*.m)
SOURCE_TESTS = $(TESTS_DIR)/TestHelpers/TestHelpers.m\
octest.m
CFLAGS=-Wall -Werror -fobjc-arc -g -v $(SOURCE_FILES) $(SOURCE_TEST_SUITE) $(SOURCE_TESTS)
LDFLAGS=$(LIBRARIES) $(FRAMEWORKS)
OUT=-o octest
all:
$(CC) $(FRAMEWORKS_PATH) $(CFLAGS) $(INCLUDE_PATHS) $(LDFLAGS) $(OUT)
runtests
#!/bin/bash
export DYLD_FRAMEWORK_PATH=/Applications/Xcode.app/Contents/Developer/Library/Frameworks
make
./octest
UPDATE TO CAPTURE EXIT CODE:
A few days after I had asked this question, Travis announced Objective-C support:
http://about.travis-ci.org/blog/introducing-mac-ios-rubymotion-testing/
Though there are default scripts they suggest to use to make builds, I have decided to take the approach described here and still use octest instead of approach with xcodebuild that Travis uses.
By default travis setup relies on the build scripts written by Justin Spahr-Summers: https://github.com/jspahrsummers/objc-build-scripts:
They use awk to capture exit code from xcodebuild's output, since it always exists with 0 exit code, even if the whole test suite has failing!
OCTest behaves the same way - it always exists with 0 code, and here is how I've used simplified version of Travis awk script for my needs of building it the way I decribed above:
octest.awk
# Exit statuses:
#
# 0 - No errors found.
# 1 - Build or test failure. Errors will be logged automatically.
BEGIN {
status = 0;
}
{
print;
fflush(stdout);
}
/[0-9]+: (error|warning):/ {
errors = errors $0 "\n";
}
/with [1-9]+ failures?/ {
status = 1;
}
END {
if (length(errors) > 0) {
print "\n*** All errors:\n" errors;
}
fflush(stdout);
exit status;
}
runtests
#!/bin/bash
export DYLD_FRAMEWORK_PATH=/Applications/Xcode.app/Contents/Developer/Library/Frameworks
make
runtests ()
{
./octest 2>&1 | awk -f "octest.awk"
local awkstatus=$?
if [ "$awkstatus" -eq "1" ]
then
echo "Test suite failed"
return $awkstatus
else
echo "Test suite passed"
return 0
fi
}
echo "*** Building..."
runtests || exit $?

Answer 1

There are ports of objcunit for non Mac platforms. You can look at the port in MidnightBSD that works with GNUstep. The patches should work for other environments too. The pkg-descr has the packages website and the Makefile will have the download URL.

Answer 2

The answer to your main question is yes:
there are command line testing tools for Objective-C
that does not depend on Xcode.
For example,
you can use OCUnit/SenTestingKit without using Xcode.
All you have to do
is to point your compiler/linker to the framework.
Consider the follwing file, octest.m:
#import <Foundation/Foundation.h>
#import <SenTestingKit/SenTestingKit.h>
int main() {
#autoreleasepool {
SenSelfTestMain();
}
return 0;
}
#interface MyTest : SenTestCase
#end
#implementation MyTest
- (void)testSomething {
STAssertEquals(1, 2, #"fail");
}
#end
Compile it:
clang -o octest octest.m -F/Applications/Xcode.app/Contents/Developer/Library/Frameworks -framework Foundation -framework SenTestingKit
Now run it
DYLD_FRAMEWORK_PATH=/Applications/Xcode.app/Contents/Developer/Library/Frameworks ./octest
and you should see something like
Test Suite '/tmp/octest(Tests)' started at 2013-04-04 12:34:49 +0000
Test Suite 'MyTest' started at 2013-04-04 12:34:49 +0000
Test Case '-[MyTest testSomething]' started.
octest.m:16: error: -[MyTest testSomething] : '1' should be equal to '2': fail
Test Case '-[MyTest testSomething]' failed (0.000 seconds).
Test Suite 'MyTest' finished at 2013-04-04 12:34:49 +0000.
Executed 1 test, with 1 failure (0 unexpected) in 0.000 (0.000) seconds
Test Suite '/tmp/octest(Tests)' finished at 2013-04-04 12:34:49 +0000.
Executed 1 test, with 1 failure (0 unexpected) in 0.000 (0.001) seconds
make: *** [default] Error 1
This example does depend on Xcode
in that it uses the bundled SenTestingKit framework
but the general process of building and running the tests
as described above
does not depend on Xcode.
Now,
to get this running on a linux system
you'd have to install SenTestingKit (and most probably GNUstep),
but with those components in place
the build and test process should be essentially the same.

Answer 3

For compiling objective-C, I know a quick and dirty (and somewhat limited) way of doing it - I do it online at http://www.compileonline.com/compile_objective-c_online.php . That said, this compiler throws back a lot of c-99 errors which I think is due to it being a strict superset of C and not including some of the more unique features of the language (e.g.: #synthesize, or even dot notation), but it gets the job done. I type up the code with Notepad++. Like I said, limited, but surgical.
*Also - it supports command line style input, which is solid.

Answer 4

Look at xctool from facebook which is a replacement for Apple's xcodebuild. We build our Objective-C library using maven and use it only for tests instead of xcodebuild and it works great. The output is much more readable as compared to xcodebuild.
From the xctool page,
xctool is a replacement for Apple's xcodebuild that makes it easier to build
and test iOS and Mac products.
One thing to note though is it does not support building targets. You can use a scheme though.