What frameworks exist to unit test Objective-C code? I would like a framework that integrates nicely with Apple Xcode.
Xcode includes XCTest, which is similar to OCUnit, an Objective-C unit testing framework, and has full support for running XCTest-based unit tests as part of your project's build process. Xcode's unit testing support is described in the Xcode Overview: Using Unit Tests.
Back in the Xcode 2 days, I wrote a series of weblog posts about how to perform some common tasks with Xcode unit testing:
Unit testing Cocoa frameworks
Debugging Cocoa framework unit tests
Unit testing Cocoa applications
Debugging Cocoa application unit tests
Despite using OCUnit rather than XCTest, the concepts are largely the same.
Finally, I also wrote a few posts on how to write tests for Cocoa user interfaces; the way Cocoa is structured makes it relatively straightforward, because you don't have to spin an event loop or anything like that in most cases.
Trust, but verify.
Unit testing Cocoa user interfaces: Target-Action
Unit testing Cocoa user interfaces: Cocoa Bindings
This makes it possible to do test-driven development for not just your model-level code but also your controller-level and even view-level code.
Check out GHUnit by Gabriel Handford:
"The goals of GHUnit are:
Runs unit tests within XCode, allowing
you to fully utilize the XCode
Debugger. A simple GUI to help you
visualize your tests. Show stack
traces. Be installable as a framework
(for Cocoa apps) with a simple (or
not) target setup; or easy to package
into your iPhone project."
I started using the Google toolbox testing rig for iPhone, and its working out great for me.
google-toolbox-for-mac
Check out OCUnit. Apple's developer network has a great introduction.
Note that the Google Toolbox for Mac (GTM) project simply extends/augments Apple's SenTestingKit framework (which is, itself based on OCUnit). As they say on the project site:
GTM has several enhancement to the
standard SenTestingKit allowing you to
do UI unit testing, automated binding
unit testing, log tracking, and unit
testing on the iPhone, as well as
tools for doing static and dynamic
testing of your code.
Note the following comment about user-interface testing:
GTM has extensive support for user
interface unit tests. It supports
testing both the imaging and/or
internal state of almost all of the
standard Cocoa/UIKit UI objects, and
makes it easy for you to extend this
support to your own UI objects.
See their "Code Verification and Unit Testing" page for instructions on how to use it.
I came to the conclusion that GHUnit is the most advanced testing framework for Objective-C. I have done a roundup of testing frameworks on my blog. It is the most flexible in terms of deployment (iphone, simulator or mac os native) and assert capabilities. Because it is based on GTM, it inherits all of GTM's advantages over SenTestingKit but also adds a lot more. Another bonus is that it is being maintained very actively.
I have conducted effort to integrate OCMock into GHUnit, it works great!. You can get the code on github.
I realize this is an old question, but if you prefer BDD-style testing (rspec, Jasmine, etc.) over xUnit-style testing (Test::Unit, JSUnit, JUnit, etc.), then you may consider checking out Cedar. Cedar brings BDD-style testing to Objective-C, now that the language supports closures.
We're happily using Cedar for our iOS projects at Pivotal Labs, and we're actively working on improving it. Any feedback or suggestions are welcome at cedar-discuss#googlegroups.com
I would also recommend using coverage tools to see which part of the code are covered with unit tests and which are not. Basic line and branch code coverage can be generated with the GCOV tool. If you want to generate nice HTML coverage reports there are LCOV and ZCOV which do just that.
I recommend gh-unit, it has a nice GUI for test results.
http://github.com/gabriel/gh-unit/tree/master
The Unit Testing support bundled within xcode (for its simple setup) combined with ocrunner (for some autotest/Growl goodness) is currently my favorite Obj-C Unit Testing setup.
here is a whole lot of them
List_of_unit_testing_frameworks in Objective-C
Sen:te (the creator of the testing framework included with Xcode) explains how to use OCUnit with an iPhone project: simple-iphone-ipad-unit-test.
Matt Gallagher of Cocoa with Love has a very good article on unit testing.
I would suggest looking into Kiwi, an open source BDD testing framework for iOS:
Kiwi
Check out the project's WIKI to start or get Daniel Steinberg's book "Test Driving iOS Development with Kiwi"
test-driving-ios-development
I use SimpleUnitTest works with iPhone and iPad libs.
http://cbess.blogspot.com/2010/05/simple-iphone-ipad-unit-test.html
It comes with a unit test Xcode template to easily add a unit test class. Wraps GTM.
You can literally drop it into an active project and start adding unit tests within 3 minutes (or less).
Specta is a modern TDD(Test Driven Development)/BDD(Behavior Driven Development) framework which runs on top of XCTest. It supports unit testing for iOS and Mac OS X projects.
I hope u can use 'SenTestKit', from which u can test each and every method.
Related
I found it over a few references that it is recommended to go with the same stack for creating automation suite that is being used for the development of the application, i.e., using Java for writing tests if application is Java based.
Is there any specific reason for the same? Or is there any specific hindrance if not followed.
I would personally say that developers making an application in one language will probably feel more familiar writing the tests in the same language. And Integration tests should preferably be written by the developer who wrote the feature.
So using the same language for your test suite and your application development increases the odds your developers will be able to write and fix tests for features they create and fix themselves.
I think that should be team decision, the same as what testing frameworks to use. But it's totally fine to use for example Spock which is groovy based framework if your application is java based.
I am working on my first MVVMCross application and I am trying to set up a good testing framework. I have looked at the testing Stuart is doing in his TwitterSearch app but I have found nothing that explains his approach or any other approach to testing MVVMCross. Has anyone come across a good post/tutorial on the proper way to test an MVVMCross application? Other than just code that a newb (like myself) may not completely understand...
I'm not sure what you are asking...
What do you mean by a 'proper way'?
Is this a question about mechanics of "how to write a unit test?" Or a question about "how many unit tests to write; Which components to test; How deep to go; etc?"
For the mechanics:
I personally use NUnit for testing (from NuGet)
I include this in a .net4.5 class library project.
I use Moq for most of my Mocking (from NuGet)
There are a few MvvmCross objects I manually mock - as shown in that TwitterSearch message
There's an MvvmCross base test class which provides IoC/ServiceLocation - but I generally only use this when I need to use real MvvmCross classes - e.g. when the class under test inherits from MvxViewModel
I only run tests within Resharper in Visual Studio
There are plenty of other approaches, including some people choose to run tests on devices - e.g. using the excellent MonoTouch Nunit test runner.
There are also plenty of people interested in BDD testing - e.g. things like Frank, Calabash (coming in Xamarin Test Cloud) and the Windows Phone Test Framework that I wrote - https://github.com/Expensify/WindowsPhoneTestFramework :)
For the philosophy, I have no strong opinion, but I like this answer:
https://stackoverflow.com/a/153565/373321
I too get paid to write code
I'm currently reading the book Growing Object-Oriented Software Guided by Tests which gives a decent introduction into test driven development.
One drawback is that the code developed through the book is based on Swing. For my next project I'd like to use Java FX 2 and I'm wondering if there are any GUI testing frameworks out there for it? The book introduces windowlicker, which allows testing Swing applications only.
There's yet another new testing library for JavaFX called TestFX.
There is a library named JemmyFX. Jemmy itself is a set of libraries and tools to test UI applications (mostly Java-based: Swing, AWT, SWT; but it's being extended to native right now). JemmyFX covers JavaFX for that matter.
JemmyFX can be downloaded as a part of the test branch of OpenJFX 2 (open-sourced part of JavaFX).
For JavaFX 8, the test branch includes support for new controls and other fixes.
There's a new test robot called MarvinFX:
MarvinFX has the goal to easily test JavaFX controls and scenes with a special attention on properties.
This other post shows how to use assertions and rules with MarvinFX.
Another library (which I have not used or tested) is TestComplete.
Automaton is another testing library for JavafX and Swing GUIs born out of the same team that did TestFX
easy tests for Swing and JavaFX applications
written for testers. Only basic coding skills required.
According to Eclipse site, the modern testing tools for JavaFX in the year 2015 are Squish and Jubula . But Squish is commercial. And Jubula is partly free - JavaFX belongs to that free part. And in the Eclipse Mars version already exists the testing distribution. Uses Jubula for UI testing.
Jubula is now totally free, and does indeed support JavaFX.
My question maybe silly, but can anyone coach me?
Except doing some test(most likely white box testing) while coding, after the App was built, do we have some testing tools or special method for doing the test?
All I can imaging for now, is only manual testing the functionality of my App.
Thanks everyone.
Update: Added section 'Automated testing for iOS4'
As a professional tester my suggestion is that you should have a healthy mix of automated and manual testing. The Examples below are in .net but it should be easy to find a tool for whatever technique you are using.
AUTOMATED TESTING
Unit Testing
Use NUnit to test your classes, functions and interaction between them.
http://www.nunit.org/index.php
Automated Functional Testing
If it's possible you should automate a lot of the functional testing. Some frame works have functional testing built into them. Otherwise you have to use a tool for it. If you are developing web sites/applications you might want to look at Selenium.
http://www.peterkrantz.com/2005/selenium-for-aspnet/
Continuous Integration
Use CI to make sure all your automated tests run every time someone in your team makes a commit to the project.
http://martinfowler.com/articles/continuousIntegration.html
Automated testing for iOS4
Automate the testing of your application by scripting touch events using the new UIAutomation Instrument.
Some links:
http://answers.oreilly.com/topic/1646-how-to-use-uiautomation-to-create-iphone-ui-tests/
http://cocoawithlove.com/2008/11/automated-user-interface-testing-on.html
http://alexvollmer.com/posts/2010/07/03/working-with-uiautomation/
MANUAL TESTING
As much as I love automated testing it is, IMHO, not a substitute for manual testing. The main reason being that an automated can only do what it is told and only verify what it has been informed to view as pass/fail. A human can use it's intelligence to find faults and raise questions that appear while testing something else.
Exploratory Testing
ET is a very low cost and effective way to find defects in a project. It take advantage of the intelligence of a human being and a teaches the testers/developers more about the project than any other testing technique i know of. Doing an ET session aimed at every feature deployed in the test environment is not only an effective way to find problems fast, but also a good way to learn and fun!
http://www.satisfice.com/articles/et-article.pdf
Take a look at automated testing tools. Supports automated and manual testing/sending feedback from within the app with annotated screen shots
I suggest you take a look at the iPhoneUnitTests sample code posted by Apple on their developer site.
FoneMonkey is a free and open source functional testing automation tool available for download from Gorilla Logic.
There a number of emerging options for automated functional testing, including Appium, Calabash, Frank, and Zucchini.
Much of testing any application is about understanding what you are testing and areas that should be tested. Some of this comes with experience, but types of things to consider testing about would be:
Functionality
iOS Design Guidelines / UI
Gestures
Connectivity
Types of devices to test on
Audio
Data
Crash reporting
Analytics
There's a big list of areas to cover.
I recommend Kiwi, its used for Behavior Driven Development. By far my favorite testing framework, makes testing much more fun, and test much readable and clear.
https://github.com/allending/Kiwi
I'm writing my first OS X Objective-C framework, but I do not know how to test it. How can I execute methods and classes in framework for testing purposes?
Chris Hanson's excellent set of articles on setting up unit testing for Frameworks in Xcode is a wonderful reference for testing frameworks. Their content has been largely incorporated (and supplanted) by the article, Automated Unit Testing with
Xcode 3 and Objective-C, on Apples' Dev site.
Make a unit test bundle target, and create test case classes in that bundle. You'll link against the OCUnit framework to get the base classes for your tests; there are a few OCUnit tutorials around, which you can find on Google.
For harder-to-test tasks, such as drawing in views, the easiest way is to make a test app and check the results yourself.
You can find several articles about unit testing on the blog of Chris Hanson.
One possibility would be to create a new executable target that depends on and calls the framework you're using.