I am working on my first MVVMCross application and I am trying to set up a good testing framework. I have looked at the testing Stuart is doing in his TwitterSearch app but I have found nothing that explains his approach or any other approach to testing MVVMCross. Has anyone come across a good post/tutorial on the proper way to test an MVVMCross application? Other than just code that a newb (like myself) may not completely understand...
I'm not sure what you are asking...
What do you mean by a 'proper way'?
Is this a question about mechanics of "how to write a unit test?" Or a question about "how many unit tests to write; Which components to test; How deep to go; etc?"
For the mechanics:
I personally use NUnit for testing (from NuGet)
I include this in a .net4.5 class library project.
I use Moq for most of my Mocking (from NuGet)
There are a few MvvmCross objects I manually mock - as shown in that TwitterSearch message
There's an MvvmCross base test class which provides IoC/ServiceLocation - but I generally only use this when I need to use real MvvmCross classes - e.g. when the class under test inherits from MvxViewModel
I only run tests within Resharper in Visual Studio
There are plenty of other approaches, including some people choose to run tests on devices - e.g. using the excellent MonoTouch Nunit test runner.
There are also plenty of people interested in BDD testing - e.g. things like Frank, Calabash (coming in Xamarin Test Cloud) and the Windows Phone Test Framework that I wrote - https://github.com/Expensify/WindowsPhoneTestFramework :)
For the philosophy, I have no strong opinion, but I like this answer:
https://stackoverflow.com/a/153565/373321
I too get paid to write code
How do you experienced developers comment these lines by Michael Feathers:
A test is not a unit test if:
It talks to the database
It communicates across the network
It touches the file system
It can't run at the same time as any of your other unit tests
You have to do special things to your environment (such as editing
config. files) to run it.
Now I was wondering if I should unit test my DAO classes...
Will I get more advantages or disadvantages by unit testing the DAO layer? Share your thoughts please.
Feathers isn't saying don't write such tests. He's saying they're not unit tests, because he defines a unit test as "small, they test a method or the interaction of a couple of methods. ... a "binary chop" that allows you to discover whether the problem is in your logic or in the things are you interfacing with." And he's right - the Agile/XP/Scrum intention of unit testing is to provide a fast red light/green light determination if a small piece of code is functioning correctly.
Are there any books or articles that show you how to use NUnit to test entire features of a program? Is there a name for this type of testing?
This is different from the typical use of NUnit for unit testing where you test individual classes. This is similar to acceptance testing except that it is written by the developer to discern that the program does what they interpreted as being what the customer wants the program to do. I don't need it to be readable by non-programmers or to produce a readable specification for non-programmers.
The problem I am having is keeping this feature testing code maintainable. I need help in organizing my feature testing code. I also need help organizing the program code to be drivable in this way. I am having a hard time being able to issue commands to the program while still having good code design.
Currently I have a class called Program with a single public method called Run. With every test I start at the beginning of the program like the user would and then get to the desired point in the program where a particular feature would be available. I then use that feature in some way and verify it did what I want. I have a class called Commands that exposes different features of the program as methods. An instance of the Commands object is passed to the program and it eventually gets passed to every Form class. These will subscribe to events from the Commands class that are called by the methods of the command class(one matching event per method). The events are subscribed to by pointed to the method that is called when a certain part of the user interface is used, thus allowing the entire program to be drivable by my tests. If you call a method on the Command object for an event that is currently not subscribed to, a FeatureMissingException is thrown.
All of this works but I don't like the Command class. It is getting too large with too many responsibilities (every feature of the program). The Commands class is also a dependency magnet (all the Form classes have an instance of it but only subscribe to the events that represent features that can be activated through their UI).
It's called integration testing. Integration tests are much more difficult to make automated, and are very often done by hand. Many simpler tests can still be done using NUnit though - you don't have to do anything special, just don't use Mocks (like you should be doing for unit tests) so you can test how the modules actually fit together.
Context/specification is a good way of organizing these tests.
What you want to do is integration testing, like the other answer suggests. This will allows you to functional/feature testing. The most common framework for this for StoryQ or SpecFlow. This allows you to develop your tests in a BDD style and can be mostly be automated against the spec that you want.
Tools like Selenium allow you to do functional testing in a browser to do what the end user would do. All of these can be driven with NUnit since NUnit is purely a framework for running tests be them Unit tests to large functional tests
Am looking for a regression test framework where I can add tests to.. Tests could be any sort of binaries that poke an application..
This really depends on what you're trying to do, but one of the features of the new Test::Harness (disclaimer: I'm the original author and still a core developer) is that if your tests output TAP (the Test Anything Protocol), you can use Test::Harness to run test suites written in multiple languages. As a result, you don't have to worry about getting "locked in" to a particular language because that's all your testing software supports. In one of my talks on the subject, I even give an example of a test suite written in Perl, C, Ruby, and HTML (yes, HTML -- you'd have to see it).
Just thought I would tell you guys what I ended up using..
QMtest ::=> http://mentorembedded.github.io/qmtest/
I found QMTest to full fill my needs. Its extensible framework, allows you to write very flexible test classes. Then, these test classes could be instantiated to large test suites to do regression testing.
QMTest is also very forward thinking, it allows for weak test dependencies and the creation of test resources. After a while of using QMTest, I started writing better quality tests. However, like any other piece of complex software, it requires some time to learn and understand the concepts, the API is documented and the User Manual give a good introduction. With sometime in your hand, I think QMTest is well worth it.
You did not indicate what language you are working in, but the xUnit family is available for a lot of different languages.
/Allan
It also depends heavily what kind of application you're working on. For a commandline app, for example, its probably easy enough to just create a shell script that calls it with a whole bunch of different options and compares its result to a previously known stable version, warning you if any of the output differs so that you can check whether the change is intentional or not.
If you want something more fancy, of course, you'll probably want some sort of dedicated testing framework.
I assume you are regression-testing a web application?
There are some tools in this kb article from Microsoft
And if I remember correctly, certain editions of Visual Studio also offer its own flavor of regression testing tools as well.
But if you just want a unit testing framework, the xUnit family does it pretty well.
Here's JUnit and NUnit.
What frameworks exist to unit test Objective-C code? I would like a framework that integrates nicely with Apple Xcode.
Xcode includes XCTest, which is similar to OCUnit, an Objective-C unit testing framework, and has full support for running XCTest-based unit tests as part of your project's build process. Xcode's unit testing support is described in the Xcode Overview: Using Unit Tests.
Back in the Xcode 2 days, I wrote a series of weblog posts about how to perform some common tasks with Xcode unit testing:
Unit testing Cocoa frameworks
Debugging Cocoa framework unit tests
Unit testing Cocoa applications
Debugging Cocoa application unit tests
Despite using OCUnit rather than XCTest, the concepts are largely the same.
Finally, I also wrote a few posts on how to write tests for Cocoa user interfaces; the way Cocoa is structured makes it relatively straightforward, because you don't have to spin an event loop or anything like that in most cases.
Trust, but verify.
Unit testing Cocoa user interfaces: Target-Action
Unit testing Cocoa user interfaces: Cocoa Bindings
This makes it possible to do test-driven development for not just your model-level code but also your controller-level and even view-level code.
Check out GHUnit by Gabriel Handford:
"The goals of GHUnit are:
Runs unit tests within XCode, allowing
you to fully utilize the XCode
Debugger. A simple GUI to help you
visualize your tests. Show stack
traces. Be installable as a framework
(for Cocoa apps) with a simple (or
not) target setup; or easy to package
into your iPhone project."
I started using the Google toolbox testing rig for iPhone, and its working out great for me.
google-toolbox-for-mac
Check out OCUnit. Apple's developer network has a great introduction.
Note that the Google Toolbox for Mac (GTM) project simply extends/augments Apple's SenTestingKit framework (which is, itself based on OCUnit). As they say on the project site:
GTM has several enhancement to the
standard SenTestingKit allowing you to
do UI unit testing, automated binding
unit testing, log tracking, and unit
testing on the iPhone, as well as
tools for doing static and dynamic
testing of your code.
Note the following comment about user-interface testing:
GTM has extensive support for user
interface unit tests. It supports
testing both the imaging and/or
internal state of almost all of the
standard Cocoa/UIKit UI objects, and
makes it easy for you to extend this
support to your own UI objects.
See their "Code Verification and Unit Testing" page for instructions on how to use it.
I came to the conclusion that GHUnit is the most advanced testing framework for Objective-C. I have done a roundup of testing frameworks on my blog. It is the most flexible in terms of deployment (iphone, simulator or mac os native) and assert capabilities. Because it is based on GTM, it inherits all of GTM's advantages over SenTestingKit but also adds a lot more. Another bonus is that it is being maintained very actively.
I have conducted effort to integrate OCMock into GHUnit, it works great!. You can get the code on github.
I realize this is an old question, but if you prefer BDD-style testing (rspec, Jasmine, etc.) over xUnit-style testing (Test::Unit, JSUnit, JUnit, etc.), then you may consider checking out Cedar. Cedar brings BDD-style testing to Objective-C, now that the language supports closures.
We're happily using Cedar for our iOS projects at Pivotal Labs, and we're actively working on improving it. Any feedback or suggestions are welcome at cedar-discuss#googlegroups.com
I would also recommend using coverage tools to see which part of the code are covered with unit tests and which are not. Basic line and branch code coverage can be generated with the GCOV tool. If you want to generate nice HTML coverage reports there are LCOV and ZCOV which do just that.
I recommend gh-unit, it has a nice GUI for test results.
http://github.com/gabriel/gh-unit/tree/master
The Unit Testing support bundled within xcode (for its simple setup) combined with ocrunner (for some autotest/Growl goodness) is currently my favorite Obj-C Unit Testing setup.
here is a whole lot of them
List_of_unit_testing_frameworks in Objective-C
Sen:te (the creator of the testing framework included with Xcode) explains how to use OCUnit with an iPhone project: simple-iphone-ipad-unit-test.
Matt Gallagher of Cocoa with Love has a very good article on unit testing.
I would suggest looking into Kiwi, an open source BDD testing framework for iOS:
Kiwi
Check out the project's WIKI to start or get Daniel Steinberg's book "Test Driving iOS Development with Kiwi"
test-driving-ios-development
I use SimpleUnitTest works with iPhone and iPad libs.
http://cbess.blogspot.com/2010/05/simple-iphone-ipad-unit-test.html
It comes with a unit test Xcode template to easily add a unit test class. Wraps GTM.
You can literally drop it into an active project and start adding unit tests within 3 minutes (or less).
Specta is a modern TDD(Test Driven Development)/BDD(Behavior Driven Development) framework which runs on top of XCTest. It supports unit testing for iOS and Mac OS X projects.
I hope u can use 'SenTestKit', from which u can test each and every method.