I have a suite of unit tests that I use before checking in my project. However, very often it's the case that only one of them finds some regression in the code. In these cases I'd like to only run that particular unit test while debugging the failure. I haven't found any way to do this in Xcode. Is it possible?
If you're happy restricting your testing to a single test class, a simple option is to create a second test target (duplicate the existing target, change the product name and remove the contents of the "Compile Sources" build phase, if you wish) and add only the test source file you're trying to fix to it.
Alternatively, you can use the "Other Test Flags" option to pass a -SenTest argument to otest, the test runner:
% /Developer/Tools/otest
2009-08-29 22:28:39.555 otest[70089:10b] Usage: otest [-SenTest Self | All | None |
<TestCaseClassName/testMethodName>] <path of unit to be tested>
More information about using this method is here.
Thanks for that push in the right direction. I ended using the same basic concept, but I added a GUI that lets you select what gets run as well as get a nice red/green status for each test. If anyone is interested, the code is at the URL below. The UI needs to more spit and polish, but it seems to be working.
http://github.com/nall/XcodeUnitTestGUI/tree/master
After I started the project above, I found this project which is really fantastic.
http://github.com/gabriel/gh-unit
For new readers: A much better way now available in Xcode is to edit the scheme for the target to be tested and select "Test" in the left hand column of the scheme pane.
Use the widgets in the Tests column to expand targets and suites.
You can disable/enable tests on a per test target, per suite or per test basis using the check boxes on the right
Related
I’m trying to run a custom test suite which includes several test cases. For example, I’ve wrote 4 test scripts(test_login_success,test_login_fail,test_register_xxx,test_register_yyy), and I just want to run test_login_* module, how to set the defaultTestSuite and add testcases to it?
The test cases you create belongs to its class. If you want to customise test runs you shall consider updating to the new Xcode 11. The new version of Xcode has test plans feature allowing you to control tests executions better.
Introduction video:
https://developer.apple.com/videos/play/wwdc2019/413/
If you prefer to stay on previous Xcode, you should add schemes for your scenarios.
Also, you can pass tests names in xcodebuild shell command.
I found a really nice set of functionality in IntelliJ but it is very manual...
use Analyze/Analyze backward dependencies on a object to find all the tests that eventually reference that class.
Create a run configuration using Test Kind "Pattern" and manually enter each of the test classes found into the "Pattern" field.
Run the test with code coverage
Navigate to the original class to view it's total test coverage.
This whole process is fairly slow and user intensive, but it could easily be automated with a single "Find test coverage for class" key-press (It would still be pretty slow, but I could go on and do something else). Does anyone know if this is in a key binding or plug-in I haven't found yet? It seems like a pretty obviously useful and easy to implement piece of functionality.
If not, can anyone suggest how I might do this with the IDE scripting console or a custom Intention (I've had no success finding really good usable documentation/examples of the IDE scripting console, haven't looked into intentions too much...)
How about the following 2 flows/options based on the Windows shortcuts (don't mind the failing stuff, it's just a quick copy-paste for the sake of brevity):
1) With the cursor placed on your class name:
CTRL+SHIFT+T (Chose test for launch)
SHIFT+END (Select all)
SHIFT+UP (Unselect Create new test...)
CTRL+SHFIT+F10 (Execute selected tests)
2) With the Group by test/production option selected in the find window and cursor placed on your class name:
ALT+F7 (Find usages)
chose the tests from the list
CTRL+SHFIT+F10 (Execute selected tests)
I've created a small (test) addition to the Dolphin Smalltalk framework
that I want to submit on GitHub later. (1 method: Integer>>isPrime)
But first, I want add my testing method of this method to the standard regression test set, with ~ 2400 tests now. (IntegerTest>>testIsPrime)
I've found the classes TestCase, DolphinTest, IntegerTest and the SUnit browser.
But I didn't find out how to add my test to the standard test set.
Can someone point me the right direction?
I assume you are working from a Git checkout and have the test classes in your image. From there the easiest thing is to modify an existing class (such as IntegerTest) in the code browser, save the package back to the file system, and then Git should show the files as modified.
The neat thing about SUnit is that by default it will include all methods that start with 'test' in the test suite. So just add the test, run the suite, and see the number of tests increase by one!
In my Mac Cocoa unit tests, I would like to output some files as part of the testing process, and delete them when the test is done, but only when there are no failures. How can this be done (and/or what's the cleanest way to do so)?
Your question made me curious so I looked into it!
I guess I would override the failWithException: method in the class SenTestCase (the class your tests run in inherits from this), and set a "keep output files" flag or something before calling the super's method.
Here's what SenTestCase.h says about that method:
/*"Failing a test, used by all macros"*/
- (void) failWithException:(NSException *) anException;
So, provided you only use the SenTest macros to test and/or fail (and chances are this is true in your case), that should cover any test failure.
I've never dug into the scripts for this, but it seems like you could customize how you call the script that actually runs your tests to do this. In Xcode 4, look at the last step in the Build Phases tab of your test target. Mine contains this:
# Run the unit tests in this test bundle.
"${SYSTEM_DEVELOPER_DIR}/Tools/RunUnitTests"
I haven't pored through the contents of this script or the many subscripts it pulls in on my machine, but presumably they call otest or some other test rig executable and the test results would be returned to that script. After a little time familiarizing yourself with those scripts you would likely be able to find a straightforward way to conditionally remove the output files based on the test results.
I have been developing a project which contains a TestLauncher class that'll read a given directory and for each file it contains, run it against my tool and yield the results.
So, when coding in Eclipse, it would show up one result for each test (as expected). Today I've been toying with Intellij, and I've decided to try to run and code a bit of this project in Intellij.
When trying to run the tests, though, it seems to be only showing up 2 results instead of the 100+ it should. Although I am sure it is running the full suite, it seems to be folding all the results of a given category in a single result. That means that if I have at least one failing test in each category, it shows up as a "failed test".
I guess this must not be a bug, but rather some configuration that I am not aware about and that is on by default in Intellij but not in Eclipse. Could anyone explain what might be going on?
Edit: I am using the latest Intellij (downloaded one of these days).
Thanks
What you're seeing is simply a difference in the way the Eclipse and IDEA plug-ins are implemented. I implemented the Eclipse plug-in to be pretty clever in its display, so it will show different things depending on various factors such as the presence of a toString() method in your test class or whether your test class implements org.testng.ITest.
I suggest you ask this question on the IDEA forums and if you don't get any response, feel free to email the testng-users list and I can put you in touch with the JetBrains engineer in charge of the TestNG plug-in.
The IntelliJ-IDEA TestNG Plugin has a filter symbol called "Hide Passed" above the output Test Results. You can toggle that to display all tests, including the passed ones.