In my daily work, I intend to write tests for the code that I change.
My workflow usually is like this:
Find code to change
Write failing test
Change code
Watch test pass
And before commit, I run all tests.
But in reality, I could change any lines unknowingly about our test coverage.
What I do now is usually to see if the CI server will show increasing test coverage. But this is a manual task which I have to do after my commit, and it also may be cluttered by others committing code.
How can IntelliJ help me? I cannot directly find a way to compare test coverage. How can I ensure my changes drive test coverage up?
How do you currently handle this aspect of your code coverage?
IntelliJ IDEA includes the code coverage support since the 6.0 version (Ultimate edition).
Through Run -> Edit Configurations -> Code Coverage you can configure the code coverage you want, then you can issue the "Run tests with coverage" command. You'll have as a result some code coverage data on the project view. When you open a source file you can also see what lines are fully covered, partially covered, or not covered at all.
You might want to customize the Colors & Fonts through File -> Settings -> Editor -> Colors & Fonts -> General modifying the following entries:
Full line coverage
Partial line coverage
Uncovered line
For example I wanted to add more evidency to the coverage, so I configured a different background for those lines.
Once you have some red lines (not covered), you can try to cover them and if you see them becoming green it means you did a good job, and so on!
Related
Coverage is a plugin for IntellijIDEA (going back many releases). It captures code coverage statistics for a given run configuration.
According to the documentation we should be able to append the results for multiple runs either by selecting it as the default behavior or by having Intellij prompt for the settings before applying coverage to the editor.
But the settings never seem to get applied. If I choose to be prompted; I'm never prompted. If I choose to append them; they're never appended. One member on my team says they are prompted but the results do not reflect their choice.
I've tried everything I can think of:
Manually changed settings for Coverage in my workspace.xml file
Deactivated and reactivated Coverage
Uninstalled and reinstalled Coverage
Tried using the other runners for Coverage (Emma and JaCoCo)
Even uninstalled and reinstalled Intellij with hopes I was carrying around faulty settings from a previous install.
Nothing works.
Am I missing something obvious; how do I configure Coverage to append coverage suites? I'm thinking it's a bug but is there perhaps some workaround possible?
There's a workaround but it doesn't involve appending the suites and it's a bit ugly.
I can't find a way to fix appending suites but coverage is applied to run configurations. So, what you can do if you have an existing suite you want to add to, you can add another run configuration and run with coverage to generate a suite for that run.
What you end up with is a number of suites you then have to merge; the merging functionality in Coverage works. Note that no coverage suites are appended; no new files are generated. It simply merges the results into the coverage view allowing a total report to be generated.
To view merged coverage data:
Press Ctrl+Alt+F6
Choose one or more coverage suites to merge
Click "Show selected"
A view of the merged suite data should appear in the editor.
Is there any way to run inspections in real time in intellij, rather than just having to select Analyze > Inspect Code. In particular, I'd like to do real time codenarc analysis (similar to how Eclipse does with checkstyle) so when I save a file it tells me if there are violations. I downloaded the codenarc plugin, but it doesn't seem to support that explicitly.
If you go to settings -> inspections, you will see there is a bunch of CodeNarc entries that you can tick.
I recommend creating a separate inspection profile if you use CodeNarc in a Continuous Integration tool, so you can focus on violations that would break the build.
Now it is true that you have to analyze the code explicitly the first time, but then it happens on the fly, and by right clicking you can also disable a given inspection or re-run it on other parts of your code.
I just started working on an existing grails project where there is a lot of code written and not much is covered by tests. The project is using Hudson with the Cobertura plugin which is nice. As I'm going through things, I'm noticing that even though there are not specific test classes written for code, it is being covered. Is there any easy way to see what tests are covering the code? It would save me a bit of time if I was able to know that information.
Thanks
What you want to do is collect test coverage data per test. Then when some block of code isn't exercised by a test, you can trace it back to the test.
You need a test coverage tool which will do that; AFAIK, this is straightforward to organize. Just run one test and collect test coverage data.
However, most folks also want to know, what is the coverage of the application given all the tests? You could run the tests twice, once to get what-does-this-test-cover information, and then the whole batch to get what-does-the-batch-cover. Some tools ( ours included) will let you combine the coverage from the individual tests, to produce covverage for the set, so you don't have to run them twice.
Our tools have one nice extra: if you collect test-specific coverage, when you modify the code, the tool can tell which individual tests need to be re-run. You need a bit of straightforward scripting for this, to compare the results of the instrumentation data for the changed sources to the results for each test.
I have a suite of unit tests that I use before checking in my project. However, very often it's the case that only one of them finds some regression in the code. In these cases I'd like to only run that particular unit test while debugging the failure. I haven't found any way to do this in Xcode. Is it possible?
If you're happy restricting your testing to a single test class, a simple option is to create a second test target (duplicate the existing target, change the product name and remove the contents of the "Compile Sources" build phase, if you wish) and add only the test source file you're trying to fix to it.
Alternatively, you can use the "Other Test Flags" option to pass a -SenTest argument to otest, the test runner:
% /Developer/Tools/otest
2009-08-29 22:28:39.555 otest[70089:10b] Usage: otest [-SenTest Self | All | None |
<TestCaseClassName/testMethodName>] <path of unit to be tested>
More information about using this method is here.
Thanks for that push in the right direction. I ended using the same basic concept, but I added a GUI that lets you select what gets run as well as get a nice red/green status for each test. If anyone is interested, the code is at the URL below. The UI needs to more spit and polish, but it seems to be working.
http://github.com/nall/XcodeUnitTestGUI/tree/master
After I started the project above, I found this project which is really fantastic.
http://github.com/gabriel/gh-unit
For new readers: A much better way now available in Xcode is to edit the scheme for the target to be tested and select "Test" in the left hand column of the scheme pane.
Use the widgets in the Tests column to expand targets and suites.
You can disable/enable tests on a per test target, per suite or per test basis using the check boxes on the right
Looking for a way to get a visual report about:
overall test success percentage over time (information about if and how quickly tests are going greener)
visualised single test results over time (to easily notice test gone red that has been green for long time or vice versa to pay attention to a test that has just gone green)
any other visual statistics that would benefit testers and the project as a whole
Basically a tool that would generate results from the whole test results directory not just off the single (daily) run.
Generally it seems it could be done using XSLT, but it doesn't seem to have much flexibility to work with multiple files at the same time.
Does such a tool exist already?
I feel fairly courageous to claim that most Continuous Integration Engines such as Hudson (for Java) provide such capability either natively or through plugins. In Hudson's case there's a few code coverage plugins available already and I think it does basic graphs from unit tests automatically by itself.
Oh and remember to configure the CI properly, for example our Hudson polls CVS every 10 minutes and if it sees any changes, it does all the associated tricks (get updated .java files, compile, run tests, verify dependencies etc.) to see if the build is still OK or not.
Hudson will do this and it will work with Nunit (here), Junit (natively), and MSTest.exe tests using the steps I outline here. It does all that you require and more. Even if you want it to ONLY run tests and give you feedback on those, it can.
There's such new report supporting NUnit \ JUnit called Allure. To retrieve information from NUnit you need to use NUnit adapter, for JUnit - read the following wiki page. You can use it with Jenkins via respective plugin.