Coverage is a plugin for IntellijIDEA (going back many releases). It captures code coverage statistics for a given run configuration.
According to the documentation we should be able to append the results for multiple runs either by selecting it as the default behavior or by having Intellij prompt for the settings before applying coverage to the editor.
But the settings never seem to get applied. If I choose to be prompted; I'm never prompted. If I choose to append them; they're never appended. One member on my team says they are prompted but the results do not reflect their choice.
I've tried everything I can think of:
Manually changed settings for Coverage in my workspace.xml file
Deactivated and reactivated Coverage
Uninstalled and reinstalled Coverage
Tried using the other runners for Coverage (Emma and JaCoCo)
Even uninstalled and reinstalled Intellij with hopes I was carrying around faulty settings from a previous install.
Nothing works.
Am I missing something obvious; how do I configure Coverage to append coverage suites? I'm thinking it's a bug but is there perhaps some workaround possible?
There's a workaround but it doesn't involve appending the suites and it's a bit ugly.
I can't find a way to fix appending suites but coverage is applied to run configurations. So, what you can do if you have an existing suite you want to add to, you can add another run configuration and run with coverage to generate a suite for that run.
What you end up with is a number of suites you then have to merge; the merging functionality in Coverage works. Note that no coverage suites are appended; no new files are generated. It simply merges the results into the coverage view allowing a total report to be generated.
To view merged coverage data:
Press Ctrl+Alt+F6
Choose one or more coverage suites to merge
Click "Show selected"
A view of the merged suite data should appear in the editor.
Related
Is there any way to run inspections in real time in intellij, rather than just having to select Analyze > Inspect Code. In particular, I'd like to do real time codenarc analysis (similar to how Eclipse does with checkstyle) so when I save a file it tells me if there are violations. I downloaded the codenarc plugin, but it doesn't seem to support that explicitly.
If you go to settings -> inspections, you will see there is a bunch of CodeNarc entries that you can tick.
I recommend creating a separate inspection profile if you use CodeNarc in a Continuous Integration tool, so you can focus on violations that would break the build.
Now it is true that you have to analyze the code explicitly the first time, but then it happens on the fly, and by right clicking you can also disable a given inspection or re-run it on other parts of your code.
In my daily work, I intend to write tests for the code that I change.
My workflow usually is like this:
Find code to change
Write failing test
Change code
Watch test pass
And before commit, I run all tests.
But in reality, I could change any lines unknowingly about our test coverage.
What I do now is usually to see if the CI server will show increasing test coverage. But this is a manual task which I have to do after my commit, and it also may be cluttered by others committing code.
How can IntelliJ help me? I cannot directly find a way to compare test coverage. How can I ensure my changes drive test coverage up?
How do you currently handle this aspect of your code coverage?
IntelliJ IDEA includes the code coverage support since the 6.0 version (Ultimate edition).
Through Run -> Edit Configurations -> Code Coverage you can configure the code coverage you want, then you can issue the "Run tests with coverage" command. You'll have as a result some code coverage data on the project view. When you open a source file you can also see what lines are fully covered, partially covered, or not covered at all.
You might want to customize the Colors & Fonts through File -> Settings -> Editor -> Colors & Fonts -> General modifying the following entries:
Full line coverage
Partial line coverage
Uncovered line
For example I wanted to add more evidency to the coverage, so I configured a different background for those lines.
Once you have some red lines (not covered), you can try to cover them and if you see them becoming green it means you did a good job, and so on!
I have been developing a project which contains a TestLauncher class that'll read a given directory and for each file it contains, run it against my tool and yield the results.
So, when coding in Eclipse, it would show up one result for each test (as expected). Today I've been toying with Intellij, and I've decided to try to run and code a bit of this project in Intellij.
When trying to run the tests, though, it seems to be only showing up 2 results instead of the 100+ it should. Although I am sure it is running the full suite, it seems to be folding all the results of a given category in a single result. That means that if I have at least one failing test in each category, it shows up as a "failed test".
I guess this must not be a bug, but rather some configuration that I am not aware about and that is on by default in Intellij but not in Eclipse. Could anyone explain what might be going on?
Edit: I am using the latest Intellij (downloaded one of these days).
Thanks
What you're seeing is simply a difference in the way the Eclipse and IDEA plug-ins are implemented. I implemented the Eclipse plug-in to be pretty clever in its display, so it will show different things depending on various factors such as the presence of a toString() method in your test class or whether your test class implements org.testng.ITest.
I suggest you ask this question on the IDEA forums and if you don't get any response, feel free to email the testng-users list and I can put you in touch with the JetBrains engineer in charge of the TestNG plug-in.
The IntelliJ-IDEA TestNG Plugin has a filter symbol called "Hide Passed" above the output Test Results. You can toggle that to display all tests, including the passed ones.
I have a suite of unit tests that I use before checking in my project. However, very often it's the case that only one of them finds some regression in the code. In these cases I'd like to only run that particular unit test while debugging the failure. I haven't found any way to do this in Xcode. Is it possible?
If you're happy restricting your testing to a single test class, a simple option is to create a second test target (duplicate the existing target, change the product name and remove the contents of the "Compile Sources" build phase, if you wish) and add only the test source file you're trying to fix to it.
Alternatively, you can use the "Other Test Flags" option to pass a -SenTest argument to otest, the test runner:
% /Developer/Tools/otest
2009-08-29 22:28:39.555 otest[70089:10b] Usage: otest [-SenTest Self | All | None |
<TestCaseClassName/testMethodName>] <path of unit to be tested>
More information about using this method is here.
Thanks for that push in the right direction. I ended using the same basic concept, but I added a GUI that lets you select what gets run as well as get a nice red/green status for each test. If anyone is interested, the code is at the URL below. The UI needs to more spit and polish, but it seems to be working.
http://github.com/nall/XcodeUnitTestGUI/tree/master
After I started the project above, I found this project which is really fantastic.
http://github.com/gabriel/gh-unit
For new readers: A much better way now available in Xcode is to edit the scheme for the target to be tested and select "Test" in the left hand column of the scheme pane.
Use the widgets in the Tests column to expand targets and suites.
You can disable/enable tests on a per test target, per suite or per test basis using the check boxes on the right
Looking for a way to get a visual report about:
overall test success percentage over time (information about if and how quickly tests are going greener)
visualised single test results over time (to easily notice test gone red that has been green for long time or vice versa to pay attention to a test that has just gone green)
any other visual statistics that would benefit testers and the project as a whole
Basically a tool that would generate results from the whole test results directory not just off the single (daily) run.
Generally it seems it could be done using XSLT, but it doesn't seem to have much flexibility to work with multiple files at the same time.
Does such a tool exist already?
I feel fairly courageous to claim that most Continuous Integration Engines such as Hudson (for Java) provide such capability either natively or through plugins. In Hudson's case there's a few code coverage plugins available already and I think it does basic graphs from unit tests automatically by itself.
Oh and remember to configure the CI properly, for example our Hudson polls CVS every 10 minutes and if it sees any changes, it does all the associated tricks (get updated .java files, compile, run tests, verify dependencies etc.) to see if the build is still OK or not.
Hudson will do this and it will work with Nunit (here), Junit (natively), and MSTest.exe tests using the steps I outline here. It does all that you require and more. Even if you want it to ONLY run tests and give you feedback on those, it can.
There's such new report supporting NUnit \ JUnit called Allure. To retrieve information from NUnit you need to use NUnit adapter, for JUnit - read the following wiki page. You can use it with Jenkins via respective plugin.