I am using the Serenity BDD tool Version 1.2.1-rc.3 with JUnit 4 to automate some regression testing. The project is working fine so far but I want to be able to rename the generated reports. At the moment they are called something like this:
62e91fe28f676d9f760e63756d4ba1d2.html
I want them to be named after the testcase creating them. I have not seen anything about this in the Serenity Reference Manual nor elsewhere I could think of. I even added serenity.compress.filenames=false to serenity.properties and it doesn't work.
Any help would be appreciate
Setting serenity.compress.filenames=false should work. However I would advise against doing this, except in debugging mode, if you are running any builds on Windows. The generated filenames will quickly exceed the maximum allowed Windows path length of 240 chars.
Related
I know you can easily do this with Selenium IDE.
But I would like to be able to export the test script programatically. That is to convert this
to this
by doing something like:
seleniumIDE_API.export(htmlFilePath, outputPath, OutputTypes.CSHARP_NUNIT_WEBDRIVER);
The reason I want this is to reduce the number of steps that has to be taken when creating tests. So all you have to do is create the test and drop the html file into some directory.
I don't think this is recommended to spend effort for. From Firefox 55 onwards, Selenium IDE will no longer work.
Try to use different recorders for this purpose. Cognizant QA solution Cognizant Intelligent Test Scripter does the same thing. It records and replays the action. I don't think a export would be possible, but surely you can by tweaking the code.
I mean, it would be very useful if I can see how many tests passed/failed just by one line, without reading build logs.
I use karma as test runner. It have a lot of reporter, but which one should I use?
Example from TeamCity:
This seems like a useful feature but the current user interface doesn't seem to support it.
You can file it as a feature request on Travis CI's GitHub page using the link below:
https://github.com/travis-ci/travis-ci/issues
Although Travis CI doesn't have its own interface for counting the number of tests passed, they do work with CodeClimate, which has it's it's interface and metrics for test coverage. It shows overall test coverage for the whole project and coverage for each file. There's some more info on that here, though it looks like their free version allows local testing only.
There are other tools out there for tracking and analyzing coverage as well, including Coveralls, which is pretty good as well. They have a free version for open source, like Travis CI, so that's can be a plus. They also show coverage as a percent and file-by-file.
My project requirements are
1.The framework must produce detailed Step Reports - which can be sent to the client through email.
2.The execution time must be less
3.Easy to write
I know behat and Cucumber
Please suggest me which framework is good ??
I would say Behat+Mink+Selenium combination. I've been using for very long time.
Behat will give you report as you wanted. We always send reports to clients where every single line is printed and either marked as success or failure. At the end of it, you get a full result where you can see overall report.
e.g. bin/behat #YourBundleName -f pretty,html --out ,report-path/behat.html. You can even get screen-shots of failed steps.
Every program can be considered as fast or slow. Result will depend on how you do things. You have a lot of options to make behat tests run fast. e.g. if you use phantomJs to run the tests and symfony2 as default session.
Behat uses Gherkin language which is easy to understand and write. You don't have to be a programmer at all.
One framework known for its pretty reports is Concordion. Please, have a look at the example to view one such report: http://concordion.org/Example.html
The Java version of Concordion utilizes JUnit to execute its tests. So you get a good integration in your development environment. Concordion support multiple technologies such as .NET, Ruby, Python, etc. http://concordion.org/Ports.html
Which technology are you using?
Concordion based on specification by example has been designed with a short learning-curve as a top priority. The purposely small command-set is simple to learn: http://concordion.org/Tutorial.html
I have been developing a project which contains a TestLauncher class that'll read a given directory and for each file it contains, run it against my tool and yield the results.
So, when coding in Eclipse, it would show up one result for each test (as expected). Today I've been toying with Intellij, and I've decided to try to run and code a bit of this project in Intellij.
When trying to run the tests, though, it seems to be only showing up 2 results instead of the 100+ it should. Although I am sure it is running the full suite, it seems to be folding all the results of a given category in a single result. That means that if I have at least one failing test in each category, it shows up as a "failed test".
I guess this must not be a bug, but rather some configuration that I am not aware about and that is on by default in Intellij but not in Eclipse. Could anyone explain what might be going on?
Edit: I am using the latest Intellij (downloaded one of these days).
Thanks
What you're seeing is simply a difference in the way the Eclipse and IDEA plug-ins are implemented. I implemented the Eclipse plug-in to be pretty clever in its display, so it will show different things depending on various factors such as the presence of a toString() method in your test class or whether your test class implements org.testng.ITest.
I suggest you ask this question on the IDEA forums and if you don't get any response, feel free to email the testng-users list and I can put you in touch with the JetBrains engineer in charge of the TestNG plug-in.
The IntelliJ-IDEA TestNG Plugin has a filter symbol called "Hide Passed" above the output Test Results. You can toggle that to display all tests, including the passed ones.
Looking for a way to get a visual report about:
overall test success percentage over time (information about if and how quickly tests are going greener)
visualised single test results over time (to easily notice test gone red that has been green for long time or vice versa to pay attention to a test that has just gone green)
any other visual statistics that would benefit testers and the project as a whole
Basically a tool that would generate results from the whole test results directory not just off the single (daily) run.
Generally it seems it could be done using XSLT, but it doesn't seem to have much flexibility to work with multiple files at the same time.
Does such a tool exist already?
I feel fairly courageous to claim that most Continuous Integration Engines such as Hudson (for Java) provide such capability either natively or through plugins. In Hudson's case there's a few code coverage plugins available already and I think it does basic graphs from unit tests automatically by itself.
Oh and remember to configure the CI properly, for example our Hudson polls CVS every 10 minutes and if it sees any changes, it does all the associated tricks (get updated .java files, compile, run tests, verify dependencies etc.) to see if the build is still OK or not.
Hudson will do this and it will work with Nunit (here), Junit (natively), and MSTest.exe tests using the steps I outline here. It does all that you require and more. Even if you want it to ONLY run tests and give you feedback on those, it can.
There's such new report supporting NUnit \ JUnit called Allure. To retrieve information from NUnit you need to use NUnit adapter, for JUnit - read the following wiki page. You can use it with Jenkins via respective plugin.