I am new to "KDB DATA testing" & I would like to build (Test scenario based) scripts using q programming language. Each test scenario is associated to each .q file. Is there any existing framework / tool which I can use to run these .q files & generate the report for the same ?
Please let me know, if you have any relevant information on this.
qStudio from Timestored also supports unit testing, though the output is not graphical. See here: qUnit
Simon Garland's k4unit may be similar to what you're looking for:
https://github.com/simongarland/k4unit
Alternatively, Daniel Nugent has another unit testing framework:
https://github.com/nugend/qspec
Kx Systems also maintains the following list of Git repos, which is generally the first place to look if you're trying to find something along these lines:
https://kxsystems.github.io/
Related
NUnit (and the like) has method attributes which allow tests to be run multiple times with different arrange values. Is something similar possible with SpecFlow?
What I am aiming for is a way to run the same scenario tests in a feature file with as many browser drivers as I can, in one test run.
You can use scenario outlines. In example of scenario outline you can mention driver name and you code logic should take action according to driver. Please see more details about scenario ouyline below
https://github.com/cucumber/cucumber/wiki/Scenario-outlines
Examples are one solution, but in your case a little cumbersome, as you have to specify them at every scenario.
In your case, please have a look at the targets feature of the SpecFlow+Runner. With that you can "multiply" your scenarios for different configurations. If you put the web driver that should be used in this configuration, you can test as many webdriver as you want.
Have a look at this example: https://github.com/techtalk/SpecFlow.Plus.Examples/tree/master/SeleniumWebTest
Full Disclosure: I am one of the developers of SpecFlow & SpecFlow+
Use scenario outlines and this tool if you want to use browsers as tags:
https://github.com/unickq/Unickq.SeleniumHelper
I mean, it would be very useful if I can see how many tests passed/failed just by one line, without reading build logs.
I use karma as test runner. It have a lot of reporter, but which one should I use?
Example from TeamCity:
This seems like a useful feature but the current user interface doesn't seem to support it.
You can file it as a feature request on Travis CI's GitHub page using the link below:
https://github.com/travis-ci/travis-ci/issues
Although Travis CI doesn't have its own interface for counting the number of tests passed, they do work with CodeClimate, which has it's it's interface and metrics for test coverage. It shows overall test coverage for the whole project and coverage for each file. There's some more info on that here, though it looks like their free version allows local testing only.
There are other tools out there for tracking and analyzing coverage as well, including Coveralls, which is pretty good as well. They have a free version for open source, like Travis CI, so that's can be a plus. They also show coverage as a percent and file-by-file.
My project requirements are
1.The framework must produce detailed Step Reports - which can be sent to the client through email.
2.The execution time must be less
3.Easy to write
I know behat and Cucumber
Please suggest me which framework is good ??
I would say Behat+Mink+Selenium combination. I've been using for very long time.
Behat will give you report as you wanted. We always send reports to clients where every single line is printed and either marked as success or failure. At the end of it, you get a full result where you can see overall report.
e.g. bin/behat #YourBundleName -f pretty,html --out ,report-path/behat.html. You can even get screen-shots of failed steps.
Every program can be considered as fast or slow. Result will depend on how you do things. You have a lot of options to make behat tests run fast. e.g. if you use phantomJs to run the tests and symfony2 as default session.
Behat uses Gherkin language which is easy to understand and write. You don't have to be a programmer at all.
One framework known for its pretty reports is Concordion. Please, have a look at the example to view one such report: http://concordion.org/Example.html
The Java version of Concordion utilizes JUnit to execute its tests. So you get a good integration in your development environment. Concordion support multiple technologies such as .NET, Ruby, Python, etc. http://concordion.org/Ports.html
Which technology are you using?
Concordion based on specification by example has been designed with a short learning-curve as a top priority. The purposely small command-set is simple to learn: http://concordion.org/Tutorial.html
I am creating a automation framework using cucumber+Java+Selenium. I have created customized report, need above said detail to print it in report.
Not sure this will answer your question, as it's too vague (as Bala says above), but Cucumber--JVM can output the test results in a JSON file, which can then be read & used in subsequent steps - see the Cucumber Reporting and Cucumber Performance plugins on Github for details & examples
Looking for a way to get a visual report about:
overall test success percentage over time (information about if and how quickly tests are going greener)
visualised single test results over time (to easily notice test gone red that has been green for long time or vice versa to pay attention to a test that has just gone green)
any other visual statistics that would benefit testers and the project as a whole
Basically a tool that would generate results from the whole test results directory not just off the single (daily) run.
Generally it seems it could be done using XSLT, but it doesn't seem to have much flexibility to work with multiple files at the same time.
Does such a tool exist already?
I feel fairly courageous to claim that most Continuous Integration Engines such as Hudson (for Java) provide such capability either natively or through plugins. In Hudson's case there's a few code coverage plugins available already and I think it does basic graphs from unit tests automatically by itself.
Oh and remember to configure the CI properly, for example our Hudson polls CVS every 10 minutes and if it sees any changes, it does all the associated tricks (get updated .java files, compile, run tests, verify dependencies etc.) to see if the build is still OK or not.
Hudson will do this and it will work with Nunit (here), Junit (natively), and MSTest.exe tests using the steps I outline here. It does all that you require and more. Even if you want it to ONLY run tests and give you feedback on those, it can.
There's such new report supporting NUnit \ JUnit called Allure. To retrieve information from NUnit you need to use NUnit adapter, for JUnit - read the following wiki page. You can use it with Jenkins via respective plugin.