Does anyone know how to use Gherkin to parse custom feature file?
I want to create a tool to process feature file and execute custom java code. I want Gherkin to process file similar to this:
SCENARIO: My Fist sample test
WHEN [this condition met]
USE [this dataset]
THEN [test this java code]
You also can refer one of the gherkin parser GherkinFileParser.java used in qaf. From the unit test below is the usage:
GherkinFileParser parser = new GherkinFileParser();
List<Scenario> scenarios = new ArrayList<Scenario>();
parser.parse("resources/features/gherkin.feature", scenarios);
for (Scenario scenario : scenarios) {
//...
}
Gherkin is designed for storing descriptions of business behaviour. Its not a tool for writing scripts or doing programming. If you feel you want to program in gherkin you should consider either
1) Moving the programming up outside of Cucumber, so you run multiple instances of Cucumber
2) Moving the programming DOWN into the step definitions and helper methods.
You would have to do some hardcore programming and basically write a new language (it wouldn't be Gherkin anymore) to do what you've described.
Related
I am referring to John's Serenity bdd book # https://serenity-bdd.github.io/theserenitybook/latest/step-libraries.html. Very in the beginning, John seem to be enclosing a set of #given #when #then step definitions inside a method, which is annotated with a tag #Test.
#Test
public void members_should_start_with_Bronze_status() {
// GIVEN
tracy.joins_the_frequent_flyer_program();
// THEN
tracy.should_have_a_status_of(Bronze);
}
1.How would you choose the name of the method ? From the step line describing Scenario:keyword?
2.Would this be possible only when you use #RunWith(SerenityRunner.class) annotation just before declaration of StepDefinition class?
3.How about if you were using #RunWith(CucumberWithSerenity.class) instead?
What is the best place of declaring this annotation ? In the TestRunner.java file or as you have shown in side the stepdefinition class file ?
4.What is the role of serenity.properties at the root of the project Vs. TestRunner.java file ? Can we use exclusively one instead of both ?
SORRY FOR THESE MANY QUESTIONS. On behalf of JUnit-Cucumber-Serenity Test Developers,
I am saying this
"We are struggling, not having a definitive documentation".
I know it is in the nature of the beast. Because Serenity can be used in conjunction with all the above
technologies and a single, simple answer would be difficult.
BTW, I looked at the John's starter project for Serenity-junit-starter-master project(Serenity-junit-starter) which to most extent illustrates his book, I could not locate the feature file anywhere. Connecting dots between feature file and the Steps document would have given much insight.
“// GIVEN” etc are just comments; they are unrelated to Cucumber. You can run Serenity with JUnit, Cucumber or JBehave, but most of the book (currently) focuses on pure JUnit implementations.
I am new to "KDB DATA testing" & I would like to build (Test scenario based) scripts using q programming language. Each test scenario is associated to each .q file. Is there any existing framework / tool which I can use to run these .q files & generate the report for the same ?
Please let me know, if you have any relevant information on this.
qStudio from Timestored also supports unit testing, though the output is not graphical. See here: qUnit
Simon Garland's k4unit may be similar to what you're looking for:
https://github.com/simongarland/k4unit
Alternatively, Daniel Nugent has another unit testing framework:
https://github.com/nugend/qspec
Kx Systems also maintains the following list of Git repos, which is generally the first place to look if you're trying to find something along these lines:
https://kxsystems.github.io/
In modern programming workflow numerous testing frameworks are used at once. For example, in PHP world, it is de-facto standard way to use unit tests, integration tests and functional/acceptance tests at once. Most of the time different frameworks are used for different test type. I am using combination of PHPSpec for unit, PHPunit for integration and CodeCeption for functional tests.
Is it possible to aggregate code coverage results that each of these frameworks return? Is there any tool that aggregates code coverage reports from different frameworks?
Or it is only possible to view individual results for each framework while they are incorrect because each code coverage report doesn't take into account other tests.
It is actually quite simple to perfrom this task. All your frameworks rely on the same library to generate the code coverage.
As you can see the generator in sebastianbergmann/php-code-coverage already supports a merge function (line 335) to merge different aggregates. Since you are part of a team using tests I assume it will be easy for you to change the test execution layer slightly to gather the code coverage in a single php process and just merge em.
There is a tool for this: phpcov. It allows to merge many coverage files with merge option:
$ parallel --gnu ::: \
'phpunit --coverage-php /tmp/coverage/FooTest.cov tests/FooTest' \
'phpunit --coverage-php /tmp/coverage/BarTest.cov tests/BarTest'
$ phpcov merge /tmp/coverage --clover /tmp/clover.xml
phpcov 2.0.0 by Sebastian Bergmann.
Generating code coverage report in Clover XML format ... done
I think we are on same boat. How we can tell how much we have converge using this all different testing tool. We start discuss with team and decide to go for
SonarSource. - For PHP Plugin and Live demo
PHP Report Stlyle - I advice you to visit live demo. It will help more.
It is very robust tool. It give us all inside of code.
The PHP Test Coverage Tool from Semantic Designs (my company) collects and combines test coverage from any
framework
test set
individual test
even ad hoc manual tests.
After running some set of tests, our tool is can be easily triggered to dump test coverage vectors to a file; you need to modify the framework slightly to invoke
TCVDump();
when the framework completes, or you can invoke a TCVDDump() by touching an easily found, special web page added by the test coverage tool. Each such call produces a time-stamped or user-named file (e.g, after the framework or test set) so they are easily distinguished
The graphical test coverage display included as part of the tool will interactively select and merge small or large sets of such files to produce a coherent whole, both display and summary. It will also compare test coverage vectors to enable one to decide if coverage from one test set include/intersects another, etc.
The test coverage display component will also export text or XML/HTML summaries of the coverage results.
You can even run tests on different subsystems and combine them. This test coverage tool is part of larger family of tools for many languages other than PHP; tests run on a multilingual application system can also be combined to provide an overview of coverage for the multilingual application.
My project requirements are
1.The framework must produce detailed Step Reports - which can be sent to the client through email.
2.The execution time must be less
3.Easy to write
I know behat and Cucumber
Please suggest me which framework is good ??
I would say Behat+Mink+Selenium combination. I've been using for very long time.
Behat will give you report as you wanted. We always send reports to clients where every single line is printed and either marked as success or failure. At the end of it, you get a full result where you can see overall report.
e.g. bin/behat #YourBundleName -f pretty,html --out ,report-path/behat.html. You can even get screen-shots of failed steps.
Every program can be considered as fast or slow. Result will depend on how you do things. You have a lot of options to make behat tests run fast. e.g. if you use phantomJs to run the tests and symfony2 as default session.
Behat uses Gherkin language which is easy to understand and write. You don't have to be a programmer at all.
One framework known for its pretty reports is Concordion. Please, have a look at the example to view one such report: http://concordion.org/Example.html
The Java version of Concordion utilizes JUnit to execute its tests. So you get a good integration in your development environment. Concordion support multiple technologies such as .NET, Ruby, Python, etc. http://concordion.org/Ports.html
Which technology are you using?
Concordion based on specification by example has been designed with a short learning-curve as a top priority. The purposely small command-set is simple to learn: http://concordion.org/Tutorial.html
If so, how? I haven't come across the proper incantation yet.
If not, what's the best approach to unit-testing Scala code from the command line? (I'm a troglodyte; I use IDEs when I have to, but I prefer to play around using Emacs and command-line tools.)
Since compiled Scala is just Java bytecode (OK, with a lot more $ characters in class names), it would be exactly as for running JUnit 4 tests against Java code, i.e. from the command line by passing the test classes as arguments to org.junit.runner.JUnitCore. As JUnit 4 out of the box only has command line support, you don't even have to worry about suppressing GUI based test runners.
That said, the specific Scala test frameworks (ScalaTest, ScalaCheck) do provide a more idiomatic set approach to testing code written in this more functional language.
You may be interested in ScalaTest.
ScalaTest is a free, open-source
testing tool for Scala and Java
programmers. It is written in Scala,
and enables you to write tests in
Scala to test either Scala or Java
code. It is released under the Apache
2.0 open source license.
Because different developers take
different approaches to creating
software, no single approach to
testing is a good fit for everyone. In
light of this reality, ScalaTest is
designed to facilitate different
styles of testing.
See the Runner documentation for how to run tests from the command line.
The suggestion of ScalaTest -- or any of the other Scala-specific frameworks, for that matter, is very good. I'd like to point to something else, though.
SBT.
SBT is a build tool, like Ant, Maven or Make. One interesting aspect of it, which will matter to us, is that it is Scala-based. I don't mean it has special capabilities to handle Scala code or that it is written in Scala, though both these things are true. I mean it uses Scala code, instead of XML like Maven and Ant, as the configuration source.
That in itself is interesting. Just today I saw a wonderful example of separating test sources from program sources, which I post here just because its so cool.
// on this project we keep all sources, whether they be Scala or Java, and whether they be
// regular classes or test classes, in a single src tree.
override def mainScalaSourcePath = "src"
override def mainJavaSourcePath = "src"
override def testScalaSourcePath = "src"
override def testJavaSourcePath = "src"
override def mainResourcesPath = "resources"
// distinguish main sources from test sources
def testSourceFilter =
"Test*.scala" | "Test*.java" |
"AbstractTest*.scala" | "AbstractTest*.java" |
"ScalaTestRunner.scala"
def mainSourceFilter = ("*.scala" | "*.java") - testSourceFilter
override def mainSources = descendents(mainSourceRoots, mainSourceFilter)
override def testSources = descendents(testSourceRoots, testSourceFilter)
But what makes it even more interesting is that SBT works like a console. You run "sbt", and you get dropped into a console-like interface, from which you can type commands like, for instance, "test", and have your tests run.