I would like to run Gatling tests (like provided in this repo) using the all-in-one jar file of Karate. I am stuck with how to do that.
Would you be able to give me a hint on how to get started here?
Gatling is not supported using the JAR. You have to have a Maven project. Here is an example: https://github.com/ptrthomas/karate-gatling-demo
Related
Is there any good way to get Karate tests to test a Spring-Boot microservice in the "integration-test" phase of a Maven build? What I mean is: is there an anti-pattern/workaround that works well OR am I out of luck because this was an afterthought of the Karate development?
Facts I have gathered so far:
It says here: "the surefire plugin is not hardcoded into Karate"
Is there a way to run Karate test during maven's integration test phase?
Running Karate tests in context of maven-failsafe-plugin does not work at all. Easy to reproduce on my test project url below. Two problems foremost:
When running from maven-failsafe, Karate will generate .txt and .html files but does not generate .json Cucumber output files.
Therefore, you can get no HTML report from the cucumber-reporting plugin (which I require)
I created a project to demonstrate these facts: https://github.com/djangofan/karate-api-sample
Running integration tests requires that I start Spring Boot before the test suite and then stop the spring boot server afterwards. Maven SureFire does not have this capability BUT the maven failsafe plugin does: the maven build helper plugin has a pre-integration-test hook for it. Limited only by Maven surefire, I am out of luck.
I see an example in the karate/karate-demo project of starting the spring boot server from code. This is not easy to accomplish when I am trying to test a web service that is already established. Is this my only option: to use surefire with a class filter and code to bootstrap?
Given no good answer, I will just revert to using cucumber-java + resteasy, which I am pretty sure will work with maven failsafe: https://maven.apache.org/surefire/maven-failsafe-plugin/examples/cucumber.html
First, this is not Karate.js not sure where that came from :)
When running from maven-failsafe, Karate will generate .txt and .html files but does not generate .json Cucumber output files.
Sorry, cannot replicate.
For failsafe, just use the naming conventions and it will work, e.g. **/IT*.java
Maven SureFire does not have this capability BUT the maven failsafe plugin does:
So use failsafe as mentioned above. If you are not able to do that, please change your project to demonstrate the issue and we promise to fix it. Also refer these instructions: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
Given no good answer, I will just revert to using cucumber-java + resteasy
That is of course your choice. At least in my (biased :) opinion you will miss all the JSON assertion value that Karate adds, and the parallel execution + aggregation of reports. I am sure you will help as far as possible so that we can make the experience better for all who need this solution - but else, hey - no worries :P
I'm a QA and I want to use Spock + Geb for my testing. As I understand I have to setup grails (or gradle, or something like that) to use Spock + Geb. Or Grails is not required? In this case what is the minimal set of programs that I need to install to successfully test any application? Where I need to write test and how to run it?
Sorry for such stupid questions, but all this stuff is new for me and I don't really understand how it works.
The Grails web framework isn't required at all, unless the web app you are testing is written in Grails.
The Gradle build tool isn't necessarily required, but then you don't want to manually download Spock and Geb, manually compile the tests, manually run the tests, or manually set up an IDE. A build tool can automate all of this, and if you aren't using one already, Gradle is the easiest choice.
The Geb Manual explains how to set up a Gradle build for Spock+Geb and links to a fully working example project. Cloning this project (or downloading the zip) will get you started quickly. You don't even have to install Gradle, but can just run it via the gradlew (*nix) or gradlew.bat (Windows) script (for example gradlew test).
I did a patch in the HtmlUnitDriver.java file, as explained here:
Is it possible to ignore JavaScript exceptions when working with WebDriver (HtmlUnit, Ruby bindings)
I've followed the explanations on the selenium build wiki and in the readme, also the tip given here:
Building the Selenium Server Project
Still, I don't have a jar file like selenium-server-standalone-2.25.0.jar which can be run and used as a headless browser for selenium tests.
... So I'm trying to recreate the selenium-server-standalone-2.xx.jar file with my patch, and use it to run my headless tests.
Can anyone explain to me how to do this? Thanks!
The best way I found is to do:
go //java/server/src/org/openqa/selenium/remote/server:server:uber //java/client/src/org/openqa/selenium:client-combined:uber
This will create a single jar for the selenium server
Then run go release.
The resulting .jar should be found in build\dist folder of the selenium source folder. (ex. ./build/java/server/src/org/openqa/grid/selenium/selenium-standalone.jar)
See also https://github.com/SeleniumHQ/selenium/wiki/Building-WebDriver#tips.
Here is a step by step building guide for selenium standalone server. The build process of selenium projects may be not so strait forward to newbies, so I recommend this detailed guide on how to create a runnable standalone jar.
http://shengwangi.blogspot.com/2014/08/how-to-build-selenium-from-source.html
Also you can refer to the official document on build selenium.
https://code.google.com/p/selenium/wiki/BuildingWebDriver#Building_Selenium_Server
I have setup Bamboo to run JBehave tests on a remote agent (with JBehave-web plugin launching test using webdriver), and everything runs fine. Only problem is after the execution is finished Bamboo shows no test executed. I can see the option in Bamboo to select the output of the test results, but it has to be a JUnit xml, and Jbehave reports are only generated in plain text or html.
Any idea how to solve this?
Thanks
I ran in the same situation about a year ago. JBehave "doesn't" integrate with Bamboo out of the box. Although, they have a plugin for Hudson CI.
In my case, such as yours, I resorted in running the tests through the Surefire plugin; the outputs are considered as JUnit tests results and Bamboo can recognize them.
Hope it helps.
There is a really simple way to do this. And I'm currently doing this for our build system.
Write a simple parse script that transforms your plain of html report into JUnit compatible results. And add that script as a task in your Bamboo task, then use Junit parser to parse the results. Boo! You're done! Plus, you've got the capability to quarantine!
This is way much faster than writing a plugin for Bamboo, which involves considerable more time to learn/write.
Setup JBehave with Maven.In Bamboo build plan use Maven task to run it. For getting results in Bamboo use JBehave Task for Bamboo. It will convert JBehave scenarios in tests in Bamboo. If scenario names contains JIRA issue ids, it will link them to JIRA issues.
https://marketplace.atlassian.com/plugins/com.mdb.plugins.jebehaveforbamboo/server/overview
Sample JBehave as Maven Project
https://bitbucket.org/vikasborse/jbehavesampleproject/overview
Download or clone this repository on your local machine.
To run navigate to this project in command line and use command:
"mvn integration-test"
I have a project 'ABC' with the main code and junit tests. I do have the requirement that i can execute the set unit tests against a older version of the product artefacts.
To solve this requirement i would create a maven project which only contains the junit tests.
Another maven product builds my product code and places the artifact into the repository.
Now i could launch my tests against any product build by changing the build dependency within the junit test project.
Is this a good solution? Are there perhaps better solutions to solve this requirement?
I think thats a pretty good approach. You could create a profile for each old version and then activate them via the profile name and test different old versions without having to change the pom file for each run. You could then also run the different profiles separately scheduled on a continuous integration server...