This question already has an answer here:
Karate framework and TestNG
(1 answer)
Closed last year.
We have implemented cucumber with rest assured for API automation. Recently we came to know Karate giving advantages when compared with Rest assured. So I thought of recommend to my organization to use Karate, hence I have prepared sample to API automation scripts with Karate. Yes, I can execute very well with JUnit, but same testrunner file if I tried to execute with TestNG am unable to execute at all, displaying Test runs are zero after my execution.
Could you please help me is there any way to run Karate runner file with TestNG framwework. Really your answer going to make us decision.
I have gone through various posts which are related TestNG with Karate. But unluckily am unable to trigger my scripts.
Note: I have extended KarateRunner class and used #KarateOptions as well in my code.
TestNG is needed only for development mode. All teams finally need to do CI and here Karate does not need even JUnit, please refer this: https://github.com/intuit/karate#parallel-execution
Now for development mode - even if you insist on using TestNG, you can mix JUnit into the same project: https://stackoverflow.com/a/19928639/143475
Please give up trying to use TestNG for Karate, it is un-necessary and a waste of your time. Also note that we have IDE support which is what most developers will use. Even the JUnit support is not needed most of the time.
Related
Is there any good way to get Karate tests to test a Spring-Boot microservice in the "integration-test" phase of a Maven build? What I mean is: is there an anti-pattern/workaround that works well OR am I out of luck because this was an afterthought of the Karate development?
Facts I have gathered so far:
It says here: "the surefire plugin is not hardcoded into Karate"
Is there a way to run Karate test during maven's integration test phase?
Running Karate tests in context of maven-failsafe-plugin does not work at all. Easy to reproduce on my test project url below. Two problems foremost:
When running from maven-failsafe, Karate will generate .txt and .html files but does not generate .json Cucumber output files.
Therefore, you can get no HTML report from the cucumber-reporting plugin (which I require)
I created a project to demonstrate these facts: https://github.com/djangofan/karate-api-sample
Running integration tests requires that I start Spring Boot before the test suite and then stop the spring boot server afterwards. Maven SureFire does not have this capability BUT the maven failsafe plugin does: the maven build helper plugin has a pre-integration-test hook for it. Limited only by Maven surefire, I am out of luck.
I see an example in the karate/karate-demo project of starting the spring boot server from code. This is not easy to accomplish when I am trying to test a web service that is already established. Is this my only option: to use surefire with a class filter and code to bootstrap?
Given no good answer, I will just revert to using cucumber-java + resteasy, which I am pretty sure will work with maven failsafe: https://maven.apache.org/surefire/maven-failsafe-plugin/examples/cucumber.html
First, this is not Karate.js not sure where that came from :)
When running from maven-failsafe, Karate will generate .txt and .html files but does not generate .json Cucumber output files.
Sorry, cannot replicate.
For failsafe, just use the naming conventions and it will work, e.g. **/IT*.java
Maven SureFire does not have this capability BUT the maven failsafe plugin does:
So use failsafe as mentioned above. If you are not able to do that, please change your project to demonstrate the issue and we promise to fix it. Also refer these instructions: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
Given no good answer, I will just revert to using cucumber-java + resteasy
That is of course your choice. At least in my (biased :) opinion you will miss all the JSON assertion value that Karate adds, and the parallel execution + aggregation of reports. I am sure you will help as far as possible so that we can make the experience better for all who need this solution - but else, hey - no worries :P
I want to know, if we can use karate with JS based UI automation framework. And if yes, what are the requisite to follow. Do we require maven in that case?
We have well matured UI automation Selenium based framework. We want to know, if we can integrate Karate DSL without using maven.
You might if you use the CLI (command line interface) provided by the stand-alone JAR / executable. Please read the documentation here: https://github.com/intuit/karate/tree/master/karate-netty#standalone-jar
Which means if you can call an OS command from node / JS - you can use Karate. But there are no guarantees - and you can assume the answer to your question is NO.
That said - we plan to have our own competitor to Selenium - but it will take a little more time: https://github.com/intuit/karate/tree/master/karate-core
Is there a way to integration test my JUnit Jupiter extension? Of course I can test a good cases of extension usage, but I would like to test things like:
Does it fail tests as expected?
Are the correct reports written on test end?
Is there some built in support for this?
There is no explicit Jupiter extension testing framework -- but with version 1.4 there'll be the Platform Test Kit that you may use to execute the Jupiter engine and your extension in one-go and assert the outcome of a test run.
For details see: https://junit.org/junit5/docs/snapshot/user-guide/#testkit and in addition to that documentation, find usages of the Platform Test Kit within the "platform-tests" project of JUnit 5. For example at: https://github.com/junit-team/junit5/blob/master/platform-tests/src/test/java/org/junit/platform/testkit/engine/ExecutionsIntegrationTests.java
You should not perform integration testing for your own extension of Junit. The Junit 5 API is very, very stable and I bet that your extension will work the same for the entire life of Junit 5. Integration testing is useful in a constantly evolving environment, but Junit only ever adds features and never changes them (unless there is a bug).
If you perform integration testing for your own Junit extension, it's like you're testing Junit itself. Why would you test Junit instead of your extension? Let Junit be tested by Junit developers, unless you think you found a bug. But testing Junit for the sake of testing Junit is a waste of your own time, even if unconsciously you think you're only testing your own extension.
So that kind of tests should be done on the long term by getting your users to report issues to you: "Hey, AwesomeExtension works with Junit 5.A.B but not with 5.X.Y", then find out why they say that and if the issue is not with your extension, then report with Junit.
However, make sure that you get your extension... unit-tested.
I'd like to use Cucumber linked with SoapUI and Selenium in order to create an automation functional test.
I only found one website about it, describing that first I have to create a SoapUI project and save it as a .xml file. Then I should include this file into the test project using Cucumber and Selenium.
My first question is what configuration do I have to do for that ? (in the pom.xml file when using Maven for example or in any configuration file for Jenkins)
My second and last question is : if I launch every night with Jenkins (let's say that as an example) my tests, do I have to launch first the SoapUI project and import it again in the test project to run my tests in a good way ?
Thanks for your answers
I think you can use REST Assured for you API testing.
Use cucumber for your BDD statement, use java for there step definition and REST Assured for all the API request stuff.
Wrap all your automation with well written Gherkin and you'll need no other documentation. Think about that. BDD (specification by example) is a requirements approach, not a testing approach.
I am setting up Selenium tests on Hudson and looking for an easy way to present nice results such as Project summary and build summary on Hudson.
Currently I have written Selenium test cases in Rspec and PHPUnit's extension for Selenium (I prefer working on Rspec over PHP).
What is the best way to present detailed report for Selenium tests on Hudson CI Server?
I prefer to use Rspec over PHPUnit. It would be very nice if I can use rspec and still integrate well with Hudson CI Server.
I would very much appreciate for detailed comments.
thank you so much for your help
Best regards
If you have the results as HTML files, you can use the HTML Publisher plugin.
Have a look at a similar question https://serverfault.com/questions/184805/how-to-view-test-results-in-hudson/185027#185027 This guy seams to have integrated the selenium tests into junit and therefore the results can be published using the xunit tests.
There are two selenium plugins, one should work for you.
This I have no idea what rspec is I entered rspec and Hudson into google. The second link was this one http://reprocessed.org/blog/easy_rails_ci_with_hudson It should contain everything you need.