I'd like to use Cucumber linked with SoapUI and Selenium in order to create an automation functional test.
I only found one website about it, describing that first I have to create a SoapUI project and save it as a .xml file. Then I should include this file into the test project using Cucumber and Selenium.
My first question is what configuration do I have to do for that ? (in the pom.xml file when using Maven for example or in any configuration file for Jenkins)
My second and last question is : if I launch every night with Jenkins (let's say that as an example) my tests, do I have to launch first the SoapUI project and import it again in the test project to run my tests in a good way ?
Thanks for your answers
I think you can use REST Assured for you API testing.
Use cucumber for your BDD statement, use java for there step definition and REST Assured for all the API request stuff.
Wrap all your automation with well written Gherkin and you'll need no other documentation. Think about that. BDD (specification by example) is a requirements approach, not a testing approach.
Related
We are building a shop for a customer on Shopware 6.3.5.2 and want to use tests to
ensure that core functionality is not broken by our customizations (static plugins)
write new tests for new functionality
There is Running End-to-End Tests but this seems to be for core development and uses psh.phar which is not available in the production template.
How should this be done?
edit
This question is meant a bit broader and concerns also Unit Tests.
Actually, you can use the E2E tests of the platform project - as Cypress itself doesn't care where to run the test against. However, as you already noticed you cannot use psh commands to run them. You may run the tests though the basic Cypress commands, setting your shop's url as baseUrl of the tests, for example via this command:
./node_modules/.bin/cypress run --config baseUrl="<your-url>"
It works with cypress open as well.
The only thing what may become troublesome is the setToInitialState command in most of the tests which takes care about the clean up of shopware's database using psh scripts, unfortunately. You may need to adjust it by overriding the command in order to reset the database of the Production template.
I hope I was able to help a bit. 🙏
There are actually two parts here:
ensure that core functionality is not broken by our customizations (static plugins)
write new tests for new functionality
re 1: For regression tests like this I would suggest end-to-end tests. Either test through the UI with tools like selenium or through the HTTP API (I don't know if the shopware API is sufficient for extensive regression tests).
re 2: Since plugins do not run on their own I would extract all relevant functionality into plain old PHP classes that are independent of shopware and test those in isolation. Explore if some of that functionality can be made visible through an API and test the plugin integration through this. Depending on the actual plugin you might have to resort to UI tests again.
Is there any good way to get Karate tests to test a Spring-Boot microservice in the "integration-test" phase of a Maven build? What I mean is: is there an anti-pattern/workaround that works well OR am I out of luck because this was an afterthought of the Karate development?
Facts I have gathered so far:
It says here: "the surefire plugin is not hardcoded into Karate"
Is there a way to run Karate test during maven's integration test phase?
Running Karate tests in context of maven-failsafe-plugin does not work at all. Easy to reproduce on my test project url below. Two problems foremost:
When running from maven-failsafe, Karate will generate .txt and .html files but does not generate .json Cucumber output files.
Therefore, you can get no HTML report from the cucumber-reporting plugin (which I require)
I created a project to demonstrate these facts: https://github.com/djangofan/karate-api-sample
Running integration tests requires that I start Spring Boot before the test suite and then stop the spring boot server afterwards. Maven SureFire does not have this capability BUT the maven failsafe plugin does: the maven build helper plugin has a pre-integration-test hook for it. Limited only by Maven surefire, I am out of luck.
I see an example in the karate/karate-demo project of starting the spring boot server from code. This is not easy to accomplish when I am trying to test a web service that is already established. Is this my only option: to use surefire with a class filter and code to bootstrap?
Given no good answer, I will just revert to using cucumber-java + resteasy, which I am pretty sure will work with maven failsafe: https://maven.apache.org/surefire/maven-failsafe-plugin/examples/cucumber.html
First, this is not Karate.js not sure where that came from :)
When running from maven-failsafe, Karate will generate .txt and .html files but does not generate .json Cucumber output files.
Sorry, cannot replicate.
For failsafe, just use the naming conventions and it will work, e.g. **/IT*.java
Maven SureFire does not have this capability BUT the maven failsafe plugin does:
So use failsafe as mentioned above. If you are not able to do that, please change your project to demonstrate the issue and we promise to fix it. Also refer these instructions: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
Given no good answer, I will just revert to using cucumber-java + resteasy
That is of course your choice. At least in my (biased :) opinion you will miss all the JSON assertion value that Karate adds, and the parallel execution + aggregation of reports. I am sure you will help as far as possible so that we can make the experience better for all who need this solution - but else, hey - no worries :P
This question already has an answer here:
Karate framework and TestNG
(1 answer)
Closed last year.
We have implemented cucumber with rest assured for API automation. Recently we came to know Karate giving advantages when compared with Rest assured. So I thought of recommend to my organization to use Karate, hence I have prepared sample to API automation scripts with Karate. Yes, I can execute very well with JUnit, but same testrunner file if I tried to execute with TestNG am unable to execute at all, displaying Test runs are zero after my execution.
Could you please help me is there any way to run Karate runner file with TestNG framwework. Really your answer going to make us decision.
I have gone through various posts which are related TestNG with Karate. But unluckily am unable to trigger my scripts.
Note: I have extended KarateRunner class and used #KarateOptions as well in my code.
TestNG is needed only for development mode. All teams finally need to do CI and here Karate does not need even JUnit, please refer this: https://github.com/intuit/karate#parallel-execution
Now for development mode - even if you insist on using TestNG, you can mix JUnit into the same project: https://stackoverflow.com/a/19928639/143475
Please give up trying to use TestNG for Karate, it is un-necessary and a waste of your time. Also note that we have IDE support which is what most developers will use. Even the JUnit support is not needed most of the time.
I want to automate SOAP UI test suite running and for that I heard that we can export test suite as a batch program. I searched over internet about it but I didn't get any results on the same. Is it possible to export it into .bat file? If yes then can someone suggest me a way to do it?
The tests can be automated using soapUI tool. For that one needs to create a project which consists one or more test suite(s). Of course, each test suite contains one or more test case(s). A test case contains one or more test step(s) of different types like soap request, rest request, http request, groovy script, jdbc etc.
SoapUI saves a project as an xml file.
From the summary of the question, it appears that there is confusion for you or mixing different things together i.e., automating tests and executing them, and note that both are different.
Also exporting test suite is not related in this context and doesn't have impact on either automating tests or executing. And not sure, what caused you this confusion.
Automating Tests:
Project can be created if you have wsdl / wadl files for soap / rest respectively. More details can be found here. Also this resource will help to get started with functional testing using SoapUI.
Exporting Test Suite:
One can export a test suite from SoapUI project, but that will be of xml file again. This is mainly useful if you want to reuse the existing tests or move it to a different soapui project altogether.
Executing the Automated Tests:
Ultimately the goal is to execute the automated tests against the application under test and create the test report.
It is possible to run the tests in the following ways:
From SoapUI - the tests suites can be executed from SoapUI tool. Find more details from here. This method is useful when some wants to try the newly automated tests and make sure all his assertions are going thru after building thier test suite in SoapUI.
From Commandline - this method can be approached when some one wants to run the tests as part of Continuous Integration or do not wish to invoke SoapUI or run the tests headlessly both on windows and unix platforms. For this SOAPUI_HOME/bin/testrunner.bat or .sh scripts can be used. For more details see here.
Hope this clarifies or helps.
I am setting up Selenium tests on Hudson and looking for an easy way to present nice results such as Project summary and build summary on Hudson.
Currently I have written Selenium test cases in Rspec and PHPUnit's extension for Selenium (I prefer working on Rspec over PHP).
What is the best way to present detailed report for Selenium tests on Hudson CI Server?
I prefer to use Rspec over PHPUnit. It would be very nice if I can use rspec and still integrate well with Hudson CI Server.
I would very much appreciate for detailed comments.
thank you so much for your help
Best regards
If you have the results as HTML files, you can use the HTML Publisher plugin.
Have a look at a similar question https://serverfault.com/questions/184805/how-to-view-test-results-in-hudson/185027#185027 This guy seams to have integrated the selenium tests into junit and therefore the results can be published using the xunit tests.
There are two selenium plugins, one should work for you.
This I have no idea what rspec is I entered rspec and Hudson into google. The second link was this one http://reprocessed.org/blog/easy_rails_ci_with_hudson It should contain everything you need.