I want to automate SOAP UI test suite running and for that I heard that we can export test suite as a batch program. I searched over internet about it but I didn't get any results on the same. Is it possible to export it into .bat file? If yes then can someone suggest me a way to do it?
The tests can be automated using soapUI tool. For that one needs to create a project which consists one or more test suite(s). Of course, each test suite contains one or more test case(s). A test case contains one or more test step(s) of different types like soap request, rest request, http request, groovy script, jdbc etc.
SoapUI saves a project as an xml file.
From the summary of the question, it appears that there is confusion for you or mixing different things together i.e., automating tests and executing them, and note that both are different.
Also exporting test suite is not related in this context and doesn't have impact on either automating tests or executing. And not sure, what caused you this confusion.
Automating Tests:
Project can be created if you have wsdl / wadl files for soap / rest respectively. More details can be found here. Also this resource will help to get started with functional testing using SoapUI.
Exporting Test Suite:
One can export a test suite from SoapUI project, but that will be of xml file again. This is mainly useful if you want to reuse the existing tests or move it to a different soapui project altogether.
Executing the Automated Tests:
Ultimately the goal is to execute the automated tests against the application under test and create the test report.
It is possible to run the tests in the following ways:
From SoapUI - the tests suites can be executed from SoapUI tool. Find more details from here. This method is useful when some wants to try the newly automated tests and make sure all his assertions are going thru after building thier test suite in SoapUI.
From Commandline - this method can be approached when some one wants to run the tests as part of Continuous Integration or do not wish to invoke SoapUI or run the tests headlessly both on windows and unix platforms. For this SOAPUI_HOME/bin/testrunner.bat or .sh scripts can be used. For more details see here.
Hope this clarifies or helps.
Related
We are building a shop for a customer on Shopware 6.3.5.2 and want to use tests to
ensure that core functionality is not broken by our customizations (static plugins)
write new tests for new functionality
There is Running End-to-End Tests but this seems to be for core development and uses psh.phar which is not available in the production template.
How should this be done?
edit
This question is meant a bit broader and concerns also Unit Tests.
Actually, you can use the E2E tests of the platform project - as Cypress itself doesn't care where to run the test against. However, as you already noticed you cannot use psh commands to run them. You may run the tests though the basic Cypress commands, setting your shop's url as baseUrl of the tests, for example via this command:
./node_modules/.bin/cypress run --config baseUrl="<your-url>"
It works with cypress open as well.
The only thing what may become troublesome is the setToInitialState command in most of the tests which takes care about the clean up of shopware's database using psh scripts, unfortunately. You may need to adjust it by overriding the command in order to reset the database of the Production template.
I hope I was able to help a bit. 🙏
There are actually two parts here:
ensure that core functionality is not broken by our customizations (static plugins)
write new tests for new functionality
re 1: For regression tests like this I would suggest end-to-end tests. Either test through the UI with tools like selenium or through the HTTP API (I don't know if the shopware API is sufficient for extensive regression tests).
re 2: Since plugins do not run on their own I would extract all relevant functionality into plain old PHP classes that are independent of shopware and test those in isolation. Explore if some of that functionality can be made visible through an API and test the plugin integration through this. Depending on the actual plugin you might have to resort to UI tests again.
I'm trying to improve the testing process where I work, but without adjusting the structure.
What we have: VSTS, Selenium IDE, Testers who write test cases, but not code.
What I'd like to do is manage a way to marry our TFS continuous integration with the Selenium tests we write. These are NOT the code-driven selenium tests, but rather the IDE version where users click through, and set assertions using the IDE (All are just UI tests). I know we can export those tests plans as a .SIDE file, but what I can't figure out, is how to have our TFS server execute those as part of a deployment or build pipeline.
Ideally, developers/devops would setup projects in TFS from the onset with whatever solution makes sense to execute these Selenium .SIDE files, but afterwards, the testers would manage adding/modifying those tests cases elsewhere.
The real goal here is to not have testers writing code, or checking in code. Only writing these UI Selenium tests, but having TFS execute those as part of CI.
Researching this on the internet drives me basically always to something that requires testers to write code.
I don't think it can automate testing without code, at lease, you need a test project containing your automated tests.
Generally, in Azure DevOps, we use Visual Studio Test task to run tests. This task supports using the following tests:
Test assembly: Use this option to specify one or more test assemblies that contain your tests. You can optionally specify a
filter criteria to select only specific tests.
Test plan: Use this option to run tests from your test plan that have an automated test method associated with it. To learn more about
how to associate tests with a test case work item, see Associate
automated tests with test cases.
Test run: Use this option when you are setting up an environment to run tests from test plans. This option should not be used when
running tests in a continuous integration/continuous deployment
(CI/CD) pipeline.
This was a question that I had as well, and I think I found an imperfect but better solution.
I wasn't able to get my Selenium IDE tests running with Jenkins, but I was able to get them to run with TeamCity, another CI.
I created a build step like the following :
Runner type: Command Line
Working Directory: where the selenium IDE .side file is located
Run: Custom Script
With the build script content that I usually use to run my Selenium IDE Tests, such as selenium-side-runner sidefile.side
I also added the following so I could output the results in Junitor another form: --output-directory=results --output-format=junit
You can also add the following so the tests are run headlessly, this only works in Chrome : -c "goog:chromeOptions.args=[--headless,--nogpu] browserName=chrome"
Finally, I also use --filter to run one test suite at a time, but that is optional too.
I then used another build step to export the results to our test manger, xray, but I think that is beyond the scope of this question.
The problem with this solution is that it runs directly from a users individual machine still, but this can be work around.
I am working on one devops project, from selenium I am running test script and a log file is getting generated. How to configure jira to read the log file generated by selenium.. I want to go with API approch but unable to do so. Jenkins I am using as a CI tool here. Any suggestions ?
Hmm. Generally it's a better approach to display your test results in Jenkins instead of creating issues for them automatically. You didn't mention what technology you use (nodejs, java, ...), but typically you let your test runner generate a test results file that jenkins can interprete, so it will display the results nicely. There are various jenkins plugins that can help with that.
If you want to go a step further and still create issues automatically, you can script that in a separate step of your jenkins job, using the JIRA REST API and a scripting language of your choice. It just comes down to parsing the results file and let your script create issues for the failures.
I'd like to use Cucumber linked with SoapUI and Selenium in order to create an automation functional test.
I only found one website about it, describing that first I have to create a SoapUI project and save it as a .xml file. Then I should include this file into the test project using Cucumber and Selenium.
My first question is what configuration do I have to do for that ? (in the pom.xml file when using Maven for example or in any configuration file for Jenkins)
My second and last question is : if I launch every night with Jenkins (let's say that as an example) my tests, do I have to launch first the SoapUI project and import it again in the test project to run my tests in a good way ?
Thanks for your answers
I think you can use REST Assured for you API testing.
Use cucumber for your BDD statement, use java for there step definition and REST Assured for all the API request stuff.
Wrap all your automation with well written Gherkin and you'll need no other documentation. Think about that. BDD (specification by example) is a requirements approach, not a testing approach.
Our department uses Visual Studio 2008 Team System, and we have a build server that integrates with our TFS source control server. It pulls the source code, builds the solution and runs the unit tests, just as we might do from within VS, and emails a report. The build server is setup using MSBuild and MSTest as the primary tools. All very sweet.
On our development machines we also run a set of selenium unit tests, and I want to include this in the test suite on the build server. I have been told that 'this is not possible using MSBuild/MSTest', but I am at a loss to understand why.
Does any one have experience of running selenium tests (they are just conventional test methods written in C#) who might be able to advise me on whether this is possible and what the gotchas are? Thinking about it, apart from giving the browser access to the desktop when the server is not logged in, once MSBuild has handed off a test list to MSTest it's exactly the same process as on our develop machines.
TIA
I know it's 3 years on, but someone might drop in on this post and not see an answer. This is possible todo.
In a similar fashion how you'd run unit test, in the build definition using default template you need to specify the name of the UnitTest DLL and ensure Run Unit Test is not disabled. Also ensure that the build is building your Automated UI Test solution.
Simples.