At the moment, it is possible to run the same test on different browser assuming that you have all those browsers installed on your machine and that those browsers have a version compatible with your drivers version.
Is there a way, such as a library or everything else, that allows us to run the same test on different versions of the same browser, downloading those browsers on demand (bit like WebDriverManager does with the drivers)?
We want a test harness runnable on every machine and every operating system and we don't want to rely on any browser already installed in that machine.
A tool that allows us to choose on what browser and what version we want to run the test, it downloads a sandboxed or virtual version of that browser, and it runs the test.
I heard also about Saucelabs, but the systems under test are not available by publicly available URLs because they are in a dev environment.
I thought we could use Docker, but in that case we have to use a Headless browser, and we want to keep to possibility to see what the test is actually doing.
Thanks in advance.
No longer do you need (as we did in the dark old days) to manage our own infrastructure and have machines running instances of browsers and configure a Selenium Grid.
SauceLabs is a Selenium Grid in the cloud and is most definitely the recommended approach here. They offer a wide (ever growing) array of platforms.
With all that infrastructure taken care for you in the cloud the problem changes to one of correctly providing SauceLabs the DesiredCapabilities
I would highly recommend you also use Jenkins and its Sauce OnDemand plugin which allows you to select your target environments and at the same time integrate cross device testing into your build pipeline.
Constructing the DesiredCapabilities objects for SauceLabs in your test code is not a straightforward exercise however and can lead to hard to maintain spaghetti code if not done with a lot of discipline.
There are some NuGet packages that solve the SauceLabs DesiredCapabilities problem and integrates with Jenkins as well.
Saucery2 for NUnit2 (NuGet, Source, Example)
Saucery3 for NUnit3 (NuGet, Source, Examples)
SauceryJ for JUnit4 (Source, Example)
Saucery is DesiredCapability factory for SauceLabs. Full disclosure: I wrote Saucery and maintain it.
Saucery pulls in all the dependencies you would want allowing you to write tests against:
Desktop browsers (simply extend SauceryBase)
with Selenium;
or native app tests against:
IOS (simply extend SauceryIOSBase); or
Android (simply extend SauceryAndroidBase) devices
with Appium.
The home page provides benefits and a How To.
Related
I made a Selenium code by Java to test a certain web-application that our company developed.
Now, developers fix the application quite often and every time they update or fix, testers should test this new version of webapp to assert all functions working fine before application release.
Let's suppose that there are 100 testers who do not know how to run Selenium code or install Java.
I decide to create a testing server so that testers can access this server and run test. They also can see all test histories and details so far as well.
Is it possible and realistic to develop a system that runs like server and client? If yes, Can Jenkins do that? Other solutions are welcome!
Thank you in advance and happy new year!
Jenkins is one of the tools, you able to use for this, since it provides a simple way to delegate some tasks to already configured envs, nodes share them for multiple users and hide technical complexity. Also this would be aligned to your CI process, e.g. first - deploy the new code to test env, next - run test automation.
But the same also might be said for some other CI tools, so I suggest to pick some CI tool which your development already uses.
The architecture could be:
1 CI task for run tests -->
2 CI Node or docker image with java, selenium,
maven(gradle), it may be some headless Linux -->
3 Selenium cluster which able to launch multiple
selenium sessions (to cover your testers needs). It could
be some selenium cloud service, or configured
onpremise env. -->
4 Selenium grid hub (may be headless)-->
5 Selenium grid nodes... The final nodes env
should match your test requirements. It could
be Docker with linux (headless or not) or
Windows/MacOs.
Pick some tools and look for quick start guides/tutorials.
Start from simple implementation and improve it continuously.
I may say that for many cases Docker + Ubuntu + Headless Chrome is fine, lightweight and rapid.
Some references (examples the tools I've used):
Jenkins + Selenium + Maven https://www.lambdatest.com/blog/selenium-maven-jenkins-integration/
Selenoid (selenium grid implementation based on docker containers) https://github.com/aerokube/selenoid
Report Portal (just reporting tool) - something more than the default testng report provides. https://reportportal.io/
This is very shortly. The same might be done with a lot of other tools.
We are building a shop for a customer on Shopware 6.3.5.2 and want to use tests to
ensure that core functionality is not broken by our customizations (static plugins)
write new tests for new functionality
There is Running End-to-End Tests but this seems to be for core development and uses psh.phar which is not available in the production template.
How should this be done?
edit
This question is meant a bit broader and concerns also Unit Tests.
Actually, you can use the E2E tests of the platform project - as Cypress itself doesn't care where to run the test against. However, as you already noticed you cannot use psh commands to run them. You may run the tests though the basic Cypress commands, setting your shop's url as baseUrl of the tests, for example via this command:
./node_modules/.bin/cypress run --config baseUrl="<your-url>"
It works with cypress open as well.
The only thing what may become troublesome is the setToInitialState command in most of the tests which takes care about the clean up of shopware's database using psh scripts, unfortunately. You may need to adjust it by overriding the command in order to reset the database of the Production template.
I hope I was able to help a bit. 🙏
There are actually two parts here:
ensure that core functionality is not broken by our customizations (static plugins)
write new tests for new functionality
re 1: For regression tests like this I would suggest end-to-end tests. Either test through the UI with tools like selenium or through the HTTP API (I don't know if the shopware API is sufficient for extensive regression tests).
re 2: Since plugins do not run on their own I would extract all relevant functionality into plain old PHP classes that are independent of shopware and test those in isolation. Explore if some of that functionality can be made visible through an API and test the plugin integration through this. Depending on the actual plugin you might have to resort to UI tests again.
I am currently working on node-webkit based desktop app. I was searching for a way to do automation testing for this . But most of the automation tools required a server to start up but in my case i have on but it runs in the chromium that comes by default. Is there a way to automate UI acceptance test for nwjs apps.
Take a look at the NW.js Wiki under How to run node webkit's test cases
You will need to setup the chrome driver in order to use selenium for example
Once you have those, check out Yaniv kessler's blog post on how to Automate NW.js UI testing using mocha.
I have to do Automated testing of Application based on Flash builder, after searching on this topic i came to know that flash builder includes Testing API's, and these API's supports HP QTP so After Including Testing API's in my application and making test cases Scripts based on VB scripts I can do automated testing, how should i do automated testing of application based on Flash builder.
You should take a look at Flex Mojos, which is a Maven Plugin. You can use it for automated builds and tests as well.
I would like to run multiple Selenium Tests (on a Jenkins server) at the same time.
It currently runs only a single test at a time cause ChromeDriver seems to communicate over a special port. So somehow I guess I have to pass some kind of port settings via Selenium to the ChromeDriver to start up multiple tests.
The Selenium website unfortunately is empty for that topic:
http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp#parallelizing-your-test-runs
From my point of view it makes no difference if the Test runs locally or on Jenkins, the problem is the same. We need to somehow configure ChromeDriver. The question is just how.
Anybody has some ideas or pointers where to look at and what files are involved to get this done?
You can run multiple instances of chromedriver locally quite easily, just instantiate multiple driver objects, chromedriver will keep the profiles separate and find a port to run on all by itself.
Here a link to an example that can run multiple tests using TestNG and Maven:
https://github.com/Ardesco/Selenium-Maven-Template
Just clone the above project and run the following in the command line:
mvn verify -Pselenium-tests -Dbrowser=chrome -Dthreads=2
It takes advantage of TestNG's ability to manage the thread pool and will open up multiple instances if specified. You can do the same thing with jUnit but you'll need to write a custom test runner to fire the tests off into individual threads.
If you decide to use gradle it can deal with managing the thread pools for you with both TestNG and jUnit and a lot of people prefer it to maven.
This is an old question, but for anyone still reading along, it is very possible to run multiple Selenium WebDriver instances in parallel without using Grid. I have successfully tested this using Chrome, FireFox, and PhantomJs (up to 5). Each WebDriver instance uses an isolated context, so session conflict should not be an issue. Be wary of server side conflicts though, depending on the requirements of your website!
For NUnit users, NUnit 3.2.1 now has a 'TestContext.Current.WorkerId' property that will allow you to isolate one WebDriver instance per NUnit worker.
Running multiple browsers on the same machine will often hinder performance, so be careful not to use too many browsers instances, or you may actually increase your testing time!
What you are looking for is Selenium Grid 2.
Grid allows you to :
scale by distributing tests on several machines ( parallel execution )
manage multiple environments from a central point, making it easy to run the tests against a vast combination of browsers / OS.
minimize the maintenance time for the grid by allowing you to implement custom hooks to leverage virtual infrastructure for instance.
I agree using grid in combination with Maven parallelized class, you can run multiple instance in one PC. Jenkins is possible when you are using Ant for your build ,then you can specify which test can be run parallel.
Its quite easy to set it up though ;)