I'm trying to understand what is the best practice for running remote e2e tests written in Selenium - for running different suites on different environments. the testing code is attached to git and basically we use TeamCity, but it doesn't have ti be used.
any suggestions would be great
thanks!
You can have a look at my protractor example project on GitHub. I'm not sure my examples would be considered "best practices" but it's how I currently do things :)
The examples cover:
page objects
Running on TravisCI
Running tests on Sauce Labs and Browserstack
Running multiple browsers at once
Related
I have some programming experience and want to switch to QA roles that suit my skills better, I have more knowledge of Java but some Python.
I am wondering if I should focus on Selenium that I have some knowledge of or Protractor or Cypress that are considered as the latest standards?
I am more of a front end guy as a developer.
Also, I generally use guru99, tutorialspoint or w3schools to get started. Any other great resources for free or less expensive for quick learning for these topics?
Will cloud upgrades affect this learning process?
Please advice.
Selenium is still the most wide used web UI automation tool, having a lot of tutorials for any level of experience etc. Also, once you know one tool, you will be able to learn other similar tools in the future easily. Also, Selenium supports and mainly used in programming languages you already know while the other tools you mentioned using JavaScript. So, I'd advice you to start (and then continue) with Selenium.
I think Protractor is being dropped by Angular in favor of Cypress.
Please see
The State of end-to-end testing with Angular
...we’ve decided to deprecate Protractor
and #cypress/schematic.
Component testing
Also consider that Cypress has first-class component testing (as well as e2e testing) which has better speed than e2e but better coverage than unit testing.
See Quickstart Angular
and Cypress Component Testing vs Angular Test Harness
I made a Selenium code by Java to test a certain web-application that our company developed.
Now, developers fix the application quite often and every time they update or fix, testers should test this new version of webapp to assert all functions working fine before application release.
Let's suppose that there are 100 testers who do not know how to run Selenium code or install Java.
I decide to create a testing server so that testers can access this server and run test. They also can see all test histories and details so far as well.
Is it possible and realistic to develop a system that runs like server and client? If yes, Can Jenkins do that? Other solutions are welcome!
Thank you in advance and happy new year!
Jenkins is one of the tools, you able to use for this, since it provides a simple way to delegate some tasks to already configured envs, nodes share them for multiple users and hide technical complexity. Also this would be aligned to your CI process, e.g. first - deploy the new code to test env, next - run test automation.
But the same also might be said for some other CI tools, so I suggest to pick some CI tool which your development already uses.
The architecture could be:
1 CI task for run tests -->
2 CI Node or docker image with java, selenium,
maven(gradle), it may be some headless Linux -->
3 Selenium cluster which able to launch multiple
selenium sessions (to cover your testers needs). It could
be some selenium cloud service, or configured
onpremise env. -->
4 Selenium grid hub (may be headless)-->
5 Selenium grid nodes... The final nodes env
should match your test requirements. It could
be Docker with linux (headless or not) or
Windows/MacOs.
Pick some tools and look for quick start guides/tutorials.
Start from simple implementation and improve it continuously.
I may say that for many cases Docker + Ubuntu + Headless Chrome is fine, lightweight and rapid.
Some references (examples the tools I've used):
Jenkins + Selenium + Maven https://www.lambdatest.com/blog/selenium-maven-jenkins-integration/
Selenoid (selenium grid implementation based on docker containers) https://github.com/aerokube/selenoid
Report Portal (just reporting tool) - something more than the default testng report provides. https://reportportal.io/
This is very shortly. The same might be done with a lot of other tools.
We are building a shop for a customer on Shopware 6.3.5.2 and want to use tests to
ensure that core functionality is not broken by our customizations (static plugins)
write new tests for new functionality
There is Running End-to-End Tests but this seems to be for core development and uses psh.phar which is not available in the production template.
How should this be done?
edit
This question is meant a bit broader and concerns also Unit Tests.
Actually, you can use the E2E tests of the platform project - as Cypress itself doesn't care where to run the test against. However, as you already noticed you cannot use psh commands to run them. You may run the tests though the basic Cypress commands, setting your shop's url as baseUrl of the tests, for example via this command:
./node_modules/.bin/cypress run --config baseUrl="<your-url>"
It works with cypress open as well.
The only thing what may become troublesome is the setToInitialState command in most of the tests which takes care about the clean up of shopware's database using psh scripts, unfortunately. You may need to adjust it by overriding the command in order to reset the database of the Production template.
I hope I was able to help a bit. 🙏
There are actually two parts here:
ensure that core functionality is not broken by our customizations (static plugins)
write new tests for new functionality
re 1: For regression tests like this I would suggest end-to-end tests. Either test through the UI with tools like selenium or through the HTTP API (I don't know if the shopware API is sufficient for extensive regression tests).
re 2: Since plugins do not run on their own I would extract all relevant functionality into plain old PHP classes that are independent of shopware and test those in isolation. Explore if some of that functionality can be made visible through an API and test the plugin integration through this. Depending on the actual plugin you might have to resort to UI tests again.
Is it possible to setup multiple cypress test runner to test chat apps? If yes, how? I have found on some links there is idea to test it using CasperJS and PhantomJS, but not really sure how to setup.
I think we need some more information to answer this question:
1. Why do you want multiple test runners?
2. The title suggests there are multiple chat apps as well, is that correct?
What I can tell you from my own experience:
We have multiple web applications in place which all have their own test runner. Those are all executed via a Docker by Jenkins. But I can execute all tests locally on my laptop as wel (not at the same time).
One of the web applications has 4 different test runners which all use the same test environment. But once again, run via Jenkins in a Docker.
I'm working with a Jenkins system, and I've recently started working to optimize the tests. There are almost a thousand Selenium tests and twice as many unit and integration tests.
I'm wondering if there is some way to find which tests are the most prone to failure so we can attack the worst offenders first. It would also be nice if there was an integrated way to track who is working on which test, and if a test has been fixed, what was done to fix it. I'm new to Jenkins, so please point me to some documentation or a plugin I can install that would help me with this.
I think what you are describing is JIRA which can tie SCMs to issues to releases to developers.
There is Jenkins to JIRA integration to create Jira issues out of Jenkins