I am aiming to use automated testing to ensure that specific pages of a website are loading, rather than look/feel of UI elements or performance testing.
I have set up a number of selenium scripts, using ruby, which are executable locally for each test. My aim is to host that somewhere and add some form of text/email notification if one of the tests fail.
What is the best way to go about this?
Presumably some sort of linux server setup with selenium running headless from it could work. Would it be best to run this from some sort of rails or sinatra app with scheduling?
Related
I am getting ready to start a new automation project and have done some reading on Cypress as a Selenium alternative. Given that Cypress apparently runs directly in the browser as opposed to Selenium's approach, is it difficult to perform test steps with Cypress tests that fall outside the browser such as communicating with a data store, interacting with services and interacting with product infrastructure such as remote file systems? With my limited exposure to Cypress, I have only seen browser tests so I was hoping someone could shed some light on this.
When it comes to automated testing for web applications, there are two main contenders: Selenium and Cypress. Both have their pros and cons, but which one is the best?
Selenium has been around for much longer than Cypress and is therefore more widely used. It is also more flexible, allowing for tests to be written in a variety of programming languages. However, Selenium can be slow and unreliable, and it is not as easy to use as Cypress.
Cypress, on the other hand, is a newer tool that is gaining popularity due to its simplicity and reliability. Cypress tests are written in JavaScript, making it easier for front-end developers to get started with automated testing. Cypress is also faster than Selenium and can run tests in parallel, making it more efficient.
So, which one should you use? It depends on your needs. If you need a more flexible tool that can be used with different programming languages, Selenium is a good choice. However, if you want a tool that is easier to use and more reliable, Cypress is the better option.
If you need access to things outside the browser, I would go with selenium. This is what I currently do, I have a webdriver wrapper which has "plugins" loaded so that I can make db statements, query the webserver and additionally issue selenium commands to the browser.
If you're looking for just test 100% within the browser, then cypress may be the way to go.
Alternatively, you could use selenium for workflow tests and cypress or even qunitjs for intra-browser unit tests.
In the app I work on, I actually ship a page which contains a qunit page with all of the in-browser tests. Then in a selenium test, in addition to the rest of the workflow, I browser to the qunit page and report on their status as well.
I have built an automation framework for testing our web app that runs as after each new deploy to our staging environment, as a regression pack. Now the issue is the tests fail whenever there's a new experiment that touches that specific part of the tests, e.g., the home page validation tests fail if there is a new home page experiment. I'd like to know how I can make my tests robust enough to resolve the issue maybe by ignoring experiments altogether or always ensuring the page loads in the current non-experiment group?
I thought maybe a possible solution would be for the web team to write a new cookie than controls the experiments, and then just set that cookie in a hook prior to my tests? Would that work or is there maybe a better way?
The solution with the cookie that controls your A/B experiments will work well with TestCafe. TestCafe allows you to work with cookies using the ClientFunctions mechanism or Client Scripts.
Testim.io is an automated testing platform. How is it different from Selenium?
Testim.io is a SaaS applying machine learning to test automation.
Usage: You use testim's plugin to record test cases or bugs (& submit to Trello/JIRA). You can then edit and add javascript or image validation to your tests, and run them via testim's site or via CLI and your CI/CD cloud to execute your tests.
Machine Learning: When you record tests, testim uses machine learning weighting rather than individual CSS Selectors or XPath to identify DOM elements to test. When you execute tests, the tests rebalance weighting, so you don't have to continually fix your !##$% tests because you (or React.js) changed the name of the selector or XPath element.
It is a browser add-on and runs inside your browser. It supports record and playback. Also, it is cloud based. All your scripts are stored in the cloud. Supports multiple locators as QTP.
I am new to testing, when doing some research these last few days i found 2 tools that enable testing a web application, here is what i understand so far:
Selenium provides a way to manipulate the browser, so in other terms it enables simulating user interaction on a webPage, we can write tests using PhpUnit-Selenium extension for example and it will make it possible to test our application as a real user would, after that those tests need to run on different browsers...
For TestSwarm i need to write my tests using tools such as (Qunit, Jasmine...) that are mainly focused on unit testing (not user interaction ...) and use TestSwarm server to push those tests to available browsers to run them (i think this is automatic so no need for a user to manually run theses tests)
My conclusion is that Selenium and TestSwarm are somewhat complementary as Selenium enables testing user interaction overall, and TestSwarm simplifies testing javascript cross Browser.
Am i getting this right?
I think you are on the right track, here is an excerpt from https://github.com/jquery/testswarm/issues/258
Okay, so you're using WebDriver and your test suite is a set of instructions (in what language do you have it stored now?) for the browser to execute (go to page X, click button Y, etc.).
Those are not unit tests but integration tests. They require bindings with the browser and/or the ability to execute code on the target computer. They can't be executed from within the browser (in that if I visit the url of your test suite in my browser, nothing happens as the driver instructions need to be run from outside the browser or from a plugin).
TestSwarm is not designed for these kind of integration tests, but for unit tests. A very different method that simply can't be performed by TestSwarm. Also, you wouldn't need any of TestSwarm's features for this and you'd miss things you need instead (like actual browsers and the ability to control them and extract the results). Where those browsers come from there usually is something like TestSwarm close by.
I'd recommend looking into SauceLabs and Jenkins (either self-hosted or perhaps a cloud based solution like CloudBees).
Check out:
• http://sauceio.com/index.php/2012/12/getting-the-most-out-of-selenium-with-cloudbees-and-sauce-labs/
• https://saucelabs.com/jenkins/1
• http://www.cloudbees.com/platform-service-saucelabsondemand.cb
In my current project we are testing our ASP.NET GUI using WatiN and Mbunit.
When I was writing the tests I realized that it would be great if we also could use all of these for stresstesting. Currently we are using Grinder to stresstest but then we have to script our cases all over again which for many reasons isent that good.
I have been trying to find a tool that can use my existing tests to create load on the site and record stats, but so far i have found noting. Is there such a tool or is there an easy way to create one?
We have issues on our build server when running WatiN tests as it often throws timeouts trying to access the Internet Explorer COM component. It seems to hang randomly while waiting for the total page to load.
Given this, I would not recommend it for stress testing as the results will be inaccurate and the tests are likely to be slow.
I would recommend JMeter for making threaded calls to the HTTP requests that your GUI is making
For load testing there is a tool which looks promising - LoadStorm. Free for 25 users. It has zero deployment needs as this is a cloud based service.
You could build a load controller for your stress testing. It could take your watin tests and run them in a multithreaded/multiprocessed way.
If you are comfortable using Selenium instead of WatiN, check out BrowserMob for browser-based load testing. I'm one of the Selenium RC authors and started BrowserMob to provide a new way to load test. By using real browsers, rather than simulated traffic, tests end up being much easier to script and maintain.