I was just wondering if this was the default behaviour when starting up Intern with a Selenium Tunnel. At the moment, before running any tests, random pages that appear to look like the Intern-tutorial, which I used to learn how to run tests with Intern, seem to come up before attempting to run any of my own tests.
I have set up a vanilla install with no tests set up to demonstrate this behaviour. Please see the video link given below as reference if I have been unclear.
https://www.youtube.com/watch?v=vC15PbjSxVw
Leadfoot, which is the WebDriver implementation used by Intern, performs feature and defect tests on the browser before testing starts. These tests tell intern about features that a particular browser might not support, or might implement incorrectly, giving Intern (well, Leadfoot) a chance to work around them.
Related
I am getting ready to start a new automation project and have done some reading on Cypress as a Selenium alternative. Given that Cypress apparently runs directly in the browser as opposed to Selenium's approach, is it difficult to perform test steps with Cypress tests that fall outside the browser such as communicating with a data store, interacting with services and interacting with product infrastructure such as remote file systems? With my limited exposure to Cypress, I have only seen browser tests so I was hoping someone could shed some light on this.
When it comes to automated testing for web applications, there are two main contenders: Selenium and Cypress. Both have their pros and cons, but which one is the best?
Selenium has been around for much longer than Cypress and is therefore more widely used. It is also more flexible, allowing for tests to be written in a variety of programming languages. However, Selenium can be slow and unreliable, and it is not as easy to use as Cypress.
Cypress, on the other hand, is a newer tool that is gaining popularity due to its simplicity and reliability. Cypress tests are written in JavaScript, making it easier for front-end developers to get started with automated testing. Cypress is also faster than Selenium and can run tests in parallel, making it more efficient.
So, which one should you use? It depends on your needs. If you need a more flexible tool that can be used with different programming languages, Selenium is a good choice. However, if you want a tool that is easier to use and more reliable, Cypress is the better option.
If you need access to things outside the browser, I would go with selenium. This is what I currently do, I have a webdriver wrapper which has "plugins" loaded so that I can make db statements, query the webserver and additionally issue selenium commands to the browser.
If you're looking for just test 100% within the browser, then cypress may be the way to go.
Alternatively, you could use selenium for workflow tests and cypress or even qunitjs for intra-browser unit tests.
In the app I work on, I actually ship a page which contains a qunit page with all of the in-browser tests. Then in a selenium test, in addition to the rest of the workflow, I browser to the qunit page and report on their status as well.
What kind of pros and con involve with headless selenium test execution. I would like to know the recommendations to run tests on real browser vs headless browsers
With a real browser you can; see whats actually going on, inspect element, test javascript on the go.
With a headless browser you can let it run in the background.
But they are both very similar. One you can see... the other you cant.
I traditionally develop using selenium with a browser to see whats going on, and if your code implements a webdriver interface you can just switch browser whenever you want.... even to go headless.
In c# you have RemoteWebDriver, which is what you want to use if you want to be able to use different browsers.
I am new to testing, when doing some research these last few days i found 2 tools that enable testing a web application, here is what i understand so far:
Selenium provides a way to manipulate the browser, so in other terms it enables simulating user interaction on a webPage, we can write tests using PhpUnit-Selenium extension for example and it will make it possible to test our application as a real user would, after that those tests need to run on different browsers...
For TestSwarm i need to write my tests using tools such as (Qunit, Jasmine...) that are mainly focused on unit testing (not user interaction ...) and use TestSwarm server to push those tests to available browsers to run them (i think this is automatic so no need for a user to manually run theses tests)
My conclusion is that Selenium and TestSwarm are somewhat complementary as Selenium enables testing user interaction overall, and TestSwarm simplifies testing javascript cross Browser.
Am i getting this right?
I think you are on the right track, here is an excerpt from https://github.com/jquery/testswarm/issues/258
Okay, so you're using WebDriver and your test suite is a set of instructions (in what language do you have it stored now?) for the browser to execute (go to page X, click button Y, etc.).
Those are not unit tests but integration tests. They require bindings with the browser and/or the ability to execute code on the target computer. They can't be executed from within the browser (in that if I visit the url of your test suite in my browser, nothing happens as the driver instructions need to be run from outside the browser or from a plugin).
TestSwarm is not designed for these kind of integration tests, but for unit tests. A very different method that simply can't be performed by TestSwarm. Also, you wouldn't need any of TestSwarm's features for this and you'd miss things you need instead (like actual browsers and the ability to control them and extract the results). Where those browsers come from there usually is something like TestSwarm close by.
I'd recommend looking into SauceLabs and Jenkins (either self-hosted or perhaps a cloud based solution like CloudBees).
Check out:
• http://sauceio.com/index.php/2012/12/getting-the-most-out-of-selenium-with-cloudbees-and-sauce-labs/
• https://saucelabs.com/jenkins/1
• http://www.cloudbees.com/platform-service-saucelabsondemand.cb
I am a newbie to Automated Testing using WebDriver, so I have a few questions just to clear some things in my head. On a few pages, I saw the samples of executing WebDriver tests on different platforms by just targeting these for the capabilities of the browser or OS.
capability= DesiredCapabilities.firefox();
capability.setBrowserName("firefox");
capability.setPlatform(org.openqa.selenium.Platform.ANY);
or
capability= DesiredCapabilities.internetExplorer();
capability.setBrowserName("iexplore");
capability.setPlatform(org.openqa.selenium.Platform.WINDOWS);
As mentioned in:
Executing tests Concurrently on different OS and Browsers with WebDriver using Java and TestNG
So, if I understand that correctly, actually it is possible to run tests and verify those on the different OS and Browsers just by using the libraries provided by the Selenium?
If so, how accurate are these tests for typical cross browser/platform html/JavaScript issues?
Thank you
This is a great question. I'm going to try to break this down into smaller packets of information so that it hopefully makes sense for old pros and newbies alike.
Without Selenium Grid:
For starters, it is possible to use individual drivers for all the different browser/OS combinations you wish to run the tests on. The drawback is you have to make some (though usually minimal) code adjustments for each browser's driver. This also means breaking the DRY principle. To learn more about writing these kinds of tests check out this documentation. (Also note that if you wanted to run these tests on each build via CI on something like Jenkins you need to have the actual browsers running on a slave on your own hardware, but these are more the DevOps concerns.)
Using Selenium Grid:
More commonly used for the sort of goals you mentioned (and referenced in the other post you linked to), Selenium Grid is a server that allows multiple instances of tests to run in different web browsers on remote machines. The more intro oriented docs for this are here and more forward looking docs are here.
Running Local or in the Cloud:
With Selenium Grid you are going to go one of two ways.
Run on your own hardware locally (or wherever your company has machines to remote into)
Use an online service like Sauce Labs or Testing Bot
A nice "what this might look like in Java" for having an online service provide the browsers is shown in this Sauce Labs page and for Testing Bot here.
Selenium Can be Written in a Ton of Languages:
Selenium follows the WebDriver API and for C#, Java, Perl, PHP, Python, Ruby, JavaScript (Node) or other languages, you still can write test scripts in any of these (and they provide the “frameworks” for some of these officially, while others are community driven) and still have the run tests run solidly in all modern browsers.
Concerning Mobile Devices
There is some good discussion over here that discusses how "close to the real thing" you want your mobile browser tests to be, since the iPhoneDriver and AndroidDriver are largely based on use through WebView, which is less close to the real thing. They are now finding themselves being replaced by ios-driver, Selendroid, and Appium.
To Sum It Up
So to answer what I think you’re getting at with,
... is possible to run tests and verify those on the different OS and Browsers just
by using the libraries provided by the Selenium
the answer is that you can use Selenium Grid and an online service or you will have to use base Selenium/Selenium Server along with a number of other libraries to test all modern browser and OS combinations, but I'm sure many shops do just that because they have the experience and expertise to pull it off.
Alternate (Non-Selenium) Option to Write Once and Test Across Browsers:
If you have a team with JavaScript experience and you're looking to hit the same goal of testing across browsers without the overhead of Selenium, Automates JavaScript Unit Testing
with Sauce Labs (formerly Browser Swarm) would be a good option.
I am looking for a tool to functionally test my Yii/ Node.js web application. The first thing I looked into was Selenium. The app runs on a headless Ubuntu server so successfully setting up xvfb and run a test was really painfull and drove me to another tool.
The error I kept getting is:
Xlib: extension "RANDR" missing on display :0
The other tool was Casperjs along with Phantomjs . Aside from the 5 minute setting up, I wrote few tests and integrated all with Jenkins CI. I really believe there should be more tools like this one. So I feel I've earned something on the short term, but I'm afraid that on the long term I'll hit a dead end. Could you give me some feedback? Am I going the wrong road?
Another thing that's crossing my mind is to setup the Selenium RC and Jenkins on a Windows machine with all browsers set up. I think this will give my tests a better and more accurate perspective.
* I would also like to be able to do some parallel functional tests (interactions) since the website is socket-driven. Does Selenium handle that?
First off, don't use Selenium RC if you can avoid it, it's officially deprecated in favor of Selenium Webdriver (also known as Selenium 2).
About headlessness - Webdriver can be easily run on top of HtmlUnit and also PhantomJS - both should work right away.
If you want to run the tests on actual browsers, people have been successful in running that on headless systems, too, but as you said, it's a pain. Instead, you can run the tests remotely on a different machines with your headless server commanding them all.