Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I am trying to understand testing framework better and been looking into Selenium. I've used HTMLUnit before, mainly when I needed to scrape some information off website or the likes.
In the context of writing test automation, what's the advantage / disadvantages of Selenium vs HTMLUnit? Looks to me Selenium is more complicated to set up than HTMLUnit, although at the same time there's a HTMLUnitDriver for Selenium which I think behave the exact same way as in HTMLUnit itself?
Selenium obviously provides more robust framework, it has the Selenium RC for pararel testing, it also has different browser drivers that can be used - although when you used the browser drivers, the test will actually open/close a browser application rather than headless.
May be I am not understanding Selenium correctly. Some directions and pointers would be great!
On another note - a separate question - I am also looking at doing automated testing on mobile browser, I see that Selenium has an IPhoneDriver for it, but then this is not a headless testing either as it requires actual iOS simulator.
Is there anyway to do headless testing on mobile sites? Would changing user-agent be sufficient? I've seen a couple posts around changing user-agent that seem to have their own challenges, eg. Set user-agent in Selenium RC
Thanks a lot!
well, would try to explain differences in detail.
Speaking about parallel testing, it better to use selenium grid.
Basic concept of selenium RC and selenium grid.
You can get into more details here
Some words about selenium webDriver:
The primary new feature in Selenium 2.0 is the integration of the WebDriver API. WebDriver is designed to providing an simpler, more concise programming interface along with addressing some limitations in the Selenium-RC API. Selenium-WebDriver was developed to better support dynamic web pages where elements of a page may change without the page itself being reloaded. WebDriver’s goal is to supply a well-designed object-oriented API that provides improved support for modern advanced web-app testing problems.
How Does WebDriver ‘Drive’ the Browser Compared to Selenium-RC?
Selenium-WebDriver makes direct calls to the browser using each browser’s native support for automation. How these direct calls are made, and the features they support depends on the browser you are using. Information on each ‘browser driver’ is provided later in this chapter.
For those familiar with Selenium-RC, this is quite different from what you are used to. Selenium-RC worked the same way for each supported browser. It ‘injected’ javascript functions into the browser when the browser was loaded and then used its javascript to drive the AUT within the browser. WebDriver does not use this technique. Again, it drives the browser directly using the browser’s built in support for automation.
WebDriver and the Selenium-Server
You may, or may not, need the Selenium Server, depending on how you intend to use Selenium-WebDriver. If you will be only using the WebDriver API you do not need the Selenium-Server. If your browser and tests will all run on the same machine, and your tests only use the WebDriver API, then you do not need to run the Selenium-Server; WebDriver will run the browser directly.
There are some reasons though to use the Selenium-Server with Selenium-WebDriver.
You are using Selenium-Grid to distribute your tests over multiple
machines or virtual machines (VMs).
You want to connect to a remote machine that has a particular browser
version that is not on your current machine.
You are not using the Java bindings (i.e. Python, C#, or Ruby) and
would like to use HtmlUnit Driver
Selenium-WebDriver’s Drivers
WebDriver is the name of the key interface against which tests should be written, but there are several implementations. These include:
HtmlUnit Driver
This is currently the fastest and most lightweight implementation of WebDriver. As the name suggests, this is based on HtmlUnit. HtmlUnit is a java based implementation of a WebBrowser without a GUI. For any language binding (other than java) the Selenium Server is required to use this driver.
Pros
Fastest implementation of WebDriver
A pure Java solution and so it is platform independent.
Supports JavaScript
Cons
Emulates other browsers’ JavaScript behaviour (see below)
JavaScript in the HtmlUnit Driver
None of the popular browsers uses the JavaScript engine used by HtmlUnit (Rhino). If you test JavaScript using HtmlUnit the results may differ significantly from those browsers.
When we say “JavaScript” we actually mean “JavaScript and the DOM”. Although the DOM is defined by the W3C each browser has its own quirks and differences in their implementation of the DOM and in how JavaScript interacts with it. HtmlUnit has an impressively complete implementation of the DOM and has good support for using JavaScript, but it is no different from any other browser: it has its own quirks and differences from both the W3C standard and the DOM implementations of the major browsers, despite its ability to mimic other browsers.
With WebDriver, we had to make a choice; do we enable HtmlUnit’s JavaScript capabilities and run the risk of teams running into problems that only manifest themselves there, or do we leave JavaScript disabled, knowing that there are more and more sites that rely on JavaScript? We took the conservative approach, and by default have disabled support when we use HtmlUnit. With each release of both WebDriver and HtmlUnit, we reassess this decision: we hope to enable JavaScript by default on the HtmlUnit at some point.
To investigate deeper into webDriver's setUp see this
From HtmlUnit documentation:
HtmlUnit is not a generic unit testing framework. It is specifically a way to simulate a browser for testing purposes and is intended to be used within another testing framework such as JUnit or TestNG.
So to conclude Selenium and HtmlUnit difference:
HtmlUnit is a java based implementation of a WebBrowser without a GUI and a way to simulate a browser for testing purposes and Selenium-WebDriver makes direct calls to the browser using each browser’s native support for automation. we can see that HtmlUnit provides API without GUI possibility for automation whereas WebDriver provides internal browsers' possibilities for automation.
Speaking about mobile automation,
Selenium also has an iPhone Driver
iPhone Driver wiki article
and Android Driver
Android Driver wiki article
See also this presentation
Unfortunately I can not give you my working experience evaluation of mobile drivers as I deal with web automation (no mobile). Also know that Cucumber (automation tool) is popular among mobile automators.
see this and this.
Hope it come a lil bit more clear for you now =)
Selenium and HTMLUnit are somewhat similar in concept, but Selenium is more mature/robust and has a lot more features.
Note that Selenium encompasses the recording (IDE) plugin for Firefox, which allows you to record tests and the RC/WebDriver automation framework which essentially drives a browser. The two can be used together to make test creation very easy.
The only advantage I could see to using HTMLUnit is that it is less resource intensive, so you could potentially run tests on less hardware, but with Selenium's parallel support even that isn't really true anymore.
When running tests from Jenkins overnight, you typically have no access to a windowing system such as X11 or Windows in which to run the web browser. I therefore see a benefit of using the HTMLUnit web driver in that case since it doesn't require access to a windowing system.
In my experience, HtmlUnit does a fine job with browsing automation, but might get a bit buggy when dealing with Javascript. I actually came to a case in which I wasn't able to automate an image download with HtmlUnit, and had to turn to Selenium, which performed beyond my expectations. The case is actually registered in an SO thread.
I have used Selenium WebDriver for automation.
There is a very easy method to causing the browser to be headless.
Simply apply ChromeOptions (In my case, other DriverOptions are available)
ChromeOptions("Headless")
There are many proficient methods that can come from using Options or Services, as another example
This will stop the Driver/CommandPrompt window from 'appearing' so it will remain 'silent' and unexposed.
ChromeDriverServices ("Silent")
At least at UX systems you can use for example Xvfb and point the browsers to that display to make them "headless"
See also http://infiniteundo.com/post/54014422873/headless-selenium-testing-with-firefox-and-xvfb
or How do I run Selenium in Xvfb?
Related
The main difference is, execution on GUI bases and non GUI bases(Headless).
I am looking for difference between all Headless browsers with each other, But unfortunately I didn't find any. I go through one by one, Which makes more confusion. It would be great if someone can share short information with differences, which makes things clear.
Browser
A Browser is an application program that provides a way to look at and interact with all the information on the World Wide Web. Technically a Browser, alternatively referred as a Web Browser or Internet Browser, is a client program that uses HTTP (Hypertext Transfer Protocol) to make requests of Web servers throughout the Internet on behalf of the Browser User.
Headless Browser
A Headless Browser is also a Web Browser but without a graphical user interface (GUI) but can be controlled programmatically which can be extensively used for automation, testing, and other purposes.
Why to use Headless Browsers?
There are a lot of advantages and disadvantages in using the Headless Browsers. Using a headless browser might not be very helpful for browsing the Web, but for Automating tasks and tests it’s awesome.
Advantages of Headless Browsers
There is a lot of advantages in using Headless Browsers. Some of tham are as follows:
A definite advantage of using Headless Browsers is that they are typically faster than real browsers. The reason for being faster is because we are not starting up a Browser GUI and can bypass all the time a real browser takes to load CSS, JavaScript and open and render HTML DOM.
Performancewise you can typically see a 2x to 15x faster performance when using a headless browser.
While Scraping Websites you don’t necessarily want to have to manually start up a website. So you can access the website headlessly and just scrape the HTML. You don’t need to render a Full Browser to do that.
Lot of developers use a Headless Browser for unit testing code changes for their websites and mobile apps. Being able to do all this from a command line without having to manually refresh or start a browser saves them lots and effort.
When You Might NOT Want to Use a Headless Browser
There can be number of reasons why you may opt to use a Real Browser instead of a Headless Browser. A couple of instances:
You need to mimic real users.
You need to visually see the test run.
If you need to do lots of debugging, headless debugging can be difficult.
Which headless browsers are better?
As you rightly pointed that ...the main difference is in the execution on GUI bases and non GUI bases(Headless)..., so from Testing Perspective a lot will depend on the Browser Engine implemented under the hood by any particular browser. For example, here are some of the Browser Engines which fully render web pages or run JavaScript in a virtual DOM.
Chromium Embedded Framework: CEF is a open source project based on the Google Chromium project with JavaScript support and BSD license.
Erik: Erik is a Headless Browser on top of Kanna and WebKit with Swift support and MIT license.
jBrowserDriver: jBrowserDriver is a Selenium-compatible Headless Browser which is WebKit-based and works with Selenium Server through Java binding support and Apache License v2.0 license.
PhantomJS: PhantomJS is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG with JavaScript, Python, Ruby, Java, C#, Haskell, Objective-C, Perl, PHP and R(via Selenium) support and BSD 3-Clause license.
Splash: Splash is a javascript rendering service with an HTTP API. It's a lightweight browser with an HTTP API, implemented in Python using Twisted and QT with almost all the laungage binding arts and BSD 3-Clause license.
You can find a related discussion in Which drivers support “no-browser”/“headless” testing?
I'm looking for a reliable application/tool for automation using headless browser.
Looked into htmlunit and httpunit and as far as I understand, they were built in and 'era'
way before modern Javascrips became available therefore using them may result some issues.
Then I looked further and apparently it is possible to use Selenium (Which I have used before)
in conjunction with the htmlunit driver.
I'm confused. Firstly, why would you use Selenium with htmlunit? Which API's are called during the test, Selenium's of htmunit's? The other thing is why would anybody chose to use htmlunit knowing it does not support modern web sites ?
Thanks in advance.
Firstly Selenium with HtmlUnit supports JS at present time, but with some limitations:
When we say "javascript" we actually mean "javascript and the DOM".
Although the DOM is defined by the W3C each browser out there has its
own quirks and differences in their implementation of the DOM and in
how javascript interacts with it. HtmlUnit has an impressively
complete implementation of the DOM and has good support for using
javascript, but it is no different from any other browser: it has its
own quirks and differences from both the W3C standard and the DOM
implementations of the major browsers, despite its ability to mimic
other browsers.
Reference
Firstly, why would you use Selenium with htmlunit?
HtmlUnitDriver is by far the fastest implementation of WebDriver. Some times you just don't care about JS execution - there really are still sites like this. There are definitely tons of reasons. For example I have used it to do simple administration tasks.
Which API's are called during the test, Selenium's of htmunit's?
You would be using WebDriver's API that would make calls to the browser.
There is something similar here.
I am a newbie to Automated Testing using WebDriver, so I have a few questions just to clear some things in my head. On a few pages, I saw the samples of executing WebDriver tests on different platforms by just targeting these for the capabilities of the browser or OS.
capability= DesiredCapabilities.firefox();
capability.setBrowserName("firefox");
capability.setPlatform(org.openqa.selenium.Platform.ANY);
or
capability= DesiredCapabilities.internetExplorer();
capability.setBrowserName("iexplore");
capability.setPlatform(org.openqa.selenium.Platform.WINDOWS);
As mentioned in:
Executing tests Concurrently on different OS and Browsers with WebDriver using Java and TestNG
So, if I understand that correctly, actually it is possible to run tests and verify those on the different OS and Browsers just by using the libraries provided by the Selenium?
If so, how accurate are these tests for typical cross browser/platform html/JavaScript issues?
Thank you
This is a great question. I'm going to try to break this down into smaller packets of information so that it hopefully makes sense for old pros and newbies alike.
Without Selenium Grid:
For starters, it is possible to use individual drivers for all the different browser/OS combinations you wish to run the tests on. The drawback is you have to make some (though usually minimal) code adjustments for each browser's driver. This also means breaking the DRY principle. To learn more about writing these kinds of tests check out this documentation. (Also note that if you wanted to run these tests on each build via CI on something like Jenkins you need to have the actual browsers running on a slave on your own hardware, but these are more the DevOps concerns.)
Using Selenium Grid:
More commonly used for the sort of goals you mentioned (and referenced in the other post you linked to), Selenium Grid is a server that allows multiple instances of tests to run in different web browsers on remote machines. The more intro oriented docs for this are here and more forward looking docs are here.
Running Local or in the Cloud:
With Selenium Grid you are going to go one of two ways.
Run on your own hardware locally (or wherever your company has machines to remote into)
Use an online service like Sauce Labs or Testing Bot
A nice "what this might look like in Java" for having an online service provide the browsers is shown in this Sauce Labs page and for Testing Bot here.
Selenium Can be Written in a Ton of Languages:
Selenium follows the WebDriver API and for C#, Java, Perl, PHP, Python, Ruby, JavaScript (Node) or other languages, you still can write test scripts in any of these (and they provide the “frameworks” for some of these officially, while others are community driven) and still have the run tests run solidly in all modern browsers.
Concerning Mobile Devices
There is some good discussion over here that discusses how "close to the real thing" you want your mobile browser tests to be, since the iPhoneDriver and AndroidDriver are largely based on use through WebView, which is less close to the real thing. They are now finding themselves being replaced by ios-driver, Selendroid, and Appium.
To Sum It Up
So to answer what I think you’re getting at with,
... is possible to run tests and verify those on the different OS and Browsers just
by using the libraries provided by the Selenium
the answer is that you can use Selenium Grid and an online service or you will have to use base Selenium/Selenium Server along with a number of other libraries to test all modern browser and OS combinations, but I'm sure many shops do just that because they have the experience and expertise to pull it off.
Alternate (Non-Selenium) Option to Write Once and Test Across Browsers:
If you have a team with JavaScript experience and you're looking to hit the same goal of testing across browsers without the overhead of Selenium, Automates JavaScript Unit Testing
with Sauce Labs (formerly Browser Swarm) would be a good option.
What's the pro's and cons for both and why should I use either of them for functional testing?
Regards,
Jonas
Selenium is for integration testing, It will test how your web application behaves in an actual browser. This can find things that HTTPUnit cannot, e.g. browser compatibility of css and JavaScript.
HTTPUnit tests web applications by directly calling the web service and manipulating the response. This is a functional test framework as it tests what your web app does, not how it behaves across different platforms.
One advantage of HTTPUnit is that it is much quicker than Selenium. Personally I would (and do) use both. HTTPUnit for complete functional testing and Selenium for selective (not complete) integration testing to check browser compatibility.
HttpUnit is very simple, extremely easy to use, and requires very little to get up and running. It's a good place to start for simple tests, however it will require more coding to create complex tests.
Selenium is more than just a library, like HttpUnit. It equips you better for more advanced, stateful tests.
Selenium will use your browser to automate tests
- Selenium http://seleniumhq.org/
httpunit goes from the html specification and simulates a standards conformant browser
- httpunit http://httpunit.sourceforge.net/
Selenium is more specific and will let you use browser specific behaviour and not limit the application under test.
With httpunit the limits are set by what httpunit supports which is far less than most browser do. E.g. the javascript capabilities are reduced to a very small set. On the other hand if your app passes httpunit tests it will probably run on quite a few browsers and environments.
So if you are interested in supporting a mininimal common set of features httpunit might be for you.
I can see that since selenium 2.0, WebDriver and Selenium RC are packaged together for download. Now I primarily use WebDriver, but can I bring in Selenium RC in my testing scripts from now and then? Is there anything that Selenium RC is capable of but WebDriver is not, or vice versa?
You should probably start your research here (though you may have already gone over this): http://seleniumhq.org/docs/03_webdriver.html
I'll assume you're contrasting Selenium-RC to WebDriver, Selenium-IDE really isn't in the same ballpark.
Selenium uses JavaScript to automate web pages. This lets it interact very tightly with web content, and was one of the first automation tools to support Ajax and other heavily dynamic pages. However, this also means Selenium runs inside the JavaScript sandbox. This means you need to run the Selenium-RC server to get around the same-origin policy, which can sometimes cause issues with browser setup.
WebDriver on the other hand uses native automation from each language. While this means it takes longer to support new browsers/languages, it does offer a much closer 'feel' to the browser. If you're happy with WebDriver, stick with it, it's the future. There are limitations and bugs right now, but if they're not stopping you, go for it.
Selenium Benefits over WebDriver
Supports many browsers and many languages, WebDriver needs native implementations for each new language/browser combo.
Very mature and complete API
Currently (Sept 2010) supports JavaScript alerts and confirms better
Benefits of WebDriver Compared to Selenium
Native automation faster and a little less prone to error and browser configuration
Does not require Selenium-RC Server to be running
Access to headless HTMLUnit can allow tests to run very fast
Great API
I see this is an old question but found this is on the Selenium HQ home page:
Selenium WebDriver is the successor of Selenium Remote Control which
has been officially deprecated. The Selenium Server (used by both
WebDriver and Remote Control) now also includes built-in grid
capabilities.
So it's settled :-)
The biggest difference is RC runs from a vs, 2.0 uses Webdriver and launches the browser, instead of using a vs. In order to you RC in 2.0, check here: http://seleniumhq.org/docs/09_webdriver.html#emulating-selenium-rc
I dont know how to take 2.0 into RC though, but were do you see they are packaged together? They are two different products. Selenium 2 is webdriver, and Selenium RC is Selenium 1.
Personally, I found 2.0 a lot easier to program with. Plus by the end of the year Javascript alert support should be implemented, which is a huge plus!