Where does the functionality of selenium to control the browser lies in? - selenium

Where does the functionality of the Selenium lie?
I've read somewhere that the language binding or API provides glue code to use Selenium in java.
Glue code or Binding code: The code that connects incompatible software components.
Jason Huggings wrote JavaScriptTestRunner to control the browser. The functionality to control the browser lie in the JavaScriptTestRunner.
It suffers from Same-origin policy. Paul Hammant created HTTP Proxy to bypass the Same-origin policy. This opened the doors to write tests in multiple languages. We are provided with an API to write tests.
Then WebDriver is created. WebDriver and RC are merged to form Selenium 2.
Where does the functionality to control the browser gone to?
Within the API?
Within the browser driver?
Why does the API is called as glue code or binding code? What does it bind?

Selenium in the general usage of the term is a library - a collection of code organized in modules and packages. In this form it's a WebDriver client - it can communicate with that kind of server, following the WebDriver protocol; thus enabling a programmer to control a browser.
Where does the functionality to control the browser gone to?
In the WebDriver protocol, and the server that translates the WebDriver commands to browser instructions, and back.
Selenium being a client gives you the ability to use this functionality, in your chosen language - conveniently, without you having to learn or directly run WebDriver requests and parse their response.
Why does the API is called as glue code or binding code? What does it bind?
In programming binding is the uniform / or well-defined API of a library, that connects lower-level code (other programs, or the OS) or protocols - WebDriver in this case, to higher-level concepts - your code. Wikipedia to the rescue, with more details.

Glue Code
Glue Code is the executable code most often the source code that serves the purpose of adapting different parts of code that would otherwise be incompatible. Glue code does not contribute any functionality towards meeting any program requirements as such. Instead, it often appears in code that lets the existing libraries or programs to interoperate among themselves, as in language bindings or foreign function interfaces. Glue code may be written in the same language as the code it is gluing together, or in a separate glue language. Glue code is very efficient in rapid prototyping environments where several components are quickly put together into a single language or framework.
JavaScriptTestRunner
Jason Huggins while testing an internal application at ThoughtWorks reduced the time which was required to manually step through the same tests with every change he made developing a Javascript library that could drive the interactions with the webpage which allowed him to automatically rerun tests against multiple browsers. He named this program as the JavaScriptTestRunner. Later, he made JavaScriptTestRunner open source. This library eventually became Selenium Core which underlies all the functionality of Selenium Remote Control (RC) and Selenium IDE.
Unfortunately, to work within the Same Origin Policy, Selenium Core must be placed in the same origin as the Application Under Test (AUT). So another ThoughtWork engineer, Paul Hammant, created a server that would act as an HTTP proxy masking the AUT under a fictional URL, embedding Selenium Core and the set of tests and delivering them as if they were coming from the same origin. This system became known as the Selenium Remote Control (Selenium RC), or Selenium 1.
Why to use HTTP proxy
Again, when web frameworks were becoming more complex and powerful, the restrictions of Web Browsers’ sandboxed Javascript environment were increasingly limiting the effectiveness of Selenium Core. Simon wanted a testing tool that spoke directly to the browser using the "native" method for the browser and operating system, thus avoiding the restrictions of a sandboxed Javascript environment. That was when WebDriver and Selenium RC were merged to form Selenium 2. All implementations of WebDriver that communicate with web browsers started using a common wire protocol. This wire protocol defined the RESTful web service using JSON over HTTP.
Conclusion
In short, the functionality to control the browser was always within the Javascript library that driven the interactions with the webpage which was part of the JavaScriptTestRunner and later Selenium Core.
As per the diagram below with respect to the different classes/interfaces the functionality to control the browser is with the selenium-api

Related

What is the difference between Selenium server and WebDriver (not RC, not Remote, not 4.0)?

Please note that I am not talking about Selenium RC or RemoteWebDriver or the old Selenium server or the standalone server. I am also not talking about Selenium 4.0 so let us not get into that because it simply dodges answering the question.
When we say Selenium-WebDriver, as per my understanding, it is an integration of Selenium code + WebDriver project (which is also the name of the W3C WebDriver specification).
Here is my understanding of the difference so far:
Selenium = the library which supports multiple programming languages. In each language the user creates instances of ChromeDriver / FirefoxDriver etc. which are client libraries that allow the user to write high-level code. This code is converted by these client libraries into an API request with JSON body containing javascript commands. This is sent to the Selenium server through API requests.
The API requests which will be sent to the Selenium server is either converted to the W3C WebDriver format, or maybe the client libraries already do that. I'm not sure which is it.
The selenium server converts the request or simply forwards it to the driver application that is running (Selenium server is supposed to act like a reverse proxy so I think that it simply forwards it) into a format that is understood by the drivers (the W3C specification).
The drivers now interact with the browser. How do they do that? Well, the browsers come built-in with JS libraries to help drive automation. And these drivers are external applications that take REST API requests, and are able to call these JS libraries based on these requests.
When we say WebDriver, it consists of 2 parts. One, the W3C specification. Two, it is also the same name that Selenium has kept for it's client. You are actually creating an instance of "WebDriver" (or "RemoteWebDriver" to be more specific) while writing code. So "WebDriver" is more than just a W3C specification over here. It is also the name of the Selenium client implementation. (I have no idea why they decided to keep the same name which made it so confusing - it's like creating a microservice called as "REST API microservice").
Qs A: Is my understanding of the concepts correct? I cannot get a clear answer anywhere because all answers/discussions on the web are simply dodging answering the question by quoting from the official docs (which are pretty vaguely worded).
Qs B: Does the Selenium server modify the response in any way before sending it to the driver? If yes, then can we say that in this case the "Selenium server" has become the client and driver has become the server?
Qs C: I had read somewhere that the browser drivers also act like a RESTful service. Is this true?

Difference between running a browser in headless vs browser [duplicate]

The main difference is, execution on GUI bases and non GUI bases(Headless).
I am looking for difference between all Headless browsers with each other, But unfortunately I didn't find any. I go through one by one, Which makes more confusion. It would be great if someone can share short information with differences, which makes things clear.
Browser
A Browser is an application program that provides a way to look at and interact with all the information on the World Wide Web. Technically a Browser, alternatively referred as a Web Browser or Internet Browser, is a client program that uses HTTP (Hypertext Transfer Protocol) to make requests of Web servers throughout the Internet on behalf of the Browser User.
Headless Browser
A Headless Browser is also a Web Browser but without a graphical user interface (GUI) but can be controlled programmatically which can be extensively used for automation, testing, and other purposes.
Why to use Headless Browsers?
There are a lot of advantages and disadvantages in using the Headless Browsers. Using a headless browser might not be very helpful for browsing the Web, but for Automating tasks and tests it’s awesome.
Advantages of Headless Browsers
There is a lot of advantages in using Headless Browsers. Some of tham are as follows:
A definite advantage of using Headless Browsers is that they are typically faster than real browsers. The reason for being faster is because we are not starting up a Browser GUI and can bypass all the time a real browser takes to load CSS, JavaScript and open and render HTML DOM.
Performancewise you can typically see a 2x to 15x faster performance when using a headless browser.
While Scraping Websites you don’t necessarily want to have to manually start up a website. So you can access the website headlessly and just scrape the HTML. You don’t need to render a Full Browser to do that.
Lot of developers use a Headless Browser for unit testing code changes for their websites and mobile apps. Being able to do all this from a command line without having to manually refresh or start a browser saves them lots and effort.
When You Might NOT Want to Use a Headless Browser
There can be number of reasons why you may opt to use a Real Browser instead of a Headless Browser. A couple of instances:
You need to mimic real users.
You need to visually see the test run.
If you need to do lots of debugging, headless debugging can be difficult.
Which headless browsers are better?
As you rightly pointed that ...the main difference is in the execution on GUI bases and non GUI bases(Headless)..., so from Testing Perspective a lot will depend on the Browser Engine implemented under the hood by any particular browser. For example, here are some of the Browser Engines which fully render web pages or run JavaScript in a virtual DOM.
Chromium Embedded Framework: CEF is a open source project based on the Google Chromium project with JavaScript support and BSD license.
Erik: Erik is a Headless Browser on top of Kanna and WebKit with Swift support and MIT license.
jBrowserDriver: jBrowserDriver is a Selenium-compatible Headless Browser which is WebKit-based and works with Selenium Server through Java binding support and Apache License v2.0 license.
PhantomJS: PhantomJS is a headless WebKit scriptable with a JavaScript API. It has fast and native support for various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG with JavaScript, Python, Ruby, Java, C#, Haskell, Objective-C, Perl, PHP and R(via Selenium) support and BSD 3-Clause license.
Splash: Splash is a javascript rendering service with an HTTP API. It's a lightweight browser with an HTTP API, implemented in Python using Twisted and QT with almost all the laungage binding arts and BSD 3-Clause license.
You can find a related discussion in Which drivers support “no-browser”/“headless” testing?

Performance/Load test using selenium webdriver prerecorded steps

I have been using selenium webdriver as my main method to do functional tests. So far its been working greate with our product.
I need to do some performance and/or load tests on the website, I was wondering if there is a tool that would incorporate my selenium tests or a tool which i can use with the recorded tests as the base.
Currently i am using selenium webdriver with C#
Any help is appreciated.
Given you have tests written in C# the most obvious way would be using Visual Studio Load Testing capabilities.
If you are looking for a free and open-source solution I would recommend going for Apache JMeter. JMeter has integration with Selenium via WebDriver Sampler plugin so you should be able to run your Selenium tests in multi-threaded manner. However you will need to convert your C# code into one of the WebDriver Sampler supported languages (default is JavaScript)
Remember that Selenium tests are very resource-intensive as real browsers consume a lot of CPU/RAM so the number of virtual users you will be able to mimic this way will be very limited. So recommended approach is creating main load on a HTTP protocol level and use one Selenium instance to check rendering speed while your application is under the load.
You can install WebDriver Sampler plugin using JMeter Plugins Manager.
Why do you need to run full browsers. HTTP layer tests are far simpler? You also should be looking at only a subset of business processes in performance which generate a preponderance of load.

Selenium vs HtmlUnit? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I am trying to understand testing framework better and been looking into Selenium. I've used HTMLUnit before, mainly when I needed to scrape some information off website or the likes.
In the context of writing test automation, what's the advantage / disadvantages of Selenium vs HTMLUnit? Looks to me Selenium is more complicated to set up than HTMLUnit, although at the same time there's a HTMLUnitDriver for Selenium which I think behave the exact same way as in HTMLUnit itself?
Selenium obviously provides more robust framework, it has the Selenium RC for pararel testing, it also has different browser drivers that can be used - although when you used the browser drivers, the test will actually open/close a browser application rather than headless.
May be I am not understanding Selenium correctly. Some directions and pointers would be great!
On another note - a separate question - I am also looking at doing automated testing on mobile browser, I see that Selenium has an IPhoneDriver for it, but then this is not a headless testing either as it requires actual iOS simulator.
Is there anyway to do headless testing on mobile sites? Would changing user-agent be sufficient? I've seen a couple posts around changing user-agent that seem to have their own challenges, eg. Set user-agent in Selenium RC
Thanks a lot!
well, would try to explain differences in detail.
Speaking about parallel testing, it better to use selenium grid.
Basic concept of selenium RC and selenium grid.
You can get into more details here
Some words about selenium webDriver:
The primary new feature in Selenium 2.0 is the integration of the WebDriver API. WebDriver is designed to providing an simpler, more concise programming interface along with addressing some limitations in the Selenium-RC API. Selenium-WebDriver was developed to better support dynamic web pages where elements of a page may change without the page itself being reloaded. WebDriver’s goal is to supply a well-designed object-oriented API that provides improved support for modern advanced web-app testing problems.
How Does WebDriver ‘Drive’ the Browser Compared to Selenium-RC?
Selenium-WebDriver makes direct calls to the browser using each browser’s native support for automation. How these direct calls are made, and the features they support depends on the browser you are using. Information on each ‘browser driver’ is provided later in this chapter.
For those familiar with Selenium-RC, this is quite different from what you are used to. Selenium-RC worked the same way for each supported browser. It ‘injected’ javascript functions into the browser when the browser was loaded and then used its javascript to drive the AUT within the browser. WebDriver does not use this technique. Again, it drives the browser directly using the browser’s built in support for automation.
WebDriver and the Selenium-Server
You may, or may not, need the Selenium Server, depending on how you intend to use Selenium-WebDriver. If you will be only using the WebDriver API you do not need the Selenium-Server. If your browser and tests will all run on the same machine, and your tests only use the WebDriver API, then you do not need to run the Selenium-Server; WebDriver will run the browser directly.
There are some reasons though to use the Selenium-Server with Selenium-WebDriver.
You are using Selenium-Grid to distribute your tests over multiple
machines or virtual machines (VMs).
You want to connect to a remote machine that has a particular browser
version that is not on your current machine.
You are not using the Java bindings (i.e. Python, C#, or Ruby) and
would like to use HtmlUnit Driver
Selenium-WebDriver’s Drivers
WebDriver is the name of the key interface against which tests should be written, but there are several implementations. These include:
HtmlUnit Driver
This is currently the fastest and most lightweight implementation of WebDriver. As the name suggests, this is based on HtmlUnit. HtmlUnit is a java based implementation of a WebBrowser without a GUI. For any language binding (other than java) the Selenium Server is required to use this driver.
Pros
Fastest implementation of WebDriver
A pure Java solution and so it is platform independent.
Supports JavaScript
Cons
Emulates other browsers’ JavaScript behaviour (see below)
JavaScript in the HtmlUnit Driver
None of the popular browsers uses the JavaScript engine used by HtmlUnit (Rhino). If you test JavaScript using HtmlUnit the results may differ significantly from those browsers.
When we say “JavaScript” we actually mean “JavaScript and the DOM”. Although the DOM is defined by the W3C each browser has its own quirks and differences in their implementation of the DOM and in how JavaScript interacts with it. HtmlUnit has an impressively complete implementation of the DOM and has good support for using JavaScript, but it is no different from any other browser: it has its own quirks and differences from both the W3C standard and the DOM implementations of the major browsers, despite its ability to mimic other browsers.
With WebDriver, we had to make a choice; do we enable HtmlUnit’s JavaScript capabilities and run the risk of teams running into problems that only manifest themselves there, or do we leave JavaScript disabled, knowing that there are more and more sites that rely on JavaScript? We took the conservative approach, and by default have disabled support when we use HtmlUnit. With each release of both WebDriver and HtmlUnit, we reassess this decision: we hope to enable JavaScript by default on the HtmlUnit at some point.
To investigate deeper into webDriver's setUp see this
From HtmlUnit documentation:
HtmlUnit is not a generic unit testing framework. It is specifically a way to simulate a browser for testing purposes and is intended to be used within another testing framework such as JUnit or TestNG.
So to conclude Selenium and HtmlUnit difference:
HtmlUnit is a java based implementation of a WebBrowser without a GUI and a way to simulate a browser for testing purposes and Selenium-WebDriver makes direct calls to the browser using each browser’s native support for automation. we can see that HtmlUnit provides API without GUI possibility for automation whereas WebDriver provides internal browsers' possibilities for automation.
Speaking about mobile automation,
Selenium also has an iPhone Driver
iPhone Driver wiki article
and Android Driver
Android Driver wiki article
See also this presentation
Unfortunately I can not give you my working experience evaluation of mobile drivers as I deal with web automation (no mobile). Also know that Cucumber (automation tool) is popular among mobile automators.
see this and this.
Hope it come a lil bit more clear for you now =)
Selenium and HTMLUnit are somewhat similar in concept, but Selenium is more mature/robust and has a lot more features.
Note that Selenium encompasses the recording (IDE) plugin for Firefox, which allows you to record tests and the RC/WebDriver automation framework which essentially drives a browser. The two can be used together to make test creation very easy.
The only advantage I could see to using HTMLUnit is that it is less resource intensive, so you could potentially run tests on less hardware, but with Selenium's parallel support even that isn't really true anymore.
When running tests from Jenkins overnight, you typically have no access to a windowing system such as X11 or Windows in which to run the web browser. I therefore see a benefit of using the HTMLUnit web driver in that case since it doesn't require access to a windowing system.
In my experience, HtmlUnit does a fine job with browsing automation, but might get a bit buggy when dealing with Javascript. I actually came to a case in which I wasn't able to automate an image download with HtmlUnit, and had to turn to Selenium, which performed beyond my expectations. The case is actually registered in an SO thread.
I have used Selenium WebDriver for automation.
There is a very easy method to causing the browser to be headless.
Simply apply ChromeOptions (In my case, other DriverOptions are available)
ChromeOptions("Headless")
There are many proficient methods that can come from using Options or Services, as another example
This will stop the Driver/CommandPrompt window from 'appearing' so it will remain 'silent' and unexposed.
ChromeDriverServices ("Silent")
At least at UX systems you can use for example Xvfb and point the browsers to that display to make them "headless"
See also http://infiniteundo.com/post/54014422873/headless-selenium-testing-with-firefox-and-xvfb
or How do I run Selenium in Xvfb?

How to setup Selenium RC to test any website on the internet?

I'm new in using Selenium.
Selenium IDE is a user-friendly firefox plugin. I have no problem in using it. However, I found that the documentation for other Selenium tools such as Selenium RC and Selenium Core is quite confusing for beginners. It seems that the author assume that the readers already have deep knowledge in using these tools.
For example, when I try to figure out how to setup Selenium RC to test a webserver, the only diagram i can find from the Selenium website is this:
http://www.sparksupport.com/blog/wp-content/uploads/2010/11/selenium-rc.png
From this diagram, i can't even see which one is the webserver under test and where should i install the Selenium components.
At first I thought this diagram is a bit weird and i should be able to get a better diagram from other websites. I was surprised to find that almost all Selenium RC setup diagram on the internet are similar to this diagram (clones). No one has ever attempted to create a different diagram or give more description for Selenium RC setup.
Appreciate if anyone can give me guidance on how to setup Selenium RC. The things that i want to know are:
Can i use Selenium RC to test any website on the Internet?
How to setup Selenium RC?
Is my current setup correct? My current setup is like this: In a LAN network which has access to the Internet, I have 3 servers. Server-1 comes with IE8, Server-2 comes with Firefox 3.6. Server-3 will be used as the Selenium RC server. So, Selenium RC in server-3 will remotely control server-1 and server-2 to start up IE and FF. Server-1 and 2 will use server-3 as the HTTP proxy to connect to any webserver on the Internet. If I want to test a website such as yahoo.com, I can write Selenium script and let it run in Server-3 to control the IE and FF in server-1 and 2.
This info is related to Selenium 1.
Selenium system consists of 3 parts:
selenium core - that is javascript library that will be used to simulate user actions
selenium RC - this is selenium-server.jar - mediation JETTY server that will receive requests from selenium client. Selenium Server RC (Remote Control) should be on the same machine where the Browser placed
Selenium client - java/ruby/... library that you will be use with your tests to communicate with Selenium RC.
It will be helpful if you provide language that you use for your tests and other technical details.
About your questions:
can
type in command line -> java selenium-server.jar
or you can use class SeleniumServer in your program
please use text formatting when ask questions.
server-1 will has IE8 and SeleiumServer
server-2 will has FF and yet one SeleniumServer
server-3 will has you client tests
FYI - you can run all together on one PC
The below diagram is of a web application test system that I've implemented on numerous occassions. This does not show you specifically details on installing Selenium RC, but it does show you, at a high level, all of the necessary system components and how they interface.
We hope you can use it to get ideas on how to implement your own systems using open source solutions like Selenium, MySQL and Perl.
Our team understands that not all web sites are created equal, and, in order for any automation initiative to be successful a thorough analysis must be performed of not only the web application, but the business as well. Since our client's QA team, while technically savvy, were not programmers we decided to implement a page object design pattern where all of the "magical selenium commands" were abstracted in a class and exposed to the test developers as methods they would call from their test scripts.
The resulting implementation, as seen in the diagram below, is currently deployed and keeping management and interested parties up to date on the status of key functional areas of the web site.
System Diagram - Click to View
In the coming weeks, we are going to be covering each implementation step in more detail. We look forward to any feedback!
Web and Mobile Automation Blog