Selenium RemoteWebDriver and Windows Authentication Dialogs - selenium

I've seen this question has been asked a few times, and lots of solutions get suggested - but none of them seem to work for the RemoteWebDriver (ie: using Selenium Grid). They're usually centered around using the local ChromeDriver/FirefoxDriver/IEDriver classes.
I am using the .NET bindings, by the way :).
What I want to do is fairly simple (in terms of requirement). I have a Selenium Server setup, and am currently using the RemoteWebDriver to perform automated UI tests on various sites. This setup is working fine.
However, some sites use NTLM/Windows Authentication, and we need to start writing automated tests for these. However, as far as I can tell, there is no solution for this.
I have seen the following "solutions", but - unless someone can correct me - they either don't work consistently, or will not work using RemoteWebDriver:
Using the IAlert functionality (like here). However, this isn't implemented in the .NET bindings, and doesn't work for all browsers as far as I can tell.
Using the Robot API to interact with the popup (like here). But this is for running on your local machine, and not supported by RemoteWebDriver.
Using AutoIt to do a similar thing to the Robot API. However, this won't work using RemoteWebDriver.
Passing the credentials in the URL (eg: http://username:password#example.com). However, this doesn't work for Windows Authentication - just normal HTTP Basic Authentication.
I can't actually see any other solutions, unless anyone else can help?
A workaround currently is to log onto the Selenium server, go to the sites in each browser, and save the credentials. But this isn't ideal, and adds a level of manual interaction to each test.
Any help would be appreciated :).

It appears I have found my own solution - use a proxy which adds the NTLM negotiation/authorisation automatically. Pretty simple to setup :).
http://cntlm.sourceforge.net/

Related

Difference between running via Firefox versus Selenium WebDriver (GeckoDriver)

I'm trying to scrape my own banking information by automating the process using Selenium in Ruby.
I'm running into a bizarre situation where performing the exact same sequence in the browser (whether just the normal browser or private/incognito) works fine, but when I try to log in under a Selenium-controlled browser I get back a strange 500 error from the server.
I've noticed the browser console logs also look different in terms of certain logging messages related to cookies, JS errors, libraries being loaded, etc.
I have found an answer on SO mentioning one possible difference in Chrome being a specific "cdc" string that might be detectable, but is there some kind of corresponding difference in Firefox/Geckodriver that could be used to detect the fact that I'm trying to automate the browser?
I'm not really sure where to look, because my understand was that running via Selenium should basically have identical behaviour to running via the browser itself.
Would love some guidance on what mechanisms may be in play to explain the difference in behaviour!

How to specify Selenium Webdriver to use current browser settings for internet

Using Selenium's WebDriver, with PhantomJSDriver, I am trying to do headless browser testing. It works fine when connected to internet WITHOUT a proxy. But when the connection to internet is via an authenticated proxy, it fails. I want to deploy this program to multiple user sites, which might be connected to internet with or without proxy, and in case of proxy, it might be authenticated or unauthenticated.
Is there a way to tell Selenium Webdriver to use the "current" browser's internet connection settings? Please note I am using phantomjs.
Thanks,
abbas
There is some more simple but very effective solution, that I've used when battling with similar issues an year ago.
Do you have these issues when using other *Drivers? If not - my proposal is to use your implementation of any other *Driver that works fine and after it passes authentication just cast it to PhantomJSDriver. Please note that is just possible workaround if your TestFramework hierarchy is built to support such an action.
In addition you can consider the following - when I used such Polymorphism the difference is speed. For FirefoxDriver and PhantomJSDriver it wasn't such a pain and if you can use it only for authentication it will not slow you down noticeable.
I'm not sure that I can help you with my solution, but it will not hurt to try it.

Capture web driver network traffic across all browsers

I want to capture all the network calls from Web Driver in Java. I am not doing any UI testing, just testing JS execution and, requests and responses of some network calls.
I tried using Browser Mob as is suggested in most forums, but I need it to work across all browsers. It worked flawlessly with Firefox, but I was facing some issues with the others. Safari driver doesn't event support a Proxy capability.
I don't want to use Fiddler as it involves some manual steps around invoking and storing the calls. Whereas, Browser Mob being an in-code proxy can be integrated in a more smoother fashion.
I also tried using the RC-like package included in Selenium standalone server package. But, I have some HTTPS calls and some nested iframes in cross domains. I am particularly interested in some cross domain POST call and it doesn't work out that well. Also, people keep saying it's not recommended to use that package.
So, I had a solution where we can use a standalone proxy server running on a machine. Using host entries, we'll point Web Driver to hit the proxy instead of the actual server. The proxy will record all the incoming calls and route them to the actual server host. Later, I can make a request to the proxy which will return me all the calls it intercepted. I am not sure whether it's still called a proxy or a router.
I came across TCPmon, but it's no longer being supported. Does anyone know some similar tools that could run on Unix systems or any alternate solutions?
We modified the Fiddler rules script to include a new exec action. If you use their native script editor, it also provide auto complete features and we were comfortably able to get around it. The syntax is similar to that of JavaScript.
The Fiddler package comes with a ExecActions.exe which can be used to pass console arguments to a running Fiddler instance using the command prompt.
The code we wrote processed all the sessions captured by Fiddler and wrote it to a file in a custom JSON format and later used GSON to deserialize it.
Please let me know, if you want further details.

Does anyone know browser emulators?

Over 2 years I tested web application with help Selenium framework. I know the best design is testing on VM.
The only one downside of this - it's very slow testing. Why?
browser only gets so much memory, if you will run several instances.
site coud be very slow.
connections can be very slow.
Would be great if there was a framework that emulated the browser (engine/core) correctly and can provide some results (api) for surf on the page.
I don't mean to simulate just on the one browser with different version (like IE). I mean to simulate for all browsers with very popular and newest version.
Does anyone know a framework/tool that can do it?
Thank you.
You can try PhantomJS for example.
From their page:
PhantomJS is a headless WebKit with JavaScript API. It has fast and
native support for various web standards: DOM handling, CSS selector,
JSON, Canvas, and SVG.
You can use it in combination with Jasmine (as well as several other frameworks) for testing.
However the selection of available engines is limited to WebKit. I doubt that Selenium will be easy to replace. By the way it looks like Selenium will probably become a W3C standard over the next years.
You can also run Selenium with Xvfb - I use it to execute test on remote server and it is going very well.

running autoit/autohotkey scripts from server? or other automation alternatives?

I have some scripts (AutoIt) browsing YouTube for list of trending videos etc. It involves no mouse clicks (just keystrokes). The script takes a long time to finish and I can't use my PC during this time (it needs the window activated to work on it).
Is there anything I can do about this? Can these scripts run from a server or some stuff like that?
I've run into a similar problem: got to run automation with AutoIt on a Windows Server and the whole thing had got to be headless. Using Remote Desktop simply didin't work because then I'd had to keep a client opened and maximized all the time.
Short solution: install a VNC server in the Windows Server, open a client from another computer, log in and close the client. As the AutoIt script was being started by a Jenkins job, before closing the client the Jenkins applet had to be started via web interface.
By the way, I've had this idea from this post: How to run remote headless GUI automation.
As Johannes said, AutoIt probably wouldn't be suitable (and likewise AutoHotkey), but you could check out the many GUI and web testing frameworks that exist for other languages. With some of those, you can run a "headless" browser (a program that navigates the web just like a browser, but has no visible window); or you can run a standard browser on a virtual display like a Xvfb X11 server. This would be easiest if the server (or wherever it's going to run) is running a Unix-like OS, but it may be possible with Windows too.
Selenium Webdriver seems to be a very popular choice for scripting and testing actual browsers. It's natively Java but has bindings for languages like Ruby. It can also hook into something called HtmlUnit, which is also Java; that's one of the more popular headless browsers. Another (a relative newcomer) is phantom.js, which is in Javascript but (again) has bindings for other languages.
As far as I know this will not work unless the user account is logged on. You could try to see if you could convert it to an exe and run this as a service, but even then I don't think this will work. Let me know if you found out!
You can either:
Hide your window (SetWindowState #SW_HIDE) or something like that...
and use ControlClicks (if the they are original controls!)
or
Hide your window and use SendControl
or
use SendKeepActive
or
use OLEObjects like ie.au3 for automation.
Good Luck