Proxy issue while running script - selenium

I'm trying to use Selenium Grid2 to run concurrent Watir WebDriver scripts. I have the started the hub successfully, as I can open it up in my browser and I've spawned two nodes, each of which show in the console.
I started the node by doing this:
java -jar selenium-server-standalone-2.46.0.jar -role node -hub http://localhost:4444/grid/register -browser browserName=WatirWebDriver
I then attempted to run a test script:
require 'watir-webdriver'
caps = Selenium::WebDriver::Remote::Capabilities.ie
caps.platform = :WINDOWS
caps[:name] = "WatirWebDriver"
browser = Watir::Browser.new(
:remote,
:url => "http://xxxxxxxx:5555",
:desired_capabilities => caps)
browser.goto "http://www.google.com"
*where xxxx is local IP"
This returns a 403 Proxy error.
I can run this script basic script locally just fine in which I've just created a new IE browser instance and then told it to go to google. However, I did try setting http_proxy with my company's proxy information and that didn't work either. That however gave me an access denied error when I ran the script both locally and through the node. However, I know the creds are correct.
Thanks in advance for any help you can provide.

My issue was that I needed to be pointing to /wd/hub foor the url: http://xxxxxxxx:5555/wd/hub
Once I did that, it worked. Should have done more research before posting question...

Related

Unable to start ChromeDriver in CI environment

I am using Selenium with ChromeDriver in .NET Core. All tests work fine locally, however fail in TeamCity with the following error:
Starting ChromeDriver 80.0.3987.106 (f68069574609230cf9b635cd784cfb1bf81bb53a-refs/branch-heads/3987#{#882}) on port 24272
Only local connections are allowed.
Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
...
OneTimeSetUp: OpenQA.Selenium.WebDriverException : Cannot start the driver service on http://localhost:24272/
After the tests have finished, chromedriver.exe isn't terminated. There is nothing in the Windows event log or further details in the build logs, so I'm now stabbing in the dark.
I am using the following arguments, some added out of desperation:
var options = new ChromeOptions();
options.AddArgument("--headless");
options.AddArgument("--remote-debugging-port=9222");
options.AddArgument("--no-sandbox");
options.AddArgument("--no-first-run");
// See https://bugs.chromium.org/p/chromium/issues/detail?id=737678
options.AddArgument("--disable-gpu");
Driver = new ChromeDriver(options);
Interestingly, the remote debugging port != the port in the first error. Not sure if this means anything. The Chromedriver documentation suggests that options are added without leading --, but removing this doesn't seem to have any effect.
FWIW the TeamCity agent is running as a local Windows service account on Windows Server 2008 R2 (yes, I know).

Selenium ChromeDriver Failed to load resource: net::ERR_CONNECTION_CLOSED

I'm using Codeception to run some acceptance tests using WebDriver and a Docker Selenium standalone server. I noticed that after one of my tests threw an error I get this logged:
[Selenium browser Logs]
13:59:52.345 SEVERE - https://ssl.google-analytics.com/ga.js - Failed to load resource: net::ERR_CONNECTION_CLOSED
Funny thing is that there isn't any google analytics on the page at all (or the app for that matter). I'm guessing that there is some anonymous usage analytics but can't tell whose it is, whether it's Selenium, Codeception or it's included libs, or even the ChromeDriver for Selenium. I run the Selenium server with --net=host and maybe that has something to do with it.
Anyone know how to turn these analytics off or let Codeception ignore failed external resources?
add the following entry to your local hosts file,
127.0.0.1 www.google-analytics.com
127.0.0.1 google-analytics.com
127.0.0.1 ssl.google-analytics.com
This works for all browsers, regardless of whether they support plugins. As long as you don't run a web server on your local machine, these connections are instantly rejected and so don't take very long to fail.
You could also try 0.0.0.0 (never personally tested by me though).
0.0.0.0 www.google-analytics.com
0.0.0.0 google-analytics.com
0.0.0.0 ssl.google-analytics.com
PS: 0.0.0.0, a null route, is better. If you're running your own web server you'll start to get 404s if your route to localhost.

Couldn't connect to selenium server to execute a webdriverIO test

i,m having a runtime error when i execute a webdriverIO test by console.
this is the error Couldn't connect to selenium server to execute a webdriverIO test.
I can access to my selenium server by browser (http://localhost:4444/wd/hub/static/resource/hub.html) but its still not working.
Please, can anyone help me.
thanks a lot for the help.
Have you tried this solution?
https://github.com/webdriverio/webdriverio/issues/602
Specifically, adding:
var options = {
desiredCapabilities: {
browserName: 'chrome'
},
host: 'localhost',
port: 4444
};
Worked for me.
Very long time after, but I faced this issue with Appium and I'd like to share what I find to solve it in case someone else face it with Appium.
Open Appium server application and start the server with host: 127.0.0.1 and port: 4723.
Open your Emulator or connect device.
Run again the instruction to execute it.
As per the docs:
Handling the Selenium server is out of scope of the actual WebdriverIO project.
So we need to either go with raw ChromeDriver or spin up the Selenium grid. When running with the latter and setting
WebDriver: {
url: '<your-vaue-here>',
browser: 'chrome',
host: '127.0.0.1',
port: 4444,
...
don't forget to disable the codecept.config.js section, like this:
services: [
['']
],
...
Otherwise both servers will try to use http://localhost:4444.
Caveat:
When executing codeceptJS tests in parallel via run-workers I observe flaky behaviour as some threads actually kill the Selenium grid while others are still running. To resolve this issue, disable the services section and run a dedicated Grid server as initial step in your CI pipeline (assuming this is the place you actually need the parallelism).

How to run selenium chrome nodes using proxy?

I'm using Docker Selenium images to run browser nodes, repo is available here https://github.com/SeleniumHQ/docker-selenium. There is no documentation on how config.json can be used to provide proxy values.
I'm using Selenium version 2.44.0.
In my infrastructure, there are certain assets that are sourced from a location which needs proxy configuration on browser to access them. I'm trying to setup proxy on a chrome node. According to this documentation here, proxy can be set like following:
java -jar selenium-2.44.0.jar -Dhttp.proxyHost=192.168.2.10 -Dhttp.proxyPort=80
My proxy does not require, usename and password hence I have ignored those values.
What is not clearly mentioned on SeleniumHQ documentation is, whether it needs proxy configuration on both hub or nodes or just the nodes. I've tried different combinations but haven't worked for me.
Actual commands i'm running are:
For Hub:
java -jar /opt/selenium/selenium-server-standalone.jar -role hub -Dhttp.proxyHost=192.168.2.10 -Dhttp.proxyPort=80 -hubConfig /opt/selenium/hubconfig.json
When I run command above, I can see -D* values being displayed on console config.
For node:
xvfb-run --server-args=":99.0 -screen 0 1360x1020x24 -ac +extension RANDR" java -jar /opt/selenium/selenium-server-standalone.jar -Dhttp.proxyHost=192.168.2.10 -Dhttp.proxyPort=80 -role node -hub http://$HUB_PORT_4444_TCP_ADDR:$HUB_PORT_4444_TCP_PORT/grid/register -nodeConfig /opt/selenium/config.json
When I run this command I can see the proxy values on console again but I the assets are not loaded by the browser.
Also, on a side note it seems like this can be done on developers side (in java code) but I'm keen to solve it on my (operations) side.
Thanks - here is what we got:
First you need a way to verify your settings made it into the browser.
chrome://net-internals/proxyservice.config#proxy
The actual command line instruction is:
/chromeexec --proxy="http=http://proxyserver:port/;https=http://proxyserver:port/"
Note that the colons will blow up on the bash command line if you don't use double-quotes.
Now if you're sending this from the Webdriver Java code programmatically - you'll need to escape out the double quotes - so the proxy server setting in Java may look like:
org.openqa.selenium.Proxy proxy = new org.openqa.selenium.Proxy();
proxy.setHttpProxy("\"http://proxyserver:port/\"")
Alternatively you can pass this in as an execution parameter.
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability("chrome.switches", Arrays.asList("--proxy \"http=http://proxyserver:port/;https=http://proxyserver:port/\""));
WebDriver driver = new ChromeDriver(capabilities);
Now your origin question was about accessing external resources with the proxy. What we did (similar to your question) was to pass a proxy exception for the site we were hitting so the external resources would go via the proxy.
So then you add an exception for your primary website - assuming the resource is 10.1.10.5 then it looks like:
--proxy-bypass-list=10.1.10.5
Which then we do in code as:
capabilities.setCapability("chrome.switches", Arrays.asList("--proxy=\"http=http://proxyserver:port/;https=http://proxyserver:port/\"" "--proxy-bypass-list=10.1.10.5"));
Note that setting username and password is a bug in Chrome. (Please star it if this holds you up. )
If you need a username and password, then the solution is a PAC file.
The syntax is:
--proxy-pac-url=file:///proxy.pac
The file format looks like:
if (host == "mylocalserver.com")
{
return 'DIRECT';
} else {
return return "PROXY wcg2.example.com:8080 ";
}
For the case of usernames and passwords in proxy settings, note the following:
Proxy auto-configuration files do not support hard-coded usernames and passwords. There's good reasoning behind this too, since providing support for hard-coded credentials would open up significant security holes, as anybody would be able to easily view the required credentials to access the proxy.
Rather configure the proxy as a transparent proxy, that way you won't need a username and password. You mention in one of your comments that the proxy server is located outside your LAN, which is why you require authentication. However, most proxies support rules based on the source IP, in which case it's a simple matter of only allowing requests originating from your corporate network.
The original proxy auto-config specification was originally drafted by Netscape in 1996. The original specification is no longer available directly, but you can still access it using The Wayback Machine's archived copy. The specification hasn't changed much, and is still largely the same as it was originally. You'll see the specification is quite simple, and that there is no provision for hard-coded credentials.
To solve this problem - you can use this tool:
https://github.com/sjitech/proxy-login-automator
This tool can create a local proxy and automatically inject user/password to real proxy server. Support PAC script.

Unable to set proxy :selenium.common.exceptions.WebDriverException:Access Denied

I want to use Selenium Webdriver and I am unable to do so because when I run my code, I get the following exception.
My code is very basic and as follows.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.google.com.bh")
assert "Google" in driver.title
driver.close()
Exception Message
selenium.common.exceptions.WebDriverException: Message: '<HTML><HEAD>\n<TITLE>Access Denied</TITLE>\n</HEAD>\n<BODY>\n<FONT face="Helvetica">\n<big><strong></strong></big><BR>\n</FONT>\n<blockquote>\n<TABLE border=0 cellPadding=1 width="80%">\n<TR><TD>\n<FONT face="Helvetica">\n<big>Access Denied (authentication_failed)</big>\n<BR>\n<BR>\n</FONT>\n</TD></TR>\n<TR><TD>\n<FONT face="Helvetica">\nYour credentials could not be authenticated: "Credentials are missing.". You will not be permitted access until your credentials can be verified.\n</FONT>\n</TD></TR>\n<TR><TD>\n<FONT face="Helvetica">\nThis is typically caused by an incorrect username and/or password, but could also be caused by network problems.\n</FONT>\n</TD></TR>\n<TR><TD>\n<FONT face="Helvetica" SIZE=2>\n<BR>\nFor assistance, contact your network support team.\n</FONT>\n</TD></TR>\n</TABLE>\n</blockquote>\n</FONT>\n</BODY></HTML>\n'
It opens firefox but after that it is unable to connect to google or any other local sites.
The exception is at driver = webdriver.Firefox()
I googled around and I followed the link on SO.
But unfortunately I still get the same error.
I cannot run as the root user. I changed my proxy settings and set No Proxy element for localhost as well as mentioned in the link.
I am using Python 2.7 and have installed selenium 2.31 version.
I also tried setting proxy.
myProxy = "*********:8080"
proxy = Proxy({
'proxyType': ProxyType.MANUAL,
'httpProxy': myProxy,
'ftpProxy': myProxy,
'sslProxy': myProxy,
'noProxy': 'localhost,127.0.0.1,*.abc'
})
driver = webdriver.Firefox(proxy=proxy)
I also tried to set the proxy to system's proxy i.e., in the above code, 'proxyType': ProxyType.SYSTEM
But it again gives the above exception message.
is there a place where I have to set my username and password?
Any help would be appreciated!
Remove proxy settings from all the browsers on the system manually. I had IE, Firefox and Google Chrome.
When I removed the proxy settings of all the browsers and enabled the proxy only on Firefox,it worked without giving any errors. I do not know the exact reason why this works like this, may be got to do with the registry settings on windows which I am not sure about.
After doing the above said, I ran the basic code and it worked fine.
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.google.com.bh")
assert "Google" in driver.title
driver.close()
I didn't set the proxy explicitly also. By default, it had taken the system's proxy settings. Hope this would help others facing similar issue.