I'm using selenium with phantomjs under Ubuntu 16.04. I want to make use of Tor proxy. There is, of course, possibility to set a proxy via host:port pair like so:
from selenium import webdriver
service_args = [
'--proxy=127.0.0.1:9050',
'--proxy-type=socks5',
]
driver = webdriver.PhantomJS(service_args=service_args)
But I wonder if there is a possibility to make phantomjs use a unix domain socket (configured in Tor, of course) instead of host:port pair.
Setting it simply like --proxy=/run/tor/tor.socket does not make the trick - the request to check.torproject.org shows that. Maybe there is some special syntax for that option?
Related
Hi Am usinf webdriver Io and trying to connect to selenium grid running as HTTPS. I have included the protocol:https option. I get the following error
RequestError: unable to verify the first certificate
If I add strictSSL: false option in wdio config, I am able to connect to selenium grid.
But I do not want to use strictSSL: false and instead want to send certificates and connect the the selenium server running in https mode.
Any pointers on how to do that will be much appreciated.
Thanks
I have Integrated the Selenium with JMeter to perform Performance Test, For this I have used Firefox driver Cofig and gecko driver. Script is executing successfully in local window machine But when trying to execute it on Linux server it giving the following error.
"The proxy moz-proxy://host:port is requesting a username and password. The site says: “LDAP”:"
Note:
gecko driver is compatible with Firefox version.
username and password passed while setting proxy on server, and also through JMeter command.
It looks like you're behind a corporate proxy server which requires authentication and unfortunately this is not something you can bypass using Firefox Driver Config
You will need to switch to JSR223 Sampler with Groovy language and instantiate the browser manually providing the proxy host/port/username/password/etc. Take a look at Proxy class JavaDoc
Example code:
import org.openqa.selenium.Proxy
import org.openqa.selenium.firefox.FirefoxDriver
import org.openqa.selenium.firefox.FirefoxOptions
Proxy proxy = new Proxy()
proxy.setHttpProxy("http://username:password#proxy-host:proxy.port")
proxy.setSslProxy("http://username:password#proxy-host:proxy.port")
FirefoxOptions options = new FirefoxOptions()
options.setCapability("proxy", proxy)
FirefoxDriver driver = new FirefoxDriver(options)
driver.get('http://example.com')
I am trying to set up sock5 proxy setup with firefox + python + selenium. I am having
IP and PORT of the proxy server. Even i added username and password it is giving pop up windows.
from selenium import webdriver
import time
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.proxy import *
profile = webdriver.FirefoxProfile()
profile.set_preference("network.proxy.type", 1)
profile.set_preference('network.proxy.ssl_port', int(PROXY_PORT))
profile.set_preference('network.proxy.ssl', PROXY_IP)
profile.set_preference("network.proxy.http", PROXY_IP)
profile.set_preference("network.proxy.http_port", int(PROXY_PORT))
profile.set_preference("network.proxy.no_proxies_on", 'localhost, 127.0.0.1')
profile.set_preference("network.proxy.socks_username", PROXY_USERNAME)
profile.set_preference("network.proxy.socks_password", PROXY_PASSWORD)
profile.update_preferences()
driver = webdriver.Firefox(firefox_profile=profile, executable_path="./geckodriver")
driver.get("https://www.whatismyip.com/")
time.sleep(10)
driver.close()
System Information:
Python 3.8.10
Ubuntu 20.04
Mozilla Firefox 99.0
That's just your proxy service, I had that same problem. I use a service called smartproxy.com. You can write your computer's IP address on the website for the authentication process. If the IP address that is running the code matches the IP address you listed on their site, there will be no pop up.
I do not know the proxy service you are using but it may have an authenticated method similar to the one I explained.
I am trying to setup the browserstack and the local testing.
I opened a tunnel using the browserstack local client like below:
./BrowserStackLocal.exe myCodeToken -proxyHost MY_PROXY_IP -proxyPort MY_PROXY_PORT -v -force -forcelocal
So it is forwardin all the trafic through my local network.
In my local /etc/hosts file I have entry like below:
127.0.0.1 dev.mysite.com
Then when I'm executing my simple selenium test the browserstack virtualmachine are able to access my domain name and everything works fine. But in some sections where I have external scripts/css/images etc. for example gooogle analitics browserstack doesn't have access to them (and to the internet as well).
The thing is that my local machine uses the MY_PROXY_IP:MY_PROXY_PORT to access the internet and it looks like the browserstack tries to forward all the trafic through my machine and it doesn't work.
Do you have any ideas how to resolve this ?
You can try removing the -forcelocal parameter, as that is responsible for routing all traffic via your machine. On doing this, any public (external) css/images would be resolved directly and not via your machine.
Alternatively, if the -forcelocal parameter is necessary, you can try using it with -only parameter. In this case, the requests for domain(s) mentioned under -only, would be routed via your machine whereas rest of the requests would be resolved publicly.
You can execute the Local Testing binaries as follows:
BrowserStackLocal.exe <automate-key> <the-proxy-parameters> -forcelocal -only host_name,port_no,ssl_flag
OR
BrowserStackLocal.exe <automate-key> <the-proxy-parameters> -forcelocal -only host1,port1,ssl_flag,host2,port2,ssl_flag (For multiple hosts)
More details on different parameters that can be used while setting up the Local Testing connection available here - https://www.browserstack.com/local-testing#modifiers
I'm using Docker Selenium images to run browser nodes, repo is available here https://github.com/SeleniumHQ/docker-selenium. There is no documentation on how config.json can be used to provide proxy values.
I'm using Selenium version 2.44.0.
In my infrastructure, there are certain assets that are sourced from a location which needs proxy configuration on browser to access them. I'm trying to setup proxy on a chrome node. According to this documentation here, proxy can be set like following:
java -jar selenium-2.44.0.jar -Dhttp.proxyHost=192.168.2.10 -Dhttp.proxyPort=80
My proxy does not require, usename and password hence I have ignored those values.
What is not clearly mentioned on SeleniumHQ documentation is, whether it needs proxy configuration on both hub or nodes or just the nodes. I've tried different combinations but haven't worked for me.
Actual commands i'm running are:
For Hub:
java -jar /opt/selenium/selenium-server-standalone.jar -role hub -Dhttp.proxyHost=192.168.2.10 -Dhttp.proxyPort=80 -hubConfig /opt/selenium/hubconfig.json
When I run command above, I can see -D* values being displayed on console config.
For node:
xvfb-run --server-args=":99.0 -screen 0 1360x1020x24 -ac +extension RANDR" java -jar /opt/selenium/selenium-server-standalone.jar -Dhttp.proxyHost=192.168.2.10 -Dhttp.proxyPort=80 -role node -hub http://$HUB_PORT_4444_TCP_ADDR:$HUB_PORT_4444_TCP_PORT/grid/register -nodeConfig /opt/selenium/config.json
When I run this command I can see the proxy values on console again but I the assets are not loaded by the browser.
Also, on a side note it seems like this can be done on developers side (in java code) but I'm keen to solve it on my (operations) side.
Thanks - here is what we got:
First you need a way to verify your settings made it into the browser.
chrome://net-internals/proxyservice.config#proxy
The actual command line instruction is:
/chromeexec --proxy="http=http://proxyserver:port/;https=http://proxyserver:port/"
Note that the colons will blow up on the bash command line if you don't use double-quotes.
Now if you're sending this from the Webdriver Java code programmatically - you'll need to escape out the double quotes - so the proxy server setting in Java may look like:
org.openqa.selenium.Proxy proxy = new org.openqa.selenium.Proxy();
proxy.setHttpProxy("\"http://proxyserver:port/\"")
Alternatively you can pass this in as an execution parameter.
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability("chrome.switches", Arrays.asList("--proxy \"http=http://proxyserver:port/;https=http://proxyserver:port/\""));
WebDriver driver = new ChromeDriver(capabilities);
Now your origin question was about accessing external resources with the proxy. What we did (similar to your question) was to pass a proxy exception for the site we were hitting so the external resources would go via the proxy.
So then you add an exception for your primary website - assuming the resource is 10.1.10.5 then it looks like:
--proxy-bypass-list=10.1.10.5
Which then we do in code as:
capabilities.setCapability("chrome.switches", Arrays.asList("--proxy=\"http=http://proxyserver:port/;https=http://proxyserver:port/\"" "--proxy-bypass-list=10.1.10.5"));
Note that setting username and password is a bug in Chrome. (Please star it if this holds you up. )
If you need a username and password, then the solution is a PAC file.
The syntax is:
--proxy-pac-url=file:///proxy.pac
The file format looks like:
if (host == "mylocalserver.com")
{
return 'DIRECT';
} else {
return return "PROXY wcg2.example.com:8080 ";
}
For the case of usernames and passwords in proxy settings, note the following:
Proxy auto-configuration files do not support hard-coded usernames and passwords. There's good reasoning behind this too, since providing support for hard-coded credentials would open up significant security holes, as anybody would be able to easily view the required credentials to access the proxy.
Rather configure the proxy as a transparent proxy, that way you won't need a username and password. You mention in one of your comments that the proxy server is located outside your LAN, which is why you require authentication. However, most proxies support rules based on the source IP, in which case it's a simple matter of only allowing requests originating from your corporate network.
The original proxy auto-config specification was originally drafted by Netscape in 1996. The original specification is no longer available directly, but you can still access it using The Wayback Machine's archived copy. The specification hasn't changed much, and is still largely the same as it was originally. You'll see the specification is quite simple, and that there is no provision for hard-coded credentials.
To solve this problem - you can use this tool:
https://github.com/sjitech/proxy-login-automator
This tool can create a local proxy and automatically inject user/password to real proxy server. Support PAC script.