I am a newbie in using selenium. When I use proxy chrome shows there is no internet connection. I have checked my internet connection and looked for possible solutions on the internet but failed. I also restarted my computer but it did not work.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from time import sleep
PROXY = '43.225.164.59:38829'
options = Options()
options.add_argument('--proxy-server=%s' % PROXY)
driver = webdriver.Chrome(options = options,
executable_path='C:\webscraping\chromedriver')
driver.get('https://whatismyipaddress.com/')
sleep(6)
driver.close()
The code is looking good, it should works.
Do you use proxy with authentification? If do, you should use another way to enable proxy - issue (Selenium doesn't support authentification of proxy).
You should also be sure, that proxy is working. Check it with curl:
If you're using public proxy:
curl https://2ip.ru/ -x "http://proxy_host:proxy_port"
If you're using private proxy with authentification:
curl https://2ip.ru/ -x "http://proxy_user:proxy_pwd#proxy_host:proxy_port"
Both of these commands will return your ip address (it should be equal to proxy address you use.
Related
Im writing an automated flow using Selenium and Java and I need to
connect via an authenticated HTTPS proxy using a "< username> " and "< password >".
Since Selenium doesn't support proxy authentication I'm using the standard technique of
running BrowserMobProxyServer and "chaining" the external proxy to it. While the code below works great with regular HTTP
For some reason it doesn't work with HTTPS and Im getting ERR_PROXY_CONNECTION_FAILED in my browser.
note that
"curl -v -x https://<username>:<password>#<proxy hostname>:<proxy HTTPS port> https://ipinfo.io"
works perfectly fine one my Ubuntu 22.04 LTS,
So I suspect that it is code error.
implementation 'org.seleniumhq.selenium:selenium-java:4.5.0'
implementation 'net.lightbody.bmp:browsermob-core:2.1.5'
public static BrowserMobProxyServer createLocalProxy(String hostname, String port,
String username, String password) {
BrowserMobProxyServer proxy = new BrowserMobProxyServer();
// Handling http and https URLs
proxy.setTrustAllServers(true);
// // remote proxy as added to the chain of locally running proxy server
proxy.setChainedProxy(new InetSocketAddress(hostname, Integer.parseInt(port)));
proxy.chainedProxyAuthorization(username, password, AuthType.BASIC);
proxy.setMitmManager(ImpersonatingMitmManager.builder().trustAllServers(true).build());
// This is a local proxy in JVM. Port is assigned automatically.
// It must be stopped using the stop() method before exiting.
proxy.start(0);
return proxy;
}
// proxy setup
BrowserMobProxy proxy =
createLocalProxy("<proxy hostname>", "<proxy HTTPS port>", "<user name>",
"<password>");
Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
seleniumProxy.setHttpProxy("localhost:" + proxy.getPort());
seleniumProxy.setSslProxy("localhost:" + proxy.getPort());
<some additional options here>
options.setProxy(seleniumProxy);
WebDriver driver = new ChromeDriver(options);
I'm running the Selenium on Kubernetes, behind the Ingress-Nginx controller.
One thing I noticed is when I try to see the live VNC through the Selenium Grid web, the browser keeps trying to connect the 4444 port of the Selenium grid server for the WebSocket connection.
Is there a way to make it point the 80 port instead?
I already tried to switch the port number for the Selenium router to 80 (SE_ROUTER_PORT to 80), but apparently, the WebSocket is still not working.
This is what I'm seeing on the browser console:
I already confirmed the functionality itself with the kube-proxy localhost proxying.
Thank you very much.
I got proxy from https://www.zyte.com/ with APIkey and followed their instruction(https://support.zyte.com/support/solutions/articles/22000203564-using-zyte-smart-proxy-manager-with-selenium), but the response time is too long due to it goes through Docker to connect the proxy.
Isn't there any other solution to connect the such proxy to my code? (host url, APIkey and port are given)
I am using Python, Scrapy, Selenium on Window 10
Is there a way to enable a proxy on running Google Chrome (for example after I log in to my mail)?
Yes this is possible:
PROXY = "proxy-here" # IP:PORT or HOST:PORT
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % PROXY)
chrome = webdriver.Chrome(chrome_options=chrome_options)
chrome.get("http://whatismyipaddress.com")
how do i set proxy for chrome in python webdriver
Running a Selenium 2 RemoteWebDriver server using java -jar selenium-server-standalone-2.15.0.jar.
I always get the error:
HTTP ERROR: 403
Forbidden for Proxy
RequestURI=/session
when connecting to it using the python WebDriver client:
import selenium.webdriver as webdriver
webdriver.Remote('http://localhost:4444', {})
or any other various RemoteWebDriver client I could find.
The solution was simple: Use the pathname /wd/hub
i.e.
import selenium.webdriver as webdriver
webdriver.Remote('http://localhost:4444/wd/hub', {})
Not a solution to the exactly Problem, but for people getting this error:
HTTP ERROR: 403
Forbidden for Proxy
RequestURI=/
Powered by Jetty://
This error appears e.g. if multiple instances of Selenium are running, so you need to shut it down by browsing to URL:
http://localhost:4444/selenium-server/driver/?cmd=shutDownSeleniumServer
If another instance was running there should appear okok
You have another process which is bound to same port i.e 4444.
Selenium Grid by default uses port:4444.
You either need to kill the process which is bound to port:4444 or else you need to use another port(below used 5555) for your hub.
Use following in command prompt:
java -jar selenium-server-standalone-2.15.0.jar -role hub -port 5555