My Selenium tests are being slowed by third-party scripts that are not necessary to the tests.
How can I block them? Preferably I'd like to block requests to everywhere but localhost.
Solutions offered elsewhere online are:
Block unwanted domains (e.g., *.facebook.com) by editing your hostfiles.
Route all your tests through BrowserMob which can be configured to filter requests.
Both options seemed like overkill to me. Editing the host files affects your whole system, and using BrowserMob introduces new problems.
Here's another way: Use a PAC file to configure the browser to connect to localhost directly, and attempt to connect to everything else through an unavailable proxy.
Selenium code (Java):
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
Proxy proxy = new Proxy();
proxy.setProxyType(Proxy.ProxyType.PAC);
proxy.setProxyAutoconfigUrl("http://localhost:8080/my-pac-file.pac");
capabilities.setCapability("proxy", proxy);
ChromeDriver driver = new ChromeDriver(INSTANCE, capabilities);
The PAC file:
function FindProxyForURL(url, host) {
if (host.toLowerCase() === "localhost"){
return "DIRECT"; // whitelisted
}
return "PROXY 127.0.0.1:9876"; // blocked (bad proxy)
}
Related
Background: Here is my situation, I have a website I access that requires me to go through a proxy. For day-to-day use, I installed the chrome extension Proxy switchyomega to switch between proxies based on a URL wildcard. For example, when I go to URL abc.xyz.com my company requires me to use a proxy at x.x.x.x:yyyy.
Issue I am trying to write an automation script using WebdriverIO (version 7.24.0) in standalone mode using chrome that will launch the website abc.xyz.com. The problem is when I run the script it is not going through the proxy, so my access is blocked. I have tried to configure a proxy in the capabilities but that does not seem to work, and I'm wondering if I'm doing something wrong in my configuration, which is below
const { remote } = require("webdriverio");
const browser = await remote({
capabilities: {
browserName: "chrome",
proxy: {
proxyType: "manual",
httpProxy: "x.x.x.x:yyyy"
}
},
});
await browser.url("abc.xyz.com");
As mentioned in normal use when I access URL abc.xyz.com it routes through the x.x.x.x:yyyy proxy and that is what I need to happen when running my script but the above configuration does not seem to do anything.
Any help would be appreciated.
I have Integrated the Selenium with JMeter to perform Performance Test, For this I have used Firefox driver Cofig and gecko driver. Script is executing successfully in local window machine But when trying to execute it on Linux server it giving the following error.
"The proxy moz-proxy://host:port is requesting a username and password. The site says: “LDAP”:"
Note:
gecko driver is compatible with Firefox version.
username and password passed while setting proxy on server, and also through JMeter command.
It looks like you're behind a corporate proxy server which requires authentication and unfortunately this is not something you can bypass using Firefox Driver Config
You will need to switch to JSR223 Sampler with Groovy language and instantiate the browser manually providing the proxy host/port/username/password/etc. Take a look at Proxy class JavaDoc
Example code:
import org.openqa.selenium.Proxy
import org.openqa.selenium.firefox.FirefoxDriver
import org.openqa.selenium.firefox.FirefoxOptions
Proxy proxy = new Proxy()
proxy.setHttpProxy("http://username:password#proxy-host:proxy.port")
proxy.setSslProxy("http://username:password#proxy-host:proxy.port")
FirefoxOptions options = new FirefoxOptions()
options.setCapability("proxy", proxy)
FirefoxDriver driver = new FirefoxDriver(options)
driver.get('http://example.com')
We are trying to run some automated tests for our Xamarin based iOS app using the Selenium IOSDriver within Visual Studio for Mac.
The error we are getting when we run them is (I've sanitised to remove the URL we are actually trying to send the request to).
A exception with a null response was thrown sending an HTTP request to the remote WebDriver server for URL https://<Test Host URL>. The status of the exception was NameResolutionFailure, and the message was: nodename nor servname provided, or not known nodename nor servname provided, or not known
Because our machines are behind a proxy we often see DNS errors like this when the code sending the request is unaware of the proxy.
We've tried various approached to setting the proxy, such as HttpClient.DefaultProxy and OpenQA.Selenium.Proxy but the error still persists.
Are there any other ways to tell Selenium that it is operating behind a proxy?
For anyone else struggling with this the answer seemed to be using a 'HttpCommandExecutor'
var uri = new Uri("https://hub.browserstack.com/wd/hub/");
var commandExecutor = new HttpCommandExecutor(uri, TimeSpan.FromSeconds(30))
{
Proxy = new WebProxy
{
Address = new Uri("http://your.proxy")
}
};
driver = new IOSDriver<IOSElement>(commandExecutor, options);
Running a test that has a long running (non-request/response action) along with polling to check the status. While this polling is going on, I'd like to make a few packets disappear. I keep seeing things that look like this might work with WebDriver only to come up short. Is there anyway to do this completely inside of selenium or do I have to go to a completely external proxy?
My thoughts were to act like an ad blocker in that I could watch what was being requested and refuse certain connections and return things like 502s or return nothing at all. But I'd like it to be under the control of the test, not an external setup.
You can manipulate requests by using a proxy. It's simplest to block requests. Depending on how your webapp is performing this might be a way to do it.
Check out Browsermob proxy usage:
Look here to get started: URL blacklisting with BrowserMobProxy in Robot Framework/Selenium?
// Start the server and get the selenium proxy object
ProxyServer server = new ProxyServer(proxy_port); // package net.lightbody.bmp.proxy
server.start();
server.setCaptureHeaders(true);
// Blacklist google analytics
server.blacklistRequests("https?://.*\\.google-analytics\\.com/.*", 410);
// Or whitelist what you need
server.whitelistRequests("https?://*.*.yoursite.com/.*. https://*.*.someOtherYourSite.*".split(","), 200);
Proxy proxy = server.seleniumProxy(); // Proxy is package org.openqa.selenium.Proxy
// configure it as a desired capability
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, proxy);
// start the driver ;
Webdriver driver = new FirefoxDriver(capabilities);
return driver;
But you should be able to dynamically configure the proxy to alternate request blocking.
I am able to create a Selenium proxy using BrowserMob everything work well on my my local PC. When I run the same code on a server (Windows Server 2008 R2 Standard) it errors our "cannot connect to tunnel".
I have tried different combinations of Chrome swithes like --ignore-certificate-errors,--user-data-dir=C:/temp/insecurechrome,--ignore-certificate-errors. I have ensured that .setTrustAllServer(true) is set. I have tried adjusting the Windows Firewal without any effect.
I will add my code I am using, however, it does work on my local PC, yet not on the server. I am hoping someone can suggest other setting on the server I can changer or something in my code I may have missed.
I first get a Chrome Browser message: wating for Proxy Tunnel. Several seconds later (15-20). I get the error: ERR_TUNNEL_CONNECTION_FAILED.
browserMobProxyServer = new BrowserMobProxyServer();
browserMobProxyServer.setTrustAllServers(true);
browserMobProxyServer.start(0);
port = browserMobProxyServer.getPort();
seleniumProxy = ClientUtil.createSeleniumProxy(browserMobProxyServer);
ChromeOptions options = new ChromeOptions();
options.addArguments("--proxy-server","--ignore-certificate-errors","--user-data-dir=C:/temp/insecurechrome");
Map<String, Object> prefs = new HashMap<String, Object>();
prefs.put("credentials_enable_service", false);
prefs.put("profile.password_manager_enabled", false);
options.setExperimentalOption("prefs", prefs);
PropertyConfigurator.configure("./resources/properties/log4j.properties");
DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
desiredCapabilities.setCapability(ChromeOptions.CAPABILITY, options);
desiredCapabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
//desiredCapabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true); //Has no effect
driverService = new ChromeDriverService.Builder().usingDriverExecutable(new File("./resources/driver/chromedriver.exe")).usingPort(Integer.parseInt(portRequested)).build();
driverService.start();
return new ChromeDriver((ChromeDriverService)driverService, desiredCapabilities);
I was able to figure out my own issue. There was an existing corporate proxy that was intercepting the traffic. This proxy has different protocols for servers and for users. When running on my PC it ran fine. When running my program on a server I needed to address proxy forwarding or chaining. I accomplished this by adding the below lines to my above code:
import java.net.InetSocketAddress;
...
...
InetSocketAddress x = new InetSocketAddress("proxy.example.com", 80);
browserMobProxyServer.setChainedProxy(x);