Running a test that has a long running (non-request/response action) along with polling to check the status. While this polling is going on, I'd like to make a few packets disappear. I keep seeing things that look like this might work with WebDriver only to come up short. Is there anyway to do this completely inside of selenium or do I have to go to a completely external proxy?
My thoughts were to act like an ad blocker in that I could watch what was being requested and refuse certain connections and return things like 502s or return nothing at all. But I'd like it to be under the control of the test, not an external setup.
You can manipulate requests by using a proxy. It's simplest to block requests. Depending on how your webapp is performing this might be a way to do it.
Check out Browsermob proxy usage:
Look here to get started: URL blacklisting with BrowserMobProxy in Robot Framework/Selenium?
// Start the server and get the selenium proxy object
ProxyServer server = new ProxyServer(proxy_port); // package net.lightbody.bmp.proxy
server.start();
server.setCaptureHeaders(true);
// Blacklist google analytics
server.blacklistRequests("https?://.*\\.google-analytics\\.com/.*", 410);
// Or whitelist what you need
server.whitelistRequests("https?://*.*.yoursite.com/.*. https://*.*.someOtherYourSite.*".split(","), 200);
Proxy proxy = server.seleniumProxy(); // Proxy is package org.openqa.selenium.Proxy
// configure it as a desired capability
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, proxy);
// start the driver ;
Webdriver driver = new FirefoxDriver(capabilities);
return driver;
But you should be able to dynamically configure the proxy to alternate request blocking.
Related
I have Integrated the Selenium with JMeter to perform Performance Test, For this I have used Firefox driver Cofig and gecko driver. Script is executing successfully in local window machine But when trying to execute it on Linux server it giving the following error.
"The proxy moz-proxy://host:port is requesting a username and password. The site says: “LDAP”:"
Note:
gecko driver is compatible with Firefox version.
username and password passed while setting proxy on server, and also through JMeter command.
It looks like you're behind a corporate proxy server which requires authentication and unfortunately this is not something you can bypass using Firefox Driver Config
You will need to switch to JSR223 Sampler with Groovy language and instantiate the browser manually providing the proxy host/port/username/password/etc. Take a look at Proxy class JavaDoc
Example code:
import org.openqa.selenium.Proxy
import org.openqa.selenium.firefox.FirefoxDriver
import org.openqa.selenium.firefox.FirefoxOptions
Proxy proxy = new Proxy()
proxy.setHttpProxy("http://username:password#proxy-host:proxy.port")
proxy.setSslProxy("http://username:password#proxy-host:proxy.port")
FirefoxOptions options = new FirefoxOptions()
options.setCapability("proxy", proxy)
FirefoxDriver driver = new FirefoxDriver(options)
driver.get('http://example.com')
We are trying to run some automated tests for our Xamarin based iOS app using the Selenium IOSDriver within Visual Studio for Mac.
The error we are getting when we run them is (I've sanitised to remove the URL we are actually trying to send the request to).
A exception with a null response was thrown sending an HTTP request to the remote WebDriver server for URL https://<Test Host URL>. The status of the exception was NameResolutionFailure, and the message was: nodename nor servname provided, or not known nodename nor servname provided, or not known
Because our machines are behind a proxy we often see DNS errors like this when the code sending the request is unaware of the proxy.
We've tried various approached to setting the proxy, such as HttpClient.DefaultProxy and OpenQA.Selenium.Proxy but the error still persists.
Are there any other ways to tell Selenium that it is operating behind a proxy?
For anyone else struggling with this the answer seemed to be using a 'HttpCommandExecutor'
var uri = new Uri("https://hub.browserstack.com/wd/hub/");
var commandExecutor = new HttpCommandExecutor(uri, TimeSpan.FromSeconds(30))
{
Proxy = new WebProxy
{
Address = new Uri("http://your.proxy")
}
};
driver = new IOSDriver<IOSElement>(commandExecutor, options);
I am able to create a Selenium proxy using BrowserMob everything work well on my my local PC. When I run the same code on a server (Windows Server 2008 R2 Standard) it errors our "cannot connect to tunnel".
I have tried different combinations of Chrome swithes like --ignore-certificate-errors,--user-data-dir=C:/temp/insecurechrome,--ignore-certificate-errors. I have ensured that .setTrustAllServer(true) is set. I have tried adjusting the Windows Firewal without any effect.
I will add my code I am using, however, it does work on my local PC, yet not on the server. I am hoping someone can suggest other setting on the server I can changer or something in my code I may have missed.
I first get a Chrome Browser message: wating for Proxy Tunnel. Several seconds later (15-20). I get the error: ERR_TUNNEL_CONNECTION_FAILED.
browserMobProxyServer = new BrowserMobProxyServer();
browserMobProxyServer.setTrustAllServers(true);
browserMobProxyServer.start(0);
port = browserMobProxyServer.getPort();
seleniumProxy = ClientUtil.createSeleniumProxy(browserMobProxyServer);
ChromeOptions options = new ChromeOptions();
options.addArguments("--proxy-server","--ignore-certificate-errors","--user-data-dir=C:/temp/insecurechrome");
Map<String, Object> prefs = new HashMap<String, Object>();
prefs.put("credentials_enable_service", false);
prefs.put("profile.password_manager_enabled", false);
options.setExperimentalOption("prefs", prefs);
PropertyConfigurator.configure("./resources/properties/log4j.properties");
DesiredCapabilities desiredCapabilities = new DesiredCapabilities();
desiredCapabilities.setCapability(ChromeOptions.CAPABILITY, options);
desiredCapabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
//desiredCapabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true); //Has no effect
driverService = new ChromeDriverService.Builder().usingDriverExecutable(new File("./resources/driver/chromedriver.exe")).usingPort(Integer.parseInt(portRequested)).build();
driverService.start();
return new ChromeDriver((ChromeDriverService)driverService, desiredCapabilities);
I was able to figure out my own issue. There was an existing corporate proxy that was intercepting the traffic. This proxy has different protocols for servers and for users. When running on my PC it ran fine. When running my program on a server I needed to address proxy forwarding or chaining. I accomplished this by adding the below lines to my above code:
import java.net.InetSocketAddress;
...
...
InetSocketAddress x = new InetSocketAddress("proxy.example.com", 80);
browserMobProxyServer.setChainedProxy(x);
My Selenium tests are being slowed by third-party scripts that are not necessary to the tests.
How can I block them? Preferably I'd like to block requests to everywhere but localhost.
Solutions offered elsewhere online are:
Block unwanted domains (e.g., *.facebook.com) by editing your hostfiles.
Route all your tests through BrowserMob which can be configured to filter requests.
Both options seemed like overkill to me. Editing the host files affects your whole system, and using BrowserMob introduces new problems.
Here's another way: Use a PAC file to configure the browser to connect to localhost directly, and attempt to connect to everything else through an unavailable proxy.
Selenium code (Java):
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
Proxy proxy = new Proxy();
proxy.setProxyType(Proxy.ProxyType.PAC);
proxy.setProxyAutoconfigUrl("http://localhost:8080/my-pac-file.pac");
capabilities.setCapability("proxy", proxy);
ChromeDriver driver = new ChromeDriver(INSTANCE, capabilities);
The PAC file:
function FindProxyForURL(url, host) {
if (host.toLowerCase() === "localhost"){
return "DIRECT"; // whitelisted
}
return "PROXY 127.0.0.1:9876"; // blocked (bad proxy)
}
Environment:
- Selenium 2.39 Standalone Server
- PHP 5.4.11
- PHPUnit 3.7.28
- Chrome V31 & ChromeDriver v2.7
I'm testing a website,which invokes a lot of Advertisement Systems,such as Google AD.
The browser takes a lot of time to connect to external AD links , even all the elements of the page has already been loaded.
If my internet network was not fast when I ran my tests on a webpage,
Selenium would wait for a very long time ,since the AD links responsed slowly.
Under this condition ,Selenium usually waits for over 60 seconds, and throws a timeout exception.
I'm not sure how Senelium works, but it seems that Selenium has to wait for a sign of webpage's full loading, then pulls the DOM to find elements.
I want to make selenium operate elements without waiting for connectiong to external AD links.
Is there a way to do that ? Thank you very much.
I would suggest that you could make use of a proxy. Browsermob integrates well with selenium, very easy to use it:
// start the proxy
ProxyServer server = new ProxyServer(4444);
server.start();
// get the Selenium proxy object
Proxy proxy = server.seleniumProxy();
// This line will automatically return http.200 for any request going to google analytics
server.blacklistRequests("https?://.*\\.google-analytics\\.com/.*", 200);
// configure it as a desired capability
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, proxy);
// start the browser up
WebDriver driver = new FirefoxDriver(capabilities);
I'm not sure how Senelium works, but it seems that Selenium has to
wait for a sign of webpage's full loading, then pulls the DOM to find
elements.
It is pretty much like this. The default loading strategy is "NORMAL" which means:
NORMAL of type DOMString
The remote end MUST wait until the "document.readyState" of the frame currently handling commands equals "complete", or there are no
more outstanding network requests other than XMLHttpRequests.
I finally found a simple solution for my condition.
I decide to block these Ad requests and tried some firewall and proxy softwares,for example,
comodo,privatefirewall, etc.
comodo is too heavy and complex ,privatefirewall doesn't support wildcards, and firewall would interrupt tests. At last I choosed a proxy software CCproxy. Trial Version is enough.
I create a rule for localhost ,to make it can request my test website domain only, and all other requests are rejected.
Running a test costs about 1-2 minutes before and only 30 seconds now ,it's apparently more stable and fast without connecting to the useless Ad links.
Here're configuration steps:
1.launch CCproxy with Administor privilege( you should set it using Adminisrator in the file property)
2.click Options, select AutoStartup,select AutoDetected for Local IP Address. click OK.
3.create a txt file ,input your domains,like " *.rong360.com*;*.rong360.*; "
4.click Account, select PermitOnly for Permit Category;
click New, input 127.0.0.1 for IP Address/Range;
select WebFilter,click the E button at right side to create a filter;
click the ... button,select the text file you create at Step3,
select PermittedSites. click OK
click OK.
5.click OK to return to the main UI of CCproxy.
6.launch IE and config the local proxy with 127.0.0.1:808
other browsers will use this config automatically too.
now you can run the tests again , you'll feel better if have same condition :)