Selenium Proxy on Grid - selenium

I have this setup that is setting proxy ok in the local browser but when i try to use the grid the proxy will not be sent to the node:
var driver = new webdriver.Builder()
.withCapabilities(webdriver.Capabilities.firefox())
.setProxy(proxy.manual({ http : 'proxy:port',
https : 'proxy:port',
}))
.build();
Result : the browser proxy is - proxy:port ;
When i add :
var driver = new webdriver.Builder()
.withCapabilities(webdriver.Capabilities.firefox())
.setProxy(proxy.manual({ http : 'proxy:port',
https : 'proxy:port',
}))
.usingServer('http://hub:port/wd/hub')
.build();
The result : the browser proxy is - it showes me the ip of the hub.
Question : does anyone knows why the proxy set manualy is not sent to the hub and why the browser doesn't use it ? Or any other solution for this problem ?

This is the solution that worked :
var webdriver = require('selenium-webdriver'),
firefox = require('selenium-webdriver/firefox'),
proxy = require('selenium-webdriver/proxy')
driver = null,
profile = new firefox.Profile();
profile.setPreference("network.proxy.type", 1); // Manual proxy config
profile.setPreference("network.proxy.http", "proxy");
profile.setPreference("network.proxy.http_port", port);
profile.setPreference("network.proxy.ssl", "proxy");
profile.setPreference("network.proxy.ssl_port", port);
var opts = new firefox.Options();
opts.setProfile(profile);
var driver = new webdriver.Builder()
.withCapabilities(webdriver.Capabilities.chrome())
.setFirefoxOptions(opts);
.build();

Related

Selenium Proxy IP Address Configuration IP Not Correct

I have paid/rented a proxy server in brazil
String proxyAddress = "myusername:myuserpass123#196.18.199.51:15464"
proxy.setAutodetect(false);
proxy.setHttpProxy(proxyAddress);
proxy.setSslProxy(proxyAddress);
chromeOptions.setCapability(CapabilityType.PROXY, proxy);
WebDriver webDriver = new ChromeDriver(chromeOptions);
I m running the webdriver on my local computer and I am in Indonesia. When the chrome browser opens up, I can debug and made sure that capabilities were set correctly: I can see the manual proxy setting set to the correct address string above.
However, when webdriver opens https://api.ipify.org/?format=json, it still returns my IP in Indonesia. What am I Missing here? My expectation is because I had configured webdriver to be proxied by a server in Brazil, https://api.ipify.org/?format=json should return Brazilian IP address?
Using Selenium 4 BiDirectional API (https://www.selenium.dev/documentation/webdriver/bidirectional/bidi_api/)
Register Basic Auth.
Some applications make use of browser authentication to secure pages. With Selenium, you can automate the input of basic auth credentials whenever they arise.
//C#
//Console App .NET 6
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
Proxy proxy = new Proxy();
var proxyAddress = "address:port";
proxy.HttpProxy = proxyAddress;
proxy.SslProxy = proxyAddress;
ChromeOptions options = new ChromeOptions();
options.Proxy = proxy;
IWebDriver driver = new ChromeDriver(options);
NetworkAuthenticationHandler handler = new NetworkAuthenticationHandler()
{
UriMatcher = (d) => d.Host.Contains("your-domain.com"), // or set it `true` to enable proxy everywhere
Credentials = new PasswordCredentials("admin", "password")
};
INetwork networkInterceptor = driver.Manage().Network;
networkInterceptor.AddAuthenticationHandler(handler);
await networkInterceptor.StartMonitoring();
driver.Navigate().GoToUrl("https://api.ipify.org/?format=json");
await networkInterceptor.StopMonitoring();

Unable to Fetch Network Traffic Logs via BrowserMob Proxy with Selenium Webdriver

I am not able to Fetch Network Logs using Browsermob Proxy when set Http proxy , it will just create a har file but i am not able to see any logs inside the file
Below is my Code :-
String strFilePath = "./PerformanceLogs/PerformanceLogs.har";
String noProxy = "localhost, 127.0.0.1";
LegacyProxyServer server = new BrowserMobProxyServer();
server.start();
server.setCaptureHeaders(true);
server.setCaptureContent(true);
Proxy proxy = server.seleniumProxy().setHttpProxy(PROXY)
.setFtpProxy(PROXY)
.setSslProxy(PROXY)
.setNoProxy(noProxy);
DesiredCapabilities capabilities = DesiredCapabilities.firefox();
capabilities.setCapability(CapabilityType.PROXY, proxy);
WebDriver driver = new FirefoxDriver(capabilities);
server.newHar("NetworkLogs");
driver.navigate().to("https://url-");
driver.findElements(xpath="").click();
pMonitor = new Prism_Selenium_Monitor_Tab(driver);
pMonitor.prism_monitor_credentialDeploy(URL,excelfile[0], excelfile[4]);
Har har = server.getHar();
File harFile = new File(strFilePath);
har.writeTo(harFile);
server.stop();
can anyone please me in this?
Thanks in advance
For some reason you're starting LegacyProxy, try starting BrowserMobProxyServer, i.e.
BrowserMobProxyServer server = new BrowserMobProxyServer();
Default proxy type is HTTP, i.e. try to delete this part:
.setNoProxy(noProxy);
It's actually shouldn't be a problem, but in default case there is no need to name your HAR, i.e. this code is redundant:
server.newHar("NetworkLogs");
you may just:
server.newHar();

Selenium 2 chrome driver fails with WebException inside HttpCommandExecutor

I am trying to setup an automated test environment using
- TestStack.Seleno v0.8.2
- TestStack.BDDfy v4.0.0
- Selenium .NET WebDriver 2.43.0.0
- Chrome v38
- ChromeDriver v2.9
While I am able to establish initial session hand shake between chrome driver and chrome browser, subsequent calls to actual web application via chrome driver are failing with timeout exceptions.
Here is the code to instantiate a SelenoHost object :
var options = new ChromeOptions();
Instance.Run(configure => configure
.WithWebServer(new InternetWebServer(String.Format("http://{0}/portal", IISServerHost)))
.WithRemoteWebDriver(() => BrowserFactory.Chrome(options))
.UsingLoggerFactory(new ConsoleFactory()));
If I debug the above method call, it fails inside SelenoApplication Initialize method :
public void Initialize()
{
_initialised = true;
_logger.Debug("Starting Webserver");
WebServer.Start();
_logger.Debug("Browsing to base URL");
Browser.Navigate().GoToUrl(WebServer.BaseUrl); >>> this line fails inside HttpCommandExecutor.CreateResponse() method
}
Not able to figure out what obvious am i missing out here.
BTW, web application is hosted on IIS 7.5 and is configured for windows authentication.
Launching chrome with 'no-sandbox' resolved the issue.
Here is how the final configuration looks like :
var options = new ChromeOptions();
options.AddArgument("ignore-certificate-errors");
options.AddArgument("no-sandbox");
var driverService = ChromeDriverService.CreateDefaultService();
driverService.EnableVerboseLogging = (ChromeDriverVerboseLogigng == "true");
driverService.LogPath = ChromeDriverLogPath;
_SelenoHostLazy.Value.Run(configure => configure
.WithWebServer(new InternetWebServer(String.Format("http://localhost/portal", IISServerHost)))
.WithRemoteWebDriver(() => new ChromeDriver(driverService, options))
.UsingLoggerFactory(new ConsoleFactory()));

Selenium chrome driver socks proxy configuration

I am having troubles in setting socks proxy for chrome driver
Proxy proxy = new Proxy();
proxy.setProxyType(Proxy.ProxyType.MANUAL);
proxy.setAutodetect(false);
proxy.setSocksProxy(ProxyHelper.PROXY_HOST + ":" + ProxyHelper.PROXY_PORT);
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability(CapabilityType.PROXY, proxy);
WebDriver chromeDriver = new ChromeDriver(capabilities);
This configuration gives:
Exception in thread "main" org.openqa.selenium.WebDriverException: unknown error: cannot parse capability: proxy from unknown error: proxyType is 'manual' but no manual proxy capabilities were found
I think it expects me to fill http, ftp and ssl proxies. But if I fill them; error doesnt raise but my proxy does not work properly too as it tries to use it like http proxy rather than socks proxy.
What can I do?
ChromeOptions options = new ChromeOptions();
options.add_argument("--proxy-server=socks5://" + host + ":" + port);
WebDriver driver = new ChromeDriver(options);
Have you tried using this chromium arg?
--proxy-server="socks5://host:port"
from selenium.webdriver.firefox.options import Options as ff_options
random_proxy = "142.54.61.98:120"
options = ff_options()
firefox_capabilities = webdriver.DesiredCapabilities.FIREFOX
firefox_capabilities['marionette'] = True
firefox_capabilities['proxy'] = {
"proxyType": "MANUAL",
"httpProxy": random_proxy,
"ftpProxy": random_proxy,
"sslProxy": random_proxy
}
profile = webdriver.FirefoxProfile()
profile.set_preference("media.peerconnection.enabled", False)
profile.set_preference("media.navigator.enabled", False)
# profile.set_preference("general.useragent.override", user_agent)
profile.update_preferences()
driver = webdriver.Firefox(capabilities=firefox_capabilities, firefox_profile=profile,
firefox_options=options)

Unable to load resource due to SSL certificates using Selenium Webdriver with BrowserMob proxy to capture har file

I'm using Selenium Webdriver (Chromedriver) in Java, along with BrowserMob Proxy Server to capture HTTP traffic in a har file. I recently encountered a problem where sections of the website would not load, and I've narrowed it down to this error:
"Failed to load resource https://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js".
It seems that when using the proxy server, the Selenium driver can't access the SSL certificate for the https link. Here is a snippet of the code I am using:
ProxyServer server = new ProxyServer(4040);
server.start();
Proxy proxy = server.seleniumProxy();
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, proxy);
driver = new ChromeDriver(capabilities);
server.newHar("myHar");
Har har = server.getHar();
server.stop();
I have tried adding "capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);" and this solves the above problem, but only if I delete "capabilities.setCapability(CapabilityType.PROXY, proxy);" and then I am no longer able to capture the har file.
I tried switching to a firefox driver and setting up a firefox profile, but either I am not doing it properly or it won't work for my purposes either.
I have also tried setting up the cybervillainsCA certificate that comes with BrowserMob proxy in Trusted Root Certifaction Authorities, but that hasn't solved the issue either.
Does anyone know how to solve this in a way that I can collect HTTP traffic as well as successfully load the resource mentioned above?
What I ended up doing was scrapping the proxy entirely, and instead switched to using a Firefox profile with the Firebug and NetExport extensions in order to export the har file. This ended up significantly slowing down the tests, and exports a har for each page visited rather than for the entire session.
Code:
//Firefox profile
FirefoxProfile ffProfile = new FirefoxProfile();
ffProfile.addExtension(new File("firebug-1.11.4-fx.xpi"));
ffProfile.addExtension(new File("netExport-0.9b3.xpi"));
//Set default Firefox preferences
ffProfile.setPreference("app.update.enabled", false);
String domain = "extensions.firebug.";
//Set default Firebug preferences
ffProfile.setPreference(domain + "currentVersion", "1.11.4");
ffProfile.setPreference(domain + "allPagesActivation", "on");
ffProfile.setPreference(domain + "defaultPanelName", "net");
ffProfile.setPreference(domain + "net.enableSites", true);
//Set default NetExport preferences
ffProfile.setPreference(domain + "netexport.alwaysEnableAutoExport", true);
ffProfile.setPreference(domain + "netexport.autoExportToFile", true);
ffProfile.setPreference(domain + "netexport.showPreview", false);
ffProfile.setPreference(domain + "netexport.defaultLogDir", "string file path");
//WebDriver, instantiated outside the method
driver = new FirefoxDriver(ffProfile);
s = new WebDriverBackedSelenium(driver, "http://www.google.ca/");
I collected the har file after each page as follows:
HarFileReader r = new HarFileReader();
HarFileWriter w = new HarFileWriter();
int count = 1;
String allHars = "";
String harFolderPath = "file path for har";
File dir = new File(harFolderPath);
for (File child : dir.listFiles()) {
HarLog log = r.readHarFile(child);
File f = new File(harFolderPath + "\\test"+count+".txt");
w.writeHarFile(log, f);
allHars = allHars + readFileAsString(f.getPath());
count++;
}
FileUtils.cleanDirectory(dir);