Selenium Proxy IP Address Configuration IP Not Correct - selenium

I have paid/rented a proxy server in brazil
String proxyAddress = "myusername:myuserpass123#196.18.199.51:15464"
proxy.setAutodetect(false);
proxy.setHttpProxy(proxyAddress);
proxy.setSslProxy(proxyAddress);
chromeOptions.setCapability(CapabilityType.PROXY, proxy);
WebDriver webDriver = new ChromeDriver(chromeOptions);
I m running the webdriver on my local computer and I am in Indonesia. When the chrome browser opens up, I can debug and made sure that capabilities were set correctly: I can see the manual proxy setting set to the correct address string above.
However, when webdriver opens https://api.ipify.org/?format=json, it still returns my IP in Indonesia. What am I Missing here? My expectation is because I had configured webdriver to be proxied by a server in Brazil, https://api.ipify.org/?format=json should return Brazilian IP address?

Using Selenium 4 BiDirectional API (https://www.selenium.dev/documentation/webdriver/bidirectional/bidi_api/)
Register Basic Auth.
Some applications make use of browser authentication to secure pages. With Selenium, you can automate the input of basic auth credentials whenever they arise.
//C#
//Console App .NET 6
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
Proxy proxy = new Proxy();
var proxyAddress = "address:port";
proxy.HttpProxy = proxyAddress;
proxy.SslProxy = proxyAddress;
ChromeOptions options = new ChromeOptions();
options.Proxy = proxy;
IWebDriver driver = new ChromeDriver(options);
NetworkAuthenticationHandler handler = new NetworkAuthenticationHandler()
{
UriMatcher = (d) => d.Host.Contains("your-domain.com"), // or set it `true` to enable proxy everywhere
Credentials = new PasswordCredentials("admin", "password")
};
INetwork networkInterceptor = driver.Manage().Network;
networkInterceptor.AddAuthenticationHandler(handler);
await networkInterceptor.StartMonitoring();
driver.Navigate().GoToUrl("https://api.ipify.org/?format=json");
await networkInterceptor.StopMonitoring();

Related

How to Configure the Ports for Communication between Selenium Server and the Browsers

I'm using Selenium Standalone Server 3.141.59 https://www.seleniumhq.org/download
In my code, when a WebDriver is created the Selenium server debugs something like: Starting ChromeDriver on port 28208
Is it possible to configure a range of ports (e.g., 28000-28100) that are allowed to be used by the Selenium server?
Use below code to configure chrome to run on other then default port.
int desiredPortNo = 22300;
ChromeDriverService service = new ChromeDriverService.Builder().usingDriverExecutable(new File("chrome_driver_path")).usingPort(desiredPortNo).build();
WebDriver driver = new ChromeDriver(service);
Update
To use with RemoteWebDriver :
int desiredPortNo = 22300;
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("...", true);
ChromeDriverService service = new ChromeDriverService.Builder()
.usingDriverExecutable(new File("driver_path")).usingPort(desiredPortNo)
.build();
service.start();
WebDriver driver = new RemoteWebDriver(service.getUrl(),capabilities);
driver.get("site_url");

Unable to Fetch Network Traffic Logs via BrowserMob Proxy with Selenium Webdriver

I am not able to Fetch Network Logs using Browsermob Proxy when set Http proxy , it will just create a har file but i am not able to see any logs inside the file
Below is my Code :-
String strFilePath = "./PerformanceLogs/PerformanceLogs.har";
String noProxy = "localhost, 127.0.0.1";
LegacyProxyServer server = new BrowserMobProxyServer();
server.start();
server.setCaptureHeaders(true);
server.setCaptureContent(true);
Proxy proxy = server.seleniumProxy().setHttpProxy(PROXY)
.setFtpProxy(PROXY)
.setSslProxy(PROXY)
.setNoProxy(noProxy);
DesiredCapabilities capabilities = DesiredCapabilities.firefox();
capabilities.setCapability(CapabilityType.PROXY, proxy);
WebDriver driver = new FirefoxDriver(capabilities);
server.newHar("NetworkLogs");
driver.navigate().to("https://url-");
driver.findElements(xpath="").click();
pMonitor = new Prism_Selenium_Monitor_Tab(driver);
pMonitor.prism_monitor_credentialDeploy(URL,excelfile[0], excelfile[4]);
Har har = server.getHar();
File harFile = new File(strFilePath);
har.writeTo(harFile);
server.stop();
can anyone please me in this?
Thanks in advance
For some reason you're starting LegacyProxy, try starting BrowserMobProxyServer, i.e.
BrowserMobProxyServer server = new BrowserMobProxyServer();
Default proxy type is HTTP, i.e. try to delete this part:
.setNoProxy(noProxy);
It's actually shouldn't be a problem, but in default case there is no need to name your HAR, i.e. this code is redundant:
server.newHar("NetworkLogs");
you may just:
server.newHar();

Browsermob proxy server crashes websites

trying to use browsermob proxy server by falling code:
final int port = 9000;
server = new ProxyServer(port);
server.start();
final DesiredCapabilities dc = new DesiredCapabilities();
dc.setCapability(CapabilityType.PROXY, server.seleniumProxy());
setName("test");
FirefoxBinary binary = new FirefoxBinary(new File("C:\\Program Files (x86)\\ff21\\firefox.exe"));
File profileDir = new File("C:\\Users\\arno\\Documents\\profiles\\firefox21.default");
FirefoxProfile profile = new FirefoxProfile(profileDir);
driver = new FirefoxDriver(binary, profile, dc);//;
server.newHar("monitis");
but it crashes the website: see the capture .
I'm using the fallowing code to find out what is going on(getting the url and the http status )
Har har = server.getHar();
for(HarEntry entry : har.getLog().getEntries()){
System.out.println(entry.getRequest().getUrl() +": " + entry.getResponse().getStatus());
}
and it gives this result:
mysite/files/js/numeral.min.js: 200
mysite/js/94842541.js: 200
mysite/files/css/page-home.min.css?v=6: -999
mysite/files/css/ui.min.css?v=18: -999
mysite/files/js/ui.min.js?also=jquery.selectric.min.js,jquery.checkradios.min.js,index.min.js&v=53: -999
Try using the non-deprecated BrowserMobProxyServer class instead of the legacy ProxyServer implementation. Make sure you're using the latest version of BMP and the browsermob-core-littleproxy module as well.
it turned out, that in Firefox settings, in network settings tab, I need to set up the option "auto detect proxy server for this network"

Selenium 2 chrome driver fails with WebException inside HttpCommandExecutor

I am trying to setup an automated test environment using
- TestStack.Seleno v0.8.2
- TestStack.BDDfy v4.0.0
- Selenium .NET WebDriver 2.43.0.0
- Chrome v38
- ChromeDriver v2.9
While I am able to establish initial session hand shake between chrome driver and chrome browser, subsequent calls to actual web application via chrome driver are failing with timeout exceptions.
Here is the code to instantiate a SelenoHost object :
var options = new ChromeOptions();
Instance.Run(configure => configure
.WithWebServer(new InternetWebServer(String.Format("http://{0}/portal", IISServerHost)))
.WithRemoteWebDriver(() => BrowserFactory.Chrome(options))
.UsingLoggerFactory(new ConsoleFactory()));
If I debug the above method call, it fails inside SelenoApplication Initialize method :
public void Initialize()
{
_initialised = true;
_logger.Debug("Starting Webserver");
WebServer.Start();
_logger.Debug("Browsing to base URL");
Browser.Navigate().GoToUrl(WebServer.BaseUrl); >>> this line fails inside HttpCommandExecutor.CreateResponse() method
}
Not able to figure out what obvious am i missing out here.
BTW, web application is hosted on IIS 7.5 and is configured for windows authentication.
Launching chrome with 'no-sandbox' resolved the issue.
Here is how the final configuration looks like :
var options = new ChromeOptions();
options.AddArgument("ignore-certificate-errors");
options.AddArgument("no-sandbox");
var driverService = ChromeDriverService.CreateDefaultService();
driverService.EnableVerboseLogging = (ChromeDriverVerboseLogigng == "true");
driverService.LogPath = ChromeDriverLogPath;
_SelenoHostLazy.Value.Run(configure => configure
.WithWebServer(new InternetWebServer(String.Format("http://localhost/portal", IISServerHost)))
.WithRemoteWebDriver(() => new ChromeDriver(driverService, options))
.UsingLoggerFactory(new ConsoleFactory()));

Unable to load resource due to SSL certificates using Selenium Webdriver with BrowserMob proxy to capture har file

I'm using Selenium Webdriver (Chromedriver) in Java, along with BrowserMob Proxy Server to capture HTTP traffic in a har file. I recently encountered a problem where sections of the website would not load, and I've narrowed it down to this error:
"Failed to load resource https://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js".
It seems that when using the proxy server, the Selenium driver can't access the SSL certificate for the https link. Here is a snippet of the code I am using:
ProxyServer server = new ProxyServer(4040);
server.start();
Proxy proxy = server.seleniumProxy();
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, proxy);
driver = new ChromeDriver(capabilities);
server.newHar("myHar");
Har har = server.getHar();
server.stop();
I have tried adding "capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);" and this solves the above problem, but only if I delete "capabilities.setCapability(CapabilityType.PROXY, proxy);" and then I am no longer able to capture the har file.
I tried switching to a firefox driver and setting up a firefox profile, but either I am not doing it properly or it won't work for my purposes either.
I have also tried setting up the cybervillainsCA certificate that comes with BrowserMob proxy in Trusted Root Certifaction Authorities, but that hasn't solved the issue either.
Does anyone know how to solve this in a way that I can collect HTTP traffic as well as successfully load the resource mentioned above?
What I ended up doing was scrapping the proxy entirely, and instead switched to using a Firefox profile with the Firebug and NetExport extensions in order to export the har file. This ended up significantly slowing down the tests, and exports a har for each page visited rather than for the entire session.
Code:
//Firefox profile
FirefoxProfile ffProfile = new FirefoxProfile();
ffProfile.addExtension(new File("firebug-1.11.4-fx.xpi"));
ffProfile.addExtension(new File("netExport-0.9b3.xpi"));
//Set default Firefox preferences
ffProfile.setPreference("app.update.enabled", false);
String domain = "extensions.firebug.";
//Set default Firebug preferences
ffProfile.setPreference(domain + "currentVersion", "1.11.4");
ffProfile.setPreference(domain + "allPagesActivation", "on");
ffProfile.setPreference(domain + "defaultPanelName", "net");
ffProfile.setPreference(domain + "net.enableSites", true);
//Set default NetExport preferences
ffProfile.setPreference(domain + "netexport.alwaysEnableAutoExport", true);
ffProfile.setPreference(domain + "netexport.autoExportToFile", true);
ffProfile.setPreference(domain + "netexport.showPreview", false);
ffProfile.setPreference(domain + "netexport.defaultLogDir", "string file path");
//WebDriver, instantiated outside the method
driver = new FirefoxDriver(ffProfile);
s = new WebDriverBackedSelenium(driver, "http://www.google.ca/");
I collected the har file after each page as follows:
HarFileReader r = new HarFileReader();
HarFileWriter w = new HarFileWriter();
int count = 1;
String allHars = "";
String harFolderPath = "file path for har";
File dir = new File(harFolderPath);
for (File child : dir.listFiles()) {
HarLog log = r.readHarFile(child);
File f = new File(harFolderPath + "\\test"+count+".txt");
w.writeHarFile(log, f);
allHars = allHars + readFileAsString(f.getPath());
count++;
}
FileUtils.cleanDirectory(dir);