selenium proxy configuration multiple times in same browser - selenium

I opened a browser(any) with selenium tool and applying proxy to that browser by this below posted code, below is for Firefox
//LINE 1 FirefoxProfile profile = new FirefoxProfile();
//LINE 2 profile.setPreference("network.proxy.http", configuration
.getProxyConfiguration().getHostname());
//LINE 3 profile.setPreference("network.proxy.http_port", configuration
.getProxyConfiguration().getPort());
//LINE 4 profile.setPreference("network.proxy.type", configuration
.getProxyConfiguration().getType().toInt());
//LINE 5 return new FirefoxDriver(profile);
Now, I want to apply another proxy configuration for the same browser(Because, If I use another browser, session will be get change, So.... I want to apply my changes to that browser itself). How to apply my proxy configuration to the same browser. When I use same code I've to return driver which uses "NEW". I showed in my code(//LINE 5). Please help me to solve this issue.
Thanks:
Ramakrishna K.C

You can create Proxy with Kind = ProxyKind.System
new Proxy { Kind = ProxyKind.System};
Then update the internet settings in the registry
var proxyServer = string.Format("http={0};https={0}", ipAddressAndPort);
var proxyEnable = enableProxy ? 1 : 0;
const string subKeyPath = #"Software\Microsoft\Windows\CurrentVersion\Internet Settings";
using (var subKey = Registry.CurrentUser.CreateSubKey(subKeyPath))
{
if (subKey == null)
{
throw new Exception(string.Format("Failed to create or open subKey. SubKeyPath: {0} ", subKeyPath));
}
subKey.SetValue("ProxyServer", proxyServer, RegistryValueKind.String);
subKey.SetValue("ProxyEnable", proxyEnable, RegistryValueKind.DWord);
}

otherwise use PAC file in firefox profile.
http://findproxyforurl.com/

Related

Additional chrome options for headless mode with Kameleo

Been testing the Kameleo.LocalApiClient now whole day. Noticed something strange.
var client = new KameleoLocalApiClient(new Uri(KameleoBaseUrl));
client.SetRetryPolicy(null);
// Search Chrome Base Profiles
var baseProfileList = await client.SearchBaseProfilesAsync("desktop", "windows", "chrome", "ru-ru");
Random rnd = new Random();
var baseId = rnd.Next(1, baseProfileList.Count);
// Create a new profile with recommended settings
// Choose one of the Firefox BaseProfiles
// You can set your proxy up in the setProxy method
var requestBody = BuilderForCreateProfile
.ForBaseProfile(baseProfileList[baseId].Id)
.SetRecommendedDefaults()
.SetProxy("http", new Server("zproxy.lum-superproxy.io", 22225, "lum-customer-joshua901-zone-zone2", "5inhblkgrzxj"))
//.SetProxy("http", new Server("zproxy.lum-superproxy.io", 22225, "lum-customer-joshua901-zone-russiashared-ip-181.214.180.215", "lqoz0ee2hbvb"))
.SetLauncher("chrome")
//.SetScreen("--window-size", 500, 783)
.Build();
var profile = await client.CreateProfileAsync(requestBody);
// Start the browser
await client.StartProfileAsync(profile.Id);
// Connect to the running browser instance using WebDriver
var uri = new Uri($"{KameleoBaseUrl}/webdriver");
var opts = new ChromeOptions();
opts.AddAdditionalOption("kameleo:profileId", profile.Id.ToString());
opts.AddExcludedArgument("enable-automation");
opts.AddArgument("--window-size=500,783");
opts.AddArgument("disable-infobars");
opts.AddArgument("--incongito");
opts.AddArgument("--disable-extensions");
opts.AddArgument("disable-gpu");
opts.AddArgument("headless");
opts.AddArgument("--ignore-certificate-errors");
opts.AddArgument("no-sandbox");
opts.AddArgument("--silent-launch");
opts.AddArgument("--no-startup-window");
//var webdriver = new RemoteWebDriver(uri, opts);
_driver = new RemoteWebDriver(uri, opts)
I want to add my additional ChromeOptions to my driver, especially be able to run things in "headless" mode.
Even if i define the ChromeOptions above and create the RemoteWebDriver with these options, the chrome browser does still popup and not run as headless.
How come and how do i add my additional options?
The browser is started by Kameleo
The RemoteWebDriver is not starting any browser, just connecting to the already started browser
Because of these, you cannot pass any arguments to the browser with the RemoteWebDriver's options.
There is way to provide additional web driver options. There is a POST /profiles/{guid}/start endpoint which can process additional arguments, options, and preferences when Kameleo is starting the browser. See the documentation or see the example code.
await client.StartProfileWithWebDriverSettingsAsync(profile.Id, new WebDriverSettings
{
Arguments = new List<string> { "mute-audio" },
Preferences = new List<Preference>
{
new Preference("profile.managed_default_content_settings.images", 2),
new Preference("profile.password_manager_enabled.images", 2),
},
AdditionalOptions = new List<Preference>
{
new Preference("pageLoadStrategy", "eager"),
}
});
There are arguments that are not supported and that may cause issues so it is better to contact the team about the arguments.
For example:
opts.AddArgument("--disable-extensions");
This cannot be added, since Kameleo's extension is needed in the browser. If you remove it, the spoofing won't work.
I see you want to use this flag as well:
opts.AddArgument("disable-gpu");
You can simply set WebGL to Block. IT will result in the same.
Currently, there is no opportunity to start the browser in headless mode. But there will be a solution soon.

How to run Selenium test using Edge headless

Some time ago, I wrote some Selenium unit tests and these used the Firefox browser (headless). After a few years, I have returned to these (and cannot now remember how they worked) and would like to update them to use the Edge browser (in headless mode). How do I do this ? Specifically, when I ask the IWebDriver to "Navigate", how does it know the root URL - specifically the port ? I presume that my URL would be "http://localhost:nnnn/blah...", but how do I determine the port (which I also presume I need) ?
My code is based upon the following:
protected IWebDriver GetBrowserEdge(bool headless = true)
{
var service = SeleniumTools.EdgeDriverService.CreateChromiumService();
service.UseVerboseLogging = true;
var options = new SeleniumTools.EdgeOptions { PageLoadStrategy = PageLoadStrategy.Normal, UseChromium = true };
if (headless) options.AddArgument("headless");
return new SeleniumTools.EdgeDriver(service, options);
}
I wasn't using "CreateChromiumService" before (and I note it is disposable - which gives me a design issue), but I note that the service it returns has a "port" property. Do I need to leverage that in my "IWebDriver.Navigate" calls ?
I am using Edge v92.0.902.73 and Microsoft.Edge.SeleniumTools v3.141.2

How to disable SmartScreen (safebrowsing in edge) on C# Selenium Edge Chromium?

I'm trying to run some selenium C# end-to-end tests on Edge Chromium browser. One of the tests does a download check, basically it downloads a xml file and check whether it exists in the downloaded location.
I'm using Microsoft.Edge.SeleniumTools EdgeOptions to construct options for the EdgeDriver.
But the issue right now is that Edge blocks downloads.
Tried different sorts of things but none of them worked.
Same issue can be solved on Chrome by disabling "safebrowsing" on UserProfilePreferences in ChromeOptions.
I know for a fact that SmartScreen does the blocking, if that is so is there any profile preference that I can use to disable SmartScreen ?
Or any other workaround to force download without the block would be very helpful.
For testing purposes, I suggest you create HashMap objects and try to set Edge preferences like below.
static void Main(string[] args)
{
HashMap<String, Object> edgePrefs = new HashMap<String, Object>();
edgePrefs.put("profile.default_content_settings.popups", 0);
edgePrefs.put("profile.default_content_setting_values.notifications", 2);
edgePrefs.put("profile.default_content_setting_values.automatic_downloads" , 1);
edgePrefs.put("profile.content_settings.pattern_pairs.*,*.multiple-automatic-downloads",1);
var options = new EdgeOptions();
egdeOptions.setExperimentalOption("prefs",edgePrefs);
options.UseChromium = true;
options.BinaryLocation = #"C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe"; // Here add the Edge browser exe path.
var driver = new EdgeDriver(#"Edge_driver_path_here...", options); // Here modify the selenium web driver path.
driver.Navigate().GoToUrl("http://website_URL_here...");
driver.FindElementById("lnk1").Click();
}
If the issue persists then I suggest you set the default download directory like below.
options.AddUserProfilePreference("download.default_directory", #"D://Folder1");
See whether it help to download the file.
I was trying to find the answer with the usage of W3C WebDriver Protocol, but it seems like the only possible way is to turn off XML Edge SmartScreen policies using Windows Registry:
public void SetEdgeXmlDownloadPolicy()
{
var keyName = #"HKEY_CURRENT_USER\Software\Policies\Microsoft\Edge\ExemptDomainFileTypePairsFromFileTypeDownloadWarnings";
var valueName = "AllowXmlDownload";
var valueData = #"{""domains"": ["" * ""], ""file_extension"": ""xml""}";
var currentValue = Registry.GetValue(keyName, valueName, string.Empty).ToString();
if (currentValue == string.Empty || !currentValue.Contains("xml"))
Registry.SetValue(keyName, valueName, valueData, RegistryValueKind.String);
}
Credits to this workaround goes to IndianCodeMaster

How to configure and setup HTMLUNIT with Selenium while using it in C#?

I setup selenium remote driver and run the selenium server.The selenium server running correctly and while I try to run my code using :
var remoteServer = new Uri("http://127.0.0.1:4444/wd/hub");
DesiredCapabilities desiredCapabilities = DesiredCapabilities.Firefox();
desiredCapabilities.IsJavaScriptEnabled = true;
myDriver = new RemoteWebDriver(remoteServer, desiredCapabilities, new TimeSpan(0,1, 30));
No error throws in the CMD log and elements can find properly, but headache comes while I try to run this using the below code :
var remoteServer = new Uri("http://127.0.0.1:4444/wd/hub");
DesiredCapabilities desiredCapabilities = DesiredCapabilities.HtmlUnit();
desiredCapabilities.IsJavaScriptEnabled = true;
myDriver = new RemoteWebDriver(remoteServer, desiredCapabilities, new TimeSpan(0, 1, 30));
in log it throws lots of error and while I try to find any element, timeout exception showing in log.
My test code is in below :
myDriver.Manage().Window.Maximize();
myDriver.Navigate().GoToUrl(tollFreeURL);
IWebElement planClick = myDriver.FindElement(By.ClassName("trial"));
planClick.Click();
IWebElement startPlan = myDriver.FindElement(By.Id("choose2000"));
startPlan.Click();
IWebElement selectValue = myDriver.FindElement(By.Name("AreaCode"));
var selectElement = new SelectElement(selectValue);
selectElement.SelectByValue("800");
IWebElement selectNumber = myDriver.FindElement(By.XPath("//*[#id='divList']/div[1]"));
tollFreeNumber = selectNumber.Text;
tollFreeNumber = stringConvert.StringRefiner(tollFreeNumber, " ");
tollFreeNumber = stringConvert.StringRefiner(tollFreeNumber, "(");
tollFreeNumber = stringConvert.StringRefiner(tollFreeNumber, ")");
tollFreeNumber = stringConvert.StringRefiner(tollFreeNumber, "-");
Console.WriteLine(tollFreeNumber);
My target is to run the application in background and generate data, so I do not have any option to run this using Firefox.HTMLUNIT is the only option for me.Help needed it will stuck me last 3-4 days.
I believe your machine is using proxy servers. Try by adding that proxy configuration while creating driver instance.
String PROXY = "xx.xx.xx.xx:8080";
OpenQA.Selenium.Proxy proxy = new OpenQA.Selenium.Proxy();
proxy.HttpProxy=PROXY;
proxy.FtpProxy=PROXY;
proxy.SslProxy=PROXY;
Add proxy setting to desired Capabilities:
desiredCapabilities.SetCapability(CapabilityType.PROXY, proxy);

This is the initial start page for the WebDriver server

same code is running in firefox but it is not executing in IE9 and displaying the String message "This is the initial start page for the WebDriver server." while no error found on others
public void setUp() throws Exception {
File file = new File("C:/Users/Sunil.Wali/Desktop/Softwares/IEDriverServer_Win32_2.37.0/IEDriverServer.exe");
System.setProperty("webdriver.ie.driver", file.getAbsolutePath());
driver = new InternetExplorerDriver();
// driver = new FirefoxDriver();
baseUrl = "https://tssstrpms501.corp.trelleborg.com:12001";
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
}
#Test
public void testLogin() throws Exception {
driver.get(baseUrl + "/ProcessPortal/login.jsp");
driver.findElement(By.id("username")).clear();
driver.findElement(By.id("username")).sendKeys("sunil.wali");
driver.findElement(By.id("password")).clear();
driver.findElement(By.id("password")).sendKeys("Trelleborg#123");
driver.findElement(By.id("log_in")).click();
driver.findElement(By.id("processPortalUserDropdown")).click();
driver.findElement(By.id("dijit_MenuItem_56_text")).click();
}
#After
public void tearDown() throws Exception {
driver.quit();
String verificationErrorString = verificationErrors.toString();
if (!"".equals(verificationErrorString)) {
fail(verificationErrorString);
}
}
Output:-
Started InternetExplorerDriver server (32-bit)
2.37.0.0
Listening on port 31651
Make sure you have same value of Protected Mode settings for each zone. Refere Required Configuration for IE.
UPDATE: setting ignoreZoomSetting and ignoreProtectedModeSettings capabilities to true helps when you don't have access to change settings.
If you are using qaf you can set capabilities as below:
driver.name=iexplorerDriver
iexplorer.additional.capabilities={'ignoreProtectedModeSettings':true,'ignoreZoomSetting':true,'nativeEvents':false,'acceptSslCerts':true}
On IE 7 or higher on Windows Vista or Windows 7, you must set the
Protected Mode settings for each zone to be the same value. The value
can be on or off, as long as it is the same for every zone. To set
the Protected Mode settings, choose "Internet Options..." from the
Tools menu, and click on the Security tab. For each zone, there will
be a check box at the bottom of the tab labeled "Enable Protected
Mode".
Additionally, "Enhanced Protected Mode" must be disabled for IE 10
and higher. This option is found in the Advanced tab of the Internet
Options dialog.
http://code.google.com/p/selenium/wiki/InternetExplorerDriver#Required_Configuration
Also make sure that your zoom is set up to 100%.
Try using an older version of IEDriverServer. I tried to everything like enabling protection mode and zoom on 100% but was still stuck on localhost page. So i downloaded and used an older version of iedriver 3.4 and Voila it worked.
Solution:
Modify the file ..\node_modules\protractor\lib\driverProviders\local.js
/*
* This is an implementation of the Local Driver Provider.
* It is responsible for setting up the account object, tearing
* it down, and setting up the driver correctly.
*
* TODO - it would be nice to do this in the launcher phase,
* so that we only start the local selenium once per entire launch.
* ------Modified by Jonathan Arias mail: jab504#gmail.com-----------
*/
var util = require('util'),
log = require('../logger.js'),
path = require('path'),
remote = require('selenium-webdriver/remote'),
fs = require('fs'),
q = require('q'),
DriverProvider = require('./driverProvider');
var LocalDriverProvider = function(config) {
DriverProvider.call(this, config);
this.server_ = null;
};
util.inherits(LocalDriverProvider, DriverProvider);
/**
* Helper to locate the default jar path if none is provided by the user.
* #private
*/
LocalDriverProvider.prototype.addDefaultBinaryLocs_ = function() {
if (!this.config_.seleniumServerJar) {
log.debug('Attempting to find the SeleniumServerJar in the default ' +
'location used by webdriver-manager');
this.config_.seleniumServerJar = path.resolve(__dirname,
'../../selenium/selenium-server-standalone-' +
require('../../config.json').webdriverVersions.selenium + '.jar');
}
if (!fs.existsSync(this.config_.seleniumServerJar)) {
throw new Error('No selenium server jar found at the specified ' +
'location (' + this.config_.seleniumServerJar +
'). Check that the version number is up to date.');
}
if (this.config_.capabilities.browserName === 'chrome') {
if (!this.config_.chromeDriver) {
log.debug('Attempting to find the chromedriver binary in the default ' +
'location used by webdriver-manager');
this.config_.chromeDriver =
path.resolve(__dirname, '../../selenium/chromedriver');
}
// Check if file exists, if not try .exe or fail accordingly
if (!fs.existsSync(this.config_.chromeDriver)) {
if (fs.existsSync(this.config_.chromeDriver + '.exe')) {
this.config_.chromeDriver += '.exe';
} else {
throw new Error('Could not find chromedriver at ' +
this.config_.chromeDriver);
}
}
}
if (this.config_.capabilities.browserName === 'internet explorer') {
if (!this.config_.IEDriverServer) {
log.debug('Attempting to find the Internet explorer binary in the default ' +
'location used by webdriver-manager');
this.config_.IEDriverServer =
path.resolve(__dirname, '../../selenium/IEDriverServer');
}
// Check if file exists, if not try .exe or fail accordingly
if (!fs.existsSync(this.config_.IEDriverServer)) {
if (fs.existsSync(this.config_.IEDriverServer + '.exe')) {
this.config_.IEDriverServer += '.exe';
} else {
throw new Error('Could not find IEDriverServer at ' +
this.config_.IEDriverServer);
}
}
}
};
/**
* Configure and launch (if applicable) the object's environment.
* #public
* #return {q.promise} A promise which will resolve when the environment is
* ready to test.
*/
LocalDriverProvider.prototype.setupEnv = function() {
var deferred = q.defer(),
self = this;
this.addDefaultBinaryLocs_();
log.puts('Starting selenium standalone server...');
// configure server
if (this.config_.chromeDriver) {
this.config_.seleniumArgs.push('-Dwebdriver.chrome.driver=' +
this.config_.chromeDriver);
}
if (this.config_.IEDriverServer) {
this.config_.seleniumArgs.push('-Dwebdriver.ie.driver=' +
this.config_.IEDriverServer);
}
this.server_ = new remote.SeleniumServer(this.config_.seleniumServerJar, {
args: this.config_.seleniumArgs,
port: this.config_.seleniumPort
});
//start local server, grab hosted address, and resolve promise
this.server_.start().then(function(url) {
log.puts('Selenium standalone server started at ' + url);
self.server_.address().then(function(address) {
self.config_.seleniumAddress = address;
deferred.resolve();
});
});
return deferred.promise;
};
/**
* Teardown and destroy the environment and do any associated cleanup.
* Shuts down the drivers and server.
*
* #public
* #override
* #return {q.promise} A promise which will resolve when the environment
* is down.
*/
LocalDriverProvider.prototype.teardownEnv = function() {
var self = this;
var deferred = q.defer();
DriverProvider.prototype.teardownEnv.call(this).then(function() {
log.puts('Shutting down selenium standalone server.');
self.server_.stop().then(function() {
deferred.resolve();
});
});
return deferred.promise;
};
// new instance w/ each include
module.exports = function(config) {
return new LocalDriverProvider(config);
};
Even I had the same issue . And it started working when I made sure that the two comments over here done properly
Make sure you have same value of Protected Mode settings for each zone
make sure that your zoom is set up to 100%.
Thanks A lot for the above two comments.
usage of 64 bit IEDriverServer fixed this issue for me.
I tried all the capabilities, still failed and at last 64 bit IE server fixed.
Use this below code to resolve the issue of type "This is the initial start page for the WebDriver server". In my machine its working fine.
System.setProperty("webdriver.ie.driver", "\\IEDriverServer.exe path");
DesiredCapabilities capabilities = DesiredCapabilities.internetExplorer();
// this line of code is to resolve protected mode issue capabilities.setCapability(InternetExplorerDriver.INTRODUCE_FLAKINESS_BY_IGNORING_SECURITY_DOMAINS, true);
capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
driver=new InternetExplorerDriver();
The below code fixed the issue for me.
DesiredCapabilities capability = DesiredCapabilities.internetExplorer();
capability.setCapability(InternetExplorerDriver.INTRODUCE_FLAKINESS_BY_IGNORING_SECURITY_DOMAINS, true);
capability.setCapability(InternetExplorerDriver.IGNORE_ZOOM_SETTING, true);