How to set the PhantomJSOptions in Selenium for proxy-type? - selenium

Using Selenium and the web driver to drive phantomjs as part of a web scraping project. From the "Known Issues" for Windows at PhantomJS, it is suggested that the proxy-type be set to "none" to speed up network performance.
I tried the following:
PhantomJSOptions options = new PhantomJSOptions();
options.AddAdditionalCapability("proxy", "{proxyType:none}");
However, this sets proxy to a string and I think I need a json object. Can someone show me how to do this correctly?

Use the driver service to set options:
var phantomJSDriverService = PhantomJSDriverService.CreateDefaultService(phantomJsDir);
phantomJSDriverService.ProxyType = "none";
var driver = new PhantomJSDriver(phantomJSDriverService);

Related

How to get Chrome Dev Tools logs with Selenium C#?

My aim is getting the transfer size of page. Explained below image.
How can I get it with C# code using selenium NuGet package? I searched chrome dev tools.
https://chromedevtools.github.io/devtools-protocol/tot/Network/
I see there is Network.dataReceived property. As I understand I need to get this.
My code
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium;
ChromeOptions options = new ChromeOptions();
//Following Logging preference helps in enabling the performance logs
options.SetLoggingPreference("performance", LogLevel.All);
//Based on your need you can change the following options
options.AddUserProfilePreference("intl.accept_languages", "en-US");
options.AddUserProfilePreference("disable-popup-blocking", "true");
options.AddArgument("test-type");
options.AddArgument("--disable-gpu");
options.AddArgument("no-sandbox");
options.AddArgument("start-maximized");
options.LeaveBrowserRunning = true;
//Creating Chrome driver instance
IWebDriver driver = new ChromeDriver(options);
var url = " https://...............";
driver.Navigate().GoToUrl(url);
//Extracting the performance logs
var logs = driver.Manage().Logs.GetLog("performance");
for (int i = 0; i < logs.Count; i++)
{
Console.WriteLine((i+1) + " - " + logs[i].Message);
}
I found some code that possibly worked in old versions. I am using Chrome 105 version(as a nuget package, and selenium web dirver 4.0.0) and Chrome should be 105 version. In C# I can not find the below code, this is Java version:
for (LogEntry entry : driver.manage().logs().get(LogType.PERFORMANCE)) {
if(entry.getMessage().contains("Network.dataReceived")) {
Matcher dataLengthMatcher = Pattern.compile("encodedDataLength\":(.*?),").matcher(entry.getMessage());
dataLengthMatcher.find();
}
Even if we can do this with C# code, I dont believe driver.Manage().Logs.GetLog("performance") returns the same with the listed lines in dev tools network section. Is there nay option about that?

ChromeDriver Access To Mic/Camera For Selenium Test

I’m trying to run an end to end test for a video chat platform, using the ChromeDriver. I need to use the actual webcam stream on my computer to run the test. I’ve tried both versions of the following code, but neither actually defaults the mic/camera settings to allow.
let options = new Options();
options.set("preferences.profile.content_settings.exceptions.media_stream_camera.'<localhost-address>,*'.setting", 1)
options.set("preferences.profile.content_settings.media_stream_mic.'<localhost-address>,*'.setting", 1);
let driver = await new Builder().setChromeOptions(options).forBrowser("chrome").build();
let options = new Options();
options.setUserPreferences({"preferences.profile.content_settings.exceptions.media_stream_camera.'<localhost-address>,*'.setting": 1,
"preferences.profile.content_settings.media_stream_mic.'<localhost-address>,*'.setting": 1});
let driver = await new Builder().setChromeOptions(options).forBrowser("chrome").build();
The pop-up still comes up and I can’t find a way to interact with it. Using a fake device/UI doesn’t resolve the issue, because it’s incompatible (to my knowledge) with Twilio, which the platform also uses.
Does anyone know how to default the ChromeDriver to allow camera/mic access during testing?
Update: The following code runs so that the pop up doesn't appear. But when the driver gets to the point that pop up would show up, the browser instance quits and the test ends. Is it impossible to stream from a webcam via Selenium?
let options = new Options();
options.setUserPreferences({"profile.default_content_setting_values.media_stream_mic": 1,
"profile.default_content_setting_values.media_stream_camera": 1});
let driver = await new Builder().setChromeOptions(options).forBrowser(browser).build();

How to enable IE mode in Chromium Edge Browser in selenium C#?

I want to automate a website in Edge which is require IE mode to be enabled. How can launch Edge in IE mode in selenium?
Below code which I currently use launches Edge in non IE mode, which won't display the website properly.
Dim edgeDriverService = Microsoft.Edge.SeleniumTools.EdgeDriverService.CreateChromiumService()
Dim edgeOptions = New Microsoft.Edge.SeleniumTools.EdgeOptions()
edgeOptions.PageLoadStrategy = PageLoadStrategy.Normal
edgeOptions.UseChromium = True
Dim driver As IWebDriver = New Microsoft.Edge.SeleniumTools.EdgeDriver(edgeDriverService, edgeOptions)
driver.Navigate().GoToUrl("http://example.com")
Tried using edgeOptions.AddAdditionalCapability("ie.edgechromium", True)but it didn't work
You could refer to the section Automating Internet Explorer mode in this article about how to use IE mode in Edge Chromium in Selenium C#.
You could refer to the following steps:
Download the latest version of IEDriverServer from Selenium site. Here I use 32 bit Windows IE version 3.150.1.
Make some preparations to use IEDriver according to this.
Create a C# console project using Visual Studio.
Install Selenium.WebDriver 3.141.0 nuget package from Nuget package manager.
Add the code below to the project and modify the paths to your owns in the code:
static void Main(string[] args)
{
var dir = "{FULL_PATH_TO_IEDRIVERSERVER}";
var driver = "IEDriverServer.exe";
if (!Directory.Exists(dir) || !File.Exists(Path.Combine(dir, driver)))
{
Console.WriteLine("Failed to find {0} in {1} folder.", dir, driver);
return;
}
var ieService = InternetExplorerDriverService.CreateDefaultService(dir, driver);
var ieOptions = new InternetExplorerOptions{};
ieOptions.AddAdditionalCapability("ie.edgechromium", true);
ieOptions.AddAdditionalCapability("ie.edgepath", "{FULL_PATH_TO_MSEDGE.EXE}");
var webdriver = new InternetExplorerDriver(ieService, ieOptions, TimeSpan.FromSeconds(30));
webdriver.Url = "http://www.example.com";
}
Run the project to test:
Notes:
Make sure to close all the Edge browser tabs and window before running the code.
Use full paths in the code. For example: ieOptions.AddAdditionalCapability("ie.edgepath", #"C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe");.

Disable the request for geolocation in Opera webdriver?

Faced c with the fall of tests due to a geolocation request. Please tell me how to launch the browser without this request?
My code is the calling browser:
System.setProperty("webdriver.opera.driver", "C:/src/test/resources/operadriver.exe");
OperaOptions options = new OperaOptions();
options.setBinary(new File("C:/Program Files/Opera/50.0.2762.67/opera.exe"));
driver = new OperaDriver(options);
WebDriverRunner.setWebDriver(driver);
Use below code:
System.setProperty("webdriver.opera.driver", "C:/src/test/resources/operadriver.exe");
OperaOptions options = new OperaOptions();
options.setBinary(new File("C:/Program Files/Opera/50.0.2762.67/opera.exe"));
options.addArguments("--disable-notifications");
driver = new OperaDriver(options);

How to disable 'This type of file can harm your computer' pop up

I'm using selenium chromedriver for automating web application.
In my application, I need to download xml files. But when I download xml file, I get 'This type of file can harm your computer' pop up. I want to disable this pop up using selenium chromedriver and I want these type of files to be downloaded always. How can this be done?
Selenium version : 2.47.1
Chromedriver version : 2.19
UPDATE it's long standing Chrome bug from 2012.
The problem with XML files started to happen to me as of Chrome 47.0.2526.80 m.
After spending maybe 6 hours trying to turn off every possible security option I tried a different approach.
Ironically, it seems that turning on the Chrome option "Protect you and your device from dangerous sites" removes the message "This type of file can harm your computer. Do you want to keep file.xml anyway?"
I am using 'Ruby' with 'Watir-Webdriver' where the code looks like this:
prefs = {
'safebrowsing' => {
'enabled' => true,
}
}
b = Watir::Browser.new :chrome, :prefs => prefs
Starting the browser like this, with safebrowsing option enabled, downloads the xml files without the message warning. The principle should be the same for Selenium with any programming language.
#####
Edited: 13-04-2017
In latest version of Google Chrome the above solution is not enough. Additionally, it is necessary to start the browser with the following switch:
--safebrowsing-disable-download-protection
Now, the code for starting the browser would look something like this:
b = Watir::Browser.new :chrome, :prefs => prefs, :switches => %w[--safebrowsing-disable-download-protection]))
I am posting below the complete code that got file download working for me:
Hope it helps :-) I am using Java-Selenium
System.setProperty("webdriver.chrome.driver", "C:/chromedriver/chromedriver.exe");
String downloadFilepath = "D:/MyDeskDownload";
HashMap<String, Object> chromePrefs = new HashMap<String, Object>();
chromePrefs.put("profile.default_content_settings.popups", 0);
chromePrefs.put("download.default_directory", downloadFilepath);
chromePrefs.put("safebrowsing.enabled", "true");
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("prefs", chromePrefs);
DesiredCapabilities cap = DesiredCapabilities.chrome();
cap.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
cap.setCapability(ChromeOptions.CAPABILITY, options);
WebDriver driver = new ChromeDriver(cap);
Following Python code works for me
chromeOptions = webdriver.ChromeOptions()
prefs = {'safebrowsing.enabled': 'false'}
chromeOptions.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(chrome_options=chromeOptions)
The accepted answer stopped working after a recent update of Chrome. Now you need to use the --safebrowsing-disable-extension-blacklist and --safebrowsing-disable-download-protection command-line switches. This is the WebdriverIO config that works for me:
var driver = require('webdriverio');
var client = driver.remote({
desiredCapabilities: {
browserName: 'chrome',
chromeOptions: {
args: [
'disable-extensions',
'safebrowsing-disable-extension-blacklist',
'safebrowsing-disable-download-protection'
],
prefs: {
'safebrowsing.enabled': true
}
}
}
});
Note that I am also disabling extensions, because they generally interfere with automated testing, but this is not strictly needed to fix the problem with downloading XML and JavaScript files.
I found these switches by reading through this list. You can also see them in the Chromium source.
I came across this recently, using Katalon Studio, Chrome version 88.
Thankfully just the enabling safebrowsing did the trick. You access the settings via "Project Settings" and you navigate to "Desired Capabilities" -> "Web UI" -> "Chrome".
If you don't have a "prefs" setting add it and set the type to Dictionary. Then in the Value add a boolean named "safebrowsing.enabled" and set the value to "true".
Result might look like:
(And you can the default directory setting has nothing to do with this example).
I used all of suggested chrome options in C# and only when my internet connected worked for me but when internet disconnected none of them work for me.
(I'm not sure.may be chrome safebrowsing need to internet connection)
By using older version of chrome(version71) and chromedriver(version 2.46) and after downloading,i saw downloaded XML file name constains 'Unconfirmed' with 'crdownload' extension
and parsing XML file wouldn't work properly.
Finally, creating wait with Thread.Sleep(1000) solved my problem.
IWebDriver Driver;
//chromedriver.exe version2.46 path
string path = #"C:\cd71";
ChromeDriverService driverService = ChromeDriverService.CreateDefaultService(path, "chromedriver.exe");
ChromeOptions options = new ChromeOptions();
// options.AddArgument("headless");
options.AddArgument("--window-position=-32000,-32000");
options.AddUserProfilePreference("download.default_directory", #"c:\xmlFiles");
options.AddUserProfilePreference("download.prompt_for_download", false);
options.AddUserProfilePreference("disable-popup-blocking", "true");
options.AddUserProfilePreference("safebrowsing.enabled", "true");
Driver = new ChromeDriver(driverService, options);
try
{
Driver.Navigate().GoToUrl(url);
Thread.Sleep(1000);
//other works like: XML parse
}
catch
{
}
For context, I had a .csv with a list of .swf flash documents and their respective urls. Since the files could only be accessed after a valid login, I couldn't use a simple requests based solution.
Just like .xml, downloading a .swf triggers a similar prompt.
None of the answers worked for me.
So I stripped away all the extra arguments and settings the above answers proposed and just made the chrome instance headless.
options.add_argument("--headless")
prefs = {
"download.default_directory": "C:\LOL\Flash",
"download.prompt_for_download": False,
"download.directory_upgrade": True,
}
options.add_experimental_option("prefs",prefs)
The only workaround that works for me:
Use argument with path to chrome profile
chromeOptions.add_argument(chrome_profile_path)
Search for file download_file_types.pb in chrome profile folder.
In my case ..chromedriver\my_profile\FileTypePolicies\36\download_file_types.pb
Backup this file and then open with any hexeditor (you can use oline).
Search for the filetype you want to download, i.e. xml and change it to anything i.e. xxx
Im using Google Version 80.0.3987.122 (Official Build) (32-bit) and ChromeDriver 80.0.3987.106. Getting the same error even after adding the below while downloading a .xml file.
$ChromeOptions = New-Object OpenQA.Selenium.Chrome.ChromeOptions
$ChromeOptions.AddArguments(#(
"--disable-extensions",
"--ignore-certificate-errors"))
$download = "C:\temp\download"
$ChromeOptions.AddUserProfilePreference("safebrowsing.enabled", "true");
$ChromeOptions.AddUserProfilePreference("download.default_directory", $download);
$ChromeOptions.AddUserProfilePreference("download.prompt_for_download", "false");
$ChromeOptions.AddUserProfilePreference("download.directory_upgrade", "true");
$ChromeDriver = New-Object OpenQA.Selenium.Chrome.ChromeDriver($chromeOptions)