Is there any way to add a header in Selenium WebDriver test? (Like with Firefox Modify Headers plugin) I cannot use HtmlUnitDriver, because the browser has to be visible.
WebDriver does not allow you to change or set the headers using any of the browser based drivers. You can find a lot of information about their decision about the headers and the response codes at the following URL.
http://code.google.com/p/selenium/issues/detail?id=141
We use Apache HTTP Client for this type of testing where we do not want to check the rendered page elements, but just the responses and header information.
You can also give browser mob proxy with your selenium tests as well as mentioned in the url above. I have used this for other purposes and it is awesome.
There are alternative methods to do this like Browsermob-Proxy
Since Browsermob-proxy has its own limitations while we work on the selenium grid, the below is how I fixed the problem in my case. Hopefully, might be helpful for anyone with a similar setup.
Add the ModHeader extension to the chrome browser
How to download the Modheader? Link
ChromeOptions options = new ChromeOptions();
options.addExtensions(new File(C://Downloads//modheader//modheader.crx));
// Set the Desired capabilities
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(ChromeOptions.CAPABILITY, options);
// Instantiate the chrome driver with capabilities
WebDriver driver = new RemoteWebDriver(new URL(YOUR_HUB_URL), options);
Go to the browser extensions and capture the Local Storage context ID of the ModHeader
Navigate to the URL of the ModHeader to set the Local Storage Context
.
// set the context on the extension so the localStorage can be accessed
driver.get("chrome-extension://idgpnmonknjnojddfkpgkljpfnnfcklj/_generated_background_page.html");
Where `idgpnmonknjnojddfkpgkljpfnnfcklj` is the value captured from the Step# 2
Now add the headers to the request using Javascript
.
((Javascript)driver).executeScript(
"localStorage.setItem('profiles', JSON.stringify([{ title: 'Selenium', hideComment: true, appendMode: '',
headers: [
{enabled: true, name: 'token-1', value: 'value-1', comment: ''},
{enabled: true, name: 'token-2', value: 'value-2', comment: ''}
],
respHeaders: [],
filters: []
}]));");
Where token-1, value-1, token-2, value-2 are the request headers and values that are to be added.
Now navigate to the required web-application.
driver.get("your-desired-website");
Here is a short example of how to use Seleniumwire with Python:
from seleniumwire import webdriver
def set_chrome_driver():
options = webdriver.ChromeOptions()
options.add_argument("--start-maximized")
options.add_argument("--disable-infobars")
options.add_argument("--no-proxy-server")
driver = webdriver.Chrome(executable_path=r'C:\Automation_package\chromedriver.exe')
driver.get('http://172.1.1.1:5000/path/api/')
driver.header_overrides = {"iv-user": "Admin", "iv-groups": "SuperAdmin", "iv-roles": "Viewers",}
driver.get('http://172.1.1.1:5000/path/api/')
Related
I'm do me code in Cromedrive in 'normal' mode and works fine. When I change to headless mode it don't download the file. I already try the code I found alround internet, but didn't work.
chrome_options = Options()
chrome_options.add_argument("--headless")
self.driver = webdriver.Chrome(chrome_options=chrome_options, executable_path=r'{}/chromedriver'.format(os.getcwd()))
self.driver.set_window_size(1024, 768)
self.driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': os.getcwd()}}
self.driver.execute("send_command", params)
Anyone have any idea about how solve this problem?
PS: I don't need to use Chomedrive necessarily. If it works in another drive it's fine for me.
First the solution
Minimum Prerequisites:
Selenium client version: Selenium v3.141.59
Chrome version: Chrome v77.0
ChromeDriver version: ChromeDriver v77.0
To download the file clicking on the element with text as Download Data within this website you can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless")
options.add_argument("--window-size=1920,1080")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe', service_args=["--log-path=./Logs/DubiousDan.log"])
print ("Headless Chrome Initialized")
params = {'behavior': 'allow', 'downloadPath': r'C:\Users\Debanjan.B\Downloads'}
driver.execute_cdp_cmd('Page.setDownloadBehavior', params)
driver.get("https://www.mockaroo.com/")
driver.execute_script("scroll(0, 250)");
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#download"))).click()
print ("Download button clicked")
#driver.quit()
Console Output:
Headless Chrome Initialized
Download button clicked
File Downloading snapshot:
Details
Downloading files through Headless Chromium was one of the most sought functionality since Headless Chrome was introduced.
Since then there were different work-arounds published by different contributors and some of them are:
Downloading with chrome headless and selenium
Python equivalent of a given wget command
Now the, the good news is Chromium team have officially announced the arrival of the functionality Downloading file through Headless Chromium.
In the discussion Headless mode doesn't save file downloads #eseckler mentioned:
Downloads in headless work a little differently. There's the Page.setDownloadBehavior devtools command to set a download folder. We're working on a way to use DevTools network interception to stream the downloaded file via DevTools as well.
A detailed discussion can be found at Issue 696481: Headless mode doesn't save file downloads
Finally, #bugdroid revision seems to have nailed the issue for us.
[ChromeDriver] Added support for headless mode to download files
Previously, Chromedriver running in headless mode would not properly download files due to the fact it sparsely parses the preference file given to it. Engineers from the headless chrome team recommended using DevTools's "Page.setDownloadBehavior" to fix this. This changelist implements this fix. Downloaded files default to the current directory and can be set using download_dir when instantiating a chromedriver instance. Also added tests to ensure proper download functionality.
Here is the revision and commit
From ChromeDriver v77.0.3865.40 (2019-08-20) release notes:
Resolved issue 2454: Headless mode doesn't save file downloads [Pri-2]
Solution
Update ChromeDriver to latest ChromeDriver v77.0 level.
Update Chrome to Chrome Version 77.0 level. (as per ChromeDriver v76.0 release notes)
Note: Chrome v77.0 is yet to be GAed/pushed for release so till then you can download and install a development build and test either from:
Chrome Canary
Latest build from the Dev Channel
Outro
However Mac OSX users have a wait for their pie as On Chromedriver, headless chrome crashes after sending Page.setDownloadBehavior on MacOSX.
Chomedriver Version: 95.0.4638.54
Chrome Version 95.0.4638.69
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless")
options.add_argument("--start-maximized")
options.add_argument("--no-sandbox")
options.add_argument("--disable-extensions")
options.add_argument('--disable-dev-shm-usage')
options.add_argument("--disable-gpu")
options.add_argument('--disable-software-rasterizer')
options.add_argument("user-agent=Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; Microsoft; Lumia 640 XL LTE) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Mobile Safari/537.36 Edge/12.10166")
options.add_argument("--disable-notifications")
options.add_experimental_option("prefs", {
"download.default_directory": "C:\\link\\to\\folder",
"download.prompt_for_download": False,
"download.directory_upgrade": True,
"safebrowsing_for_trusted_sources_enabled": False,
"safebrowsing.enabled": False
}
)
What seemed to work was that I used "\\" instead of "/" for the address. The latter approach didn't throw any error, but didn't download any documents either. But, using double back slashes did the job.
For javascript use below code:
const chrome = require('selenium-webdriver/chrome');
let options = new chrome.Options();
options.addArguments('--headless --window-size=1500,1200');
options.setUserPreferences({ 'plugins.always_open_pdf_externally': true,
"profile.default_content_settings.popups": 0,
"download.default_directory": Download_File_Path });
driver = await new webdriver.Builder().setChromeOptions(options).forBrowser('chrome').build();
Then switch tabs as soon as you click the download button:
await driver.sleep(1000);
var Handle = await driver.getAllWindowHandles();
await driver.switchTo().window(Handle[1]);
This C# works for me
Note the new headless option https://www.selenium.dev/blog/2023/headless-is-going-away/
private IWebDriver StartBrowserChromeHeadlessDriver()
{
var chromeOptions = new ChromeOptions();
chromeOptions.AddArgument("--headless=new");
chromeOptions.AddArgument("--window-size=1920,1080");
chromeOptions.AddUserProfilePreference("download.default_directory", downloadFolder);
var chromeDownload = new Dictionary<string, object>
{
{ "behavior", "allow" },
{ "downloadPath", downloadFolder }
};
var driver = new ChromeDriver(driverFolder, chromeOptions, TimeSpan.FromSeconds(timeoutSecs));
driver.ExecuteCdpCommand("Browser.setDownloadBehavior", chromeDownload);
return driver;
}
import pathlib
from selenium.webdriver import Chrome
driver = Chrome()
driver.execute_cdp_cmd("Page.setDownloadBehavior", {
"behavior": "allow",
"downloadPath": str(pathlib.Path.home().joinpath("Downloads"))
})
I don't think you should be using the browser for downloading content, leave it to Chrome developers/testers.
I believe you should rather get href attribute of the element you want to download and obtain it using requests library
If your site requires authentication you could fetch cookies from the browser instance and pass them to requests.Session.
I am using protractor and selenium with browser stack and trying to automate a webrtc web application, I need to get rid of browser asking for permission and using a fake stream instead of real camera and mic as available in chrome.
I have tried using these options they both do not work.
Option 1:
var capabilities = {
'browserName': 'iPhone',
'device': 'iPhone 6S',
'realMobile': 'true',
'os_version': '11.4',
"media.navigator.permission.disabled": true,
"media.navigator.streams.fake": true
};
Option 2
var capabilities = {
'browserName': 'iPhone',
'device': 'iPhone 6S',
'realMobile': 'true',
'os_version': '11.4',
'safariOptions': {
'args': ["--use-fake-ui-for-media-stream", '--use-fake-device-for-media-stream']
}
};
For building options I use:
var driver = new webdriver.Builder()
usingServer('http://hub-cloud.browserstack.com/wd/hub').
withCapabilities(capabilities).
build();
Currently, there is no such BrowserStack specific custom capability to pass fake media stream on Safari. Also, passing fake stream is not yet supported on Safari browsers. You can read about the issues below:
https://github.com/web-platform-tests/results-collection/issues/125
https://github.com/web-platform-tests/wpt/issues/7424
Also, there seem to be no such arguments supported for Safari browser. I reviewed the same in the sample SafariOptions examples here
I wrote test using TestNG and selenium.
code...
actions.sendKeys(Keys.chord(Keys.CONTROL, "a"));
actions.sendKeys(Keys.BACK_SPACE);
actions.build().perform();
code...
I wanted to delete text in login window using these sendKeys, with DataProvider
#DataProvider(name = "inputs")
public Object[][] getData() {
return new Object[][]{
{"000000000", true},
{"000000000", true}
};
}
Html:
<div><input type="tel" class="valid TextInput-kJdagG iVdKHC" name="recoveryPhone" id="eb69ff0b-3427-6986-7556-b7af40ffb156" aria-describedby="eb69ff0b-3427-6986-7556-b7af40ffb156" value="+48 "></div>
Error message:
Unable to read VR
Path1523545392670 Marionette INFO Enabled via --marionette
1523545393744 Marionette INFO Listening on port 52644
1523545394180 Marionette WARN TLS certificate errors will be ignored for this session
Test work as I expected on Chrome, but on firefox these sendKeys not always mark the text, and clear this text. In project I have to use action class. Why the test runs differently?
Check this post https://github.com/mozilla/geckodriver/issues/665 against your versions (browser, browser driver, selenium etc which are always sensible to include in any question), it could be a known bug with the geckodriver.
The post includes a work around of creating the chord in a different way, using:
List<CharSequence> keyWithModifiers = new ArrayList<CharSequence>();
keyWithModifiers.add(Keys.CONTROL);
keyWithModifiers.add("a");
String ctrlA = Keys.chord(keyWithModifiers);
textFieldElem.sendKeys(ctrlA);
This approach worked for me using Selenium 3.7.1 Java bindings, gecko driver 0.18.0 (64 bit) and Firefox 57.0.2 - 59.0
It is always suggested to set the firefox profile in DesiredCapabilities and pass that through the wire ,where the hub is running . Like below
DesiredCapabilities caps = DesiredCapabilities.firefox();
FirefoxProfile profile=new FirefoxProfile(new File("Local Path to firefox profile folder"));
caps.setCapability(FirefoxDriver.PROFILE, profile);
URL url = new URL("http://localhost:4444/wd/hub");
WebDriver driver= new RemoteWebDriver(url,caps );
But sending the huge 87-90 mb profile info to hub over http ,for each selenium test case slowing down the test case execution .
I have tried configuring the grid node with "Dwebdriver.firefox.profile=E:\\Firefox_Profile_Location":"", property in json node config file like below.
{
"configuration":
{
.//Other Settings
.//Other Settings
.//Other Settings
"Dwebdriver.firefox.profile=E:\\Firefox_Profile_Location":"",
"maxSession":7,
"registerCycle":5000,
"register":true
},
"capabilities":
[
{"browserName":"firefox",
"seleniumProtocol":"WebDriver",
"maxInstances":5,
"platform":"VISTA"
}
]
}
But running with the above configuration is throwing below error .
WebDriverException: Firefox profile 'E:\Firefox_Profile_Location'
named in system property 'webdriver.firefox.profile' not found
Advanced thanks for any help on how to configure the firefox profile from the node side .
You need to provide the profile in the capabilities object as a base64 encoded zip:
var fs = require('fs');
capabilities: [
{
browserName: 'firefox',
seleniumProtocol: 'WebDriver',
maxInstances: 5,
platform: 'VISTA',
firefox_profile: new Buffer(fs.readFileSync("./profile.zip")).toString('base64')
}
]
Moreover Firefox creates the missing files for a given profile. So you should keep just the necessary files in the profile depending on your needs:
Preferences: user.js
Passwords: key3.db
logins.json
Cookies: cookies.sqlite
Certificate: cert8.sqlite
Extensions: extensions/
I think you'll have to use firefox profile name and not the location.
"webdriver.firefox.profile":"default"
Have a look at this and this and this
If you want know how to create a profile follow this and this
I'm using selenium chromedriver for automating web application.
In my application, I need to download xml files. But when I download xml file, I get 'This type of file can harm your computer' pop up. I want to disable this pop up using selenium chromedriver and I want these type of files to be downloaded always. How can this be done?
Selenium version : 2.47.1
Chromedriver version : 2.19
UPDATE it's long standing Chrome bug from 2012.
The problem with XML files started to happen to me as of Chrome 47.0.2526.80 m.
After spending maybe 6 hours trying to turn off every possible security option I tried a different approach.
Ironically, it seems that turning on the Chrome option "Protect you and your device from dangerous sites" removes the message "This type of file can harm your computer. Do you want to keep file.xml anyway?"
I am using 'Ruby' with 'Watir-Webdriver' where the code looks like this:
prefs = {
'safebrowsing' => {
'enabled' => true,
}
}
b = Watir::Browser.new :chrome, :prefs => prefs
Starting the browser like this, with safebrowsing option enabled, downloads the xml files without the message warning. The principle should be the same for Selenium with any programming language.
#####
Edited: 13-04-2017
In latest version of Google Chrome the above solution is not enough. Additionally, it is necessary to start the browser with the following switch:
--safebrowsing-disable-download-protection
Now, the code for starting the browser would look something like this:
b = Watir::Browser.new :chrome, :prefs => prefs, :switches => %w[--safebrowsing-disable-download-protection]))
I am posting below the complete code that got file download working for me:
Hope it helps :-) I am using Java-Selenium
System.setProperty("webdriver.chrome.driver", "C:/chromedriver/chromedriver.exe");
String downloadFilepath = "D:/MyDeskDownload";
HashMap<String, Object> chromePrefs = new HashMap<String, Object>();
chromePrefs.put("profile.default_content_settings.popups", 0);
chromePrefs.put("download.default_directory", downloadFilepath);
chromePrefs.put("safebrowsing.enabled", "true");
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("prefs", chromePrefs);
DesiredCapabilities cap = DesiredCapabilities.chrome();
cap.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
cap.setCapability(ChromeOptions.CAPABILITY, options);
WebDriver driver = new ChromeDriver(cap);
Following Python code works for me
chromeOptions = webdriver.ChromeOptions()
prefs = {'safebrowsing.enabled': 'false'}
chromeOptions.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(chrome_options=chromeOptions)
The accepted answer stopped working after a recent update of Chrome. Now you need to use the --safebrowsing-disable-extension-blacklist and --safebrowsing-disable-download-protection command-line switches. This is the WebdriverIO config that works for me:
var driver = require('webdriverio');
var client = driver.remote({
desiredCapabilities: {
browserName: 'chrome',
chromeOptions: {
args: [
'disable-extensions',
'safebrowsing-disable-extension-blacklist',
'safebrowsing-disable-download-protection'
],
prefs: {
'safebrowsing.enabled': true
}
}
}
});
Note that I am also disabling extensions, because they generally interfere with automated testing, but this is not strictly needed to fix the problem with downloading XML and JavaScript files.
I found these switches by reading through this list. You can also see them in the Chromium source.
I came across this recently, using Katalon Studio, Chrome version 88.
Thankfully just the enabling safebrowsing did the trick. You access the settings via "Project Settings" and you navigate to "Desired Capabilities" -> "Web UI" -> "Chrome".
If you don't have a "prefs" setting add it and set the type to Dictionary. Then in the Value add a boolean named "safebrowsing.enabled" and set the value to "true".
Result might look like:
(And you can the default directory setting has nothing to do with this example).
I used all of suggested chrome options in C# and only when my internet connected worked for me but when internet disconnected none of them work for me.
(I'm not sure.may be chrome safebrowsing need to internet connection)
By using older version of chrome(version71) and chromedriver(version 2.46) and after downloading,i saw downloaded XML file name constains 'Unconfirmed' with 'crdownload' extension
and parsing XML file wouldn't work properly.
Finally, creating wait with Thread.Sleep(1000) solved my problem.
IWebDriver Driver;
//chromedriver.exe version2.46 path
string path = #"C:\cd71";
ChromeDriverService driverService = ChromeDriverService.CreateDefaultService(path, "chromedriver.exe");
ChromeOptions options = new ChromeOptions();
// options.AddArgument("headless");
options.AddArgument("--window-position=-32000,-32000");
options.AddUserProfilePreference("download.default_directory", #"c:\xmlFiles");
options.AddUserProfilePreference("download.prompt_for_download", false);
options.AddUserProfilePreference("disable-popup-blocking", "true");
options.AddUserProfilePreference("safebrowsing.enabled", "true");
Driver = new ChromeDriver(driverService, options);
try
{
Driver.Navigate().GoToUrl(url);
Thread.Sleep(1000);
//other works like: XML parse
}
catch
{
}
For context, I had a .csv with a list of .swf flash documents and their respective urls. Since the files could only be accessed after a valid login, I couldn't use a simple requests based solution.
Just like .xml, downloading a .swf triggers a similar prompt.
None of the answers worked for me.
So I stripped away all the extra arguments and settings the above answers proposed and just made the chrome instance headless.
options.add_argument("--headless")
prefs = {
"download.default_directory": "C:\LOL\Flash",
"download.prompt_for_download": False,
"download.directory_upgrade": True,
}
options.add_experimental_option("prefs",prefs)
The only workaround that works for me:
Use argument with path to chrome profile
chromeOptions.add_argument(chrome_profile_path)
Search for file download_file_types.pb in chrome profile folder.
In my case ..chromedriver\my_profile\FileTypePolicies\36\download_file_types.pb
Backup this file and then open with any hexeditor (you can use oline).
Search for the filetype you want to download, i.e. xml and change it to anything i.e. xxx
Im using Google Version 80.0.3987.122 (Official Build) (32-bit) and ChromeDriver 80.0.3987.106. Getting the same error even after adding the below while downloading a .xml file.
$ChromeOptions = New-Object OpenQA.Selenium.Chrome.ChromeOptions
$ChromeOptions.AddArguments(#(
"--disable-extensions",
"--ignore-certificate-errors"))
$download = "C:\temp\download"
$ChromeOptions.AddUserProfilePreference("safebrowsing.enabled", "true");
$ChromeOptions.AddUserProfilePreference("download.default_directory", $download);
$ChromeOptions.AddUserProfilePreference("download.prompt_for_download", "false");
$ChromeOptions.AddUserProfilePreference("download.directory_upgrade", "true");
$ChromeDriver = New-Object OpenQA.Selenium.Chrome.ChromeDriver($chromeOptions)