How to ignore ssl certificate in codeception chrome headless browser? - selenium

I added in acceptance.suite.yml
chromeOptions:
args: ["--headless", "--disable-gpu","--test-type","--ignore-certificate-errors"],
but no luck? what can i do now
description edit:
When i start tests in headless mode they get stuck, in the _output file (fail.png) i get blank page. So i think that they get to "Insecure Connection" page and can't get through it, and my question is how to avoid that page

I suspect the argument you want is allow-insecure-localhost. This line worked for me to configure my acceptance.suite.yml file in CodeCeption.
- WebDriver:
url: xxx.com
window_size: false # disabled in ChromeDriver
port: 9515
browser: chrome
capabilities:
goog:chromeOptions:
args: ["allow-insecure-localhost","headless","start-maximized"]
This page lists all the options that chrome supports https://peter.sh/experiments/chromium-command-line-switches/#allow-insecure-localhost. Google themselves link to that (3rd party) page from their own page describing ChromeDriver config https://sites.google.com/a/chromium.org/chromedriver/capabilities.

Not sure if this already answered but according to codeception documentation
https://codeception.com/docs/modules/WebDriver
modules:
enabled:
- WebDriver:
config:
url: 'http://localhost/'
browser: chrome
capabilities:
acceptInsecureCerts: true

This works for me,
ChromeOptions options = (ChromeOptions) caps.getCapability(ChromeOptions.CAPABILITY);
options.addArguments("--headless", "--disable-gpu", "--window-size=1366,768", "--no-sandbox");
caps.setAcceptInsecureCerts(true);

Please try following code .
ChromeOptions options = new ChromeOptions();
options.addArguments("--disable-dev-shm-usage");
options.addArguments("--no-sandbox");
options.addArguments("--headless", "--window-size=1920,1200", "--ignore-certificate-errors");
options.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
options.setCapability(CapabilityType.ACCEPT_INSECURE_CERTS, true);
driver = new ChromeDriver(options);

Related

How to turn off w3c in chromedriver to address the error unknown command: Cannot call non W3C standard command while in W3C

With version 75 of Chrome just released, our tests no longer run properly. They give the stacktrace pasted below. We are using ruby on rails v. 5.1.6.2 with rspec, selenium-webdriver 3.8.0.
Stacktrace:
Selenium::WebDriver::Error::UnknownCommandError:
unknown command: Cannot call non W3C standard command while in W3C mode
# 0 chromedriver 0x000000010c46e8e9 chromedriver + 3594473
# 1 chromedriver 0x000000010c3fe543 chromedriver + 3134787
# 2 chromedriver 0x000000010c1aa29f chromedriver + 692895
# 3 chromedriver 0x000000010c11a691 chromedriver + 104081
# 4 chromedriver 0x000000010c11b7d5 chromedriver + 108501
# 5 chromedriver 0x000000010c42d555 chromedriver + 3327317
# 6 chromedriver 0x000000010c438e60 chromedriver + 3374688
# 7 chromedriver 0x000000010c438bf8 chromedriver + 3374072
# 8 chromedriver 0x000000010c40cd39 chromedriver + 3194169
# 9 chromedriver 0x000000010c4396d8 chromedriver + 3376856
# 10 chromedriver 0x000000010c420f27 chromedriver + 3276583
# 11 chromedriver 0x000000010c456064 chromedriver + 3493988
# 12 chromedriver 0x000000010c474617 chromedriver + 3618327
# 13 libsystem_pthread.dylib 0x00007fff7744c2eb _pthread_body + 126
# 14 libsystem_pthread.dylib 0x00007fff7744f249 _pthread_start + 66
# 15 libsystem_pthread.dylib 0x00007fff7744b40d thread_start + 13
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.8.0/lib/selenium/webdriver/remote/response.rb:69:in `assert_ok'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.8.0/lib/selenium/webdriver/remote/response.rb:32:in `initialize'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.8.0/lib/selenium/webdriver/remote/http/common.rb:81:in `new'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.8.0/lib/selenium/webdriver/remote/http/common.rb:81:in `create_response'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.8.0/lib/selenium/webdriver/remote/http/default.rb:104:in `request'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.8.0/lib/selenium/webdriver/remote/http/common.rb:59:in `call'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.8.0/lib/selenium/webdriver/remote/bridge.rb:166:in `execute'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.8.0/lib/selenium/webdriver/remote/oss/bridge.rb:579:in `execute'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.8.0/lib/selenium/webdriver/remote/oss/bridge.rb:526:in `element_displayed?'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/selenium-webdriver-3.8.0/lib/selenium/webdriver/common/element.rb:199:in `displayed?'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/capybara-2.17.0/lib/capybara/selenium/node.rb:148:in `visible?'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/capybara-2.17.0/lib/capybara/node/element.rb:269:in `block in visible?'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/capybara-2.17.0/lib/capybara/node/base.rb:81:in `synchronize'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/capybara-2.17.0/lib/capybara/node/element.rb:269:in `visible?'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/capybara-2.17.0/lib/capybara/queries/selector_query.rb:84:in `matches_filters?'
# /Users/julie/.rvm/gems/ruby-2.5.1/gems/capybara-2.17.0/lib/capybara/result.rb:29:in `block in initialize'
Our driver configuration:
File.write(LOG_FILE_PATH, '')
Selenium::WebDriver.logger.level = :debug
Selenium::WebDriver.logger.output = LOG_FILE_PATH
Capybara.register_driver :selenium do |app|
# from https://github.com/SeleniumHQ/selenium/issues/3738
capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(loggingPrefs: {browser: 'ALL'})
options = Selenium::WebDriver::Chrome::Options.new
options.add_argument '--disable-infobars' # hide info bar about chrome automating test
# if we don't use this flag, every selenium test will die with the error:
# "unknown error: Chrome failed to start: exited abnormally"
options.add_argument '--no-sandbox'
options.add_argument '--headless' if ENV.fetch("HEADLESS", nil).present?
options.add_argument '--window-size=1600,2400'
options.add_argument '-–allow-file-access-from-files' # TODO Julie - may help with file specs?
options.add_preference('homepage', 'about:blank') # TODO is this working?
options.add_preference('profile.default_content_settings.popups', 0)
options.add_preference('download.default_directory', DownloadHelpers::PATH.to_s)
Capybara::Selenium::Driver.new(
app,
clear_local_storage: true,
clear_session_storage: true,
browser: :chrome,
options: options,
desired_capabilities: capabilities,
)
end
UPDATE:
I was able to get our tests to work temporarily using capabilities = { "chromeOptions" => {'w3c' => false} }.
After updating chromedriver, we began receiving the error "unknown error: DevToolsActivePort file doesn't exist". In order to fix this problem, we upgraded our selenium-webdriver gem to 3.142.3 and this fixed the issue, allowing us to use w3c without any additional parameters.
First the solution
As promised by John Chen [Owner - WebDriver for Google Chrome] yesterday, new versions of ChromeDriver 75.0.3770.90 and 76.0.3809.25 have been released, and are now available at the ChromeDriver Downloads site. These versions include the following bug fixes over the previous releases of ChromeDriver 75 and 76:
Fixed a bug that incorrectly rejected POST requests with empty body in OSS mode
Added new endpoints for retrieving Chrome log
In addition, version 76.0.3809.25 also includes the following change:
Added endpoint for Is Displayed command in W3C mode
Email Snapshot
Details
It will be against the best practices to turn off w3c in chromedriver to address the error:
Selenium::WebDriver::Error::UnknownCommandError:
unknown command: Cannot call non W3C standard command while in W3C mode
as the current implementation of ChromeDriver requests a W3C-compliant session to the client.
However, this error message implies that the ChromeDriver was unable to invoke a non W3C standard command while in W3C mode while initiating/spawning a new WebBrowser i.e. Chrome Browser session.
The main issue is, when ChromeDriver's client requests a W3C-compliant session, the response from ChromeDriver does not conform to the W3C spec, and causes errors in language APIs.
As per the discussion in ChromeDriver response in W3C mode is not standard compliant John Chen (Owner - WebDriver for Google Chrome) mentioned Simon Stewart (Creator - WebDriver) have updated that:
The new session response for a w3c session should look like:
{
"value": {
"sessionId": "some-uuid",
"capabilities": {
"browserName": "chrome",
...
}
}
}
But when starting a new session with the w3c option set to true in the chromeOptions, the returned response looked like:
{
"sessionId": "af4656c27fb94485b7872e1fc616923a",
"status": "ok",
"value": {
"browserName": "chrome",
...
}
}
Which is neither a correctly formed response for the JSON Wire Protocol (where "status" would be an integer), nor a correctly formed W3C response and without a correctly formed response, the w3c compatible cannot be used.
This revision and this commit addressed this issue.
This usecase
Presumably you are using ChromeDriver v75.x with Chrome v75.x and in case you are still seeing the error, you need to pass the ExperimentalOption w3c as true explicitly as follows:
Ruby code sample:
capabilities = { "chromeOptions" => {'w3c' => true} }
Java code sample:
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
public class W3c {
public static void main(String[] args) throws Exception {
ChromeOptions opt = new ChromeOptions();
opt.setExperimentalOption("w3c", true);
ChromeDriver driver = new ChromeDriver(opt);
driver.get("https://www.google.co.in");
}
}
Python code sample:
from selenium import webdriver
opt = webdriver.ChromeOptions()
opt.add_experimental_option('w3c', True)
driver = webdriver.Chrome(chrome_options=opt)
driver.get('https://www.google.co.in')
Update
Till ChromeDriver v74.x, Chrome and ChromeDriver combo was running in w3c mode by default but there was bug with in the chromedriver/server/http_handler.cc. As per the details in goog:chromeOptions.w3c=false doesn't work for POST request with empty body:
Method HttpHandler::HandleCommand checks the value of the kW3CDefault constant instead of session goog:chromeOptions.w3c value. As a result, JSON Wire protocol support was broken, where POST requests with an empty body are allowed.
JSON Wire protocol will be in demand until displayed endpoint is resumed in the w3c mode. It should be noted that W3C WebDriver specification doesn't forbid the use of 'displayed' endpoint and this feature is actively used in some APIs.
As Is Element Displayed command is not part of W3C spec, but is still used by some APIs, and its functionality can be difficult to replicate in those APIs. This Change List [revision and commit] re-enables this command in W3C mode to ease transition to W3C mode.
#John have already confirmed us to expect an update to ChromeDriver v75.0 tomorrow with the fix.
For the Javascript people (I specifically use WebdriverIO) make sure you use 'goog:chromeOptions'
capabilities: {
browserName: 'chrome',
'goog:chromeOptions': {
'w3c': false
}
}
Else you'll get
unknown error: Illegal key values seen in w3c capabilities: [chromeOptions]
After doing options = Selenium::WebDriver::Chrome::Options.new
you can do options.add_option('w3c', false)
I'm facing the same issue.
I tried to disable using capabilities = Selenium::WebDriver::Remote::Capabilities.chrome({ "chromeOptions" => {'w3c' => false} }) but it didn't work.
Then I changed to capabilities = { "chromeOptions" => {'w3c' => false} } and now it works.
Maybe it can help you.
This is how it can be done in Behat with Mink: #behat #mink
Behat\MinkExtension:
base_url: "your_site_url"
browser_name: 'chrome'
goutte: ~
javascript_session: selenium2
selenium2:
wd_host: http://127.0.0.1:4444/wd/hub
capabilities:
browser: chrome
extra_capabilities:
chromeOptions:
args: ['--headless', '--disable-gpu']
w3c: false
PHP Behat-Mink-Selenium users see this post for information:
https://medium.com/#alex.designworks/chromedriver-75-enforces-w3c-standard-breaking-behat-tests-460cad435545
and GitHub issue https://github.com/minkphp/MinkSelenium2Driver/issues/293
As of this post, the "workaround" is to fall back to Chrome 74 for those using Behat-Mink-Selenium.
{
'platformName':'Android',
'platformVersion':'8.0.0',
'deviceName':'Samsung Galaxy S9',
'deviceType':'Phone',
'nativeWebTap': 'True',
'browser' : 'Chrome',
"goog:chromeOptions": {'w3c': False}
}
use goog:chromeOptions options to set w3c True or False. on console you will see the same getting passed.
I recently upgraded appium to 1.18 , using chromeDriver version 84 and my chrome browser version is also 84. I use appium python library to start a browser. My capabilities are like below but I get error -"WebDriverException: Message: 'chromeOptions' must be of type object". Please let me know whats wrong with my capabilities.
{
'platformName':'Android',
'platformVersion':'8.0.0',
'deviceName':'Samsung Galaxy S9',
'deviceType':'Phone',
'nativeWebTap': 'True',
'browser' : 'Chrome',
'chromeOptions' : '{args: [ 'w3c : false']}'
}

Start headless chrome in incognito

Under SpecFlow/Selenium, I'm starting Chrome like this:
var options = new ChromeOptions();
options.AddArguments("--headless", "--window-size=1920,1080", "--disable-gpu", "--disable-extensions", "--no-sandbox", "-incognito");
var svc = ChromeDriverService.CreateDefaultService();
svc.Start();
Context.Driver = new ChromeDriver(svc, options);
As the -incognito flag on the Chrome executable starts the browser in incognito mode I was thinking this was do the same for headless mode. I was wrong. Can it be done with headless? If so, how?
I have also tried --incognito and incognito as well as placing the flag at different indices within the AddArguments() call.
D'oh! Turns out I was missing a single -...
"--incognito"

How to set the firefox profile at the node end in remote webdriver/grid configuration

It is always suggested to set the firefox profile in DesiredCapabilities and pass that through the wire ,where the hub is running . Like below
DesiredCapabilities caps = DesiredCapabilities.firefox();
FirefoxProfile profile=new FirefoxProfile(new File("Local Path to firefox profile folder"));
caps.setCapability(FirefoxDriver.PROFILE, profile);
URL url = new URL("http://localhost:4444/wd/hub");
WebDriver driver= new RemoteWebDriver(url,caps );
But sending the huge 87-90 mb profile info to hub over http ,for each selenium test case slowing down the test case execution .
I have tried configuring the grid node with "Dwebdriver.firefox.profile=E:\\Firefox_Profile_Location":"", property in json node config file like below.
{
"configuration":
{
.//Other Settings
.//Other Settings
.//Other Settings
"Dwebdriver.firefox.profile=E:\\Firefox_Profile_Location":"",
"maxSession":7,
"registerCycle":5000,
"register":true
},
"capabilities":
[
{"browserName":"firefox",
"seleniumProtocol":"WebDriver",
"maxInstances":5,
"platform":"VISTA"
}
]
}
But running with the above configuration is throwing below error .
WebDriverException: Firefox profile 'E:\Firefox_Profile_Location'
named in system property 'webdriver.firefox.profile' not found
Advanced thanks for any help on how to configure the firefox profile from the node side .
You need to provide the profile in the capabilities object as a base64 encoded zip:
var fs = require('fs');
capabilities: [
{
browserName: 'firefox',
seleniumProtocol: 'WebDriver',
maxInstances: 5,
platform: 'VISTA',
firefox_profile: new Buffer(fs.readFileSync("./profile.zip")).toString('base64')
}
]
Moreover Firefox creates the missing files for a given profile. So you should keep just the necessary files in the profile depending on your needs:
Preferences: user.js
Passwords: key3.db
logins.json
Cookies: cookies.sqlite
Certificate: cert8.sqlite
Extensions: extensions/
I think you'll have to use firefox profile name and not the location.
"webdriver.firefox.profile":"default"
Have a look at this and this and this
If you want know how to create a profile follow this and this

WebDriver Chrome Browser: Avoid 'Do you want chrome to save your password' pop up

Every time my webdriver tests login into the application, 'Do you want chrome to save your password' pop up appears.. Is there a way to avoid this??
Please help.
Thanks,
Mike
You need to configure the following chrome driver options:
chromeOptions: {
prefs: {
'credentials_enable_service': false,
'profile': {
'password_manager_enabled': false
}
}
}
I'm using Python, and this worked for me:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_experimental_option('prefs', {
'credentials_enable_service': False,
'profile': {
'password_manager_enabled': False
}
})
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get('https://google.com')
Just add these preferences to your chrome driver options:
Map<String, Object> prefs = new HashMap<String, Object>();
prefs.put("credentials_enable_service", false);
prefs.put("password_manager_enabled", false);
options.setExperimentalOption("prefs", prefs);
Yeah I just found the answer. I had to look into the Chrome's user data directory and find all the available chromeOptions the Preferences file. I'm on Centos 7 so the path looks like this:
~/.config/google-chrome/Default/Preferences
In order to remove the save password dialog, the config JSON chromeOptions section needs to have this:
chromeOptions: {
prefs: {
profile: {
password_manager_enabled: false
}
}
}
It really makes me happy that I have finally found these options, however, it still is disappointing that google or selenium didn't list all the configurable preferences.
Thanks to #karanvir Kang comment above, I added the following to my conf.js which I use when I call protractor. Example
protractor tests/conf.js --specs /tests/e2e/myspec.spec.js
And in my conf.js
exports.config = {
seleniumAddress: 'http://localhost:4444/wd/hub',
seleniumPort: '4455',
baseUrl: url,
directConnect: false,
//getMultiCapabilities: helper.getFirefoxProfile,
capabilities: {
browserName: 'chrome',
chromeOptions: {
prefs: {
'credentials_enable_service': false,
'profile': {
'password_manager_enabled': false
}
},
args: [
'--disable-cache',
'--disable-application-cache',
'--disable-offline-load-stale-cache',
'--disk-cache-size=0',
'--v8-cache-options=off'
]
}
},
You can also start the chromedriver in incognito mode to stop the infobars from appearing. Please note that the experience will be like the incognito mode. Command will be
chrome.exe --incognito if you are running from command line
you can add --incognito to chromeswitch array for executing from webdriver.
To provide a more complete picture, here is a working configuration for Watir in a Selenium Grid:
RSpec.configure do |config|
config.before :all do
capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(
chromeOptions: {
prefs: {
'credentials_enable_service': false,
'profile': {
'password_manager_enabled': false
}
}
}
)
#browser = Watir::Browser.new(
:remote,
url: "http://#{ENV.fetch('HUB_HOST')}/wd/hub",
desired_capabilities: capabilities
)
end
config.after :all do
#browser&.close
end
end
See a full proof of concept on github at docker-grid-watir.
I know this is pretty old, it has been answered correctly and all. Just wanted to give my 5 cents. If you are using Robot Framework, bellow is the way to do it.
open-browser
${chrome_options}= Evaluate sys.modules['selenium.webdriver'].ChromeOptions() sys
${cred_dict}= Create Dictionary credentials_enable_service=${FALSE}
Call Method ${chrome_options} add_experimental_option prefs ${cred_dict}
Create Webdriver Chrome chrome chrome_options=${chrome_options}

Headers and Selenium Webdriver 2

Is there any way to add a header in Selenium WebDriver test? (Like with Firefox Modify Headers plugin) I cannot use HtmlUnitDriver, because the browser has to be visible.
WebDriver does not allow you to change or set the headers using any of the browser based drivers. You can find a lot of information about their decision about the headers and the response codes at the following URL.
http://code.google.com/p/selenium/issues/detail?id=141
We use Apache HTTP Client for this type of testing where we do not want to check the rendered page elements, but just the responses and header information.
You can also give browser mob proxy with your selenium tests as well as mentioned in the url above. I have used this for other purposes and it is awesome.
There are alternative methods to do this like Browsermob-Proxy
Since Browsermob-proxy has its own limitations while we work on the selenium grid, the below is how I fixed the problem in my case. Hopefully, might be helpful for anyone with a similar setup.
Add the ModHeader extension to the chrome browser
How to download the Modheader? Link
ChromeOptions options = new ChromeOptions();
options.addExtensions(new File(C://Downloads//modheader//modheader.crx));
// Set the Desired capabilities
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(ChromeOptions.CAPABILITY, options);
// Instantiate the chrome driver with capabilities
WebDriver driver = new RemoteWebDriver(new URL(YOUR_HUB_URL), options);
Go to the browser extensions and capture the Local Storage context ID of the ModHeader
Navigate to the URL of the ModHeader to set the Local Storage Context
.
// set the context on the extension so the localStorage can be accessed
driver.get("chrome-extension://idgpnmonknjnojddfkpgkljpfnnfcklj/_generated_background_page.html");
Where `idgpnmonknjnojddfkpgkljpfnnfcklj` is the value captured from the Step# 2
Now add the headers to the request using Javascript
.
((Javascript)driver).executeScript(
"localStorage.setItem('profiles', JSON.stringify([{ title: 'Selenium', hideComment: true, appendMode: '',
headers: [
{enabled: true, name: 'token-1', value: 'value-1', comment: ''},
{enabled: true, name: 'token-2', value: 'value-2', comment: ''}
],
respHeaders: [],
filters: []
}]));");
Where token-1, value-1, token-2, value-2 are the request headers and values that are to be added.
Now navigate to the required web-application.
driver.get("your-desired-website");
Here is a short example of how to use Seleniumwire with Python:
from seleniumwire import webdriver
def set_chrome_driver():
options = webdriver.ChromeOptions()
options.add_argument("--start-maximized")
options.add_argument("--disable-infobars")
options.add_argument("--no-proxy-server")
driver = webdriver.Chrome(executable_path=r'C:\Automation_package\chromedriver.exe')
driver.get('http://172.1.1.1:5000/path/api/')
driver.header_overrides = {"iv-user": "Admin", "iv-groups": "SuperAdmin", "iv-roles": "Viewers",}
driver.get('http://172.1.1.1:5000/path/api/')