Selenium headless chromedriver unable to locate element - selenium

Helllo everyone, I'm on raspberry pi with no gui and I'm trying to use Selenium chromedriver in headless mode. I keep getting an exception saying that driver is unable to locate what i'm searching for. The fact is that without headless mode I have no problem and my code works fine.
class Bot:
def __init__(self):
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--window-size=1920,1080');
self.drivers = []
for j in range(2):
self.drivers.append(webdriver.Chrome(options=options,executable_path="/usr/lib/chromium-browser/chromedriver"))
def clearCache(self, index):
self.drivers[index].get('chrome://settings/clearBrowserData')
time.sleep(1)
try:
settings = self.drivers[index].find_element_by_xpath('//settings-ui')
except Exception as e:
print(e)
The error:
Message: no such element: Unable to locate element: {"method":"xpath","selector":"//settings-ui"}
(Session info: headless chrome=88.0.4324.187)
EDIT
I tried taking a screenshot
self.drivers[index].get('chrome://settings/clearBrowserData')
self.drivers[index].get_screenshot_as_file("screenshot.png")
and I get a blank image.
EDIT2
I am noticing that it seems to have troubles getting settings page
self.drivers[index].get('chrome://settings/clearBrowserData')
because elements searched in other pages can be found with no exception. I just needed to go to the settings in order to delete the cache (because my bot will loop refreshing a page each time deleting the cache).
The strange thing is that without headless mode, all works fine and self.drivers[index].get('chrome://settings/clearBrowserData') does not end in a blank screenshot.

can you try to add the below arguments. Alternatively You may add the URL onetime manually by joining to Internat Option->Security-> Trusted Sites-> add the URL.
--allow-running-insecure-content

Related

TimeoutException: Message: Timed out waiting for page to load

I want to Open browser in Edge with IE mode.
My environment: IE7, windows 11, Python 3.10.4, Edge version 108.0.1462.46
And I follow the required configuration from below:
https://www.selenium.dev/documentation/ie_driver_server/
I made the same value for Enhanced Protected Mode by setting the REG_DWORD 2500 value to 0 in Zones 0,1,2,3,4:
Registry Editer path: Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Internet Settings\Zones
I add the IEDriverServer to my PATH
Is there any wrong steps about my configuration?
Below is my code:
*** Settings ***
Library SeleniumLibrary
*** Variables ***
${IEDriver} D:\\IEDriver\\64bits\\IEDriverServer.exe
*** Test Cases ***
Example Test
Open Browser https://www.google.com.tw/ ie executable_path=${IEDriver} options=ignore_zoom_level=True; attach_to_edge_chrome=True; edge_executable_path="C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe"
Maximize Browser Window
After I executing my code, I can open google in Edge with IE mode, but after that web page stuck and always get thie error message:
TimeoutException: Message: Timed out waiting for page to load.
And I found something interesting: I use python and selenium and below is my code:
ieOptions = webdriver.IeOptions()
ieOptions.add_additional_option("ie.edgechromium", True)
ieOptions.add_additional_option("ie.edgepath",'C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe')
driver = webdriver.Ie(options=ieOptions)
driver.maximize_window()
driver.get('https://google.com')
driver.quit()
If I remove this line "driver.get('https://google.com')" and my code runs perfectly.
But If I add it back, the page will go to goole and stuck there (It means that this code will not do driver.quit()
This option may work for you in Windows 11 IE mode automation. Check it out:
ignore_protected_mode_settings = True
UPDATE
Or you can simply try setting the page load timeout according to your requirement:
*** Variables ***
${orig timeout} 15 seconds
*** Test Cases ***
Example Test
Set Selenium Timeout ${orig timeout}
If the element you'd like to interact with has been fully loaded, you can skip the page loading and continue. A native Python example:
driver.set_page_load_timeout(10)
try:
driver.get('https://google.com')
except TimeoutException:
print
'!!!!!!time out after 10 seconds!!!!!!'
driver.execute_script("window.stop()")
In Robot Framework:
*** Test Cases ***
Example Test
Set Selenium Timeout ${orig timeout}
TRY
Open Browser https://www.google.com ie options=ignore_zoom_level=True; attach_to_edge_chrome=True; edge_executable_path="C:\\Program Files (x86)\\Microsoft\\Edge\\Application\\msedge.exe"
EXCEPT '!!!!!!time out after 10 seconds!!!!!!'
Execute Javascript window.stop()
END
Maximize Browser Window

Unable to close download notification(after successfully download) in mozilla firefox through selenium python 3

Selenium Version = 3.141,
Mozilla Firefox Version : 101.0.1
Automation Framework : Pytest
After successfully downloading the file, we want the notification to be closed.
I am able to download the file, but unable to perform any operation because of the notification that appears at the right top of the mozilla firefox browser. I want to close that notification through selenium code but it seems not working.
Have tried following solutions:
options = webdriver.FirefoxOptions()
# options = FirefoxProfile()
options.set_preference("dom.webnotifications.enabled", False)
options.set_preference("dom.push.enabled", False)
options.set_preference("browser.helperApps.neverAsk.saveToDisk","application/pptx, application/csv, application/ris, text/csv, image/png, application/pdf, text/html, text/plain, application/zip, application/x-zip, application/x-zip-compressed, application/download, application/octet-stream")
options.set_preference("browser.download.panel.shown", False)
options.set_preference("browser.download.folderList", 2)
options.set_preference("browser.download.manager.showWhenStarting", False)
options.set_preference("browser.download.dir", os.path.join(parent_folder, 'Downloads'))
options.set_preference("browser.download.manager.closeWhenDone", True)
options.set_preference("browser.download.manager.showAlertOnComplete", False)
# options.add_argument("--headless")
# options.headless = True
web_driver = webdriver.Firefox(firefox_options=options,executable_path=GeckoDriverManager().install())
Solution 2:
After successfully downloading the file, we are trying to press ESC key to dismiss the notification but this also seems not working
self.driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + Keys.ENTER)
OR
ActionChains(self.driver).send_keys(Keys.ESCAPE).perform()
Solution 3
Tried to Refresh the screen as well, but notification is still there.
Image is attached below:
File is being successfully downloaded but unable to close this notification through selenium python.
You are using the incorrect MIME type for pptx, so that may be causing the notification to hang around, so do following update to see if it resolves the issue:
from application/pptx
to application/vnd.openxmlformats-officedocument.presentationml.presentation
More info on MIME types can be found here: Common MIME types - HTTP | MDN
UPDATE
Mozilla has introduced changes from Firefox 97+, one of them is to the download panel and the behaviour is what you experienced. For selenium purposes, you can stop the panel from opening using the following option:
options.set_preference("browser.download.alwaysOpenPanel", False)

Is there a way to add -Dchrome.switches chrome properties to serenity.properties or serenity.conf files?

I tried to add the below in the serenity.conf file to always load the chrome browser with these options but it fails to load the browser. When I pass in the below options via command line like so "gradle test -Dchrome.switches="--no-sandbox,--ignore-certificate-errors,--homepage=about:blank,--no-first-run" the browser starts successfully.
"-Dchrome.switches="--no-sandbox,--ignore-certificate-errors,--homepage=about:blank,--no-first-run"
Is there a way to always open chrome browser without having to pass this via command line or have the chrome driver as part of the framework?
serenity.conf
#
# WebDriver configuration
#
webdriver {
driver = chrome
autodownload = true
}
#headless.mode = true
serenity.test.root = java
#
# Chrome options can be defined using the chrome.switches property
#
chrome.switches = """--start-maximized;--test-type;--no-sandbox;--ignore-certificate-errors;
--disable-popup-blocking;--disable-default-apps;--disable-extensions-file-access-check;
--disable-web-security;--incognito;--disable-infobars,--disable-gpu,--homepage=about:blank,--no-first-run"""
Thanks!
Try
chrome {
switches = "--start-maximized;--enable-automation;--no-sandbox;--disable-popup-blocking;--disable-default-apps;--disable-infobars;--disable-gpu;--disable-extensions;"
preferences {
download: "{prompt_for_download: false,directory_upgrade: true,default_directory:'${user.dir}/downloaded-files'}"
}
}
Thank you!
I have switched to
#Managed
WebDriver driver;
https://serenity-bdd.github.io/theserenitybook/latest/web-testing-in-serenity.html#_a_simple_selenium_web_test
Below is the information on manned drivers in serenity.
Serenity reduces the amount of code you need to write and maintain when you write web tests. For example, it takes care of creating WebDriver instances, and of opening and closing the browser for you. The following is a very simple Selenium web test using Serenity:

Selenium not opening Firefox

So I am creating a simple bot to like people's posts, just for proof of concept and learning. I am using pycharm, along with selenium and geckodriver as my driver. As of now, I am simply trying to get the firefox web browser to open.
I have already added geckodriver to my PATH, and my computer recognizes Geckodriver. I have also tried using chromedriver and have it open chrome. The same thing happens, it simply does not open. Therefore, I feel as though there is a problem in my code or my external libraries, but I just can't seem to find it. Am I supposed to have some external library other than Python 3.7?
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
class InstagramBot:
def __init__(self, username, password):
self.username = username
self.password = password
self.driver = webdriver.Firefox()
def closeBrowser(self):
self.driver.close()
def login(self):
driver = self.driver
driver.get("https://www.instagram.com/")
time.sleep(2)
# "//a[#href'accounts/login']"
# "//input [#name='username']"
# "//input [#name='password']"
ig = InstagramBot("scumbag_scarbs, Sunny9999")
ig.login()
I expected it to open firefox and navigate to instagram.com. I do not recieve an error, but it just shows the program as running and not doing anything.
Please help, I found a bunch of people with problems similar to mine but nothing exactly the same.

Issue in taking screenshot in selenium while using chromedriver 73 for chrome version 73

When i tried to take screenshot of a webpage using selenium in python, i get error message selenium.common.exceptions.TimeoutException: Message: timeout: Timed out receiving message from renderer: 10.000.
Code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
indi_url = 'http://www.google.com'
options = Options()
options.add_argument("disable-infobars")
options.add_argument("--start-maximized")
options.add_argument("--disable-popup-blocking")
options.add_argument("disable-popup-blocking")
options.add_argument("--disable")
driver = webdriver.Chrome(options=options)
driver.get(indi_url)
driver.implicitly_wait(30)
driver.save_screenshot("new.png")
Error message:
I'm using Chrome version 73, chromedriver version 73.
Note: code was working fine (ie.screenshot)in lower version of chrome and chrome driver.
Help me out in fixing this issue for new version of chrome driver.
Thanks in advance
As the error shows, your filename for screenshot does not match the template extensions .png
Here is an example how to make a screenshot.
Java:
File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
FileUtils.copyFile(scrFile, new File(".\\Screenshots\\example_screenshot.png"));
Python:
driver.save_screenshot("screenshot.png")
This error message...
UserWarning: name used for saved screenshot does not match file type. It should end with a .png extension
"type. It should end with a .png extension", UserWarning)
...implies that the Selenium-Python client encountered an issue while invoking get_screenshot_as_file() method.
get_screenshot_as_file()
get_screenshot_as_file() saves a screenshot of the current window to a PNG image file. Returns False if there is any IOError, else returns True. Use full paths in your filename.
Args:
filename: The full path you wish to save your screenshot to. This should end with a .png extension.
Usage:
driver.get_screenshot_as_file('/Screenshots/foo.png')
Defination:
if not filename.lower().endswith('.png'):
warnings.warn("name used for saved screenshot does not match file "
"type. It should end with a `.png` extension", UserWarning)
png = self.get_screenshot_as_png()
try:
with open(filename, 'wb') as f:
f.write(png)
except IOError:
return False
finally:
del png
return True
Analysis
As per the snapshot of the error stack trace:
You have used the command as:
driver.get_screenshot_as_file('new.jpeg')
The issues were:
The filename didn't end with .png
The desired full path of your filename wasn't provided.
Even if you desire to use save_screenshot() this method in-turn invokes get_screenshot_as_file(filename)
Solution
Create a directory within your project as Screenshots and provide the absolute path of the filename you desire for the screenshot while invoking either of the methods as follows:
driver.get_screenshot_as_file("./Screenshots/YakeshrajM.png")
driver.save_screenshot("./Screenshots/YakeshrajM.png")
Update
Currently GAed Chrome v73 have some issues and you may like to downgrade to Chrome v72. You can find a couple of relevant discussions in:
Getting Timed out receiving message from renderer: 600.000 When we execute selenium scripts using Jenkins windows service mode
Timed out receiving message from renderer: 10.000 while capturing screenshot using chromedriver and chrome through Jenkins on Windows