Is there a way to run selenium headless but have it display the window for example at the start of the application ? like:
show browser
login
do captcha
go --headless
execute tasks
Actually, this is possible.
You can peek into a headless browser by using the "--remote-debugging-port" argument in ChromeOptions. For example,
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--remote-debugging-port=9222') # Recommended is 9222
driver = webdriver.Chrome(options=chrome_options)
Then while Selenium is running, you can go to http://localhost:9222 in your regular browser to view the headless session. Your browser will display links to each tab Chrome has open, and you can view/interact with each page by clicking on the links. You'll also be able to monitor Selenium/Chrome as Selenium runs its test.
Edit: As of Chrome 100 or so, you now need to go to the link chrome://inspect to view the headless session in your browser while Selenium is running.
You'll need to do some extra configuration as well. Basically:
Check the box for "Discover network Targets"
Hit "Configure..."
Add the link "localhost:9222" to the list of servers
CREDITS: https://stackoverflow.com/a/58045991/1136132
Related
I have coded a script to perform some actions while logged into my youtube account, using my Chrome user data folder:
from selenium.webdriver.chrome.options import Options
from selenium import webdriver
options = Options()
options.add_argument("user-data-dir=C:\\Users\\Username\\AppData\\Local\\Google\\Chrome\\User Data")
options.add_argument('--headless')
driver = webdriver.Chrome(chrome_options=options)
driver.get(https://youtube.com)
driver.get_screenshot_as_file("screenshot.png")
thing = driver.find_element_by_xpath(
'someXpath')
thing.click()
It works perfectly without the line:
options.add_argument('--headless')
However, when "headless" is added as an option, the screenshot taken by driver.get_screenshot_as_file shows that chrome is prompting a login to a google account, indicating that the specified chrome profile is not being used.
If possible, I'd like to use both headless mode and use my chrome profile at the same time. Thanks for any help.
For a web-test i call an online-shop which uses a special GDPR-cookie-banner. When I call this online store in a normal chrome browser, it is loaded and displayed.
However, when I call this online store with the test software (chromedriver, Selenium, Python), it is loaded but not displayed.
What is the reason and what can I do to display this banner?
Online-shop: https://www.uwaldu.de/
Browser snapshot with normal chrome browser:
Browser snapshot with webdriver:
Not sure why you won't see the special GDPR-cookie-banner. But when I access the website using a Selenium driven ChromeDriver initiated google-chrome Browsing Context the GDPR-cookie-banner is displayed perfectly everytime.
Code Block:
options = Options()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--disable-blink-features=AutomationControlled')
s = Service('C:\\BrowserDrivers\\chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.get("https://uwaldu.de/")
Browser Snapshot:
You can enable the "reader-mode" by opening chrome://flags/#enable-reader-mode in Google Chrome.
Then, you can toggle the "reader mode" while browsing a webpage:
How to get the "reader mode" version of a web page using Selenium and chromedriver?
Not really the answer you probably want. You can enable the "reader-mode" feature by using ChromeOptions.
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--enable-features=ReaderMode")
driver = webdriver.Chrome(options=options)
driver.get(url)
Then you have to toggle it somehow. The Selenium documentation is quite clear that it is for not for testing browser functionality see https://stackoverflow.com/a/49801432/839338 so you are left with using another automation tool like AutoIt https://www.autoitscript.com/site/ or Sikuli http://doc.sikuli.org/ to find and toggle the "reader-mode" menu item. I'm not sure how you go about that using scrapy.
I'm using Selenium to automate a navigation in Google Meet website for a user who can't use the keyboard. However, Google Meet won't let me enter a meeting when using Chrome in test mode. If I configure Chrome webdriver to run as a regular browser, I can navigate on the website a little but eventually I can't enter a meeting at all. Here is the python code I use to initialize Chrome as a non-test browser:
options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches", ['enable-automation'])
browser = webdriver.Chrome()
browser.get("https://meet.google.com/some_meeting_id")
time.sleep(3)
txtName = browser.find_element_by_css_selector('#jd\.anon_name')
txtName.send_keys("user_name" + Keys.RETURN) // redirects to an error page
My next hope is to use Firefox, but when I load it from webdriver, it opens with an orange address bar, indicating that it is being run by a test tool. Is there a way to run Firefox in normal mode from Selenium (just as I did with Chrome) or, even better, is there any additional configuration I can do in Chrome webdriver to make this work?
May be try this to switch the headless mode on firefox to normal mode
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
options = Options()
options.headless = False
driver = webdriver.Firefox(options=options)
driver.get("http://stackoverflow.com")
I'm trying to make a web crawler that click on ads (yes, i know), it's very sophisticated, but, I realise that Google Ads aren't showed when javascript is disabled. Today, i use Mechanize, and it doesn't "accept" javasript.
I heard selenium use another system to crawl the net.
The only thing I want to do is access my page, and click on the ad (generated by javascript).
Can Selenium do it ?
Selenium is a browser automation tool. You can basically automate everything you can do in your browser. Start with going through the Getting Started section of the documentation.
Example:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://www.python.org")
print driver.title
driver.close()
Besides automating common browsers, like Chrome, Firefox, Safari or Internet Explorer, you can also use PhantomJS headless browser.