I have a code in cmd that opens an arbitrary Chrome profile and it works very well.
start "" G:\Multi_Chrome_Part_1\ChromeFirstProVip\GoogleChromePortable.exe" --profile-directory="Profile 17". Tt shows up as usual:
But I'm having a lot of trouble opening with Selenium. I use the code below:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("user-data-dir=G:\\Multi_Chrome_Part_1\\ChromeFirstProVip\\Data\\profile\\Profile 17")
driver = webdriver.Chrome(service=Service("D:\\Python\\FirstProject\\chromedriver_99_0_4844_51.exe"), options=options)
driver.get("https://www.google.co.in")
With selenium, it cannot open a complete profile, missing gmail. It looks like this:
As you can see, in addition to not accepting pre-imported gmail, it also lacks the extensions that I have installed before, as well as other components. Some websites are also not accessible with Selenium
I want to ask is there a way to turn the code above into the bottom? Is there a way to make Selenium like a normal Chrome profile? I don't understand why opening with cmd is perfect, but selenium has problems. Thank you very much if you can help
Related
I disabled loading images in chrome while using webdriver with selenium now cant enable it.
I was using python to webscrape on instagram so thought it would be a good idea to disable images.
The commands i used:
options = webdriver.ChromeOptions()
options.add_argument("--disable-gpu=true")
options.add_argument("--blink-settings=imagesEnabled=false")
And now I cannot change it from chrome settings.
Screenshot of chrome settings page.
Please Help.
Edit: This happens only in my default Chrome Profile. Other Profiles work fine even though the profile I use for selenium is a different one.
After a long day on google finally found the solution.
options.add_experimental_option("prefs", {'profile.managed_default_content_settings.images': 1})
Is there a way to run selenium headless but have it display the window for example at the start of the application ? like:
show browser
login
do captcha
go --headless
execute tasks
Actually, this is possible.
You can peek into a headless browser by using the "--remote-debugging-port" argument in ChromeOptions. For example,
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--remote-debugging-port=9222') # Recommended is 9222
driver = webdriver.Chrome(options=chrome_options)
Then while Selenium is running, you can go to http://localhost:9222 in your regular browser to view the headless session. Your browser will display links to each tab Chrome has open, and you can view/interact with each page by clicking on the links. You'll also be able to monitor Selenium/Chrome as Selenium runs its test.
Edit: As of Chrome 100 or so, you now need to go to the link chrome://inspect to view the headless session in your browser while Selenium is running.
You'll need to do some extra configuration as well. Basically:
Check the box for "Discover network Targets"
Hit "Configure..."
Add the link "localhost:9222" to the list of servers
CREDITS: https://stackoverflow.com/a/58045991/1136132
I have coded a script to perform some actions while logged into my youtube account, using my Chrome user data folder:
from selenium.webdriver.chrome.options import Options
from selenium import webdriver
options = Options()
options.add_argument("user-data-dir=C:\\Users\\Username\\AppData\\Local\\Google\\Chrome\\User Data")
options.add_argument('--headless')
driver = webdriver.Chrome(chrome_options=options)
driver.get(https://youtube.com)
driver.get_screenshot_as_file("screenshot.png")
thing = driver.find_element_by_xpath(
'someXpath')
thing.click()
It works perfectly without the line:
options.add_argument('--headless')
However, when "headless" is added as an option, the screenshot taken by driver.get_screenshot_as_file shows that chrome is prompting a login to a google account, indicating that the specified chrome profile is not being used.
If possible, I'd like to use both headless mode and use my chrome profile at the same time. Thanks for any help.
My goal is it to make a request by using the seleniumrequest module. This is working fine. But now I often get captchas that are harder than the normal ones. So I assume the site is able to detect that I use selenium. So the next step would be that I change the Chrome Options and disable the "attribute" that I am using an automated software. I know how to do it for selenium alone, but it seems that seleniumrequest does not support that. Can somebody confirm this or can tell me what I did wrong.
from seleniumrequests import Chrome
webdriver = Chrome()
option = webdriver.ChromeOptions()
option.add_argument("--disable-blink-features=AutomationControlled")
webdriver = webdriver.Chrome(executable_path=r"chromedriver.exe",options=option)```
option.add_experimental_option("excludeSwitches", ["enable-automation"])
Would be the options for switching off automated software.
Also ChromeOptions is depreciated use Options instead.
from selenium.webdriver.chrome.options import Options
option = Options()
You can enable the "reader-mode" by opening chrome://flags/#enable-reader-mode in Google Chrome.
Then, you can toggle the "reader mode" while browsing a webpage:
How to get the "reader mode" version of a web page using Selenium and chromedriver?
Not really the answer you probably want. You can enable the "reader-mode" feature by using ChromeOptions.
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--enable-features=ReaderMode")
driver = webdriver.Chrome(options=options)
driver.get(url)
Then you have to toggle it somehow. The Selenium documentation is quite clear that it is for not for testing browser functionality see https://stackoverflow.com/a/49801432/839338 so you are left with using another automation tool like AutoIt https://www.autoitscript.com/site/ or Sikuli http://doc.sikuli.org/ to find and toggle the "reader-mode" menu item. I'm not sure how you go about that using scrapy.