I have been using Selenium in Google Colab to do some scrapping for some time now. I use:
# install chromium, its driver, and selenium
!apt update
!apt install chromium-chromedriver
!pip install selenium
# set options to be headless, ..
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
# open it, go to a website, and get results
wd = webdriver.Chrome('chromedriver',options=options)
wd.get("https://www.website.com")
print(wd.page_source) # results
And until today it worked perfectly, but today I ran this code and it started giving this error:
ImportError: cannot import name 'Literal' from 'typing' (/usr/lib/python3.7/typing.py)
I have tried manually installing typing.py, I've tried updating the python version, but nothing has worked. Can some one please help me to run Selenium (with Chrome) in Google Colab?
I've noticed that the same error happens in all answers I have found about this topic such as:
How to use Selenium on Colaboratory Google?
https://colab.research.google.com/github/restrepo/ComputationalMethods/blob/master/tools/selenium.ipynb
https://gist.github.com/korakot/5c8e21a5af63966d80a676af0ce15067
Related
Writing a simple selenium script to click on links on aa website. The script is written like so:
from selenium import webdriver
import time
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)
try:
browser.get("https://www.google.com")
print("Page title was '{}'".format(browser.title))
finally:
browser.quit()
Now the issue is the actual chrome driver itself I get the following exception
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 94
Current browser version is 93.0.4577.82 with binary path /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
I went to the chromedriver downloads site. I still get the same error though.
Compatibility issue.
Your chrome driver version is 94.0.4606.41 and this driver version supports Chrome browser 94
Please do anyone of the following.
Update the chrome browser version to 94
Degrade the driver version to 93 (Download 93 version from here https://chromedriver.storage.googleapis.com/index.html?path=93.0.4577.63/)
This error occurred because you have different versions of Google Chrome and driver. It is better to update the driver, rather than install the old version of Google, since in the future it will be constantly updated (why do you want to use outdated technologies?).
I usually use :
ChromeDriverManager
because at any time without going to the web driver website you can simply download the driver with the following command:
driver = webdriver.Chrome(ChromeDriverManager().install())
Further, using the path given by this command, you can use the freshly installed version:
driver = webdriver.Chrome(executable_path=r"C:\path_to_chrome_driver_executable\chromedriver.exe")
This made me go crazy I solved it like this we are using selenium npm module.
Run the code below and it will tell you what executable path you are using.
const { Builder, By, Key, util } = require("selenium-webdriver");
const chrome = require("selenium-webdriver/chrome");
console.log(chrome.getDefaultService().executable_);
I had installed chromedriver globally
npm i chromedriver -g
The executable path showed me it was using this and an older version.
Uninstalled this.
npm uninstall chromedriver -g
Now it started using the version I dowloaded and added to me PATH.
Download the latest chromedriver from here.
https://chromedriver.chromium.org/downloads
Add it to your path in the .zshrc file.
export PATH=/Users/dave/SeleniumWebdrivers:$PATH
Drag your downloaded drivers into this folder.
If you're using Mac run
brew reinstall chromedriver
I think there is another way to solve this problem. Uninstall the protarctor and reinstall it and see magic.
npm uninstall protractor
npm install protractor
Session not created: This version of ChromeDriver only supports
Issue is seen in selenium, selenium module not found even after installation of selenium via pip.
I have already installed pip in windows (global). It is working in Pycharm and other apps.
Driver Fine
working fine in Pycharm.
# Importing Selenium
from selenium import webdriver
import time
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.select import Select
# Terminal :
PS C:\Users\sudip neupane> pip3 install selenium
***Requirement already satisfied: selenium in c:\users\sudip*** neupane\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (3.141.0)
Requirement already satisfied: urllib3 in c:\users\sudip neupane\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from selenium) (1.26.5)
PS C:\Users\sudip neupane>
# ERROR
from selenium import webdriver
ModuleNotFoundError: ***No module named 'selenium***'
I found this issue was caused by having two python versions installed in my machine.
This worked after Reinstallation and Removing path from VS code while installing.
this can be solved by :
open file in VS Code and click CTRL+SHIFT + P
search -> Python: Select Interpreter
select any interpreter from list or and new Interpreter.
verify that selected interpreter have selenium modules.
Thanks!
Whenever I try to use Selenium module after pip installing the module it shows this error. I am not able to run my code and it continuously shows this error. It shows Win32 error. It also shows chromedriver executable needs to be in Path.
Try to use webdriver-manager package:
pip install webdriver-manager
the code should look like this:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(ChromeDriverManager().install())
The module download driver automatically and store it at known path for selenium. It will solve all of kinda troubles with driver path
Whenever I try to use Selenium module after pip installing the module it shows this error.
I am not able to run my code and it continuously shows this error. It shows Win32 error.
It also shows chromedriver executable needs to be in Path.
Try to use webdriver-manager package:
pip install webdriver-manager
the code should look like this:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(ChromeDriverManager().install())
The module download driver automatically and store it at known path for selenium. It will solve all of kinda troubles with driver path
"Using Selenium on PythonAnywhere" says:
Firefox only, selenium 2, geckodriver not required (…)
That (Firefox v17.0) is quite an old version, but it
works for most sites.
In my case it did not work. Is there a way to use Google Chrome (headless) anyway on PythonAnywhere?
I found this forum entry with the hint that it is not only possible to use Chrome, but even "You'll need to upgrade Selenium to the most recent version".
I simply wrote an email to the PythonAnywhere support and they enabled that for my (payed) account within hours.
Don't forget to start a new console or restart your web app! Then the following code should work:
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-gpu")
browser = webdriver.Chrome(options=chrome_options)
try:
browser.get("https://www.google.com")
print(f'Page title was {browser.title}')
finally:
browser.quit()