Why did Selenium stop working today on Colab? - selenium

Screen Shot
Link to workbook showing issue:`
https://colab.research.google.com/drive/1Zf8YTHRLOL6-ez1SWvzZEVZpkelrG2Kz?usp=sharing
!apt-get update
!apt install chromium-chromedriver
!pip install selenium
from selenium import webdriver
Google Colab + Account on Chrome Browserenter image description here
I have run various scripts for the last two years and no issues.
Today I get the following error:
ImportError: cannot import name 'Literal' from 'typing' (/usr/lib/python3.7/typing.py)

The error relates to the typing library. I do not know exactly why this is happening and I hope someone has a better solution than mine, but editing the file that calls this library worked for me:
# Install Normal Selenium Stuff
!pip install selenium
!apt-get update # to update ubuntu to correctly run apt install
!apt install chromium-chromedriver
!cp /usr/lib/chromium-browser/chromedriver /usr/bin
import sys
sys.path.insert(0,'/usr/lib/chromium-browser/chromedriver')
## This edits the part of the file that gives the error:
with open('/usr/local/lib/python3.7/dist-packages/selenium/webdriver/common/virtual_authenticator.py', 'r') as file:
# read a list of lines into data
data = file.readlines()
data[21] = 'from typing_extensions import Literal\n'
with open('/usr/local/lib/python3.7/dist-packages/selenium/webdriver/common/virtual_authenticator.py', 'w') as file:
file.writelines( data )
# Calls Selenium (It is important to edit the file before calling seleninum):
from selenium import webdriver
I know this is probably not the best solution, but it worked for me. I hope someone can come up with a better solution and explain why this is happening. Meanwhile, this will probably work for most people.

Related

No module named 'stable_baseline3' even when it is installed in google colab

I am trying to set up stable baselines 3 in google colab. The document is connected to a local runtime on my pc through jupyter notebooks. On my pc i have installed stable baselines 3 using anaconda, and got the output saying essentially stable baselines 3 is installed. I have also run the cells:
!pip install stable-baselines3[extra]
!pip install stable-baselines3
and
!pip install stable-baselines3 --upgrade
Despite this, when i run the cell:
import stable_baseline3
from stable_baselines3 import DQN
etc...
I get the error on line 1 of ModuleNotFoundError: No module named 'stable_baseline3'. I dont understand why this would be happening, does anybody know how it could be solved?
i had the same problem
try to import stable-baselines3 first in alone cell and it should work
!pip install stable-baselines3

why does matplotlib refuses to show plots in pycharm [duplicate]

I am trying to plot a simple graph using pyplot, e.g.:
import matplotlib.pyplot as plt
plt.plot([1,2,3],[5,7,4])
plt.show()
but the figure does not appear and I get the following message:
UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
I saw in several places that one had to change the configuration of matplotlib using the following:
import matplotlib
matplotlib.use('TkAgg')
import matplotlib.pyplot as plt
I did this, but then got an error message because it cannot find a module:
ModuleNotFoundError: No module named 'tkinter'
Then, I tried to install "tkinter" using pip install tkinter (inside the virtual environment), but it does not find it:
Collecting tkinter
Could not find a version that satisfies the requirement tkinter (from versions: )
No matching distribution found for tkinter
I should also mention that I am running all this on Pycharm Community Edition IDE using a virtual environment, and that my operating system is Linux/Ubuntu 18.04.
I would like to know how I can solve this problem in order to be able to display the graph.
Solution 1: is to install the GUI backend tk
I found a solution to my problem (thanks to the help of ImportanceOfBeingErnest).
All I had to do was to install tkinter through the Linux bash terminal using the following command:
sudo apt-get install python3-tk
instead of installing it with pip or directly in the virtual environment in Pycharm.
Solution 2: install any of the matplotlib supported GUI backends
solution 1 works fine because you get a GUI backend... in this case the TkAgg
however you can also fix the issue by installing any of the matplolib GUI backends like Qt5Agg, GTKAgg, Qt4Agg, etc
for example pip install pyqt5 will fix the issue also
NOTE:
usually this error appears when you pip install matplotlib and you are trying to display a plot in a GUI window and you do not have a python module for GUI display.
The authors of matplotlib made the pypi software deps not depend on any GUI backend because some people need matplotlib without any GUI backend.
In my case, the error message was implying that I was working in a headless console. So plt.show() could not work. What worked was calling plt.savefig:
import matplotlib.pyplot as plt
plt.plot([1, 2, 3], [5, 7, 4])
plt.savefig("mygraph.png")
I found the answer on a github repository.
If you use Arch Linux (distributions like Manjaro or Antegros) simply type:
sudo pacman -S tk
And all will work perfectly!
Simple install
pip3 install PyQt5==5.9.2
It works for me.
Try import tkinter because pycharm already installed tkinter for you, I looked Install tkinter for Python
You can maybe try:
import tkinter
import matplotlib
import matplotlib.pyplot as plt
matplotlib.use('TkAgg')
plt.plot([1,2,3],[5,7,4])
plt.show()
as a tkinter-installing way
I've tried your way, it seems no error to run at my computer, it successfully shows the figure. maybe because pycharm have tkinter as a system package, so u don't need to install it. But if u can't find tkinter inside, you can go to Tkdocs to see the way of installing tkinter, as it mentions, tkinter is a core package for python.
I added %matplotlib inline
and my plot showed up in Jupyter Notebook.
The answer has been given a few times but it is not obvious, one needs to install graphics, this works.
pip3 install PyQt5
I too had this issue in PyCharm. This issue is because you don't have tkinter module in your machine.
To install follow the steps given below (select your appropriate os)
For ubuntu users
sudo apt-get install python-tk
or
sudo apt-get install python3-tk
For Centos users
sudo yum install python-tkinter
or
sudo yum install python3-tkinter
for Arch Users
sudo pacman -S tk
or
sudo pamac install tk
For Windows, use pip to install tk
After installing tkinter restart your Pycharm and run your code, it will work
This worked with R reticulate. Found it here.
1: matplotlib.use( 'tkagg' )
or
2: matplotlib$use( 'tkagg' )
For example:
import numpy as np
import matplotlib.pyplot as plt
from matplotlib import style
import matplotlib
matplotlib.use( 'tkagg' )
style.use("ggplot")
from sklearn import svm
x = [1, 5, 1.5, 8, 1, 9]
y = [2, 8, 1.8, 8, 0.6, 11]
plt.scatter(x,y)
plt.show()
If using Jupyter notebook try the following:
%matplotlib inline
This should render the plot even if not specifying the
plt.show()
command.
None of these answers worked for me using Pycharm Professional edition 2021.3
Regular matplotlib graphs did work on the scientific view, but it did not allow me to add images to the plots.
What did work for me is adding this line before I try plotting anything:
plt.switch_backend('TkAgg')
issue = “UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.”
And this worked for me
import matplotlib
import matplotlib.pyplot as plt
matplotlib.use('Qt5Agg')
For Windows 10, if using pip install tk does not work for you, try:
Download and run official python installer for windows. Even if you
already have it downloaded, run it again.
When (re)installing python, make sure you chose "advanced" options, and
set the checkbox "tcl/tk and IDLE" to true.
If you already had python installed, select the "Modify" option, and
make sure that checkbox is selected.
Source of my fix:
https://stackoverflow.com/a/59970646/2506354
I have solved it by putting matplotlib.use('TkAgg') after all import statements.
I use python 3.8.5 VSCODE and anaconda.
No other tricks worked.
I installed python3-tk , on Ubuntu 20.04 and using WSL2
import numpy as np
import matplotlib.pyplot as plt
import matplotlib
matplotlib.use( 'tkagg')
and then I installed GWSL from the Windows Store which seems to solve problem of WSL2 rendering out of the box
This will solve the issue. It works well in jupyter.
%matplotlib inline
The comment by #xicocaio should be highlighted.
tkinter is python version-specific in the sense that sudo apt-get install python3-tk will install tkinter exclusively for your default version of python. Suppose you have different python versions within various virtual environments, you will have to install tkinter for the desired python version used in that virtual environment. For example, sudo apt-get install python3.7-tk. Not doing this will still lead to No module named ' tkinter' errors, even after installing it for the global python version.
On Mac OS, I made it work with:
import matplotlib
matplotlib.use('MacOSX')
Ubuntu 20.04 command line setup. I install the following to make Matplotlib stop throwing the error UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
I installed python-tk through the steps:
apt-get update
apt-get install python3.8-tk
Just in case if this helps anybody.
Python version: 3.7.7
platform: Ubuntu 18.04.4 LTS
This came with default python version 3.6.9, however I had installed my own 3.7.7 version python on it (installed building it from source)
tkinter was not working even when the help('module') shows tkinter in the list.
The following steps worked for me:
sudo apt-get install tk-dev.
rebuild the python:
1. Navigate to your python folder and run the checks:
cd Python-3.7.7
sudo ./configure --enable-optimizations
Build using make command:
sudo make -j 8 --- here 8 are the number of processors, check yours using nproc command.
Installing using:
sudo make altinstall
Don't use sudo make install, it will overwrite default 3.6.9 version, which might be messy later.
Check tkinter now
python3.7 -m tkinter
A windows box will pop up, your tkinter is ready now.
After upgrading lots of packages (Spyder 3 to 4, Keras and Tensorflow and lots of their dependencies), I had the same problem today! I cannot figure out what happened; but the (conda-based) virtual environment that kept using Spyder 3 did not have the problem. Although installing tkinter or changing the backend, via matplotlib.use('TkAgg) as shown above, or this nice post on how to change the backend, might well resolve the problem, I don't see these as rigid solutions. For me, uninstalling matplotlib and reinstalling it was magic and the problem was solved.
pip uninstall matplotlib
... then, install
pip install matplotlib
From all the above, this could be a package management problem, and BTW, I use both conda and pip, whenever feasible.
You can change the matplotlib using backend using the from agg to Tkinter TKAgg using command
matplotlib.use('TKAgg',warn=False, force=True)
Works if you use some third party code in your project. It probably contains the following line
matplotlib.use('Agg')
Search for it and comment it out.
If you have no clue about what it is you are probably not using this part of the code.
Solutions about using another backend GUI may be cleaner, so choose your fighter.
The solution that worked for me:
Install tkinter
import tkinter into the module
make sure that matplotlib uses (TkAgg) instead of (Agg)
matplotlib.use('TkAgg')
execute the following command before plotting
%matplotlib inline
Try:
%matplotlib inline
I had the same problem and it worked for me. I tested it on my Jupyter notebooks and visual studio code, so you should have no problems.
On WSL with X server
Make sure that your X server work. Matplotlib indicate this error if he can't connect to the X display.
Windows Firewall configuration
Pay attention to the windows firewall ! I changed from WSL Debian to Ubuntu and didn't remember about the firewall rule.
I use this post to configure the windows firewall rule to make the X server work. This method avoid too permisive rule that able anyone to use your X server.
It said :
If you already had installed an X11 server, Windows may have created firewall rules that will mess with the above configuration. Search for them and delete them in "Windows Defender Firewall with Advanced Security."
You will now need to configure Windows Firewall to permit connections from WSL2 to the X11 display server. You will install the display server in the next step. We do this step first to avoid Windows Firewall from auto-creating an insecure firewall rule when you run the X11 display server. Many guides on X11 forwarding and WSL2 make this firewall rule too permissive, allowing connections from any computer to your computer. This means someone could theoretically, if they are on your same network, start sending graphical display information to your computer.
To avoid this, we will make Windows Firewall only accept internet traffic from the WSL2 instance.
To set this up, you can copy the below to a script and run it from within WSL2:
#!/bin/sh
LINUX_IP=$(ip addr | awk '/inet / && !/127.0.0.1/ {split($2,a,"/"); print a[1]}')
WINDOWS_IP=$(ip route | awk '/^default/ {print $3}')
# Elevate to administrator status then run netsh to add firewall rule
powershell.exe -Command "Start-Process netsh.exe -ArgumentList \"advfirewall firewall add rule name=X11-Forwarding dir=in action=allow program=%ProgramFiles%\VcXsrv\vcxsrv.exe localip=$WINDOWS_IP remoteip=$LINUX_IP localport=6000 protocol=tcp\" -Verb RunAs"
Manual method :
Alternatively, you can manually add the rule through a GUI by doing the following:
Open "Windows Defender Firewall with Advanced Security"
Click add new rule brings up the New Rule Wizard (next to navigate between each section):
Rule type: Custom
Program: "This program path:" %ProgramFiles%\VcXsrv\vcxsrv.exe
Protocol and ports
Protocol type: TCP
Local port: 6000
Remote port: any
Scope
Local IP address: Obtain the IP address to put in by running the below command in WSL2
ip route | awk '/^default/ {print $3}'
remote IP addresses
Obtain IP address to enter by running the below in WSL2
ip addr | awk '/inet / && !/127.0.0.1/ {split($2,a,"/"); print a[1]}'
Action: "Allow the connection
Profile: Selection Domain, Private, and Public
Name: "X11 forwarding"
Linux Mint 19. Helped for me:
sudo apt install tk-dev
P.S. Recompile python interpreter after package install.
When I ran into this error on Spyder, I changed from running my code line by line to highlighting my block of plotting code and running that all at once. Voila, the image appeared.
If you install python versions using pyenv on Debian-based systems, be sure to run sudo apt install tk-dev before pyenv install. If it's already installed, remove it with pyenv uninstall and install it again after install tk-dev. Therefore, there is no need to set any env variables when running pyenv install.

install "pip undetected-chromedriver" for selenium python

I'm trying to make an autofiller using selenium, but it couldn't be done. so I decided to use undetected chromedriver to finish the automation.
I am having some difficulty here to import the undetected-chromedriver.
I already downloaded it by inputting the command line: pip install undetected-chromedriver
But when I put the import undetected_chromedriver as uc, the complier doesn't recognize it.
Below is the Error message after trying to import undetected-chromedriver:
import undetected_chromedriver as uc
ModuleNotFoundError: No module named 'undetected_chromedriver'
Use the following command to check if the undetected_chromedriver package is in the list
pip list
or
pip3 list
Try the following
# navigate into the project directory with your python script
cd presearch
# create virtual environment
python3 -m venv venv
# activate the virtual environment
source venv/bin/activate
# install required pip packages
pip3 install undetected-chromedriver
If you have multiple python versions installed, you might check if you actually installed it in the right one.

"cannot find Chrome binary error" when running Selenium chrome driver in a Google Cloud instance (on jupyter Lab)

I'm using a Jupyter Lab notebook which run on google cloud instance. After I installed chromedriver using the following command line:
wget https://chromedriver.storage.googleapis.com/85.0.4183.87/chromedriver_linux64.zip
then I unzip it with:
unzip chromedriver_linux64.zip
After which I run the following cell:
from selenium import webdriver
path = '/home/jupyter/chromedriver'
driver = webdriver.Chrome(path)
And this is where I get the error : WebDriverException: Message: unknown error: cannot find Chrome binary (I'm pretty sure the path to Chromedriver is right though)
This work perfectly on my local machine but I'm a bit lost as to why I can't have it work on a Google Cloud instance. Can you please help me out with this issue ? (Also I have tried other things by looking at other posts online but without success). Thank you
Here is an example on how to install Selenium on a Jupyter notebook.
It is recommended to install it wit pip:
!pip install selenium
and to import os and sys:
import os
import sys
os.path.dirname(sys.executable)

Can't run pytest + selenium

Please help me!
When I trying to run my simple test with pytest+selenium it is not starting. Just nothing happens:
import pytest
def test_example(selenium):
selenium.get('http://www.example.com')
Screenshot https://gyazo.com/01d93c221cc6acff0446f1fb473000cb
Note: I'm doing the same at my work PC - it works!
I'm idiot (((
Install pytest-selenium
To install pytest-selenium using pip:
$ pip install pytest-selenium
I thought it was enough:
pip install pytest
pip install selenium
I want to note that you can use pytest-selenium but you can also just use pytest and selenium (e.g. selenium==3.0.2 within your requirements.txt) file. The problem that you were facing just looked like a dependency issue.
I also want to note that the screenshot you uploaded contains sensitive information with your staging environment. You should edit it or delete the file entirely.