Why won't the VScode ChromeDriver display the website url I am trying to open? - beautifulsoup

I am working on a scraping project and have all recent downloads of python (3.9.5), VScode, Selenium, and BeautifulSoup. All the modules seem to be working correctly and I receive no errors when running the code. However, the URL is not opened by the web driver when I run the code - nothing happens? Please assist me in what I am missing so I am able to see my control window/display.
Code:
import csv
from bs4 import BeautifulSoup
from selenium import webdriver
# Startup the webdriver
driver = webdriver.Chrome()
url = 'https://www.amazon.com'
driver.get(url)
Output:
[Running] python -u "/Library/Frameworks/Python. framework/Versions/3.9/lib/python3.9/site-packages/
ScrapingTest.py"
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/bs4/element.py:16:
UserWarning: The soupsieve package is not installed. csS selectors cannot be used.
'The soupsieve package is not installed. CSS selectors cannot be used.'
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/ScrapingTest.py",
line 2, in <module>
from bs4 import BeautifulSoup
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/bs4/_init_-py",
line 32, in <module>
from .builder import builder_registry, ParserRejectedMarkup
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/bs4/builder/
_init_.py", line 7, in <module>
from bs4.element import (
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/bs4/element.py",
line 19, in <module>
from bs4. formatter import (
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/bs4/formatter.py",
line 1, in <module>
from bs4.dammit import EntitySubstitution
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/bs4/dammit.py",
line 68, in <module>
class EntitySubstitution(object):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/bs4/dammit.py",
line 97, in EntitySubstitution
CHARACTER_TO_HTML_ENTITY_RE) = _populate_class_variables ()
File "/Library/Frameworks/Python. framework/Versions/3.9/lib/python3.9/site-packages/bs4/dammit.py",
line 83, in _populate_class_variables
character = chr(codepoint)
ValueError: chr() arg not in range (256)
[Done] exited with code=1 in 0.198 seconds

The error message states that:
UserWarning: The soupsieve package is not installed. css selectors cannot be used.
Hence, installing the soupsieve package using pip install soupsieve should fix the error.

Related

matplotlib import error : libwebp.so.6: cannot open shared object file: No such file or directory

I have installed matplotlib via pip3 in venv. After importing I get the error as below. Have changed the directory long path to ...PATH.
>>> import matplotlib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "...PATH/lib/python3.11/site-packages/matplotlib/__init__.py", line 113, in <module>
from . import _api, _version, cbook, _docstring, rcsetup
File "...PATH/lib/python3.11/site-packages/matplotlib/rcsetup.py", line 27, in <module>
from matplotlib.colors import Colormap, is_color_like
File "...PATH/lib/python3.11/site-packages/matplotlib/colors.py", line 51, in <module>
from PIL import Image
File "...PATH/lib/python3.11/site-packages/PIL/Image.py", line 103, in <module>
from . import _imaging as core
ImportError: libwebp.so.6: cannot open shared object file: No such file or directory
After looking at pillow website I learnt that
Pillow >= 2.1.0 no longer supports import _imaging. Please use from
PIL.Image import core as _imaging instead.
I am using pillow 9.4.0. And the last line of matplotlib error says that it is importing _imaging, which is no more valid in pillow. Matplotlib version is 3.6.3.
For the missing shared library (libwebp), being complained in the error, I have verified that it is already installed via dev package.

Selenium ERROR in Run the simple program in Linux

I'm new in python and Linux also. I try to install selenium in python 3.7
but i got the error:
from selenium import webdriver
browser = webdriver.Firefox()
browser.get('http://www.ubuntu.com/')
output
Traceback (most recent call last):
File "test.py", line 1, in <module>
from selenium import webdriver
File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/__init__.py", line 18, in <module>
from .firefox.webdriver import WebDriver as Firefox # noqa
File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/firefox/webdriver.py", line 29, in <module>
from selenium.webdriver.remote.webdriver import WebDriver as RemoteWebDriver
File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 26, in <module>
from .webelement import WebElement
File "/usr/local/lib/python3.7/dist-packages/selenium/webdriver/remote/webelement.py", line 37, in <module>
from StringIO import StringIO as IOStream File
"/usr/lib/python3.7/StringIO.py", line 40
raise ValueError, "I/O operation on closed file"
^ SyntaxError: invalid syntax
you need to downgrade to python 3.6
Or if you do not have gecko driver go through following steps.
wget https://github.com/mozilla/geckodriver/releases/download/v0.23.0/geckodriver-v0.23.0-linux64.tar.gz
sudo sh -c 'tar -x geckodriver -zf geckodriver-v0.23.0-linux64.tar.gz -O > /usr/bin/geckodriver'
sudo chmod +x /usr/bin/geckodriver
rm geckodriver-v0.23.0-linux64.tar.gz

ImportError: object file libiconv.so.2 missing

I was trying out QuotesSpider example given in docs (page# 5, here), but am having a hard time to get it running. I installed scrapy from conda in the root environment. I am on Ubuntu 14.04, 64 bit machine. As soon as I run the given code snippet with the following command:
$ scrapy runspider quotes_spider.py -o quotes.json
I get the following error:
Traceback (most recent call last):
File "/home/rip/miniconda2/bin/scrapy", line 4, in <module>
import scrapy.cmdline
File "/home/rip/miniconda2/lib/python2.7/site-packages/scrapy/__init__.py", line 34, in <module>
from scrapy.spiders import Spider
File "/home/rip/miniconda2/lib/python2.7/site-packages/scrapy/spiders/__init__.py", line 10, in <module>
from scrapy.http import Request
File "/home/rip/miniconda2/lib/python2.7/site-packages/scrapy/http/__init__.py", line 11, in <module>
from scrapy.http.request.form import FormRequest
File "/home/rip/miniconda2/lib/python2.7/site-packages/scrapy/http/request/form.py", line 9, in <module>
import lxml.html
File "/home/rip/miniconda2/lib/python2.7/site-packages/lxml/html/__init__.py", line 54, in <module>
from .. import etree
ImportError: libiconv.so.2: cannot open shared object file: No such file or directory
As apparent, an object file seems to be missing. Do I have to build scrapy from source, or is there an alternative?

numpy pandas not working in IDLE / work in Anaconda

I am running Python 3.5.1 on Windows 10 and have Anaconda and IDLE installed.
If I run any Python files, pandas and numpy seem to work in Jupyter notebooks, but not in IDLE.
I have ensured that the PATH and PYTHONPATH environment variables include access to the folders where pandas and numpy are located (Lib/site-packages).
I receive the following error in IDLE when trying to import pandas:
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
import pandas
File "C:\Users\me\AppData\Local\Continuum\Anaconda3\Lib\site-packages\pandas\__init__.py", line 18, in <module>
raise ImportError("Missing required dependencies {0}".format(missing_dependencies))
ImportError: Missing required dependencies ['numpy']
When I try to import numpy I get the following error:
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
import numpy
File "C:\Users\me\AppData\Local\Continuum\Anaconda3\Lib\site-packages\numpy\__init__.py", line 180, in <module>
from . import add_newdocs
File "C:\Users\me\AppData\Local\Continuum\Anaconda3\Lib\site-packages\numpy\add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "C:\Users\me\AppData\Local\Continuum\Anaconda3\Lib\site-packages\numpy\lib\__init__.py", line 8, in <module>
from .type_check import *
File "C:\Users\me\AppData\Local\Continuum\Anaconda3\Lib\site-packages\numpy\lib\type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "C:\Users\me\AppData\Local\Continuum\Anaconda3\Lib\site-packages\numpy\core\__init__.py", line 14, in <module>
from . import multiarray
ImportError: cannot import name 'multiarray'
Can someone help?
try this:
I had the same problem
https://stackoverflow.com/a/40637960/6376603
maybe the above answer will help.
I had similar problem. I uninstalled anaconda from my system completely.
Steps - Delete app, delete anaconda3 folder, empty trash
Then I reinstalled older version of it and it started working perfectly. Maybe it is some kind of bug in the newer version.

How to install and run pandas python library in Eclipse Juno?

I am using pydev plugin in Eclipse Juno for my python programming in windows 7 and i am using python 3.2, it works fine while running python application which using standard python packages. For my one of my project i have to use pandas library, for that i download and install numpy and pandas Windows installer for python 3. But while running even a small program it shows error message. So anyone have any idea about how to install and test pandas in Windows 7 by using eclipse, just pass it to me.
The error message is like this:
Traceback (most recent call last):
import numpy
File "C:\Python32\lib\site-packages\numpy\__init__.py", line 137, in <module>
from . import add_newdocs
File "C:\Python32\lib\site-packages\numpy\add_newdocs.py", line 9, in <module>
from numpy.lib import add_newdoc
File "C:\Python32\lib\site-packages\numpy\lib\__init__.py", line 4, in <module>
from .type_check import *
File "C:\Python32\lib\site-packages\numpy\lib\type_check.py", line 8, in <module>
import numpy.core.numeric as _nx
File "C:\Python32\lib\site-packages\numpy\core\__init__.py", line 40, in <module>
from numpy.testing import Tester
File "C:\Python32\lib\site-packages\numpy\testing\__init__.py", line 8, in <module>
from unittest import TestCase
File "C:\Python32\lib\unittest\__init__.py", line 59, in <module>
from .case import (TestCase, FunctionTestCase, SkipTest, skip, skipIf,
File "C:\Python32\lib\unittest\case.py", line 6, in <module>
import pprint
EOFError: EOF read where not expected
Thanks in advance for your time
I think you have Install panda: here you can find panda package for windows here
https://pypi.python.org/pypi/pandas#downloads