Scrapy pyOpenSSL Windows 7x64 - scrapy

I have installed pyOpenSSL on x64 windows 7 machine.
But every time I try to access a 'https' website I get the error
HTTPS not supported: install pyopenssl library
If you traceback the error this is where it is coming from:
scrapy\core\downloader\handlers\http.py", line 34, in _connect
raise NotSupported("HTTPS not supported: install pyopenssl library")
I look up http.py and find this:
from scrapy import optional_features
ssl_supported = 'ssl' in optional_features
if ssl_supported:
from twisted.internet.ssl import ClientContextFactory
When I go look for optional_features to add the path of pyopenssl I cannot find it.
Where is the optional_features file?
The idea was to find the file and add the path.
Is this a right way to go about this?
Any suggestions?
EDIT:
I uninstalled pyOpenSSL and attempted to reinstall via
pip install pyopenssl
but keep coming up short.
The error log: http://sebsauvage.net/paste/?4066d45d10e18a4f#AwpphSnXEhq7Sc9fi6yZdt6O9YEHnHYZl+2k2QgITjQ=
Any help would be greatly appreciated.

try:
easy_install pyOpenSSL==0.15

Related

Error installing library of Scrapy in PyCharm

I can install other packages, but can't install Scrapy. I get the following errors:
warning: build_py: byte-compiling is disabled, skipping.
running build_ext
building 'lxml.etree' extension
error: Microsoft Visual C++ 10.0 is required (Unable to find vcvarsall.bat).
However, C++ is installed, which I installed numerous of times. I have x86 and 64 bit installations (not sure if it's 10.0) but I have 2013-2017 versions installed.
Please upgrade your pip by following command.
python -m pip install --upgrade pip
Then install Scrapy by following command.
pip install Scrapy
download latest twisted package and install with pip.
https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted
after that install scrapy
In my case, I found that pywin32 was not installed...
So I did
download the latest Twisted package from https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted
You want to use the amd64 if you have Windows 64 (regardless if it's an Intel processor or not)
You can use any browser for the download and copy/paste the file into the project folder of your current pycharm project.
Then in pycharm type this:
pip install Twisted-20.3.0-cp39-cp39-win_amd64.whl
(assuming that your package was Twisted-20.3.0-cp39-cp39-win_amd64.whl)
then proceed with:
pip install Scrapy

Upload package to pypi moans "must use HTTPS"

When executing this from the command-line of within my package:
python setup.py sdist bdist_egg upload
I get:
Server response (403): Must access using HTTPS instead of HTTP
This used to work many times until now. Searching for the err-msg didn't give me helpful infos, has anyone a clue what's going on?
Update: Use twine for uploading distributions to pypi.
Are you using a .pypirc file?
If you are maybe change the urls to point to the https links?
[distutils]
index-servers =
pypi
pypitest
[pypi]
repository=https://pypi.python.org/pypi
username=your_username
password=your_password
[pypitest]
repository=https://testpypi.python.org/pypi
username=your_username
password=your_password
Updating setuptools let's the error dissapear:
pip install setuptools -U
Then running the upload-command ends with:
Submitting dist/my.packagename-1.3.tar.gz to https://upload.pypi.org/legacy/
error: None
But still, no new version is available at pypi.

libgeos_c.so undefined symbol: GEOSisClosed

I was trying to set up the CKAN with couple extensions. The main extension is spatial ( https://github.com/ckan/ckanext-spatial)> During the tests server returns code 500.
The log is:
AttributeError: /usr/lib/libgeos_c.so.1: undefined symbol: GEOSisClosed
I couldn't find similar issue / attribute. Does anyone faced similar error?
There appears to be an issue with the recent versions of Shapely, according to this: https://github.com/Toblerity/Shapely/issues/176
It looks like this became a problem for installers during September, because the ckanext-spatial's pip-requirements.txt will get you the latest version "Shapely>=1.2.13". It sounds like the latest version is fixed though, so try that - 1.4.3 (released 1st Oct 2014) or failing that, an earlier one (e.g. 1.3.3).
(pyenv) $ pip install 'Shapely>=1.4.3'
Check if you installed the GEOS package
sudo apt-get install libgeos-c1
If still no luck try installing the development libraries:
sudo apt-get install libgeos-dev
I have shapely 1.5.9 installed on Ubuntu and I was receiving a similar error...
AttributeError: /usr/lib/libgeos_c.so.1: undefined symbol: GEOSCovers
I had to revert to a previous version of Shapely to get this to work. Try this command.
sudo pip install 'Shapely==1.4.3'

lxml on python-3.3.0 ImportError: undefined symbol: xmlBufContent

I am having a hard time installing lxml(3.1.0) on python-3.3.0. It installs without errors and I can see the lxml-3.1.0-py3.3-linux-i686.egg in the correct folder (/usr/local/lib/python3.3/site-packages/), but when I try to import etree, I get this:
from lxml import etree
Traceback (most recent call last):
File "", line 1, in
ImportError: /usr/local/lib/python3.3/site-packages/lxml-3.1.0-py3.3-linux-i686.egg/lxml/etree.cpython-33m.so: undefined symbol: xmlBufContent
I did try to install with apt-get, I tried "python3 setup.py install" and I did via easy_install. I have to mention that I have 3 versions installed (2.7, 3.2.3 and 3.3.0.), but I am too much of a beginner to tell if this has to do with it.
I did search all over, but I could not find any solution to this.
Any help is greatly appreciated!
best,
Uhru
You should probably mention the specific operating system you're trying to install on, but I'll assume it's some form of Linux, perhaps Ubuntu or Debian since you mention apt-get.
The error message you mention is typical on lxml when the libxml2 and/or libxslt libraries are not installed for it to link with. For whatever reason, the install procedure does not detect when these are not present and can give the sense the install has succeeded even though those dependencies are not satisfied.
If you issue apt-get install libxml2 libxml2-dev libxslt libxslt-dev that should eliminate this error.

How to install modules for Python 2.7 on Ubuntu 10.10?

On Ubuntu 10.10, I am unable to install lxml to python 2.7. Here are the steps I take.
sudo su -
apt-get install python2.7
apt-get install python-lxml
Note when running the install for python-lxml package, the following appeared:
INFO: using unknown version '/usr/bin/python2.7' (debian_defaults not up-to-date?)"
Importing the module in python2.6 (the version that comes standard with Ubuntu) works. However, importing the module under python2.7 does not. So how does one install Python modules to a non-default Python installation?
Try to install libxml2, libxml2-dev, libxslt, libxslt-dev, python-dev. These are header files. Then try to install lxml again.
On Ubuntu 10.10 the python packages installed from the repositories get installed to /usr/lib/python2.6/dist-packages so one option is to add this path to your $PYTHONPATH environmental variable so python2.7 will look to the python2.6 directory for the libs.
What I've done on Ubuntu 10.10 is add
export PYTHONPATH="$PYTHONPATH:/usr/lib/python2.6/dist-packages"
to my .bashrc file, and also to my .gnomerc file. This sets the $PYTHONPATH for python instances started from the shell or from the gnome desktop. You should then be able to import the python libs which you have installed from the Ubuntu repositories in python2.7.
.bashrc and .gnomerc are both located in your home directory; you might have to create .gnomerc if it doesn't already exist. And one caution: I had a syntax error in my .gnomerc which stopped the gnome desktop from loading, and I couldn't log in. I had to use a recovery console to fix this syntax error and then I could log in again.
This seems a little hackish to me, so I'm interested in hearing better solutions.
Another solution might be to use the following code:
try:
from lxml import etree
except ImportError:
try:
# Python 2.5
import xml.etree.cElementTree as etree
except ImportError:
try:
# Python 2.5
import xml.etree.ElementTree as etree
except ImportError:
try:
# normal cElementTree install
import cElementTree as etree
except ImportError:
try:
# normal ElementTree install
import elementtree.ElementTree as etree
except ImportError:
print("Failed to import ElementTree from any known place")
[Source]
This will import lxml if it is available, or the original ElementTree otherwise.
I use this code for my application on Google App Engine (using Python 2.7): on the server it will use lxml, on my machine it will use ElementTree.
I have one easiest trick Just open synaptic package manager type "python-lxml" in search box it will show you all the dependencies and available packages select packages which you want to install and hit apply.