I upgraded to the latest BQ (2.0.14) then downloaded the latest gsutil tar package and manually updated gsutil thus:
"python setup.py install"
When I run gsutil I now get the following error message:
Traceback (most recent call last):
File "/usr/local/bin/gsutil", line 8, in
load_entry_point('gsutil==3.31', 'console_scripts', 'gsutil')()
File "build/bdist.linux-i686/egg/pkg_resources.py", line 318, in load_entry_point
File "build/bdist.linux-i686/egg/pkg_resources.py", line 2221, in load_entry_point
File "build/bdist.linux-i686/egg/pkg_resources.py", line 1954, in load
File "/usr/local/lib/python2.7/site-packages/gsutil-3.31-py2.7.egg/gslib/main.py", line 32, in
from gslib import util
File "/usr/local/lib/python2.7/site-packages/gsutil-3.31-py2.7.egg/gslib/util.py", line 28, in
from oauth2client.client import HAS_CRYPTO
ImportError: cannot import name HAS_CRYPTO
I couldn't find a way to actually uninstall gsutil so I'm stuck.
Any ideas?
Thanks
Have you tried upgrading oauth2client?
pip install -U oauth2client
Related
I've installed DBT CLI on GCP cloud shell using pip install dbt after that I had added the path to dbt in my .profile file too -
.profile file -
if [ -d "$HOME/.local/bin" ] ; then
PATH="$HOME/.local/bin:$HOME/.local/lib/python3.9/site-packages:$PATH"
fi
export $PATH
After the above steps, now when I test the commands like - dbt --version it gives below error -
my_name#cloudshell:~$ dbt --init
Traceback (most recent call last):
File "/home/my_name/.local/bin/dbt", line 5, in <module>
from dbt.main import main
File "/home/my_name/.local/lib/python3.9/site-packages/dbt/main.py", line 11, in <module>
import dbt.version
File "/home/my_name/.local/lib/python3.9/site-packages/dbt/version.py", line 11, in <module>
import dbt.semver
File "/home/my_name/.local/lib/python3.9/site-packages/dbt/semver.py", line 8, in <module>
import dbt.utils
File "/home/my_name/.local/lib/python3.9/site-packages/dbt/utils.py", line 9, in <module>
import jinja2
File "/home/my_name/.local/lib/python3.9/site-packages/jinja2/__init__.py", line 12, in <module>
from .environment import Environment
File "/home/my_name/.local/lib/python3.9/site-packages/jinja2/environment.py", line 25, in <module>
from .defaults import BLOCK_END_STRING
File "/home/my_name/.local/lib/python3.9/site-packages/jinja2/defaults.py", line 3, in <module>
from .filters import FILTERS as DEFAULT_FILTERS # noqa: F401
File "/home/my_name/.local/lib/python3.9/site-packages/jinja2/filters.py", line 13, in <module>
from markupsafe import soft_unicode
ImportError: cannot import name 'soft_unicode' from 'markupsafe' (/usr/local/lib/python3.9/dist-packages/markupsafe/__init__.py)
Where have I failed? Can anyone please correct me?
Try pip install dbt-[adapter] (eg dbt-bigquery) which will install the latest version of dbt, rather than the pre-1.0 version you’re getting now (which has become broken due to an issue with a dependency).
I am attempting to package up my dependencies for a spark program I am creating. I have a requirements.txt file as below
pandas
I then run
pip3 install -t dependencies -r requirements.txt
cd dependencies
zip -r ../dependencies.zip .
pyspark --py-files dependencies.zip
And run the line -
import pandas
And I get the error -
Traceback (most recent call last):
File "/mnt/tmp/spark-REDACTED/userFiles-REDACTED/dependencies.zip/pandas/__init__.py", line 31, in <module>
File "/mnt/tmp/spark-REDACTED/userFiles-REDACTED/dependencies.zip/pandas/_libs/__init__.py", line 3, in <module>
File "/mnt/tmp/spark-REDACTED/userFiles-REDACTED/dependencies.zip/pandas/_libs/tslibs/__init__.py", line 3, in <module>
ModuleNotFoundError: No module named 'pandas._libs.tslibs.conversion'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/mnt/tmp/spark-REDACTED/userFiles-REDACTED/dependencies.zip/pandas/__init__.py", line 36, in <module>
ImportError: C extension: No module named 'pandas._libs.tslibs.conversion' not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
Any ideas on how to fix this?
In order to ship dependency on the worker, there are two ways one is exactly what you did, zip the file, or simple py file then use --py-file. The problem you encountered is because of missing C dependency on the worker side. Pkg like NumPy/pandas all have c dependency.
In order to solve this, create the virtualenv, and zip the virtualenv including
the python executable
PYSPARK_DRIVER_PYTHON = <path to current working python>
PYSPARK_PYTHON = './venv/<path to python executable>'
pyspark --archives = <path to zip file>#venv
or follow this link
I have both python 2.7.16 and python 3.5.2 installed on my ubuntu 16.04 LTS and numpy used to work well on both of them but recently something went wrong with numpy on python 2 while it still running smoothly on python 3
I have tried to uninstall and install the numpy package and this did not fix anything
The exact stdout I am getting is as follow:
python -c "import numpy as np"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/ahmed/.local/lib/python2.7/site-packages/numpy/__init__.py", line 153, in <module>
from . import add_newdocs
File "/home/ahmed/.local/lib/python2.7/site-packages/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/home/ahmed/.local/lib/python2.7/site-packages/numpy/lib/__init__.py", line 18, in <module>
from .polynomial import *
File "/home/ahmed/.local/lib/python2.7/site-packages/numpy/lib/polynomial.py", line 19, in <module>
from numpy.linalg import eigvals, lstsq, inv
File "/home/ahmed/.local/lib/python2.7/site-packages/numpy/linalg/__init__.py", line 50, in <module>
from .linalg import *
File "/home/ahmed/.local/lib/python2.7/site-packages/numpy/linalg/linalg.py", line 29, in <module>
from numpy.linalg import lapack_lite, _umath_linalg
ImportError: liblapack.so.3gf: cannot open shared object file: No such file or directory
Indeed Linux error while loading shared libraries: cannot open shared object file: No such file or directory resolved the problem. I had have only to do the following:
sudo find / -iname liblapack.so.3gf
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:*path_to_lib_as_found*
sudo ldconfig -v
Thanks to Abdur Rehman for the hint
Cheers.
Does your Ubuntu have the latest lapack library? Try installing sudo apt install liblapack3gf liblapack-dev. Sometimes you need to run sudo ldconfig after the install.
I'd like to make use of TensorBoard.I already saved training and validation data in the same directory as the rest of the project data using the tensorflow.keras.callbacks library.
I already pip reinstalled numpy, didn't work out. Then I deleted tb-nightly and tensorboard as I had them both for some reason and just pip reinstalled tb-nightly.
(Tensorflow) C:\Users\alias>python -c "print(__import__('numpy').__version__)"
1.16.2
(Tensorflow) C:\Users\alias>python -c "print(__import__('tensorflow').__version__)"
2.0.0-alpha0
(Tensorflow) C:\Users\alias>python -c "print(__import__('tensorboard.version').version.VERSION)"
1.14.0a20190301
When calling it via Anaconda prompt on a Win10-OS , the following happens:
(Tensorflow) C:\Users\alias>tensorboard --logdir=logs\
TensorBoard 1.14.0a20190301 at http://LAPTOP-4E1BJCAV:6006 (Press CTRL+C to quit)
Traceback (most recent call last):
File "c:\users\alias\anaconda3\envs\tensorflow\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "c:\users\alias\anaconda3\envs\tensorflow\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\alias\Anaconda3\envs\Tensorflow\Scripts\tensorboard.exe\__main__.py", line 9, in <module>
File "c:\users\alias\anaconda3\envs\tensorflow\lib\site-packages\tensorboard\main.py", line 58, in run_main
app.run(tensorboard.main, flags_parser=tensorboard.configure)
File "c:\users\alias\anaconda3\envs\tensorflow\lib\site-packages\absl\app.py", line 300, in run
_run_main(main, args)
File "c:\users\alias\anaconda3\envs\tensorflow\lib\site-packages\absl\app.py", line 251, in _run_main
sys.exit(main(argv))
File "c:\users\alias\anaconda3\envs\tensorflow\lib\site-packages\tensorboard\program.py", line 228, in main
self._register_info(server)
File "c:\users\alias\anaconda3\envs\tensorflow\lib\site-packages\tensorboard\program.py", line 274, in _register_info
manager.write_info_file(info)
File "c:\users\alias\anaconda3\envs\tensorflow\lib\site-packages\tensorboard\manager.py", line 269, in write_info_file
payload = "%s\n" % _info_to_string(tensorboard_info)
File "c:\users\alias\anaconda3\envs\tensorflow\lib\site-packages\tensorboard\manager.py", line 129, in _info_to_string
for k in _TENSORBOARD_INFO_FIELDS
File "c:\users\alias\anaconda3\envs\tensorflow\lib\site-packages\tensorboard\manager.py", line 129, in <dictcomp>
for k in _TENSORBOARD_INFO_FIELDS
File "c:\users\alias\anaconda3\envs\tensorflow\lib\site-packages\tensorboard\manager.py", line 51, in <lambda>
(dt - datetime.datetime.fromtimestamp(0)).total_seconds()),
OSError: [Errno 22] Invalid argument
Would be glad about some help.
Thanks in advance!
Did you do a fresh install and or update recently?
Not sure if this applies to your case or not? Yesterday I installed the latest conda, for python 2.7, from scratch and updated to the latest packages after installation. Running python from pycharm or windows powershell and importing numpy would throw a multiarray import error. The fix for me was to downgrade numpy from ?1.16.12? to ?1.5.14?. Sorry, but I am away from my PC's were I encountered the error but I think those versions are correct.
I experienced the same error in my local Jupyter notebook. Upgrading Numpy package worked for me.
Try upgrading Numpy as below
pip install numpy==1.16
If the above doesn't work, upgrade tensorflow with below command and try again
pip install tensorflow --upgrade
The issue was solved # Github https://github.com/tensorflow/tensorboard/issues/2092
Thanks for the support
So I have tried multiple ways to install mysql-python and MYSQLdb. I have tried downloading through this link: https://sourceforge.net/projects/mysql-python/ and then when I run python setup.py install I get the following errors:
Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.28.tar.gz
Traceback (most recent call last):
File "/Users/Jackson/Google Drive/CS520/HW/MySQL-python-1.2.4b4/distribute_setup.py", line 143, in use_setuptools
raise ImportError
ImportError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "setup.py", line 7, in <module>
use_setuptools()
File "/Users/Jackson/Google Drive/CS520/HW/MySQL-python-1.2.4b4/distribute_setup.py", line 145, in use_setuptools
return _do_download(version, download_base, to_dir, download_delay)
File "/Users/Jackson/Google Drive/CS520/HW/MySQL-python-1.2.4b4/distribute_setup.py", line 124, in _do_download
to_dir, download_delay)
File "/Users/Jackson/Google Drive/CS520/HW/MySQL-python-1.2.4b4/distribute_setup.py", line 194, in download_setuptools
src = urlopen(url)
File "/Users/Jackson/anaconda3/lib/python3.6/urllib/request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "/Users/Jackson/anaconda3/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/Users/Jackson/anaconda3/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/Users/Jackson/anaconda3/lib/python3.6/urllib/request.py", line 570, in error
return self._call_chain(*args)
File "/Users/Jackson/anaconda3/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/Users/Jackson/anaconda3/lib/python3.6/urllib/request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: SSL is required
I could not find a resolution that worked so I tried pip install mysql-python and then I get the following
Traceback (most recent call last):
File "setup.py", line 6, in <module>
req = urllib.request.Request(url, headers={'User-Agent': 'Mozilla/5.0'})
NameError: name 'url' is not defined
Jacksons-MacBook-Pro:MySQL-python-1.2.4b4 Jackson$ pip install mysql-python
Collecting mysql-python
Using cached MySQL-python-1.2.5.zip
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/tmp/pip-build-kdgbz3qe/mysql-python/setup.py", line 13, in <module>
from setup_posix import get_config
File "/private/tmp/pip-build-kdgbz3qe/mysql-python/setup_posix.py", line 2, in <module>
from ConfigParser import SafeConfigParser
ModuleNotFoundError: No module named 'ConfigParser'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/tmp/pip-build-kdgbz3qe/mysql-python/
When I look this problem up I tried running:
pip install --upgrade setuptools
pip install ez_setup
easy_install -U setuptools
With no problems, but then when I tried rerunning the command pip install mysql-python I get the same error as before. I tried downloading the mysqlclient and the mysql/python connector and nothing is working. I just want a fix for trying to install mysql-python, mysqlclient, MYSQLdb, mysql/python connector or anything that will actually work. I already have mysql installed and running as well as Sequel Pro.
I could solve the issue by running the combination of #Devstr an #jsteele. I realized that the python module to connect from python to mysql is not mysql but ```mysql-connector``.
Follow these steps:
brew cask install mysql-connector-python
pip search mysql-connector | grep --color mysql-connector-python
pip install mysql-connector-python-rf
then in python:
import mysql.connector
...
found some useful info: What's the difference between MySQLdb, mysqlclient and MySQL connector/Python?
Maybe use brew?
brew cask install mysql-connector-python
see https://apple.stackexchange.com/questions/130324/how-to-install-mysql-utilities-using-brew-or-brew-cask#136180