ImportError: Can't determine version for tables - pandas

After I upgraded python2 to python3.7, I cannot use pandas to load hdf files any more. The following codes had no problems before, but after updating to python3.7, I got error message of "Can't determine version for tables."
My python version is 3.7, but there are still previous python2.7 paths. Please see the following:
$ python --version
Python 3.7.3
$ whereis python
python: /usr/bin/python /usr/bin/python2.7 /usr/bin/python2.7-config /usr/lib/python2.7 /usr/lib64/python2.7 /etc/python /usr/include/python2.7 /home/yun.wei/anaconda3/bin/python /home/yun.wei/anaconda3/bin/python3.7 /home/yun.wei/anaconda3/bin/python3.7-config /home/yun.wei/anaconda3/bin/python3.7m /home/yun.wei/anaconda3/bin/python3.7m-config /usr/share/man/man1/python.1.gz
Is this error due to old python versions?
import pandas as pd
filename = 'filename.h5'
df = pd.read_hdf(filename, key='data', mode='r')
ImportError: Can't determine version for tables

I ran into this error because I did not have pytables installed. I solved it by running
conda install pytables
Or if you use pip you should be able to run
pip install tables

Related

How to get most recent version of Pandas in Google Colab?

I have successfully downloaded a version of Python 3.10 to Colab, and Colab says I have the most up to date version of pandas. However, whenever I import pandas, the version is 1.3.5, while I need it to be up to date.
I tried restarting the kernel, but this did not fix things. The code and its output are below, excepting the long list I get when I download the newest version of Python.
!wget https://github.com/korakot/kora/releases/download/v0.10/py310.sh
!bash ./py310.sh -b -f -p /usr/local
!python -m ipykernel install --name "py310" --user
!python --version
Python 3.10.6
!pip3 install pandas --upgrade --no-deps
Requirement already satisfied: pandas in /usr/local/lib/python3.10/site-packages (1.5.2)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
import pandas as pd
print(pd.__version__)
1.3.5
Thank you so much for your time.

tensorflow-data-validation cannot be pip installed

Since I'm moving away from pandas DataFrames to TensorFlow datasets, I'd like to use tensorflow-data-validation instead of the more traditional pandas-profiling when it comes to data exploration and validation.
However, pip install tensorflow-data-validation gives the following error:
ERROR: Could not find a version that satisfies the requirement tensorflow-data-validation (from versions: none)
ERROR: No matching distribution found for tensorflow-data-validation
What could be the problem? This old GitHub issue explains how this could be due to the Python version, but Apache Beam (on which tensorflow-data-validation presumably relies) is now fully compatible with Python 3, so it must be something else.
My environment is as follows:
Python 3.9.2
TensorFlow 2.6.0
Debian GNU/Linux 11 (bullseye)
pip 21.3
I got the same error when using Python 3.9. After downgrading to Python 3.8, pip install tensorflow-data-validation ran successfully.
Regarding your comment about Apache Beam, it looks like the Python SDK currently supports Python 3.8 (and earlier) but not yet Python 3.9.
My environment:
Python 3.8.10
TensorFlow 2.8.0
macOS Monterey (12.0.1)
pip 21.1.1
Try this
pip install --upgrade --force-reinstall tensorflow-data-validation[all]
It might be a version compatibility issue with tensorflow==2.6.0.
Try
pip install tensorflow-data-validation==1.3.0
I was able to install the tensorflow_data_validation library successfully, via the below command in my Google Colab file.
!pip install -U tensorflow \
tensorflow-data-validation \
apache-beam[gcp]

How to fix Python 2.7 and Numpy syntax error in tmp directory

I am installing my setup.py dependency file on a Ubuntu 16 instance. When I run the setup.py file below is the error I am getting.
File "/tmp/easy_install-z7cdA1/pandas-1.0.3/setup.py", line 42
f"numpy >= {min_numpy_ver}"
The problem happening is that file is in a tmp directory which I am not able to debug. From the error I am guessing it is some numpy version issue with Python 2.7. Any help fixing this issue will be helpful.
pandas removed support for Python 2.7 starting from version 0.25.0.
The most up-to-date version you can install for 2.7 is
pip install pandas==0.24.2
You can see the error is because there's an f-string, which is a python 3.6 feature. pd.__version__ == '1.0.3' officially supports Python 3.6.1 and above, 3.7, and 3.8.

Error installing pandas with pip: Could not find a version that satisfies the requirement numpy==1.9.3

I'm trying to install pandas. When I run: pip install pandas in cmd, I get the following error message: Could not find a version that satisfies the requirement numpy==1.9.3. Not sure how to fix this.
It's likely you have a different version of numpy installed, try upgrade numpy first with:
pip install numpy==1.9.3 --upgrade
then run pip install pandas. Also check this github issue. Maybe your python version is not supported.

Theano fails due to NumPy Fortran mixup under Ubuntu

I installed Theano on my machine, but the nosetests break with a Numpy/Fortran related error message. For me it looks like Numpy was compiled with a different Fortran version than Theano. I already reinstalled Theano (sudo pip uninstall theano + sudo pip install --upgrade --no-deps theano) and Numpy / Scipy (apt-get install --reinstall python-numpy python-scipy), but this did not help.
What steps would you recommend?
Complete error message:
ImportError: ('/home/Nick/.theano/compiledir_Linux-2.6.35-31-generic-x86_64-with-Ubuntu-10.10-maverick--2.6.6/tmpIhWJaI/0c99c52c82f7ddc775109a06ca04b360.so: undefined symbol: _gfortran_st_write_done'
My research:
The Installing SciPy / BuildingGeneral page about the undefined symbol: _gfortran_st_write_done' error:
If you see an error message
ImportError: /usr/lib/atlas/libblas.so.3gf: undefined symbol: _gfortran_st_write_done
when building SciPy, it means that NumPy picked up the wrong Fortran compiler during build (e.g. ifort).
Recompile NumPy using:
python setup.py build --fcompiler=gnu95
or whichever is appropriate (see python setup.py build --help-fcompiler).
But:
Nick#some-serv2:/usr/local/lib/python2.6/dist-packages/numpy$ python setup.py build --help-fcompiler
This is the wrong setup.py file to run
Used software versions:
scipy 0.10.1 (scipy.test() works)
NumPy 1.6.2 (numpy.test() works)
theano 0.5.0 (several tests fails with undefined symbol: _gfortran_st_write_done')
python 2.6.6
Ubuntu 10.10
[UPDATE]
So I removed numpy and scipy from my system with apt-get remove and using find -name XXX -delete of what was left.
Than I installed numpy and scipy from the github sources with sudo python setpy.py install.
Afterwards I entered again sudo pip uninstall theano and sudo pip install --upgrade --no-deps theano.
Error persists :/
I also tried the apt-get source ... + apt-get build-dep ... approach, but for my old Ubuntu (10.10) it installs too old version of numpy and scipy for theano: ValueError: numpy >= 1.4 is required (detected 1.3.0 from /usr/local/lib/python2.6/dist-packages/numpy/__init__.pyc)
I had the same problem, and after reviewing the source code, user212658's answer seemed like it would work (I have not tried it). I then looked for a way to deploy user212658's hack without modifying the source code.
Put these lines in your theanorc file:
[blas]
ldflags = -lblas -lgfortran
This worked for me.
Have you tried to recompile NumPy from the sources?
I'm not familiar with the Ubuntu package system, so I can't check what's in your dist-packages/numpy. With a clean archive of the NumPy sources, you should have a setup.py at the same level as the directories numpy, tools and benchmarks (among others). I'm pretty sure that's the one you want to use for a python setup.py build.
[EDIT]
Now that you have recompiled numpy with the proper --fcompiler option, perhaps could you try to do the same with Theano, that is, compiling directly from sources without relying on a apt-get or even pip. You should have a better control on the build process that way, which will make debugging/trying to find a solution easier.
I had the same problem. The solution I found is to add a hack in theano/gof/cmodule.py to link against gfortran whenever 'blas' is in the libs. That fixed it.
class GCC_compiler(object):
...
#staticmethod
def compile_str(module_name, src_code, location=None,
include_dirs=None, lib_dirs=None, libs=None,
preargs=None):
...
cmd.extend(['-l%s' % l for l in libs])
if 'blas' in libs:
cmd.append('-lgfortran')
A better fix is to remove atlas and install openblas. openblas is faster then atlas. Also, openblas don't request gfortran and is the one numpy was linked with. So it will work out of the box.