Upgrading Tensorflow to 1.3.0 and cuDNN - tensorflow

I upgraded my TensorFlow 1.0.1 with
$ pip install --upgrade tensorflow-gpu
on Ubuntu 16.04, cuDNN 5.1.5 and CUDA 8.0
However, there's error message when I import tensorflow in python module as below:
>> import tensorflow as tf
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/tensorflow/__init__.py", line 24, in <module>
from tensorflow.python import *
File "/usr/local/lib/python2.7/dist-
packages/tensorflow/python/__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 52, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 41, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>_pywrap_tensorflow_internal = swig_import_helper()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
ImportError: libcudnn.so.6: cannot open shared object file: No such file or directory
Failed to load the native TensorFlow runtime.
I think this is the problem with cuDNN version compatibility.
Anyone knows how to upgrade cuDNN and make compatible with tensorflow 1.3.0??

TensorFlow 1.3 requires cuDNN version 6. Follow the installation instructions for your platform on NVIDIA's cuDNN website. Once cuDNN 6 is installed successfully, your installation of TensorFlow 1.3 should start working.

I wrote a script which can be used to upgrade the cuDNN version with a clean install. You can download the script from:
https://github.com/dnzzcn/cuDNNv
This is what the script does:
#!/bin/bash
rm -f /usr/include/cudnn.h
rm -f /usr/lib/x86_64-linux-gnu/*libcudnn*
rm -f /usr/local/cuda-*/lib64/*libcudnn*
cp -P packages/cudnn/include/cudnn.h /usr/include
cp -P packages/cudnn/lib64/libcudnn* /usr/lib/x86_64-linux-gnu/
chmod a+r /usr/lib/x86_64-linux-gnu/libcudnn*
rm -rf packages/cudnn
It performs the installation operations automatically for the version you need.
If you have TensorFlow 1.3.0, which requires cuDNN v6.0, installing cuDNN v6.0 by using the script will be sufficient.

Related

ImportError: cannot import name 'abs' from tensorflow.python.keras._impl.keras.backend

Current Version Stacks:
python 3.5
tensorflow 1.14.0
tensorflow-estimator 1.14.0
tensorflow-gpu 1.14.0
tensorflow-tensorboard 0.1.8
protobuf 3.9.1
Keras 2.2.4
Keras-Applications 1.0.7
Keras-Preprocessing 1.0.9
Drivers & Machine:
Ubuntu 16.04 , NVIDIA-SMI 396.54 , CUDA - 9.0 , CuDnn 7 , libcudnn - 7.4.1
Summary-
Not able to import abs from tensorflow.python.keras._impl.keras.backend on
tensorflow-gpu == 1.14.0
And Not able to get CheckpointableBase attribute on tensorflow-gpu == 1.8.0
Which tensorflow-gpu and keras version supports both CheckpointableBase attribute and 'abs' from tensorflow.python.keras._impl.keras.backend ?
Details -
I'm currently training matterport mask rcnn with Keras==2.2.4. According to some threads in order to get abs I have tried updating tensorflow-gpu to 1.8.0 , updating protobuf. Although I was then able to import abs, but end up reciveing " AttributeError: module 'tensorflow.python.training.checkpointable' has no attribute 'CheckpointableBase' " on tensorflow-gpu to 1.8.0. Then in order to get CheckpointableBase some threads pointed to again update tensorflow-gpu == 1.14.0.
Traceback :
Traceback (most recent call last):
File "/home/kalpit/Desktop/palash/m-rcnn-traning/data/libraries/bacteria_rcnn_train.py", line 53, in <module>
import keras
File "/usr/local/lib/python3.5/dist-packages/keras/__init__.py", line 3, in <module>
from . import utils
File "/usr/local/lib/python3.5/dist-packages/keras/utils/__init__.py", line 6, in <module>
from . import conv_utils
File "/usr/local/lib/python3.5/dist-packages/keras/utils/conv_utils.py", line 9, in <module>
from .. import backend as K
File "/usr/local/lib/python3.5/dist-packages/keras/backend/__init__.py", line 84, in <module>
from .tensorflow_backend import *
File "/usr/local/lib/python3.5/dist-packages/keras/backend/tensorflow_backend.py", line 5, in <module>
import tensorflow as tf
File "/usr/local/lib/python3.5/dist-packages/tensorflow/__init__.py", line 28, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/__init__.py", line 83, in <module>
from tensorflow.python import keras
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/__init__.py", line 26, in <module>
from tensorflow.python.keras import activations
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/activations/__init__.py", line 22, in <module>
from tensorflow.python.keras._impl.keras.activations import elu
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/_impl/keras/__init__.py", line 21, in <module>
from tensorflow.python.keras._impl.keras import activations
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/_impl/keras/activations.py", line 23, in <module>
from tensorflow.python.keras._impl.keras import backend as K
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/_impl/keras/backend.py", line 38, in <module>
from tensorflow.python.layers import base as tf_base_layers
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/layers/base.py", line 25, in <module>
from tensorflow.python.keras import backend
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/backend/__init__.py", line 22, in <module>
from tensorflow.python.keras._impl.keras.backend import abs
ImportError: cannot import name 'abs'
Hi,
I was able to resolve this issue by removing all the tensorFlow and keras python3 packages from dist-packages dir itself , uninstalling using pip3 and then again installing tensorflow==1.8.0
tensorflow-gpu==1.8.0 , Keras==2.2.4
( The Potential reason which caused this specific erro was python referring multiple tensoflow-gpu,keras version packages files in dis-packages dir. Although installing a specific tensoflow-gpu,keras version via pip3 should ideally remove previous existing versions. But In my case that didn't occurred.)
Solution:
Locate tensorflow , keras packages in dist-packages dir using $ locate --basename '\tensorflow' command (provided you are using Ubuntu). In my case it is
$ locate --basename '\tensorflow'
outputs -
/usr/local/lib/python3.5/dist-packages/tensorflow
/usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow
/usr/share/nginx/html/projects/python/machine-learning/objectDetection/models-master/research/syntaxnet/tensorflow
&
$ locate --basename '\keras'
outputs-
/usr/local/lib/python3.5/dist-packages/keras
/usr/local/lib/python3.5/dist-packages/tensorflow/keras
/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/keras
/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/keras/api/keras
/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras
/usr/local/lib/python3.5/dist-packages/tensorflow/python/keras/_impl/keras
/usr/local/lib/python3.5/dist-packages/tensorflow/tools/api/generator/api/keras
/usr/share/nginx/html/projects/python/Playground/lambda/testkeras/keras
Now that you are aware of the desired packages location. Simply remove the tensorflow and keras dir's and uninstall packages using below commands
Removing dir's
sudo rm -R /usr/local/lib/python3.5/dist-packages/tensorflow/
sudo rm -R /usr/share/nginx/html/projects/python/machine-learning/objectDetection/models-master/research/syntaxnet/tensorflow
sudo rm -R /usr/local/lib/python3.5/dist-packages/keras/
sudo rm -R /usr/share/nginx/html/projects/python/Playground/lambda/testkeras/keras
Uninstall using pip3
sudo pip3 uninstall tensorflow
sudo pip3 uninstall tensorflow-gpu
sudo pip3 uninstall keras
Again Installing tensorflow, tensorflow-gpu ,keras
sudo pip3 install tensorflow==1.8.0
sudo pip3 install tensorflow-gpu==1.8.0
sudo pip3 install Keras==2.2.4

AttributeError: 'module' object has no attribute 'ceil'

I have installed the module onnx_tf from this link.
After that when I am verifying the installation as python -c "import onnx_tf" I am encountering the following error. How do I resolve it? Thank you
testuser#testuser:~$ python -c "import onnx_tf"
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/testuser/.local/lib/python2.7/site-packages/onnx_tf/__init__.py", line 1, in <module>
from . import frontend
File "/home/testuser/.local/lib/python2.7/site-packages/onnx_tf/frontend.py", line 18, in <module>
from onnx_tf.common.handler_helper import get_all_frontend_handlers
File "/home/testuser/.local/lib/python2.7/site-packages/onnx_tf/common/handler_helper.py", line 5, in <module>
from onnx_tf.handlers.backend import * # noqa
File "/home/testuser/.local/lib/python2.7/site-packages/onnx_tf/handlers/backend/ceil.py", line 10, in <module>
#tf_func(tf.ceil)
AttributeError: 'module' object has no attribute 'ceil'
Check your tensorflow version
pip show tensorflow
If it's 2.x, you may need to change to older version.
pip install tensorflow-gpu==1.14
or if you are using a cpu...
pip install tensorflow==1.14
The following resolved this issue for me:
pip uninstall onnx-tf
pip install git+https://github.com/onnx/onnx-tensorflow.git
pip install tensorflow-addons
(Using TensorFlow 2.3 and Python 3.8)

Tensorflow-gpu 1.5 with cuda 8.0 and cudnn v7.1 for cuda 8.0

I first installed cuda 9.0 and cudnn for cuda 9.0 and the latest tensorflow-gpu 1.6. But I noticed my nvidia-driver 384.111 doesn't support cuda 9.0. SO I uninstalled cuda 9.0, cudnn and tensorflow-gpu 1.5 and reinstalled tensorflow-gpu 1.5, cuda8.0 and cudnn v7.1 for cuda 8.0. But when I import tensorflow, it always shows:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/tensorflow/__init__.py", line 24, in <module>
from tensorflow.python import *
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
ImportError: libcublas.so.9.0: cannot open shared object file: No such file or directory
Failed to load the native TensorFlow runtime.
Seems like tensorflow is trying to find libcublas.so.9.0 however I only have cuda 8.0 installed. It's clear that I can use tensorflow 1.5 with cuda 8.0.
Does anyone know how to solve this?
Update:
Looks like tensorflow officially doesn't provide binaries compiled with cuda 8.0 and cudnn v7.1 after tensorflow 1.4. Does any one know where to download those binaries?
I assume that you are installing TF using pip install. Tensorflow install page (currently, version 1.6) mentions than that CUDA® Toolkit 9.0 along with cuDNN v7.0 are the requirements for your installation.
Since TF version 1.5 CUDA9+CUDNN7 are among new requirements. Since CUDA9 relies on NVIDIA driver 384+ it causes a chain of updates of software: driver, CUDA, CUDNN, TF, etc. CUDA 9.1 would require nvidia driver 390 or later.
Ways to solve the issue you can follow one of following paths:
Update nvidia driver and switch back to CUDA 9.0 and cuDNN 7.0. You can have multiple versions of CUDA and CUDNN coexisting on your system.
Install TF 1.4 which requires exactly CUDA 8.0 and cuDNN v6.0
Compile TF from source. It requires investing some time and installing additional software. In my experience can easily take hours even if it is not your first time.
Find a wheel of TF compiled for your system configuration.
I have had the same problem when installing tensorflow-gpu-1.6, i have resolved my problem like this :
step(1): for tensorflow-gpu-1.6 we can use cuda9.0 and cudnn7 so download cuda9.0 debian if you are using Ubuntu, you can download cuda9.0 from here https://developer.nvidia.com/cuda-90-download-archive
step(2): enter the following command sudo apt-get --purge remove cuda
step(3): sudo apt autoremove
step(4): and now install the downloaded cuda9.0 file
step(5): and nvidia driver 384 is ok for cuda9.0, so no worries
step(6): edit the path for cuda9.0 in bashrc gedit ~/.bashrc
I have done the same procedure like i removed every thing related to cuda from my pc, and then tried installing cuda9.0 and it did worked for me.
Let me know whether problem is resolved or not, Thank You.

Tensorflow Image Retraining Tutorial Bazel Error

I am following the Tensorflow image retraining tutorial located here.
When I try to run this command using bazel:
bazel-bin/tensorflow/examples/image_retraining/retrain --image_dir ~/flower_photos
I get the following error:
Traceback (most recent call last):
File "/home/j/tensorflow/bazel-bin/tensorflow/examples/image_retraining/retrain.runfiles/org_tensorflow/tensorflow/examples/image_retraining/retrain.py", line 108, in <module>
import tensorflow as tf
File "/home/j/tensorflow/bazel-bin/tensorflow/examples/image_retraining/retrain.runfiles/org_tensorflow/tensorflow/__init__.py", line 24, in <module>
from tensorflow.python import *
File "/home/j/tensorflow/bazel-bin/tensorflow/examples/image_retraining/retrain.runfiles/org_tensorflow/tensorflow/python/__init__.py", line 63, in <module>
from tensorflow.python.framework.framework_lib import *
File "/home/j/tensorflow/bazel-bin/tensorflow/examples/image_retraining/retrain.runfiles/org_tensorflow/tensorflow/python/framework/framework_lib.py", line 76, in <module>
from tensorflow.python.framework.ops import Graph
File "/home/j/tensorflow/bazel-bin/tensorflow/examples/image_retraining/retrain.runfiles/org_tensorflow/tensorflow/python/framework/ops.py", line 28, in <module>
from autograd import core as ag_core
ModuleNotFoundError: No module named 'autograd'
However, when I run it using the python command
python tensorflow/examples/image_retraining/retrain.py --image_dir ~/flower_photos
It runs correctly and does not generate this error.
I am running both of the above commands from the ~/tensorflow directory (the top of the git repo)
Environment:
Ubuntu 17.04
Python 3.6.1
Anaconda 4.4.0
Bazel 0.5.4
Tensorflow 1.3 (no GPU) (v1.3.0-rc2-20-g0787eee 1.3.0)
I solved this issue. The autograd dependency was not installed. This can be solved by running pip install autograd
You have to configure TensorFlow before compiling, so it set up all the dependencies: ./configure

Install odoo8.0 on Mac OSX 10.11,I got error "unsafe use of relative rpath libpq.5.dylib"

I have downloaded Odoo 8.0 from git ,all my steps:
python --version
Python 2.7.10
Installed PostgresApp into Applications and executed
export PATH=$PATH:/Applications/Postgres.app/Contents/Versions/latest/bin
Installed pip
sudo easy_install pip
Finished processing dependencies for pip
and installed homebrew and Xcode-select
Installed other libs
brew install autoconf automake libtool
brew install libxml2 libxslt libevent
Installed Python dependencies
`sudo easy_install -U setuptools
Finished processing dependencies for setuptools
cd odoo/
sudo pip install --user -r requirements.txt
Successfully installed Mako-1.0.1 Pillow-2.7.0 Werkzeug-0.9.6 `
Running odoo
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
./odoo.py --addons-path=addons --db-filter=mydb
And i got this error:
Traceback (most recent call last):
File "./odoo.py", line 160, in <module>
main()
File "./odoo.py", line 156, in main
import openerp
File "/Users/Charlie/Desktop/Python_Workspace/odoo-8.0/openerp/__init__.py", line 76, in <module>
import modules
File "/Users/Charlie/Desktop/Python_Workspace/odoo-8.0/openerp/modules/__init__.py", line 27, in <module>
from . import db, graph, loading, migration, module, registry
File "/Users/Charlie/Desktop/Python_Workspace/odoo-8.0/openerp/modules/graph.py", line 32, in <module>
import openerp.osv as osv
File "/Users/Charlie/Desktop/Python_Workspace/odoo-8.0/openerp/osv/__init__.py", line 22, in <module>
import osv
File "/Users/Charlie/Desktop/Python_Workspace/odoo-8.0/openerp/osv/osv.py", line 23, in <module>
from .orm import Model, TransientModel, AbstractModel
File "/Users/Charlie/Desktop/Python_Workspace/odoo-8.0/openerp/osv/orm.py", line 5, in <module>
from ..models import (
File "/Users/Charlie/Desktop/Python_Workspace/odoo-8.0/openerp/models.py", line 56, in <module>
import psycopg2
File "/Library/Python/2.7/site-packages/psycopg2-2.6.2-py2.7-macosx-10.11-intel.egg/psycopg2/__init__.py", line 50, in <module>
from psycopg2._psycopg import BINARY, NUMBER, STRING, DATETIME, ROWID
ImportError: dlopen(/Library/Python/2.7/site-packages/psycopg2-2.6.2-py2.7-macosx-10.11-intel.egg/psycopg2/_psycopg.so, 2): Library not loaded: libpq.5.dylib
Referenced from: /Library/Python/2.7/site-packages/psycopg2-2.6.2-py2.7-macosx-10.11-intel.egg/psycopg2/_psycopg.so
Reason: unsafe use of relative rpath libpq.5.dylib in /Library/Python/2.7/site-packages/psycopg2-2.6.2-py2.7-macosx-10.11-intel.egg/psycopg2/_psycopg.so with restricted binary
I have worked on this about two days.Thanks a lot.