In the inference mode, Is C++ API faster than Python API? In the some issues, I've seen the C++ slower than python, why? Is this right?
OS : Ubuntu 16.04
Tensorflow : 1.12.0 binary
CUDA/CUDNN : 9.0 / 7.0
Issue 1
Issue 2
I think you just need to do few warm-up predictions. After the second or third prediction, you will notice the time will decrease.
Related
Environment:
Windows 8.1 64 bit
Python 3.7.7 64 bit
TensorFlow 2.3
Keras 2.4
CUDA: 10.1
CuDNN: 7
Error : InvalidArgumentError: assertion failed: [0] [Op:Assert] name: EagerVariableNameReuse.
Problem : Above error at a simple statement model = Sequential() at GPU execution
But if I run the script without GPU (not setting CUDA/CuDnn environment variables it works as expected.
So effectively, I have a GPU but can't use with TensorFlow(2.3)
Can someone help?
From what I've seen, this issue comes up for the following deadly combination.
Windows OS
TensorFlow 2.3.x
I've been struggling to solve this for quite sometime and only figured it out recently.
Easiest fix
Easiest solution would be to go for TensorFlow 2.4.0rcx. It is still in pre-release. But haven't given me the aforementioned issue. So,
pip uninstall tensorflow
pip install tensorflow==2.4.0rc0
Solution if you want to stay with TF 2.3.x
If you're really adamant about using TensorFlow 2.3.x the only solutions seem to be,
Trying a Linux based OS (Ubuntu)
Compiling TensorFlow from source (According to this helpful comment)
I attached RTX 3080 to my computer. but when training on keras 2.3.1 and tensorflow 1.15, I got some error "failed to run cuBLAS_STATUS_EXECUTION_FAILED, did not mem zero GPU location . . . check failed:start_event !=nullptr && stop_event != nullptr" I think the problem is that recently released rtx 3080 and CUDA 11 is not yet support the keras 2.xx and tensorflow 1.xx. is this right? And what make that problem?
At the moment of writing this, currently Nvidia 30xx series only fully support CUDA version 11.x, see https://forums.developer.nvidia.com/t/can-rtx-3080-support-cuda-10-1/155849/2
Tensorflow 1.15 wasn't fully supported on CUDA since version 10.1 and newer, for probably similar reason as described in the link above. Unfortunately TensorFlow version 1.x is no longer supported or maintained, see https://github.com/tensorflow/tensorflow/issues/43629#issuecomment-700709796
TensorFlow 2.4 is your best bet with an Ampere GPU. It has now a stable release, and it has official support for CUDA 11.0, see https://www.tensorflow.org/install/source#gpu
As TensorFlow 1.x is never going to be updated or maintained by TensorFlow team, I would strongly suggest moving to TensorFlow 2.x, excluding personal preferences, it's better in almost every way and has tf.compat module for backwards compatibility with TensorFlow 1.x code, if rewriting you code base is not an option. However, even that module is no longer maintained, really showing that version 1.x is dead, see https://www.tensorflow.org/guide/versions#what_is_covered
However, if you're dead set on using TensorFlow 1.15, you might have a chance with Nvidia Tensorflow, which apparently has support for version 1.15 on Ampere GPUs, see https://developer.nvidia.com/blog/accelerating-tensorflow-on-a100-gpus/
I am a university professor trying to learn deep learning for a possible class in the future. I have been using google colab with GPU support for the past couple of months. Just recently, the GPU device is not found. But, I am doing everything that I have done in the past. I can't imagine that I have done anything wrong because I am just working through tutorials from books and the tensorflow 2.0 tutorials site.
tensorflow 2 on Colab GPU was broken recently due to an upgrade from CUDA 10.0 to CUDA 10.1. As of this afternoon, the issue should be resolved for the tensorflow builds bundled with Colab. That is, if you run the following magic command:
%tensorflow_version 2.x
then import tensorflow will import a working, GPU-compatible tensorflow 2.0 version.
Note, however, if you attempt to install a version of tensorflow using pip install tensorflow-gpu or similar, the result may not work in Colab due to system incompatibilities.
See https://colab.research.google.com/notebooks/tensorflow_version.ipynb for more information.
I love my iMac and do not mind paying top dollars for it. However I need to run Tensorflow, Keras, and Pytorch for deep learning projects. Can I run them on the latest and maxed-out spec iMac Pro ?
tensorflow 1.8 supports ROCm, IDK how it performs next to nvidia's CUDA
but that means that if you have GPU (radeon) that supports ROCm you can use tensorflow gpu
running tensorflow on gpu is possible but extremely slow and can be added to the definition of torture
I would like to optimize a graph using Tensorflow's transform_graph tool. I tried optimizing the graph from MultiNet (and others with similar encoder-decoder architectures). However, the optimized graph is actually slower when using quantize_weights, and even much slower when using quantize_nodes. From Tensorflow's documentation, there may be no improvements, or it may even be slower, when quantizing. Any idea if this is normal with the graph/software/hardware below?
Here is my system information for your reference:
OS Platform and Distribution: Linux Ubuntu 16.04
TensorFlow installed from: using TF source code (CPU) for graph conversion, using binary-python(GPU) for inference
TensorFlow version: both using r1.3
Python version: 2.7
Bazel version: 0.6.1
CUDA/cuDNN version: 8.0/6.0 (inference only)
GPU model and memory: GeForce GTX 1080 Ti
I can post all the scripts used to reproduce if necessary.
It seems like quantization in Tensorflow only happens on CPUs. See: https://github.com/tensorflow/tensorflow/issues/2807
I got same problem in PC enviroment. My model is 9 times slower than not quantize.
But when I porting my quantized model into android application, its ok to speed up.
Seems like current only work on CPU and only ARM base CPU such as android phone.