Anyone know if Tensorflow Lite has GPU support for Python? I've seen guides for Android and iOS, but I haven't come across anything about Python. If tensorflow-gpu is installed and tensorflow.lite.python.interpreter is imported, will GPU be used automatically?
According to this thread, it is not.
one solution is to convert tflite to onnx and use onnxruntime-gpu
convert to onnx with https://github.com/onnx/tensorflow-onnx:
pip install tf2onnx
python3 -m tf2onnx.convert --opset 11 --tflite path/to/model.tflite --output path/to/model.onnx
then pip install onnxruntime-gpu
and run like:
session = onnxruntime.InferenceSession(('/path/to/model.onnx'))
raw_output = self.detection_session.run(['output_name'], {'input_name': img})
you can get the input and output names by:
for i in range(len(session.get_inputs)):
print(session.get_inputs()[i].name)
and the same but replace 'get_inputs' with 'get_outputs'
You can force the computation to take place on a GPU:
import tensorflow as tf
with tf.device('/gpu:0'):
for i in range(10):
t = np.random.randint(len(x_test) )
...
Hope this helps.
Related
I tried
%tensorflow_version 1.15
I used this code a couple days ago, but it doesn't work anymore since today.
The outcomes are
ValueError Traceback (most recent call last)
<ipython-input-6-24c52e77c597> in <module>()
----> 1 get_ipython().magic('tensorflow_version 1.15')
2 frames
/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py in magic(self, arg_s)
2158 magic_name, _, magic_arg_s = arg_s.partition(' ')
2159 magic_name = magic_name.lstrip(prefilter.ESC_MAGIC)
-> 2160 return self.run_line_magic(magic_name, magic_arg_s)
2161
2162 #-------------------------------------------------------------------------
/usr/local/lib/python3.7/dist-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line)
2079 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals
2080 with self.builtin_trap:
-> 2081 result = fn(*args,**kwargs)
2082 return result
2083
/usr/local/lib/python3.7/dist-packages/google/colab/_tensorflow_magics.py in _tensorflow_version(line)
39
40 Your notebook should be updated to use Tensorflow 2.
---> 41 See the guide at https://www.tensorflow.org/guide/migrate#migrate-from-tensorflow-1x-to-tensorflow-2."""
42 ))
43
ValueError: Tensorflow 1 is unsupported in Colab.
Your notebook should be updated to use Tensorflow 2.
See the guide at https://www.tensorflow.org/guide/migrate#migrate-from-tensorflow-1x-to-tensorflow-2.
Is there any method I can try to fix it or does it means that colab will not support tensorflow 1.x anymore?
Google Colab removed support for Tensorflow 1, and it is not possible to use %tensorflow_version 1.x magic anymore. You must remove this instruction from your code if you have it.
Also the default python version as I update this answer is python 3.8 which is not compatible with tensorflow 1.x.
To make everything work you first have to downgrade python. Python 3.6 should work. As suggested by #s-abbaasi here's a guide on how to do so:
%%bash
MINICONDA_INSTALLER_SCRIPT=Miniconda3-4.5.4-Linux-x86_64.sh
MINICONDA_PREFIX=/usr/local
wget https://repo.continuum.io/miniconda/$MINICONDA_INSTALLER_SCRIPT
chmod +x $MINICONDA_INSTALLER_SCRIPT
./$MINICONDA_INSTALLER_SCRIPT -b -f -p $MINICONDA_PREFIX
Then add to path:
import sys
_ = (sys.path.append("/usr/local/lib/python3.6/site-packages"))
At this point you can manually uninstall and re-install tensorflow through pip:
!pip uninstall tensorflow
!pip install tensorflow-gpu==1.15
Doing just so I sometimes encounter some errors due to the Cuda version. If this happens to you, you can execute the following:
!apt install --allow-change-held-packages libcudnn7=7.4.1.5-1+cuda10.0
The most appropriate version of cuda and libcudnn to use with the tensorflow version you want to install can be found here.
The versions available of libcudnn can be found with the following command:
!apt list -a libcudnn7
This will list all libcudnn7 versions available.
I was having the same problems while trying to use StyleGAN2-ADA, which only supports TensorFlow 1.
I found out that unfortunately Google Colab removed support for TensorFlow 1 in their latest release of 2022/8/11.
'Removed support for TensorFlow 1'
You can find more information in their notebook Release-Notes: https://colab.research.google.com/notebooks/relnotes.ipynb
I have trained one object detection model in tensorflow.
My Environment--
tf version == 1.15, network== ssd mobilnet v2
Now i want to convert my saved_model(.pb) file to tfjs(.json) format.
I followed below steps--
pip install tensorflowjs==0.8.6 # not sure if it's compatible with tf version 1.15
command==
tensorflowjs_converter --input_format=tf_saved_model --output_format=tfjs_graph_model --signature_name=serving_default --saved_model_tags=serve exported_path/saved_model exported_path/web_model_path
Error== AttributeError: module 'keras_applications' has no attribute 'set_keras_submodules'
Then i degrade keras_application version.
Now getting error as
usage: TensorFlow.js model converters. [-h]
[--input_format {keras,tf_session_bundle,keras_saved_model,tf_hub,tf_saved_model,tensorflowjs,tf_frozen_model}]
[--output_format {keras,tensorflowjs}]
[--output_node_names OUTPUT_NODE_NAMES]
[--signature_name SIGNATURE_NAME]
[--saved_model_tags SAVED_MODEL_TAGS]
[--quantization_bytes {1,2}]
[--split_weights_by_layer] [--version]
[--skip_op_check SKIP_OP_CHECK]
[--strip_debug_ops STRIP_DEBUG_OPS]
[--output_json OUTPUT_JSON]
[input_path] [output_path]
TensorFlow.js model converters.: error: argument --output_format: invalid choice: 'tfjs_graph_model' (choose from 'keras', 'tensorflowjs')
So there is no option for tf_graph_model in output_format.
Now when i am installing pip install tensorflowjs (not passing any specific version), then it installs tfjs==3.3.0, and uninstaling my current tf1.15 and installing new tf2.x version. which i need to avoid at any cost.
Can somebody please guide me , how to convert the saved_model to tf_js format in version tensorflow==1.15.
Thanks in Advance.
Can you please upgrade tensorflow to latest version tf 2.x, stable versions tf 2.6/2.7 and let us know if this issue exist as tf 1.x is not supported any more and hence these errors.
I can't seem to find exact question about what I am about to ask here. I just started following a Tensorflow tutorial on YouTube and got stuck at the very beginning. I wrote in my spyder IDE the below code:
import tensorflow as tf
a = tf.constant(2)
b = tf.constant(3)
x = tf.add(a,b)
#writer = tf.summary.FileWriter('./graphs', tf.get_default_graph())
with tf.Session() as sess:
writer = tf.summary.FileWriter('./graphs', sess.graph)
print(sess.run(x))
writer.close()
And via anaconda terminal I activated my env (which I newly created, installed all packages required, spyder as well) I typed python tftuts.py and got 2018-10-05 11:50:49.431174: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
5
Then I typed tensorboard --logdir="./graphs" --port 6006 as suggested in tutorial I am watching.
Now, when I go to http://localhost:6006/ the page shows
I am on Win10, using python 3.6.6 in Anaconda env, tensorflow 1.10.0.
How can solve this issue?
Following the tutorial on TensorFlow for Poets (Android) (https://codelabs.developers.google.com/codelabs/tensorflow-for-poets-2/#0)
Attempting to use Inception model instead of Mobilenet
Trying to strip DecodeJpeg Op from the retrained model using strip_unused.py, but encountered the following error.
Error:
/home/user/tensorflow/bin/python: Error while finding spec for 'tensorflow.python.tools.strip_unused.py' (AttributeError: module 'tensorflow.python.tools.strip_unused' has no attribute '__path__')
Command line:
python -m tensorflow.python.tools.strip_unused.py --input_graph=tf_files/retrained_graph.pb --output_graph=tf_files/stripped_graph.pb --input_node_names="Mul" --output_node_names="final_result" --input_binary=true
Machine:
Ubuntu 16.04 LTS
Python 3.5.2
TensorFlow 1.4.1
Any assistance is greatly appreciated. Thanks!
Might be due to the typo mistake as file extension .py was specified. This seemed to work:
python -m tensorflow.python.tools.strip_unused --input_graph=tf_files/retrained_graph.pb --output_graph=tf_files/stripped_graph.pb --input_node_names="Mul" --output_node_names="final_result" --input_binary=true
Result: 997 ops in the final graph.
I'm doubting whether tensorflow is correctly configured on my gpu box, since it's about 100x slower per iteration to train a simple linear regression model (batchsize = 32, 1500 input features, 150 output variables) on my fancy gpu machine than on my laptop.
I'm using a Titan X, with a modern cpu, etc. nvidia-smi says that I'm only at 10% gpu utilization, but I expect that's because of the small batchsizes. I'm not using a feed_dict to move data into the computation graph. Everything is coming via a tf.decode_csv and tf.train.shuffle_batch.
Does anyone have any recommendations for how to easily test whether my install is correct? Are there any simple speed benchmarks? The speed difference between my laptop and the gpu machine is so dramatic that I'm expecting that things aren't configured properly.
Try tensorflow/tensorflow/models/image/mnist/convolutional.py, that'll print per-step timing.
On Tesla K40c that should get about 16 ms per step, while about 120 ms for CPU-only on my 3 year old machine
Edit: This moved to the models repositories: https://github.com/tensorflow/models/blob/master/tutorials/image/mnist/convolutional.py.
The convolutional.py file is now at models/tutorials/image/mnist/convolutional.py
Extending Yaroslavs answer:
Here is how to do the entire testing process (CUDA and cudNN installed already)
git clone https://github.com/tensorflow/models.git
Create a Virtual Environment for tensorflow and install tensorflow
virtualenv --system-site-packages -p python3 tf-venv3
source tf-venv3/bin/activate
pip install --upgrade pip
pip install --upgrade tensorflow-gpu
Run the model within your Virtual Environment
python models/tutorials/image/mnist/convolutional.py
My GTX 1070 needs ~5ms per step
Note: On Geforce 1050 Ti it takes ~10ms per step
The answer given by Yaroslav and extended by Patrice may be simply extended once again to work with TensorFlow v2.
Just in line 35 of convolutional.py, instead of:
import tensorflow as tf
type in:
import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
The file convolutional.py was removed from the tensorflow repo but it can still be found in the repo history.