I have already had a tflite model, like 'model.TFLITE', and now I want to convert it to int8. Is there any convert funcs or tools to implement it? (tf.lite.TFLiteConverter only has from_savemodel,from_keras,from_func)
(tf.lite.TFLiteConverter only has from_savemodel,from_keras,from_func)
Related
Is there a way to convert support vector classifier with rbf kernel to tf model?
I am aware of converting support vector classifier with linear kernel because there exists coef_ where we can find parameters and assign to tf model. Got this idea from how to convert saved model from sklearn into tensorflow/lite.
However _coef wont be there for rbf , so i am not sure on how to convert this model to an tf model.
I have my own pytorch, onnx, tf1.x(frozen_graph.pb), and tf2.x(saved_model.pb) models.
Each of those model outputs is exactly same as I expected.
The original model is PyTorch model, but I converted it into the following sequences to run the model on my objective embedded platform:
PyTorch -> onnx -> tf2.x saved_model.pb -> tf1.x frozen_graph.pb
The problem is that. The objective embedded platform only supports data in NHWC format. But my whole models have NCHW format.
So the question is, is there any way to convert the data formats of the trained model from NCHW to NHWC using tf1.x or tf2.x?
Please advice.
I am converting a frozen pb model to onnx, but onnx does not support uint8. How do I go about replacing uint8 with int8 in the model file? I can also retrain if I have to, but I am not sure which file to modify and how. I would have guessed that I needed to modify the file which contains the neural net architecture:
https://github.com/tensorflow/models/blob/master/research/slim/nets/mobilenet/mobilenet_v2.py
but that does not seem to be the case...
There's a MobileNet v2 ONNX Model in the ONNX model zoo that has already been converted (although the source model was originally from mxnet). This might be useful.
ONNX operators do support uint8 datatypes, as described in the ONNX IR documentation.
I am trying to convert DeepLab trained on the Cityscapes dataset from here to TFLite. From viewing the frozen graph in Netron, the input and output tensors both are of type uint8. I was able to use the default DeepLab model provided for the TFLite GPU delegate, which had float32 input and output tensors. I didn't think the model was supposed to be quantized, so when trying the following code without the commented lines, I got this error:
F tensorflow/lite/toco/tooling_util.cc:2241] Check failed: array.data_type == array.final_data_type Array "ImageTensor" has mis-matching actual and final data types (data_type=uint8, final_data_type=float).
After this, I found that I should try to quantize the model. I inserted the commented lines to use uint8 instead of float32, but I got this error, which seems like an unsupported op.
F ./tensorflow/lite/toco/toco_tooling.h:38] Check failed: s.ok() Unimplemented: this graph contains anoperator of type Cast for which the quantized form is not yet implemented. Sorry, and patches welcome (that's a relatively fun patch to write, mostly providing the actual quantized arithmetic code for this op).
Is it right to use the quantized script? The off-the-shelf TFLite DeepLab model provided uses float32. Thanks!
I've trained a simple CNN model on Cifar-10 in tensorflow with fake quantization (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/quantize). I then generated a .tflite file using toco. Now I want to use a python interpreter to test the tflite model.
Since I used tf.image.per_image_standardization to subtract mean and divide by variance during training. I need to do the same thing to the testing data right? But, the problem is, my model is already fully quantized by tflite, and it only takes uint8 data as inputs. To do image standardization, I need to convert my image to float32. So how do I convert it back to uint8, or is image standardization even necessary for the testing data in this case? Thanks.
So, it turns out I need to do standardization on the testing data for a good accuracy.
To do it, I directly feed uint8 input images to the tf.image.per_image_standardization function. The function would convert the uint8 data to float32, and then do standardization (subtract mean, divide by std). You can find source code of the function here: https://github.com/tensorflow/tensorflow/blob/r1.11/tensorflow/python/ops/image_ops_impl.py
Now, I have the standardized float32 input images. What I did is writing a quantization function to quantize the float32 images back to uint8. The math comes from this paper: https://arxiv.org/abs/1803.08607
Now, I have the standardized uint8 input images, I then use tflite interpreter python API to test the model. It works as expected.