I have a model much like the tensorflow speech command demo except it takes a variable sized 1D array as input. Now I find it difficult to convert this model to TF lite using tflite_convert which requires input_shape for input.
It's said that tf lite requires fixed size input for efficiency and you can resize input during inference as part of your model. However, I think it would involve truncating the input which I don't want. Is there any way to make this work with TF lite?
You can convert your model using a fixed shape as in --input_shape=64, then at inference-time you would do:
interpreter->ResizeInputTensor(interpreter->inputs()[0], {128});
interpreter->AllocateTensors();
// ... populate your input tensors with 128 entries ...
interpreter->Invoke();
// ... read your output tensor ...
Related
I am training a tensorflow model with with multiple images as input and a segmentation mask at the output. I wanted to perform random rotation augmentation in my dataset pipeline.
I have a list of parallel image file names (input output files aligned) for which I convert into tf dataset object using tf.data.Dataset.from_generator and then and use Dataset.map function to load images with tf.image.decode_png and tf.io.read_file commands.
How can I perform random rotations on the input-output images. I tried using random_transform function of ImageDataGenerator class but it expects numpy data as input and does not work on Tensors (Since tensorflow does not support eager execution in data pipeline I cannot convert it into numpy as well). I suppose I can use tf.numpy_function but I expect there should be some simple solution for this simple problem.
I am trying to convert DeepLab trained on the Cityscapes dataset from here to TFLite. From viewing the frozen graph in Netron, the input and output tensors both are of type uint8. I was able to use the default DeepLab model provided for the TFLite GPU delegate, which had float32 input and output tensors. I didn't think the model was supposed to be quantized, so when trying the following code without the commented lines, I got this error:
F tensorflow/lite/toco/tooling_util.cc:2241] Check failed: array.data_type == array.final_data_type Array "ImageTensor" has mis-matching actual and final data types (data_type=uint8, final_data_type=float).
After this, I found that I should try to quantize the model. I inserted the commented lines to use uint8 instead of float32, but I got this error, which seems like an unsupported op.
F ./tensorflow/lite/toco/toco_tooling.h:38] Check failed: s.ok() Unimplemented: this graph contains anoperator of type Cast for which the quantized form is not yet implemented. Sorry, and patches welcome (that's a relatively fun patch to write, mostly providing the actual quantized arithmetic code for this op).
Is it right to use the quantized script? The off-the-shelf TFLite DeepLab model provided uses float32. Thanks!
I've trained a simple CNN model on Cifar-10 in tensorflow with fake quantization (https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/quantize). I then generated a .tflite file using toco. Now I want to use a python interpreter to test the tflite model.
Since I used tf.image.per_image_standardization to subtract mean and divide by variance during training. I need to do the same thing to the testing data right? But, the problem is, my model is already fully quantized by tflite, and it only takes uint8 data as inputs. To do image standardization, I need to convert my image to float32. So how do I convert it back to uint8, or is image standardization even necessary for the testing data in this case? Thanks.
So, it turns out I need to do standardization on the testing data for a good accuracy.
To do it, I directly feed uint8 input images to the tf.image.per_image_standardization function. The function would convert the uint8 data to float32, and then do standardization (subtract mean, divide by std). You can find source code of the function here: https://github.com/tensorflow/tensorflow/blob/r1.11/tensorflow/python/ops/image_ops_impl.py
Now, I have the standardized float32 input images. What I did is writing a quantization function to quantize the float32 images back to uint8. The math comes from this paper: https://arxiv.org/abs/1803.08607
Now, I have the standardized uint8 input images, I then use tflite interpreter python API to test the model. It works as expected.
I don't know how to convert the PyTorch method adaptive_avg_pool2d to Keras or TensorFlow. Anyone can help?
PyTorch mehod is
adaptive_avg_pool2d(14,[14])
I tried to use the average pooling, the reshape the tensor in Keras, but got the error:
ValueError: total size of new array must be unchanged
I'm not sure if I understood your question, but in PyTorch, you pass the spatial dimensions to AdaptiveAvgPool2d. For instance, if you want to have an output sized 5x7, you can use nn.AdaptiveAvgPool2d((5,7)).
If you want a global average pooling layer, you can use nn.AdaptiveAvgPool2d(1). In Keras you can just use GlobalAveragePooling2D.
For other output sizes in Keras, you need to use AveragePooling2D, but you can't specify the output shape directly. You need to calculate/define the pool_size, stride, and padding parameters depending on how you want the output shape. If you need help with the calculations, check this page of CS231n course.
If the input data is in numpy array format, then we can declare a placeholder in the graph and feed the placeholder with the numpy array data. However, if the input data is already in Tensor format (this is the case when we load jpg files using tf.image.decode_jpeg), then we can't feed a Tensor to a placeholder. In this case, should we use non trainable TF Variables as placeholders, and feed the Tensor to these Variables by tf.assign?
Figured it out. You can simply feed batches of Tensors to the model. The model probably has a line that looks similar to op = optimizer.minimize(loss). Then, each time sess.run(op) is called, the model will be trained on the batch provided to it. Also, each time sess.run(op) is called, we should have a different batch if we use tf.train.batch to provide the batch.