Keras Upsampling2d operation is converted into this with additional operations and undefined shape
Tensorflow however converts without this operations with correct shape
This leads to undefined overall model output shape and leads to errors on device. How can this be fixed?
This behavior is described here https://github.com/tensorflow/tensorflow/issues/45090
Keras by default sets dynamic batch size to true.
That means that the model input shape is [*,28,28] not [1,28,28].
The old(deprecated) converter used to ignore the dynamic batch and override this to 1 - which is wrong since this is not what the original model has - you can imagine how bad it will be when you try to resize the inputs at runtime.
The current converter instead handles the dynamic batch size correct, and the model generated can be resized at runtime correct.
That's why the sequence of "Shape, StridedSlice, Pack" wasn't constant folded, since the shape is dependent on the shape defined at runtime.
For single input model this can be fixed by setting constant shape for keras model before saving
model.input.set_shape(1 + model.input.shape[1:])
Related
I am having a keras model trained on my own dataset. However after loading weights the summary shows None as the first dimension(the batch size).
I want to know the process to fix the shape to batch size of 1, as it is compulsory for me to fix it so i can convert the model to tflite with GPU support.
What worked for me was to specify batch size to the Input layer, like this:
input = layers.Input(shape=input_shape, batch_size=1, dtype='float32', name='images')
This then carried through the rest of the layers.
The bad news is that despite this "fix" the tfl runtime still complains about dynamic tensors. I get these non-fatal errors in logcat when it runs:
E/tflite: third_party/tensorflow/lite/core/subgraph.cc:801 tensor.data.raw != nullptr was not true.
E/tflite: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#26 is a dynamic-sized tensor).
E/tflite: Ignoring failed application of the default TensorFlow Lite delegate indexed at 0.
The good news is that despite these errors it seems to be using the GPU anyway, based on performance testing.
I'm using:
tensorflow-lite-support:0.2.0'
tensorflow-lite-metadata:0.2.1'
tensorflow-lite:2.6.0'
tensorflow:tensorflow-lite-gpu:2.3.0'
Hopefully, they'll fix the runtime so it doesn't matter whether the batch size is 'None'. It shouldn't matter for doing inference.
I am trying to use the Basic Text Classification example from Tensorflow on my own dataset. Training and verification have gone well and I am to the point in the tutorial for exporting the model. The model compiles and works on an array of strings.
After that, I'd like to save the model in h5 format for use in other projects. At this point, the tutorial refers you to save and load keras models tutorial.
This second tutorial essentially says to do this:
model.save('path/saved_model.h5')
This fails with
ValueError: Weights for model sequential_X have not yet been created. Weights are created when the Model is first called on inputs or build() is called with an input_shape.
So next I attempt to do this:
model.build((None, max_features))
model.save('path/saved_model.h5')
There are several errors with this:
ValueError: Tensor conversion requested dtype string for Tensor with dtype float32: <tf.Tensor 'Placeholder:0' shape=(None, 45000) dtype=float32>
TypeError: Input 'input' of 'StringLower' Op has type float32 that does not match expected type of string.
ValueError: You cannot build your model by calling build if your layers do not support float type inputs. Instead, in order to instantiate and build your model, call your model on real tensor data (of the correct dtype).
I think this essentially means the input I defined to pass into model.build defaults to float and needs to be string. I think I have two options:
Somehow define my input layer to be string, which I cannot see how to do. This feels like the correct thing to do.
Use model.call. However I am not sure how to 'call my model on real tensor data' because tensors can't be strings and that is the input to the network.
I've seen one other person with this issue here, with no solution other than to rebuild the model in functional style with mixed results. I am not sure of the point of rebuilding in the functional style since I don't fully understand the problem.
I'd prefer to have the TextVectorization layer built into the final model to simplify deployment. This is exactly the reason the docs give for doing this in the example in the first place. (The model will save without it.)
I am a novice with this so I might be making a simple mistake. How can I get this model to save?
I trained a tensorflow model for object detection with the input as a placeholder with the dimension [1,None,None,3] since my training images have various sizes. I then converted the frozen graph (.pb file) to a tensorRT graph for faster inference, but tensorRT gave me the warning that the input tensor has unknown non-batch dimension and thus the node would fall back to TF. The error message is:
2019-05-22 08:59:56.628216: W tensorflow/contrib/tensorrt/convert/convert_nodes.cc:3710] Validation failed for TensorRTInputPH_0 and input slot 0: Input tensor with shape [1,?,?,3] has an unknown non-batch dimension at dim 1
2019-05-22 08:59:56.628262: W tensorflow/contrib/tensorrt/convert/convert_graph.cc:1021] TensorRT node TRTEngineOp_0 added for segment 0 consisting of 160 nodes failed: Invalid argument: Validation failed for TensorRTInputPH_0 and input slot 0: Input tensor with shape [1,?,?,3] has an unknown non-batch dimension at dim 1. Fallback to TF...
I know I can set the is_dynamic_op to True in trt.create_inference_graph, but this increases the runtime. I want to use this model for the inference of a video, in which all frames have the same height and width. Is there a way to fix the input placeholder dimensions to static values without having to re-train the model? My input tensor can be accessed by get_tensor_by_name
I think you are using the static mode.
The default operating mode of TF-TRT is called static mode, and it is active whenever the parameter is_dynamic_op is set to False. In static mode, it is required that all shapes in the model are fully defined (dimensions cannot be None or -1). When the argument is_dynamic_op is set to True, TF-TRT will operate in dynamic mode. In dynamic mode, the TF-TRT API (create_inference_graph in TF <= 1.13) returns a modified graph, but no TensorRT engines are created at this time. Instead, the TensorRT engines are created during runtime when you send an input to the graph.
The main advantage of dynamic mode is that it allows you to have unknown shapes in your model, despite the fact that TensorRT requires all shapes to be fully defined. In this mode, TF-TRT creates a new TensorRT engine for each unique input shape that is supplied to the model.
We are trying to build an image segmentation deep learning model using Google Colab TPU. Our model is Mask R-CNN.
TPU_WORKER = 'grpc://' + os.environ['COLAB_TPU_ADDR']
import tensorflow as tf
tpu_model = tf.contrib.tpu.keras_to_tpu_model(
model.keras_model,
strategy=tf.contrib.tpu.TPUDistributionStrategy(
tf.contrib.cluster_resolver.TPUClusterResolver(TPU_WORKER)))
However I am running into issues while converting our Mask R-CNN model to TPU model as pasted below.
ValueError:
Layer <keras.engine.topology.InputLayer object at 0x7f58574f1940> has a
variable shape in a non-batch dimension. TPU models must
have constant shapes for all operations.
You may have to specify `input_length` for RNN/TimeDistributed layers.
Layer: <keras.engine.topology.InputLayer object at 0x7f58574f1940>
Input shape: (None, None, None, 3)
Output shape: (None, None, None, 3)
Appreciate any help.
Google recently released a tutorial on getting Mask R-CNN going on their TPUs. For this, they are using an experimental model for Mask RCNN on Google's TPU github repository (under models/experimental/mask_rcnn). Looking through the code, it looks like they define the model with a fixed input size to overcome the issue you are seeing.
See below for more explanation:
As #aman2930 points out, the shape of your input tensor is not static. This won't work because Tensorflow compiles models with XLA to use a TPU and XLA must have all tensor shapes defined at compile time. In the link above, the website specifically calls this out:
Static shapes
During regular usage TensorFlow attempts to determine the shapes of each tf.Tensor during graph construction. During
execution any unknown shape dimensions are determined dynamically, see
Tensor Shapes for more details.
To run on Cloud TPUs TensorFlow models are compiled using XLA. XLA
uses a similar system for determining shapes at compile time. XLA
requires that all tensor dimensions be statically defined at compile
time. All shapes must evaluate to a constant, and not depend on
external data, or stateful operations like variables or a random
number generator.
That side, further down the document, they mention that the input function is run on the CPU, so isn't limited by static XLA sizes. They point to batch size being the issue, not image size:
Static shapes and batch size
The input pipeline generated by your
input_fn is run on CPU. So it is mostly free from the strict static
shape requirements imposed by the XLA/TPU environment. The one
requirement is that the batches of data fed from your input pipeline
to the TPU have a static shape, as determined by the standard
TensorFlow shape inference algorithm. Intermediate tensors are free to
have a dynamic shapes. If shape inference has failed, but the shape is
known it is possible to impose the correct shape using tf.set_shape().
So you could fix this by reformulating your model to have fixed batch size or to use
tf.contrib.data.batch_and_drop_remainder as they suggest.
Could you please share the input data function. It is hard to tell the exact issue, but it seems that the shape of tensor representing input sample is not static.
I want to do real time data augmentation by chaining different image transformation operators in tensorflow. My code begins with image decoding and then runs different transformations but it throw a ValueError('\'image\' must be fully defined.'). Here is an example to reproduce this error :
def decode_and_augment(image_raw):
decoded = tf.image.decode_jpeg(image_raw)
flipped = tf.image.random_flip_left_right(decoded)
return flipped
This error arises because the tf.image.random_flip_left_right() op checks the static shape of its input when you build the graph, and tf.image.decode_jpeg() produces tensors that have a data dependency on the contents of image_raw so it the shape isn't statically known. Currently the only way to work around this is to set the static shape of the decoded tensor using Tensor.set_shape(), as follows:
decoded = tf.image.decode_jpeg(image_raw)
decoded.set_shape([IMAGE_HEIGHT, IMAGE_WIDTH, NUM_CHANNELS])
flipped = tf.image.random_flip_left_right(decoded)
The downside of this is that all images must now have the same size (and number of channels).
Many of the image ops don't follow the same gradual and dynamic shape inference as the rest of TensorFlow (which allows you to have unknown shapes or dimensions, assumes that the program is correct as you build the graph, and checks the real shapes at runtime). This is considered a bug at the present time, and we'll figure out a way to fix it.