I am using efficientdet_d1_coco17_tpu-32 model after fine-tuning it on my images for object detection. I have used TFOD(tensorflow object detection) API.
I can successfully pass a single image for inference and get the bounding boxes.
I want to do inference for a batch of images. For that, I wrote the code given below:
efficientdet = tf.saved_model.load(custom_saved_model_path)
pred_files_ds = tf.data.Dataset.list_files(str(IMAGE_FOLDER/"*"))
pred_imgs_ds = pred_files_ds.map(lambda x: parse_image(x,pred_img_height, pred_img_width))
for fblb, names in pred_imgs_ds.batch(2):
print(f"Shape of image for classification: {fblb.shape}")
detections = efficientdet(fblb)
On executing the above code, I get the following error:
ValueError: Python inputs incompatible with input_signature:
inputs: (
tf.Tensor(
[[[[255. 229. 236. ]
[255. 229.60938 236.40625 ]
[255. 231.01562 237.34375 ]
...
...
[ 57.5 50.5 44.5 ]
[ 56.909966 49.909966 43.909966]
[ 56.099976 49.099976 43.099976]]]], shape=(2, 640, 640, 3), dtype=float32))
input_signature: (
TensorSpec(shape=(1, None, None, 3), dtype=tf.uint8, name='input_tensor'))
I am guessing that because the input signature specification has 1 as batch size and the actual input has 2 as batch size shape, I am getting this error.
Can I change the input signature spec of the efficientdet model somehow? I have observed that in pipeline.config file in the eval_config section, there is a batch_size key set to 1. If I change it to some value > 1, will I be able to pass a batch of images for inference ?
I am using tensorflow 2.6.x
Related
I am trying to use TF to solve a custom gym environment, all within Google Colab.
The main script is the TF "DQN Tutorial" available here.
In place of env_name = "CartPole-v0" I am using env_name = "gym_examples/GridWorld-v0", where gym_examples/GridWorld-v0 is the sample custom environment described in the gym documentation here. (That example uses gym v0.25.0 but TF requires gym <= v0.23.0, so I also had to tweak the rendering code a bit to make it work in v0.23.0.)
The environment loads fine via env = suite_gym.load(env_name), and subsequent code cells run fine as well, until the following two cells:
fc_layer_params = (100, 50)
action_tensor_spec = tensor_spec.from_spec(env.action_spec())
num_actions = action_tensor_spec.maximum - action_tensor_spec.minimum + 1
# Define a helper function to create Dense layers configured with the right
# activation and kernel initializer.
def dense_layer(num_units):
return tf.keras.layers.Dense(
num_units,
activation=tf.keras.activations.relu,
kernel_initializer=tf.keras.initializers.VarianceScaling(
scale=2.0, mode='fan_in', distribution='truncated_normal'))
# QNetwork consists of a sequence of Dense layers followed by a dense layer
# with `num_actions` units to generate one q_value per available action as
# its output.
dense_layers = [dense_layer(num_units) for num_units in fc_layer_params]
q_values_layer = tf.keras.layers.Dense(
num_actions,
activation=None,
kernel_initializer=tf.keras.initializers.RandomUniform(
minval=-0.03, maxval=0.03),
bias_initializer=tf.keras.initializers.Constant(-0.2))
q_net = sequential.Sequential(dense_layers + [q_values_layer])
optimizer = tf.keras.optimizers.Adam(learning_rate=learning_rate)
train_step_counter = tf.Variable(0)
agent = dqn_agent.DqnAgent(
train_env.time_step_spec(),
train_env.action_spec(),
q_network=q_net,
optimizer=optimizer,
td_errors_loss_fn=common.element_wise_squared_loss,
train_step_counter=train_step_counter)
agent.initialize()
After that cell, I get an error:
ValueError: Exception encountered when calling layer "sequential_2" (type Sequential).
Layer "dense_6" expects 1 input(s), but it received 2 input tensors. Inputs received: [<tf.Tensor: shape=(1, 2), dtype=int64, numpy=array([[2, 2]])>, <tf.Tensor: shape=(1, 2), dtype=int64, numpy=array([[3, 2]])>]
Call arguments received by layer "sequential_2" (type Sequential):
• inputs={'agent': 'tf.Tensor(shape=(1, 2), dtype=int64)', 'target': 'tf.Tensor(shape=(1, 2), dtype=int64)'}
• network_state=()
• kwargs={'step_type': 'tf.Tensor(shape=(1,), dtype=int32)', 'training': 'None'}
In call to configurable 'DqnAgent' (<class 'tf_agents.agents.dqn.dqn_agent.DqnAgent'>)
I'm too much of a TF novice to understand what's going on here. I suspect it's because the action state changed from 2 states (in CartPole) to 4 (in the custom GridWorld environment). But beyond that I cannot figure it out.
This can be solved by using an Embedding layer as your first layer. In this example (Embedding(16, 4)), 16 is the grid size (4x4), and 4 is the output dimension.
dense_layers = [dense_layer(num_units) for num_units in fc_layer_params]
For example, replacing the above line with the code below will eradicate the error.
dense_layers = [
# First layer
tf.keras.layers.Embedding(16, 4),
# Other layers
tf.keras.layers.Dense(100, activation=tf.keras.activations.relu)
]
Source and for further explanation:
https://martin-ueding.de/posts/reinforcement-learning-with-frozen-lake/
I implemented a simple TF model. The model received a serialized tensor of a gray image (simply a 2d ndarray), and restored it to a 2-d tensor. After then, some inference is applied on this 2-d tensor.
I deployed the mode with TensorFlow Model Serving, and tried to send a JSON string to the REST port as follows:
{
"instances": [
{"b64": bin_str},
]
}
I tried something like tf.io.serialize_tensor etc to convert input image into a serialized tensor and to pass it to the server, but all failed.
I would like to know how to send a serialized tensor to the serving server.
And my saved model has following signature:
signatures = {
"serving_default": _get_serve_tf_examples_fn(
model,
transform_output).get_concrete_function(
# explicitly specify input signature of serve_tf_examples_fn
tf.TensorSpec(
shape=[None],
dtype=tf.string,
name="examples")),
}
and the definition of _get_serve_tf_examples_fn is,
def _get_serve_tf_examples_fn(model: tf.keras.models.Model,
transform_output: tft.TFTransformOutput):
# get the Transform graph from the component
model.tft_layer = transform_output.transform_features_layer()
#tf.function
def serve_tf_examples_fn(serialized: str) -> Dict:
''' Args: serialized: is serialized image tensor.
'''
feature_spec = transform_output.raw_feature_spec()
# remove label spec.
feature_spec.pop("label")
# Deserialize the image tensor.
parsed_features = tf.io.parse_example(
serialized,
feature_spec)
# Preprocess the example using outputs of Transform pipeline.
transformed_features = model.tft_layer(parsed_features)
outputs = model(transformed_features)
return {"outputs": outputs}
return serve_tf_examples_fn
The above code segment received a serialized tensor of a gray image (simply a 2d ndarray), and restored it to a 2-d tensor. After then, the model is doing inference on this 2-d tensor.
I would like to know how to send a serialized tensor to the REST port of the serving server.
Any help would be appreciated.
I have successfully setup Google Cloud and deployed a pre-trained ML model that takes an input tensor (image) of shape=(?, 224, 224, 3) and dtype=float32. It works well but this is inefficient when making REST requests and should really use a base64 encoded string. The challenge is that I am using transfer learning and cannot control the input of the original pre-trained model. To get around this with adding additional infrastructure I created a small graph (wrapper) that handles the base64 to array conversion and connected it to my pre-trained model graph yielding a new single graph. The small graph takes an input tensor with the shape=(), dtype=string and return a tensor with the shape=(224, 224, 3), dtype=float32 which can then be passed to the original model. The model compiles to .pb file without errors and successfully deploys but I get the following error when making my Post request:
{'error': 'Prediction failed: Error during model execution: AbortionError(code=StatusCode.INVALID_ARGUMENT, details="Index out of range using input dim 0; input has only 0 dims\n\t [[{{node lambda/map/while/strided_slice}}]]")'}
Post request body:
{'instances': [{'b64': 'iVBORw0KGgoAAAANSUhEUgAAAOAA...'}]}`
This error leads me to believe the post request is incorrectly formatted for handling the base64 string or my base conversion graph input is setup incorrectly. I can run the code locally by calling predict on my combined model and pass it a tensor in the form of shape=(), dtype=string constructed locally and get a result successfully.
Here is my code for combining the 2 graphs:
import tensorflow as tf
# Local dependencies
from myProject.classifier_models import mobilenet
from myProject.dataset_loader import dataset_loader
from myProject.utils import f1_m, recall_m, precision_m
with tf.keras.backend.get_session() as sess:
def preprocess_and_decode(img_str, new_shape=[224,224]):
#img = tf.io.decode_base64(img_str)
img = tf.image.decode_png(img_str, channels=3)
img = (tf.cast(img, tf.float32)/127.5) - 1
img = tf.image.resize_images(img, new_shape, method=tf.image.ResizeMethod.AREA, align_corners=False)
# If you need to squeeze your input range to [0,1] or [-1,1] do it here
return img
InputLayer = tf.keras.layers.Input(shape = (1,),dtype="string")
OutputLayer = tf.keras.layers.Lambda(lambda img : tf.map_fn(lambda im : preprocess_and_decode(im[0]), img, dtype="float32"))(InputLayer)
base64_model = tf.keras.Model(InputLayer,OutputLayer)
tf.keras.backend.set_learning_phase(0) # Ignore dropout at inference
transfer_model = tf.keras.models.load_model('./trained_model/mobilenet_93.h5', custom_objects={'f1_m': f1_m, 'recall_m': recall_m, 'precision_m': precision_m})
sess.run(tf.global_variables_initializer())
base64_input = base64_model.input
final_output = transfer_model(base64_model.output)
new_model = tf.keras.Model(base64_input,final_output)
export_path = '../myModels/001'
tf.saved_model.simple_save(
sess,
export_path,
inputs={'input_class': new_model.input},
outputs={'output_class': new_model.output})
Tech: TensorFlow 1.13.1 & Python 3.5
I have looked at a bunch of related posts such as:
https://stackoverflow.com/a/50606625
https://stackoverflow.com/a/42859733
http://www.voidcn.com/article/p-okpgbnul-bvs.html (right-click translate to english)
https://cloud.google.com/ml-engine/docs/tensorflow/online-predict
Any suggestions or feedback would be greatly appreciated!
Update 06/12/2019:
Inspecting the 3 graph summaries everything appears correctly merged
Update 06/14/2019:
Ended up going with this alternative strategy instead, implementing a tf.estimator
I'm using Conv2d in Keras to do some classification for gray-scale images. Each image is stored as a 240*300 matrix, (namely a list [ A_1, A_2,..., A_240 ] and each A_k is a list of length 300
How should I specify the input_shape of the first layer of my ConvNet?
Thanks
ValueError: Input 0 of layer conv2d is incompatible with the layer:
expected ndim=4, found ndim=3. Full shape received
: [None, 240, 300]
First, you need to reshape your data, adding a dimension at the end with size of one, which represents one channel (a grayscale image). Assuming data has shape (samples, 240, 300):
data = data.reshape((-1, 240, 300, 1))
This will make data have shape (samples, 240, 300, 1). Then to your first layer you should give input_shape=(240, 300, 1)
I am implementing a custom layer in Keras. If I print the shape of the input passed to the call method, I get None as the first element. Why is that? Shouldn't the first element be the batch size?
def call(self, x):
print(x.shape) # (None, ...)
When I call model.fit, I am passing the batch size
batch_size = 50
model.fit(x_train, y_train, ..., batch_size=batch_size)
So, when is the method call actually called? And what is the recommended way of getting the batch size in the method call?
None means it is a dynamic shape. It can take any value depending on the batch size you choose.
When you define a model by default it is defined to support any batch size you can choose. This is what the None means. In TensorFlow 1.* the input to your model is an instance of tf.placeholder().
If you don't use the keras.InputLayer() with specified batch size you get the first dimension None by default:
import tensorflow as tf
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(units=2, input_shape=(2, )))
print(model.inputs[0].get_shape().as_list()) # [None, 2]
print(model.inputs[0].op.type == 'Placeholder') # True
When you do use keras.InputLayer() with specified batch size you can define the input placeholder with fixed batch size:
import tensorflow as tf
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.InputLayer((2,), batch_size=50))
model.add(tf.keras.layers.Dense(units=2, input_shape=(2, )))
print(model.inputs[0].get_shape().as_list()) # [50, 2]
print(model.inputs[0].op.type == 'Placeholder') # True
When you specify the batch size to the model.fit() method these input placeholders have already been defined and you cannot modify their shape. The batch size for model.fit() is used only to split the data you provided to batches.
If you define your input layer with batch size 2 and then you pass different value of a batch size to the model.fit() method you will get ValueError:
import tensorflow as tf
import numpy as np
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.InputLayer((2,), batch_size=2)) # <--batch_size==2
model.add(tf.keras.layers.Dense(units=2, input_shape=(2, )))
model.compile(optimizer=tf.keras.optimizers.Adam(),
loss='categorical_crossentropy')
x_train = np.random.normal(size=(10, 2))
y_train = np.array([[0, 1] for _ in range(10)])
model.fit(x_train, y_train, batch_size=3) # <--batch_size==3
This will raise:
ValueError: Thebatch_sizeargument value 3 is incompatible with the specified batch size of your Input Layer: 2
I faced the same issue and found that using tf.shape(your_variable) instead of your_variable.shape solved the problem. As tf.shape(your_variable) is dynamically evaluated later when the fit function is called.
reference
https://github.com/tensorflow/tensorflow/issues/36991#issuecomment-590448880
To get the value in integers consider output_shape variable.
Minimum working example
import tensorflow as tf
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.InputLayer((2,), batch_size=50))
model.add(tf.keras.layers.Dense(units=2, input_shape=(2, )))
print(model.output_shape)
print(type(model.output_shape[0]), type(model.output_shape[1]))
Output:
(50, 2)
<class 'int'> <class 'int'>