I am using tensorflow=1.14. During periodic evaluation, I save best model using tf.estimator.BestExporter. I have following questions.
1)
When I try to convert saved_model.pb stored by BestExporter during training to frozen graph using freeze_graph() function, usual input/output node names ("image_tensor" / ['detection_boxes', 'detection_classes', 'detection_scores', 'num_detections']) are not present in saved_model. when I inspect using saved model cli, input / output names are completely different than the saved model stored by export_inference_graph.py using checkpoint and graph with pipeline.config.
"""
export_inference_graph's saved model saverDef
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['serialized_example'] tensor_info:
dtype: DT_STRING
shape: ()
name: tf_example:0
The given SavedModel SignatureDef contains the following output(s):
outputs['detection_boxes'] tensor_info:
dtype: DT_FLOAT
shape: (1, 150, 4)
name: Postprocessor/BatchMultiClassNonMaxSuppression/stack_4:0
outputs['detection_classes'] tensor_info:
dtype: DT_FLOAT
shape: (1, 150)
name: Postprocessor/BatchMultiClassNonMaxSuppression/stack_6:0
outputs['detection_scores'] tensor_info:
dtype: DT_FLOAT
shape: (1, 150)
name: Postprocessor/BatchMultiClassNonMaxSuppression/stack_5:0
outputs['num_detections'] tensor_info:
dtype: DT_FLOAT
shape: (1)
name: Postprocessor/ToFloat_3:0
Method name is: tensorflow/serving/predict
BestExporter saverdef
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs'] tensor_info:
dtype: DT_UINT8
shape: (-1, -1, -1, 3)
name: image_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['detection_boxes'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 150, 4)
name: detection_boxes:0
outputs['detection_classes'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 150)
name: detection_classes:0
outputs['detection_scores'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 150)
name: detection_scores:0
outputs['num_detections'] tensor_info:
dtype: DT_FLOAT
shape: (-1)
name: num_detections:0
Method name is: tensorflow/serving/predict
"""
As one can see, both have different ip/op names.
2)
Alternate approach I tried which is directly using saved_model.pb saved by BestExporter for inference. By inspecting .pb file using saved model cli, input shape it refers to is string with no dimension, which again prevent me using this approach because when passed numpy image for inference, it raised shape mismatch error. (From above)
Can someone help me out how can I use saved_model from BestExporter for inference or convert it to frozen graph with correct ip/op so It can be used for inference.
Let me know If you need more information.
Thank you
I'm currently implement the sequantial deep matching model (https://arxiv.org/abs/1909.00385) using tensorflow 2.3. And I included the preprocessing layer as part of the model via subclassing keras.layers.Layer.
The preprocessing part of code is listed below
class Preprocessing(keras.layers.Layer):
def __init__(self, str_columns, hash_bins, float_columns, float_buckets, embedding_dim, user_columns, short_seq_columns, prefer_seq_columns, item_key_feats,
item_key_hash_bucket_size, series_feats, series_feats_hash_bucket_size, deviceid_num, device_list, **kwargs):
super(Preprocessing, self).__init__(**kwargs)
self.str_columns = str_columns
self.hash_bins = hash_bins
self.float_columns = float_columns
self.float_buckets = float_buckets
self.embedding_dim = embedding_dim
self.user_columns = user_columns
self.short_seq_columns = short_seq_columns
self.prefer_seq_columns = prefer_seq_columns
self.item_key_feats = item_key_feats
self.item_key_hash_bucket_size = item_key_hash_bucket_size
self.series_feats = series_feats
self.series_feats_hash_bucket_size = series_feats_hash_bucket_size
self.deviceid_num = deviceid_num
self.device_list = device_list
self.user_outputs = {}
self.short_outputs = {}
self.prefer_outputs = {}
deviceid_lookup = keras.layers.experimental.preprocessing.StringLookup(vocabulary=device_list, mask_token=None, oov_token="-1")
deviceid_embedding = keras.layers.Embedding(input_dim=deviceid_num, output_dim=embedding_dim)
item_key_hashing = keras.layers.experimental.preprocessing.Hashing(num_bins=item_key_hash_bucket_size)
item_key_embedding = keras.layers.Embedding(input_dim=item_key_hash_bucket_size, output_dim=embedding_dim)
series_hashing = keras.layers.experimental.preprocessing.Hashing(num_bins=series_feats_hash_bucket_size)
series_embedding = keras.layers.Embedding(input_dim=series_feats_hash_bucket_size, output_dim=embedding_dim)
for i in str_columns:
if i == "device_id":
process = [deviceid_lookup, deviceid_embedding]
elif i in item_key_feats:
process = [item_key_hashing, item_key_embedding]
elif i in series_feats:
process = [series_hashing, series_embedding]
else:
hashing = keras.layers.experimental.preprocessing.Hashing(num_bins=hash_bins[i])
embedding = keras.layers.Embedding(input_dim=hash_bins[i], output_dim=embedding_dim)
process = [hashing, embedding]
if i in user_columns:
self.user_outputs[i] = process
if i in short_seq_columns:
self.short_outputs[i] = process
if i in prefer_seq_columns:
self.prefer_outputs[i] = process
for l in float_columns:
discrete = keras.layers.experimental.preprocessing.Discretization(bins=float_buckets[l])
embedding = keras.layers.Embedding(input_dim=len(float_buckets[l]) + 1, output_dim=embedding_dim)
if l in user_columns:
self.user_outputs[l] = [discrete, embedding]
if l in short_seq_columns:
self.short_outputs[l] = [discrete, embedding]
if l in prefer_seq_columns:
self.prefer_outputs[l] = [discrete, embedding]
#staticmethod
def get_embedding(input_tmp, name, embed_dict):
func = embed_dict[name]
if len(func) < 2:
print(func)
raise Exception('Not enough function to retrieve embedding')
output = func[0](input_tmp)
output = func[1](output)
return output
def call(self, inputs):
user_embedding = tf.concat([tf.reduce_mean(self.get_embedding(inputs[i], i, self.user_outputs), axis=[1, 2]) for i in self.user_columns], axis=-1)
short_embedding = tf.concat([tf.squeeze(self.get_embedding(inputs[l], l, self.short_outputs), axis=1).to_tensor() for l in self.short_seq_columns], axis=-1)
prefer_embedding = {k: tf.squeeze(self.get_embedding(inputs[k], k, self.prefer_outputs).to_tensor(), axis=1) for k in self.prefer_seq_columns}
return user_embedding, short_embedding, prefer_embedding
And also my input code:
def read_row(csv_row):
record_defaults = [[0.]] * numeric_feature_size + [['']] * category_feature_size + [['0-0']] + [['0']]
row = tf.io.decode_csv(csv_row, record_defaults=record_defaults, field_delim='', use_quote_delim=False)
features = []
for i, feature in enumerate(row):
if i < numeric_feature_size:
features.append(feature)
elif i < numeric_feature_size + category_feature_size:
tmp_tf = tf.strings.split([feature], ";")
features.append(tmp_tf)
res = OrderedDict(zip(numeric_columns + category_columns, features))
res['target'] = [tf.cast(row[-2], tf.string)]
return res
The other part of code is not giving here, cause I believe it's right, and might be too much to list here.
The model is working correctly during training using model.compile then model.fit, however, after I saved it with model.save(path), the resulting Graph gets many unknown inputs and none of the inputs name is saved.
saved_model_cli show --dir ./ --tag_set serve --signature_def serving_default
The given SavedModel SignatureDef contains the following input(s):
inputs['args_0'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0:0
inputs['args_0_1'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_1:0
inputs['args_0_10'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_10:0
inputs['args_0_11'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_11:0
inputs['args_0_12'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_12:0
inputs['args_0_13'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_13:0
inputs['args_0_14'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_14:0
inputs['args_0_15'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_15:0
inputs['args_0_16'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_16:0
inputs['args_0_17'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_17:0
inputs['args_0_18'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_18:0
inputs['args_0_19'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_19:0
inputs['args_0_2'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_2:0
inputs['args_0_20'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_20:0
inputs['args_0_21'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_21:0
inputs['args_0_22'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_22:0
inputs['args_0_23'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_23:0
inputs['args_0_24'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_24:0
inputs['args_0_25'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_25:0
inputs['args_0_26'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_26:0
inputs['args_0_27'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_27:0
inputs['args_0_28'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_28:0
inputs['args_0_29'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_29:0
inputs['args_0_3'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_3:0
inputs['args_0_30'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_30:0
inputs['args_0_31'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_31:0
inputs['args_0_32'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_32:0
inputs['args_0_33'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_33:0
inputs['args_0_34'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_34:0
inputs['args_0_35'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_35:0
inputs['args_0_36'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_36:0
inputs['args_0_37'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_37:0
inputs['args_0_38'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_38:0
inputs['args_0_39'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_39:0
inputs['args_0_4'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_4:0
inputs['args_0_40'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_40:0
inputs['args_0_41'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_41:0
inputs['args_0_42'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_42:0
inputs['args_0_43'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_43:0
inputs['args_0_44'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_44:0
inputs['args_0_45'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_45:0
inputs['args_0_46'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_46:0
inputs['args_0_47'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_47:0
inputs['args_0_48'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_48:0
inputs['args_0_49'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_49:0
inputs['args_0_5'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_5:0
inputs['args_0_50'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_50:0
inputs['args_0_6'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_6:0
inputs['args_0_7'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_7:0
inputs['args_0_8'] tensor_info:
dtype: DT_INT64
shape: (-1)
name: serving_default_args_0_8:0
inputs['args_0_9'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_args_0_9:0
The given SavedModel SignatureDef contains the following output(s):
outputs['output_1'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 64)
name: StatefulPartitionedCall:0
In this model, I only used the categorical features with dtype as tf.string, so all the inputs with dtype of DT_INT64 is not part of my model inputs.
Can anyone help me with this?
I finally got this work.
The error came from my tf.io.decode_csv method. tf.strings.split()will return a RaggedTensor by default, which will also contain two int variables. That's why the input signatures contains 17 string type and 34 int type.
The RaggedTensor will also harm the keras serialization, which is why all of the input names were missing.
I transformed all the RaggedTensor to EagerTensor, and all things worked.
However, that's not the only error I've encountered when I tried to load the model.
I also encountered
The same saveable will be restored with two names error, which cost me tons of time to resolved it. And it turns out to be as bug of keras.layers.experimental.preprocessing module, the same function in it cannot be used twice, cause variables will be recorded as the same name, and result in a non-loadable savedmodel.
It is easy, I test with multiple ways and found name is the most significant thing that happen.
The custom class and RNN with lstm cell are not working, I spend time testing also that takes a lot of time because tracing on and RNN .
[ Sample ]:
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Model Initialize
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
model = tf.keras.models.Sequential([
tf.keras.layers.InputLayer(input_shape=( 32, 32, 3 ), batch_size=1, name='layer_input'),
tf.keras.layers.Normalization(mean=3., variance=2., name='layer_normalize_1'),
tf.keras.layers.Normalization(mean=4., variance=6., name='layer_normalize_2'),
tf.keras.layers.Conv2DTranspose(2, 3, activation='relu', padding="same", name='layer_conv2dT_1'),
tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=(1, 1), padding='valid', name='layer_maxpool_1'),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(4 * 256, name='layer_dense_1'),
tf.keras.layers.Reshape((4 * 256, 1)),
tf.keras.layers.LSTM(128, name='layer_4', return_sequences=True, return_state=False),
tf.keras.layers.LSTM(128, name='layer_5'),
tf.keras.layers.Dropout(0.2),
])
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(64, activation='relu', name='layer_dense_2'))
model.add(tf.keras.layers.Dense(7, name='layer_dense_3'))
model.summary()
# Loads the weights
if exists(target_saved_model) :
model = load_model(target_saved_model)
print("model load: " + target_saved_model)
input("Press Any Key!")
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
: Training
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""
history = model.fit(x_train, y_train, epochs=10, batch_size=1 ,validation_data=(x_test, y_test))
model.save(target_saved_model)
print('.................')
[ Output ]: 1. Error messages from layer lstm
Epoch 10/10
20/20 [==============================] - 1s 71ms/step - loss: 0.7332 - val_loss: 0.8078
2022-04-06 23:23:26.017439: W tensorflow/python/util/util.cc:368] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
WARNING:absl:Found untraced functions such as lstm_cell_layer_call_fn, lstm_cell_layer_call_and_return_conditional_losses, lstm_cell_1_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses while saving (showing 4 of 4). These functions will not be directly callable after loading.
WARNING:absl:<keras.layers.recurrent.LSTMCell object at 0x00000145F1C83130> has the same name 'LSTMCell' as a built-in Keras object. Consider renaming <class 'keras.layers.recurrent.LSTMCell'> to avoid naming conflicts when loading with `tf.keras.models.load_model`. If renaming is not possible, pass the object in the `custom_objects` parameter of the load function.
WARNING:absl:<keras.layers.recurrent.LSTMCell object at 0x00000145F1C83C10> has the same name 'LSTMCell' as a built-in Keras object. Consider renaming <class 'keras.layers.recurrent.LSTMCell'> to avoid naming conflicts when loading with `tf.keras.models.load_model`. If renaming is not possible, pass the object in the `custom_objects` parameter of the load function.
...
[ Output 2 ]: 2. Error message removed
Epoch 10/10
100/100 [==============================] - 6s 63ms/step - loss: 0.3954 - val_loss: 0.5108
.................
...
I am new to tensorflow serving, here's the data I post:
data_0 = {'inputs/input_x': mfcc[0], "inputs/is_training":False, "inputs/keep_prob":1}
data = json.dumps({"signature_name":'serving_default', 'instances':[data_0]})
headers = {"Content-type": "application/json"}
json_response = requests.post(URL, data=data, headers=headers)
I got an error which comes from the is_training flag of batch normalization (and I think the same for the dropout rate):
{ "error": "The second input must be a scalar, but it has shape [1]\n\t [[{{node conv_layer2/conv2/batch_normalization/cond/Switch}}]]" }
Then I saw a similar issue and modified my code into
data_0 = {'inputs/input_x': mfcc[0], "inputs/is_training":[False], "inputs/keep_prob":[1]}
Then I got one more dimension:
{ "error": "The second input must be a scalar, but it has shape [1,1]\n\t [[{{node conv_layer2/conv2/batch_normalization/cond/Switch}}]]" }
And I tried to post without [] like :
data = json.dumps({"signature_name":'serving_default', 'instances':data_0})
I got :
"error": "JSON Value:{...} Excepting \'instances\' to be an list/array" }
Informations of my model:
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['inputs/input_x'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 34, 20)
name: inputs/input_x:0
inputs['inputs/is_training'] tensor_info:
dtype: DT_BOOL
shape: unknown_rank
name: inputs/is_training:0
inputs['inputs/keep_prob'] tensor_info:
dtype: DT_FLOAT
shape: unknown_rank
name: inputs/keep_prob:0
The given SavedModel SignatureDef contains the following output(s):
outputs['Softmax'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 2)
name: Softmax:0
Method name is: tensorflow/serving/predict
Need some help here, thanks!
For now I haven't got an answer and I re-trained my model without is_training flag, and that works.
So it's maybe that there is a problem about boolean values.
When using instances as input key, all inputs should have same 0-th dimension. Try inputs instead of instances. For example,
data = json.dumps({"signature_name":'serving_default', 'inputs':data_0})
I'm trying to save faster R-CNN hub model and use it with AI-platform gcloud ai-platform local predict. The error I'm getting is:
Failed to run the provided model: Exception during running the graph: [_Derived_] Table not initialized.\n\t [[{{node hub_input/index_to_string_1_Lookup}}]]\n\t [[StatefulPartitionedCall_1/StatefulPartitionedCall/model/keras_layer/StatefulPartitionedCall]] (Error code: 2)\n'
The code for saving the model:
model_url = "https://tfhub.dev/google/faster_rcnn/openimages_v4/inception_resnet_v2/1"
input = tf.keras.Input(shape=(), dtype=tf.string)
decoded = tf.keras.layers.Lambda(
lambda y: tf.map_fn(
lambda x: tf.image.resize(
tf.image.convert_image_dtype(
tf.image.decode_jpeg(x, channels=3), tf.float32), (416, 416)
),
tf.io.decode_base64(y), dtype=tf.float32)
)(input)
results = hub.KerasLayer(model_url, signature_outputs_as_dict=True)(decoded)
model = tf.keras.Model(inputs=input, outputs=results)
model.save("./saved_model", save_format="tf")
The model works when I load with tf.keras.models.load_model("./saved_model") and predict with it, but not with ai-platform local predict.
Command for ai-platform local predictions:
gcloud ai-platform local predict --model-dir ./saved_model --json-instances data.json --framework TENSORFLOW
Versions:
python 3.7.0
tensorflow==2.2.0
tensorflow-hub==0.7.0
Output of saved_model_cli:
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['__saved_model_init_op']:
The given SavedModel SignatureDef contains the following input(s):
The given SavedModel SignatureDef contains the following output(s):
outputs['__saved_model_init_op'] tensor_info:
dtype: DT_INVALID
shape: unknown_rank
name: NoOp
Method name is:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['image_bytes'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_image_bytes:0
The given SavedModel SignatureDef contains the following output(s):
outputs['keras_layer'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 4)
name: StatefulPartitionedCall_1:0
outputs['keras_layer_1'] tensor_info:
dtype: DT_STRING
shape: (-1, 1)
name: StatefulPartitionedCall_1:1
outputs['keras_layer_2'] tensor_info:
dtype: DT_INT64
shape: (-1, 1)
name: StatefulPartitionedCall_1:2
outputs['keras_layer_3'] tensor_info:
dtype: DT_STRING
shape: (-1, 1)
name: StatefulPartitionedCall_1:3
outputs['keras_layer_4'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 1)
name: StatefulPartitionedCall_1:4
Method name is: tensorflow/serving/predict
Any ideas on how to fix the error?
The problem is that your input is being interpreted as a scalar. Do:
input = tf.keras.Input(shape=(1,), dtype=tf.string)
I am trying to run a SavedModel using the C-API.
When it comes to running TF_SessionRun it always fails on various input nodes with the same error.
TF_SessionRun status: 3:Input to reshape is a tensor with 6 values, but the requested shape has 36
TF_SessionRun status: 3:Input to reshape is a tensor with 19 values, but the requested shape has 361
TF_SessionRun status: 3:Input to reshape is a tensor with 3111 values, but the requested shape has 9678321
...
As can be seen, the number of requested shape values is always the square of the expected input size. It's quite odd.
The model runs fine with the saved_model_cli command.
The inputs are all either scalar DT_STRING or DT_FLOATs, I'm not doing image recogition.
Here's the output of that command:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['f1'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: f1:0
inputs['f2'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: f2:0
inputs['f3'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: f3:0
inputs['f4'] tensor_info:
dtype: DT_FLOAT
shape: (-1)
name: f4:0
inputs['f5'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: f5:0
The given SavedModel SignatureDef contains the following output(s):
outputs['o1_probs'] tensor_info:
dtype: DT_DOUBLE
shape: (-1, 2)
name: output_probs:0
outputs['o1_values'] tensor_info:
dtype: DT_STRING
shape: (-1, 2)
name: output_labels:0
outputs['predicted_o1'] tensor_info:
dtype: DT_STRING
shape: (-1, 1)
name: output_class:0
Method name is: tensorflow/serving/predict
Any clues into what's going on are much appreciated. The saved_model.pb file is coming from AutoML, my code is merely querying that model. I don't change the graph.
It turns out that the issue was caused by me not using the TF_AllocateTensor function correctly.
The original code was like:
TF_Tensor* t = TF_AllocateTensor(TF_STRING, nullptr, 0, sz);
when it appears it should have been:
int64_t dims = 0;
TF_Tensor* t = TF_AllocateTensor(TF_STRING, &dims, 1, sz);