tensorflow serving restful request how to post a list Object - tensorflow-serving

Question

I made a tensorflow Model and i use tensorflow serving to deploy this model, but when i can build the restful request params the model need
curl -d '{"instances": [[1], [1], [1], [1], [1], [1], [1], [1], [1], [1]]}'
-X POST http://localhost:8501/v1/models/shipping_predict:predict
it call back
"error": "instances is a plain list, but expecting list of objects as multiple input tensors required as per tensorinfo_map"
this is my model
# prepare each input head
in_layers = list()
em_layers = list()
#
# customer
in_layer_customer = Input(shape=(1,))
em_layer_customer = Embedding(5000, 10)(in_layer_customer)
em_layer_customer = layers.Reshape([1 * 10])(em_layer_customer)
in_layers.append(in_layer_customer)
em_layers.append(em_layer_customer)
# salesman
in_layer_sale = Input(shape=(1,))
em_layer_sale = Embedding(500, 10)(in_layer_sale)
em_layer_sale = layers.Reshape([1 * 10])(em_layer_sale)
in_layers.append(in_layer_sale)
em_layers.append(em_layer_sale)
# business_type
in_layer_businessType = Input(shape=(1,))
em_layer_businessType = Embedding(100, 10)(in_layer_businessType)
em_layer_businessType = layers.Reshape([1 * 10])(em_layer_businessType)
in_layers.append(in_layer_businessType)
em_layers.append(em_layer_businessType)
# 20
in_layer_20 = Input(shape=(1,))
em_layer_20 = layers.Dense(16, activation='relu')(in_layer_20)
in_layers.append(in_layer_20)
em_layers.append(em_layer_20)
# 40
in_layer_40 = Input(shape=(1,))
em_layer_40 = layers.Dense(16, activation='relu')(in_layer_40)
in_layers.append(in_layer_40)
em_layers.append(em_layer_40)
# 45
in_layer_45 = Input(shape=(1,))
em_layer_45 = layers.Dense(16, activation='relu')(in_layer_45)
in_layers.append(in_layer_45)
em_layers.append(em_layer_45)
# other
in_layer_other = Input(shape=(1,))
em_layer_other = layers.Dense(16, activation='relu')(in_layer_other)
in_layers.append(in_layer_other)
em_layers.append(em_layer_other)
# dischargingPort
in_layer_dischargingPortName = Input(shape=(1,))
em_layer_dischargingPortName = Embedding(5000, 10)(in_layer_dischargingPortName)
em_layer_dischargingPortName = layers.Reshape([1 * 10])(em_layer_dischargingPortName)
in_layers.append(in_layer_dischargingPortName)
em_layers.append(em_layer_dischargingPortName)
# MBL Method
in_layer_mbl = Input(shape=(1,))
em_layer_mbl = layers.Dense(16, activation='relu')(in_layer_mbl)
in_layers.append(in_layer_mbl)
em_layers.append(em_layer_mbl)
# atdMouth
in_layer_atdMouth = Input(shape=(1,))
em_layer_atdMouth = Embedding(100, 10)(in_layer_atdMouth)
em_layer_atdMouth = layers.Reshape([1 * 10])(em_layer_atdMouth)
in_layers.append(in_layer_atdMouth)
em_layers.append(em_layer_atdMouth)
merge = Concatenate()(em_layers)
dense = Dense(32, activation='relu')(merge)
output = Dense(1)(dense)
model = Model(inputs=in_layers, outputs=output)

Answer 1

instances is a plain list, but expecting list of objects as multiple
input tensors
It sounds like your model, for whatever reason, is expecting named tensors. This is not something I've worked with before but there appears to be another way of sending requests to your your model.
curl -X POST -i 'http://192.168.1.16:8501/v1/models/export:predict' --data '
{
"signature_name": "serving_default",
"inputs": [
{
"tokens_0" :["text text text text text text text text text text"],
"length_0": [1],
"tokens_1": ["01 01 01 01 01 01 01 01 01 01"],
"length_1": [1],
"tokens_2": ["4 4 4 1 1 4 4 4 4 4"],
"length_2": [1]
}
]
}'
I've just copied this example from here (credit to #ishaan-sharma).
Your model is non-trivial so I won't try and create the exact request for you. If you're unsure about the tensor names etc, you can check out the expected shape using the saved_model_cli:
saved_model_cli show [-h] --dir DIR [--all]

Answer 2

A bit late for this answer, but I hope it helps to someone. I have saved a Keras model where it excepts multiple inputs. The model expects two inputs: text and a vector of integers
import tensorflow as tf
from tensorflow.keras.models import Model
model = Model(inputs=[text_input, input2], outputs=out)
### some code for training and data preparation
export_path = "serving/5/"
tf.saved_model.save(model, export_path)
This finally saved the model to be served using Tensorflow serving. A brief summary of the model is as follows:
$ saved_model_cli show --dir ./serving/5 --tag_set serve --signature_def serving_default
The given SavedModel SignatureDef contains the following input(s):
inputs['input_1'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 23)
name: serving_default_input_1:0
inputs['text'] tensor_info:
dtype: DT_STRING
shape: (-1)
name: serving_default_text:0
The given SavedModel SignatureDef contains the following output(s):
outputs['dense_2'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 1)
name: StatefulPartitionedCall:0
Method name is: tensorflow/serving/predict
Finally to post my request to the serving container, I did:
import requests
import json
params = json.dumps(
{
"instances": [
{
"text": [features[0]],
"input_1": features[1:]
}
]
}
)
response = requests.post(
"http://localhost:8038/v1/models/serving:predict",
headers={"Content-Type": "application/json"},
data=params
)
The main issue was that I had to wrap all my inputs as a JSON in instances with their tensor names (which I got from saved_model_cli).
Important thing to note is that the values needs to be given as a list. features[0] is a text so I wrapped it up in a list while features[1:] itself is a vector which does fine.