How to send image as JSON to tensorflow_model_server? - tensorflow

Question

I have trained resnet with an additional layer to predict a dog vs cat model and serving it from localhost using
tensorflow_model_server --port=8500 --rest_api_port=8501 \
--model_name=resnet_model \
--model_base_path=/home/pc3/deep_learning/models/resnet
$ saved_model_cli show --all --dir resnet/1 shows this signature:
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['__saved_model_init_op']:
The given SavedModel SignatureDef contains the following input(s):
The given SavedModel SignatureDef contains the following output(s):
outputs['__saved_model_init_op'] tensor_info:
dtype: DT_INVALID
shape: unknown_rank
name: NoOp
Method name is:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['keras_layer_1_input'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 256, 256, 3)
name: serving_default_keras_layer_1_input:0
The given SavedModel SignatureDef contains the following output(s):
outputs['dense_2'] tensor_info:
dtype: DT_FLOAT
shape: (-1, 2)
name: StatefulPartitionedCall:0
Method name is: tensorflow/serving/predict
Concrete Functions:
Function Name: '__call__'
Option #1
Callable with:
Argument #1
inputs: TensorSpec(shape=(None, 256, 256, 3), dtype=tf.float32, name='inputs')
Argument #2
DType: bool
Value: False
Argument #3
DType: NoneType
Value: None
Option #2
Callable with:
Argument #1
keras_layer_1_input: TensorSpec(shape=(None, 256, 256, 3), dtype=tf.float32, name='keras_layer_1_input')
Argument #2
DType: bool
Value: True
Argument #3
DType: NoneType
Value: None
Option #3
Callable with:
Argument #1
inputs: TensorSpec(shape=(None, 256, 256, 3), dtype=tf.float32, name='inputs')
Argument #2
DType: bool
Value: True
Argument #3
DType: NoneType
Value: None
Option #4
Callable with:
Argument #1
keras_layer_1_input: TensorSpec(shape=(None, 256, 256, 3), dtype=tf.float32, name='keras_layer_1_input')
Argument #2
DType: bool
Value: False
Argument #3
DType: NoneType
Value: None
Function Name: '_default_save_signature'
Option #1
Callable with:
Argument #1
keras_layer_1_input: TensorSpec(shape=(None, 256, 256, 3), dtype=tf.float32, name='keras_layer_1_input')
Function Name: 'call_and_return_all_conditional_losses'
Option #1
Callable with:
Argument #1
keras_layer_1_input: TensorSpec(shape=(None, 256, 256, 3), dtype=tf.float32, name='keras_layer_1_input')
Argument #2
DType: bool
Value: False
Argument #3
DType: NoneType
Value: None
Option #2
Callable with:
Argument #1
keras_layer_1_input: TensorSpec(shape=(None, 256, 256, 3), dtype=tf.float32, name='keras_layer_1_input')
Argument #2
DType: bool
Value: True
Argument #3
DType: NoneType
Value: None
Option #3
Callable with:
Argument #1
inputs: TensorSpec(shape=(None, 256, 256, 3), dtype=tf.float32, name='inputs')
Argument #2
DType: bool
Value: False
Argument #3
DType: NoneType
Value: None
Option #4
Callable with:
Argument #1
inputs: TensorSpec(shape=(None, 256, 256, 3), dtype=tf.float32, name='inputs')
Argument #2
DType: bool
Value: True
Argument #3
DType: NoneType
Value: None
When post request with JSON to http://localhost:8501/models/resnet_model/1 with this body:
{
"signature_name": "serving_default",
"instances": [{"b64": "iVBORw0KGgoAAAA..."}]
}
where iVBORw0KGgoAAAA... is base64 encoded value of a PNG image which I've already resized to 256x256, I get this error in response:
{
"error": "Failed to process element: 0 of 'instances' list. Error: INVALID_ARGUMENT: JSON Value: {\n \"b64\": \"iVBORw0KGgoAAAA...\"\n} Type: Object is not of expected type: float"
}
and when I try "signature_name": "__saved_model_init_op", instead, I get
{
"error": "Failed to get input map for signature: __saved_model_init_op"
}
After tons of googling, I could not find any tutorial or code example about this particular scenario. So I'm left clueless what is the correct way of post an image to this model?

Answer 1

Take a look at this resnet_client.
According to the signature_def of your model, it does not accept input as base64, so MODEL_ACCEPT_JPG is False for your case. follow the link to see how data is sent to the server.