I'm trying to send a job up to my object detection model on CloudML to get predictions. I'm following the guide at https://cloud.google.com/ml-engine/docs/online-predict but I'm getting an error when submitting the request:
RuntimeError: Prediction failed: Error processing input: Expected uint8, got '\xf6>\x00\x01\x04\xa4d\x94...(more bytes)...\x00\x10\x10\x10\x04\x80\xd9' of type 'str' instead.
This is my code:
img = base64.b64encode(open("file.jpg", "rb").read()).decode('utf-8')
json = {"b64": img}
result = predict_json(project, model, json, "v1")
My fault, I forgot to add --input_type encoded_image_string_tensor when I exported the graph.
Related
I followed this tutorial and had the model fine-tuned.
the model-saving part of serving model is like this:
saved_model_path = 'dogs_and_cats_yamnet/yamnet-model/00000001'
input_segment = tf.keras.layers.Input(shape=(), dtype=tf.float32, name='audio')
embedding_extraction_layer = hub.KerasLayer(yamnet_model_handle,
trainable=False, name='yamnet')
_, embeddings_output, _ = embedding_extraction_layer(input_segment)
serving_outputs = my_model(embeddings_output)
serving_outputs = ReduceMeanLayer(axis=0, name='classifier')(serving_outputs)
serving_model = tf.keras.Model(input_segment, serving_outputs)
serving_model.save(saved_model_path, include_optimizer=False)
Then followed this page, uploading the model to S3 and deploying the model.
!tar -C "$PWD" -czf dogs_and_cats_yamnet.tar.gz dogs_and_cats_yamnet/
model_data = Session().upload_data(path="dogs_and_cats_yamnet.tar.gz", key_prefix="model")
model = TensorFlowModel(model_data=model_data, role=sagemaker_role, framework_version="2.3")
predictor = model.deploy(initial_instance_count=1, instance_type="ml.c5.xlarge")
Deployment seems successful, but when I try to do inference,
waveform = np.zeros((3*48000), dtype=np.float32)
result = predictor.predict(waveform)
the following error occurs.
ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (400) from primary with message "{
"error": "The first dimension of paddings must be the rank of inputs[1,2] [1,144000]\n\t [[{{node yamnet_frames/tf_op_layer_Pad/Pad}}]]"
I have no idea why this happens. I am struggling with it for hours and coming up with no clue.
YAMNet works fine when I pulled the model from tf hub directly and take inference with it.
This is kind of a minor question I guess, but I would appreciate any helpful answers.
Thank you in advance.
trying to get a custom tensorFlowJS model to work in JS but getting error with the shape, kindly note that the model is already trained and converted to model.json but getting below error
Error: Error: Error in concat4D: Shape of tensors[1] (1,30,40,256) does not match the shape of the rest (1,15,20,832) along the non-concatenated axis 1.
below is the code:
const RGB = await imageToRgbaMatrix(imageUrl);
const model = await tf.loadLayersModel(modelUrl);
const ImageData = await tf.tensor([RGB]);
const predictions = model.predict(ImageData);
in brief i am trying to implement the tfjs model (model.json) with binary file with it gives:
Error in concat4D: Shape of tensors[1] (1,30,40,256) does not match the shape of the rest (1,15,20,832).
I'm trying to use tf.data.experimental.TFRecordWriter to save dataset on Google cloud bucket using TPU. The code from the example in documentation works:
dataset = tf.data.Dataset.range(3)
dataset = dataset.map(tf.io.serialize_tensor)
writer = tf.data.experimental.TFRecordWriter("gs://oleg-zyablov/test.tfrec")
writer.write(dataset)
But I have dataset of tuples (string, int64), where first is jpg-encoded image and second is label. When I pass it to writer.write() method, it says: 'tuple' object has no attribute 'is_compatible_with'.
I guess I have to pack image and label into tf.train.Example to make it work. I use the following code:
def serialize(image, class_idx):
tfrecord = tf.train.Example(features = tf.train.Features(feature = {
'image': tf.train.Feature(bytes_list = tf.train.BytesList(value = [image.numpy()])),
'class': tf.train.Feature(int64_list = tf.train.Int64List(value = [class_idx.numpy()]))
}))
return tfrecord.SerializeToString()
#saving_pipeline is a dataset of (string, int64) tuples
saving_pipeline_serialized = saving_pipeline.map(serialize)
writer = tf.data.experimental.TFRecordWriter("gs://oleg-zyablov/car-classification/train_tfrecords/test.tfrecord")
writer.write(saving_pipeline_serialized)
But I get the following error:
'Tensor' object has no attribute 'numpy'
Although I didn't turn off eager mode and this code tf.constant([], dtype = float).numpy() works. Maybe TPU works not in eager mode? Ok, I changed .numpy() to .eval() in the code above. Then I get the foloowing error:
Cannot evaluate tensor using `eval()`: No default session is registered. Use `with sess.as_default()` or pass an explicit session to `eval(session=sess)`
What session does TPU use and how do I specify it? When I run the code below:
with tf.compat.v1.Session():
saving_pipeline_serialized = saving_pipeline.map(serialize)
I get an error:
Cannot use the default session to evaluate tensor: the tensor's graph is different from the session's graph. Pass an explicit session to `eval(session=sess)`.
But I don't know how to get the current graph and pass it to tf.compat.v1.Session(). When I go another way and type:
image.eval(session = tf.compat.v1.get_default_session())
It says:
Cannot evaluate tensor using `eval()`: No default session is registered. Use `with sess.as_default()` or pass an explicit session to `eval(session=sess)`
Is it possible to use .eval() on TPU? Or how do I perform my task another way?
I have a model saved in SavedModel format (.pb). After serving the model without problems i try to make a prediction via tensorflow serving. TF Serving requires me to input the data via a list, otherwise the answer i receive is TypeError: Object of type 'ndarray' is not JSON serializable
. But when i input a list the response is an error
The input is
value = [1, 2, 3, 4, 5]
body = {"signature_name": "serving_default",
"instances": [[values]]}
res = requests.post(url=url, data=json.dumps(body))
and the answer { "error": "In[0] is not a matrix. Instead it has shape [1,1,5]\n\t [[{{node sequential/dense/Relu}}]]" }
I know the model works, the input without using tensorflow serving is
value = np.array([1,2,3,4,5])
model.predict([[value]])
So the problem is how can use tensorflow serving if it requires to use a list as input but the model requires a np.array as input.
I suppose you should do it in this way
value = <ndarray>
data = value.tolist()
body = {
"signature_name": "serving_default",
"instances": data}
The XGboost model was trained on AWS sagemaker and deployed successfully but I keep getting the following error: ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (415) from model with message "could not convert string to float: ".
Any thoughts?
Test data is as following:
size mean
269 5600.0 17.499633
103 1754.0 9.270272
160 4968.0 14.080601
40 4.0 17.500000
266 36308.0 11.421855
test_data_array = test_data.drop(['mean'], axis=1).as_matrix()
test_data_array = np.array([np.float32(x) for x in test_data_array])
xgb_predictor.content_type = 'text/csv'
xgb_predictor.serializer = csv_serializer
def predict(data, rows=32):
split_array = np.array_split(data, int(data.shape[0] / float(rows) + 1))
#print(split_array)
predictions = ''
for array in split_array:
print(array[0], type(array[0]))
predictions = ','.join([predictions, xgb_predictor.predict(array[0]).decode('utf-8')])
return np.fromstring(predictions[1:], sep=',')
predictions = predict(test_data_array)
SageMaker XGBoost cannot handle csv input with header. Please make sure the string header was removed before sending the data to the endpoint.
Also for csv prediction, SageMaker XGBoost assumes that CSV input does not have the label column. So please remove the label column in the input data as well.