Hello i have followed all the steps to make an inference and successfully run it on the model in this link : https://pjreddie.com/media/files/yolov3.weights
but when i tried it on a model i trained with darknet i get this error :
[ INFO ] Creating Inference Engine...
[ INFO ] Loading network files:
newyolo.xml
newyolo.bin
[ INFO ] Preparing inputs
[ INFO ] Loading model to the plugin
[ INFO ] Starting inference...
To close the application, press 'CTRL+C' here or switch to the output window and press ESC key
To switch between sync/async modes, press TAB key in the output window
yolo_original.py:280: DeprecationWarning: shape property of IENetLayer is deprecated. Please use shape property of DataPtr instead objects returned by in_data or out_data property to access shape of input or output data on corresponding ports
out_blob = out_blob.reshape(net.layers[net.layers[layer_name].parents[0]].shape)
[ INFO ] Layer detector/yolo-v3/Conv_14/BiasAdd/YoloRegion parameters:
[ INFO ] classes : 10
[ INFO ] num : 3
[ INFO ] coords : 4
[ INFO ] anchors : [55.0, 56.0, 42.0, 87.0, 68.0, 81.0]
Traceback (most recent call last):
File "yolo_original.py", line 363, in <module>
sys.exit(main() or 0)
File "yolo_original.py", line 286, in main
args.prob_threshold)
File "yolo_original.py", line 153, in parse_yolo_region
h_scale=orig_im_h, w_scale=orig_im_w))
File "yolo_original.py", line 99, in scale_bbox
xmin = int((x - w / 2) * w_scale)
ValueError: cannot convert float NaN to integer
knowing that i have provided the right shape and changed yolo_v3.json to match my model
here is the content of my yolo_v3.json:
[
{
"id": "TFYOLOV3",
"match_kind": "general",
"custom_attributes": {
"classes": 10,
"anchors": [18,22,31,33,33,50,55, 56,42,87,68,81,111,98,73,158,156,202],
"coords": 4,
"num": 9,
"masks":[[6, 7, 8], [3, 4, 5], [0, 1, 2]],
"entry_points": ["detector/yolo-v3/Reshape", "detector/yolo-v3/Reshape_4", "detector/yolo-v3/Reshape_8"]
}
}
]
i have tried multiple things to debug this like not providing the jsonfile ....etc
ps : yolo_original.py is the same demo that comes with openvino just renamed,
i'm using openvino version 2020.1
transforming NaN to float or skipping values with Nan didn't slove the problem.
Related
I made a model in python and exported it to tensorflow js.
The model is classifying facial images to emotions, and there should be 7 categories.
I made a tensor from the image and used predict like so:
const prediction = model.predict(imageTensor);
my prediction is :
Tensor {
"dataId": Object {},
"dtype": "float32",
"id": 415,
"isDisposedInternal": false,
"kept": false,
"rankType": "2",
"scopeId": 197,
"shape": Array [
1,
7,
],
"size": 7,
"strides": Array [
7,
],
}
how do I extract the result from here?
Prediction is a tensor. If you want to access the data, you need to use prediction.dataSync()
found the answer, I was using typescript and it was needed:
const prediction = (model.predict(
imageTensor
) as tf.Tensor).dataSync();
My understanding is that I should be able to grab a TensorFlow model from Google's AI Hub, deploy it to TensorFlow Serving and use it to make predictions by POSTing images via REST requests using curl.
I could not find any bbox predictors on AI Hub at this time but I did find one on the TensorFlow model zoo:
http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz
I have the model deployed to TensorFlow serving, but the documentation is unclear with respect to exactly what should be included in the JSON of the REST request.
My understanding is that
The SignatureDefinition of the model determines what the JSON should look like
I should base64 encode the images
I was able to get the signature definition of the model like so:
>python tensorflow/tensorflow/python/tools/saved_model_cli.py show --dir /Users/alexryan/alpine/git/tfserving-tutorial3/model-volume/models/bbox/1/ --all
MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:
signature_def['serving_default']:
The given SavedModel SignatureDef contains the following input(s):
inputs['in'] tensor_info:
dtype: DT_UINT8
shape: (-1, -1, -1, 3)
name: image_tensor:0
The given SavedModel SignatureDef contains the following output(s):
outputs['out'] tensor_info:
dtype: DT_FLOAT
shape: unknown_rank
name: detection_boxes:0
Method name is: tensorflow/serving/predict
I think the shape info here is telling me that the model can handle images of any dimensions?
The input layer looks like this in Tensorboard:
But how do I convert this SignatureDefinition to a valid JSON request?
I'm assuming that I'm supposed to use the predict API ...
and Google's doc says ...
URL
POST
http://host:port/v1/models/${MODEL_NAME}[/versions/${MODEL_VERSION}]:predict
/versions/${MODEL_VERSION} is optional. If omitted the latest version
is used.
Request format
The request body for predict API must be JSON object
formatted as follows:
{
// (Optional) Serving signature to use.
// If unspecifed default serving signature is used.
"signature_name": <string>,
// Input Tensors in row ("instances") or columnar ("inputs") format.
// A request can have either of them but NOT both.
"instances": <value>|<(nested)list>|<list-of-objects>
"inputs": <value>|<(nested)list>|<object>
}
Encoding binary values JSON uses UTF-8 encoding. If you have input
feature or tensor values that need to be binary (like image bytes),
you must Base64 encode the data and encapsulate it in a JSON object
having b64 as the key as follows:
{ "b64": "base64 encoded string" }
You can specify this object as a value for an input feature or tensor.
The same format is used to encode output response as well.
A classification request with image (binary data) and caption features
is shown below:
{ "signature_name": "classify_objects", "examples": [
{
"image": { "b64": "aW1hZ2UgYnl0ZXM=" },
"caption": "seaside"
},
{
"image": { "b64": "YXdlc29tZSBpbWFnZSBieXRlcw==" },
"caption": "mountains"
} ] }
Uncertainties include:
should I use "instances" in my JSON
should I base64 encode a JPG or PNG or something else?
Should the image be of a particular
width and height?
In Serving Image-Based Deep Learning Models with TensorFlow-Serving’s RESTful API this format is suggested:
{
"instances": [
{"b64": "iVBORw"},
{"b64": "pT4rmN"},
{"b64": "w0KGg2"}
]
}
I used this image:
https://tensorflow.org/images/blogs/serving/cat.jpg
and base64 encoded it like so:
# Download the image
dl_request = requests.get(IMAGE_URL, stream=True)
dl_request.raise_for_status()
# Compose a JSON Predict request (send JPEG image in base64).
jpeg_bytes = base64.b64encode(dl_request.content).decode('utf-8')
predict_request = '{"instances" : [{"b64": "%s"}]}' % jpeg_bytes
But when I use curl to POST the base64 encoded image like so:
{"instances" : [{"b64": "/9j/4AAQSkZJRgABAQAASABIAAD/4QBYRXhpZgAATU0AKgAA
...
KACiiigAooooAKKKKACiiigAooooA//Z"}]}
I get a response like this:
>./test_local_tfs.sh
HEADER=|Content-Type:application/json;charset=UTF-8|
URL=|http://127.0.0.1:8501/v1/models/saved_model/versions/1:predict|
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8501 (#0)
> POST /v1/models/saved_model/versions/1:predict HTTP/1.1
> Host: 127.0.0.1:8501
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Type:application/json;charset=UTF-8
> Content-Length: 85033
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 400 Bad Request
< Content-Type: application/json
< Date: Tue, 17 Sep 2019 10:47:18 GMT
< Content-Length: 85175
<
{ "error": "Failed to process element: 0 of \'instances\' list. Error: Invalid argument: JSON Value: {\n \"b64\": \"/9j/4AAQSkZJRgABAQAAS
...
ooooA//Z\"\n} Type: Object is not of expected type: uint8" }
I've tried converting a local version of the same file to base64 like so (confirming that the dtype is uint8) ...
img = cv2.imread('cat.jpg')
print('dtype: ' + str(img.dtype))
_, buf = cv2.imencode('.jpg', img)
jpeg_bytes = base64.b64encode(buf).decode('utf-8')
predict_request = '{"instances" : [{"b64": "%s"}]}' % jpeg_bytes
But posting this JSON generates the same error.
However, when the json is formated like so ...
{'instances': [[[[112, 71, 48], [104, 63, 40], [107, 70, 20], [108, 72, 21], [109, 77, 0], [106, 75, 0], [92, 66, 0], [106, 80, 0], [101, 80, 0], [98, 77, 0], [100, 75, 0], [104, 80, 0], [114, 88, 17], [94, 68, 0], [85, 54, 0], [103, 72, 11], [93, 62, 0], [120, 89, 25], [131, 101, 37], [125, 95, 31], [119, 91, 27], [121, 93, 29], [133, 105, 40], [119, 91, 27], [119, 96, 56], [120, 97, 57], [119, 96, 53], [102, 78, 36], [132, 103, 44], [117, 88, 28], [125, 89, 4], [128, 93, 8], [133, 94, 0], [126, 87, 0], [110, 74, 0], [123, 87, 2], [120, 92, 30], [124, 95, 33], [114, 90, 32],
...
, [43, 24, 33], [30, 17, 36], [24, 11, 30], [29, 20, 38], [37, 28, 46]]]]}
... it works.
The problem is this json file is >11 MB in size.
How do I make the base64 encoded version of the json work?
UPDATE: It seems that we have to edit the pretrained model to accept base64 images at the input layer
This article describes how to edit the model ...
Medium: Serving Image-Based Deep Learning Models with TensorFlow-Serving’s RESTful API
... unfortunately, it assumes that we have access to the code which generated the model.
user260826's solution provides a work-around using an estimator but it assumes the model is a keras model. Not true in this case.
Is there a generic method to make a model ready for TensorFlow Serving REST interface with a base64 encoded image that works with any of the TensorFlow model formats?
The first step is to export the trained model in the appropriate format. Use export_inference_graph.py like this
python export_inference_graph \
--input_type encoded_image_string_tensor \
--pipeline_config_path path/to/ssd_inception_v2.config \
--trained_checkpoint_prefix path/to/model.ckpt \
--output_directory path/to/exported_model_directory
in the above code snippet, it is important to specify
--input_type encoded_image_string_tensor
after exporting the model, run the tensorflow server as usual with the newly exported model.
The inference code will look like this:
from __future__ import print_function
import base64
import requests
SERVER_URL = 'http://localhost:8501/v1/models/vedNet:predict'
IMAGE_URL = 'test_images/19_inp.jpg'
def main():
with open(IMAGE_URL, "rb") as image_file:
jpeg_bytes = base64.b64encode(image_file.read()).decode('utf-8')
predict_request = '{"instances" : [{"b64": "%s"}]}' % jpeg_bytes
response = requests.post(SERVER_URL, predict_request)
response.raise_for_status()
prediction = response.json()['predictions'][0]
if __name__ == '__main__':
main()
As you mentioned JSON is a very inefficient approach, as payload normally exceeds original filesize, you need to convert the model to be able to process the image bytes written to a string using Base64 encoding:
{"b64": base64_encoded_string}
This new conversion will reduce the prediction time and bandwidth utilization used to transfer image from prediction client to your infrastructure.
I recently used a Transfer Learning model with TF Hub and Keras which was using a JSON as input, as you mentioned this is not optimal for prediction.
I used the following function to overwrite it:
Using the following code we add a new serving function which will be able to process Base64 encoded images.
Using TF estimator model:
h5_model_path = os.path.join('models/h5/best_model.h5')
tf_model_path = os.path.join('models/tf')
estimator = keras.estimator.model_to_estimator(
keras_model_path=h5_model_path,
model_dir=tf_model_path)
def image_preprocessing(image):
"""
This implements the standard preprocessing that needs to be applied to the
image tensors before passing them to the model. This is used for all input
types.
"""
image = tf.expand_dims(image, 0)
image = tf.image.resize_bilinear(image, [HEIGHT, WIDTH], align_corners=False)
image = tf.squeeze(image, axis=[0])
image = tf.cast(image, dtype=tf.uint8)
return image
def serving_input_receiver_fn():
def prepare_image(image_str_tensor):
image = tf.image.decode_jpeg(image_str_tensor, channels=CHANNELS)
return image_preprocessing(image)
input_ph = tf.placeholder(tf.string, shape=[None])
images_tensor = tf.map_fn(
prepare_image, input_ph, back_prop=False, dtype=tf.uint8)
images_tensor = tf.image.convert_image_dtype(images_tensor, dtype=tf.float32)
return tf.estimator.export.ServingInputReceiver(
{'input': images_tensor},
{'image_bytes': input_ph})
export_path = os.path.join('/tmp/models/json_b64', version)
if os.path.exists(export_path): # clean up old exports with this version
shutil.rmtree(export_path)
estimator.export_savedmodel(
export_path,
serving_input_receiver_fn=serving_input_receiver_fn)
A good example here
I have been struggling with the same problem. Finally I could make it work. I just had to add a new signature to the model:
import tensorflow as tf
model = tf.saved_model.load("/path/to/the/original/model")
# This is the current signature, that only accepts image tensors as input
signature = model.signatures["default"]
#tf.function()
def my_predict(image_b64):
# Model doesn't support batch!!
img_dec = tf.image.decode_png(image_b64[0], channels=3)
img_tensor = tf.image.convert_image_dtype(img_dec, tf.float32)[tf.newaxis, ...]
prediction = signature(img_tensor)
return prediction
# Create new signature, to read b64 images
new_signature = my_predict.get_concrete_function(
image_b64=tf.TensorSpec([None], dtype=tf.string, name="image_b64")
)
tf.saved_model.save(
model,
export_dir="/path/to/the/saved/model",
signatures=new_signature
)
Finally, after serving I can make predictions passing an input like this:
{
"instances": [
{
"b64": "youBase64ImageHere"
}
]
}
I want to use Lucid to analyze the feature extraction of a detection model I trained using the tensorflow Object Detection API on my own dataset. The model used is one from the Tensorflow Object Detection Zoo, namely faster_rcnn_resnet101.
I followed the Lucid tutorial to import my own model and saved a frozen graph of the model with the node /all_class_predictions_with_background as output_node.
I'm having trouble finding the input node of the graph to make Lucid run on it.
Furthermore I don't really think I have the right approach. Maybe I should first extract all the classification part of the detection model and freeze a new graph with only this part before going to Lucid.
Or maybe I should just import a resnet_101 classification model and copy/paste the correct weights from the detection model on it?
But I don't really know how to do those kind of things.
Can someone help me? I really want to try running Lucid on my detection network.
Yes, you should export an inference (frozen) graph to work with in Lucid.
I use the following script to export a graph from the training checkpoint files.
Useful information about the nodes in the exported file is logged to the console.
training_model="ssd_mnet_v2_ppn_512x288.config"
model_signature="eb_13_v09_ppmn2_13_256_adam_512x288_tf_1.14_200k"
# the specific checkpoint to export from
checkpoint_path="/TRAIN/models/model/train/model.ckpt-200000"
# directory to export into
output_path="/XYZ/graphs/${model_signature}"
# ensure these graph nodes are exported, and everything in between
additional_output_tensor_names="Preprocessor/sub,concat_1"
#
python export_inference_graph.py \
--input_type=image_tensor \
--pipeline_config_path /TRAIN/models/model/$training_model \
--trained_checkpoint_prefix=$checkpoint_path \
--output_directory=$output_path \
--additional_output_tensor_names=$additional_output_tensor_names
I found it convenient to make my own Lucid Model class, after reviewing the examples in the Lucid model zoo.
You have to examine your graph carefully as you need to specify the input node, and provide a list of layers that Lucid can work with.
from lucid.modelzoo.vision_base import Model, _layers_from_list_of_dicts
# the input node "Preprocessor/sub" is appropriate for image injection
class SSD_Mnet2_PPN( Model ):
def __init__(self, image_shape=None, graph_path=None, labels_path=None ):
self.model_path = graph_path
self.labels_path = labels_path
self.image_shape = image_shape
self.image_value_range = (-1, 1)
self.input_name = "Preprocessor/sub"
super().__init__()
# a hand-crafted list of layers - by inspection of the graph
SSD_Mnet2_PPN.layers = _layers_from_list_of_dicts(SSD_Mnet2_PPN, [
{ 'id': 0, 'tags': ['conv'], 'name': 'FeatureExtractor/MobilenetV2/expanded_conv_2/add', 'depth': 24, 'shape': [ 1, 72, 128, 24 ], 'transform_id': 2 },
{ 'id': 2, 'tags': ['conv'], 'name': 'FeatureExtractor/MobilenetV2/expanded_conv_5/add', 'depth': 32, 'shape': [ 1, 36, 64, 32 ], 'transform_id': 2 },
{ 'id': 5, 'tags': ['conv'], 'name': 'FeatureExtractor/MobilenetV2/expanded_conv_9/add', 'depth': 64, 'shape': [ 1, 18, 32, 64 ], 'transform_id': 2 },
{ 'id': 7, 'tags': ['conv'], 'name': 'FeatureExtractor/MobilenetV2/expanded_conv_12/add', 'depth': 96, 'shape': [ 1, 18, 32, 96 ], 'transform_id': 2 },
{ 'id': 9, 'tags': ['conv'], 'name': 'FeatureExtractor/MobilenetV2/expanded_conv_15/add', 'depth': 160, 'shape': [ 1, 9, 16, 160 ], 'transform_id': 2 },
{ 'id': 11, 'tags': ['concat'], 'name': 'concat_1', 'depth': 13, 'shape': [ 1, 1212, 13 ], 'transform_id': 4 },
])
def model_for_version( version=None, path=None ):
if "320x180" in version:
return SSD_Mnet2_PPN( graph_path=path, image_shape=[ 320, 180, 3 ] )
if "480x270" in version:
return SSD_Mnet2_PPN( graph_path=path, image_shape=[ 480, 270, 3 ] )
if "512x288" in version:
return SSD_Mnet2_PPN( graph_path=path, image_shape=[ 512, 288, 3 ] )
if "720x405" in version:
return SSD_Mnet2_PPN( graph_path=path, image_shape=[ 720, 405, 3 ] )
raise ValueError( "No model for graph_version: {}".format( version ) )
Then you can write code as follows:
from lucid.optvis import render
model = model_for_version(
version = "eb_13_v09_ppmn2_13_256_adam_512x288_tf_1.14",
path = "/XYZ/graphs/eb_13_v09_ppmn2_13_256_adam_512x288_tf_1.14_200k/frozen_inference_graph.pb"
)
model.load_graphdef()
_ = render.render_vis( model, "FeatureExtractor/MobilenetV2/expanded_conv_15/add:17", thresholds=( 32, 256, 1024 ) )
Inevitably, one has to experiment quite a bit.
I trained model(cnn) for text classification.
so i want to test using keyboard input(what i typed)
like below
while True:
input=raw_input("input text ")
input = funciontForencoding(input)
print (sess.run(predict, {input_x:input}))
but logic die after show result value(prediction) one or two times
error stake is below,
model take two sentence list and return result of each sentence
please input to test : blah blah
please input to test :
[[-1.73195827 1.9319526 ]
[-1.73195827 1.9319526 ]]
please input to test :
please input to test :
Traceback (most recent call last):
File "test_eval.py", line 213, in
scores = sess.run(scores, {input_x: tdata, dropout_keep_prob:1.0})
File "/home/tech/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 340, in run
run_metadata_ptr)
File "/home/tech/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 523, in _run
processed_fetches = self._process_fetches(fetches)
File "/home/tech/anaconda2/envs/tensorflow/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 493, in _process_fetches
% (subfetch, fetch, type(subfetch), str(e)))
TypeError: Fetch argument array([[-1.73195827, 1.9319526 ],
[-1.73195827, 1.9319526 ]], dtype=float32) of array([[-1.73195827, 1.9319526 ],
[-1.73195827, 1.9319526 ]], dtype=float32) has invalid type , must be a string or Tensor. (Can not convert a ndarray into a Tensor or Operation.)
is there any caution or instruction should i follow? like clear session after each run operation
I'm trying to use a combination of geopandas, Pandas and Folium to create a polygon map that I can embed incorporate into a web page.
For some reason, it's not displaying.
The steps I've taken:
Grabbed a .shp from the UK's OS for Parliamentary boundaries.
I've then used geopandas to change the projection to epsg=4326 and then exported as GeoJSON which takes the following format:
{ "type": "Feature", "properties": { "PCON13CD": "E14000532", "PCON13CDO": "A03", "PCON13NM": "Altrincham and Sale West" }, "geometry": { "type": "Polygon", "coordinates": [ [ [ -2.313999519326579, 53.357408280545918 ], [ -2.313941776174758, 53.358341455420039 ], [ -2.31519699483377, 53.359035664493433 ], [ -2.317953152796459, 53.359102954309151 ], [ -2.319855973429864, 53.358581917200119 ],... ] ] ] } },...
Then what I'd like to do is mesh this with a dataframe of constituencies in the following format, dty:
constituency count
0 Burton 667
1 Cannock Chase 595
2 Cheltenham 22
3 Cheshire East 2
4 Congleton 1
5 Derbyshire Dales 1
6 East Staffordshire 4
import folium
mapf = folium.Map(width=700, height=370, tiles = "Stamen Toner", zoom_start=8, location= ["53.0219392","-2.1597434"])
mapf.geo_json(geo_path="geo_json_shape2.json",
data_out="data.json",
data=dty,
columns=["constituency","count"],
key_on="feature.properties.PCON13NM.geometry.type.Polygon",
fill_color='PuRd',
fill_opacity=0.7,
line_opacity=0.2,
reset="True")
The output from mapf looks like:
mapf.json_data
{'../../Crime_data/staffs_data92.json': [{'Burton': 667,
'Cannock Chase': 595,
'Cheltenham': 22,
'Cheshire East': 2,
'Congleton': 1,
'Derbyshire Dales': 1,
'East Staffordshire': 4,
'Lichfield': 438,
'Newcastle-under-Lyme': 543,
'North Warwickshire': 1,
'Shropshire': 17,
'South Staffordshire': 358,
'Stafford': 623,
'Staffordshire Moorlands': 359,
'Stoke-on-Trent Central': 1053,
'Stoke-on-Trent North': 921,
'Stoke-on-Trent South': 766,
'Stone': 270,
'Tamworth': 600,
'Walsall': 1}]}
Although the mapf.create_map() function successfully creates a map, the polygons don't render.
What debugging steps should I take?
#elksie5000, Try mplleaflet it is extremely straightforward.
pip install mplleaflet
in Jupyter/Ipython notebook:
import mplleaflet
ax = geopandas_df.plot(column='variable_to_plot', scheme='QUANTILES', k=9, colormap='YlOrRd')
mplleaflet.show(fig=ax.figure)