We are experimenting to have our own MLIR stack to import TFL models and compile them for a specific accelerator. We are also building our own runtime/simulator to run these imported models. Our current way of working is that we freeze the TF.keras model, convert to TFL and then use flatbuffer_translate to get MLIR tfl dialect.
Towards this goal, however, I need to pass some attributes with some operations special to our target architecture. I initially wanted to pass these attributes with an operation such as conv2d. However, I don't know a way (if at all possible) to extend such operations that are natively defined / supported by tfl.
I then tried to define and register a custom TF operation with its custom attributes. The semantics of the operation would be an identity function but I just intended to use it as a placeholder to pass my attributes. Once I tried this, I saw that the resulting TFL MLIR contains my custom op, however, the attributes are encoded into an opaque type with a byte stream as its value.
I could not find much documentation related to how I can decode these attributes. I'd appreciate any tips on decoding or any other suggestion to help achieving our goal.
Thanks!
How are you encoding it in the input? I'm guessing you are seeing it encoded as an AttrValue (https://github.com/tensorflow/tensorflow/blob/b1e813e2ec9634ec0e6562b836e372e393f3de43/tensorflow/core/framework/attr_value.proto#L18) and so you'd decode it as you would a protobuf normally.
Related
Issue in short
In Python version of Tensorflow there is a tf.random.categorical() method that draws samples from a categorical distribution. But I can't find a similar method in TensorFlow.js API. So, what is the proper way to draw samples from a categorical distribution in TensorFlow.js?
Issue in details
In Text generation with an RNN tutorial the tf.random.categorical() method is being used in generate_text() function to decide what character should be passed next to the RNN input to generate a sequence.
predicted_id = tf.random.categorical(predictions, num_samples=1)[-1,0].numpy()
I'm experimenting with TensorFlow.js and trying trying to generate a "random" Shakespeare-like writing but in the browser. All parts of the tutorial seems to work well together except the step with using a tf.random.categorical() method.
I guess writing the alternative to tf.random.categorical() function manually should not be that hard, and also there are couple of 3rd-party JavaScript libraries that implement this functionality already, but it looks pretty logical to have it as a part of TensorFlow.js API.
I think you can use tf.multinomial instead.
I peeked at the source code and with name and seed parameters set to None, it is essentially the same as tf.multinomial with some random seeding going on, I guess.
I am trying to use an object detection model, that was created using the TF Object Detection API, in TensorFlow.js.
I converted the model using:
tensorflowjs_converter --input_format=tf_saved_model inference_graph/saved_model inference_graph/web_model
It gets converted without any problems and loads in my javascript code.
Now I am a bit unsure about what my next steps should be.
I have to translate the Python into JavaScript, but certain areas I am unsure about.
With the object detection API in python, there are many steps; (1)preprocessing the image, such as convert to RGB, numpy array reshape, expand dimensions (I have an idea of how I would approach it) and (2) the run inference for single image function, I am not sure how I would go about it in tensorflow.js.
I tried to find some general information about deploying an object detection model in tensorflow.js, but I could not find much, except with pre-trained models.
Any information about this topic would be great!
Thanks!
as mentioned by #edkeveked you will need to perform similar input processing and output processing in JavaScript as is being done in Python. i can't say exactly what you will need to do since i am not familiar with the model. however, you can find an example using a specific object detection model here:
https://github.com/vabarbosa/tfjs-model-playground/blob/master/object-detector/demo/object-detector.js
see also
https://medium.com/codait/bring-machine-learning-to-the-browser-with-tensorflow-js-part-iii-62d2b09b10a3
You would need to replicate the same process in javascript before giving it to the model. In js, the image use by default the RGB channel, so there is no need to make that conversion again.
While serving TensorFlow models via TensorFlow-Serving, I need to expose custom meta data to the clients (e.g. a model's input data requirements, training information...).
I tried adding the information via tf.add_to_collection( <my_custom_key>, <value> ) before saving the model, and, sure enough, the information showed up in the .pb(txt) file used by the server.
However, currently it looks as if the response to querying metadata (e.g. via GET http://localhost:8501/v1/models/<my_model>/metadata) only returns the contents of the signature_def section (which also cannot be extended, the validator prevents that), and I know of no way to query contents of other sections.
Is there a way to serve/query custom meta data for TF Serving?
Unfortunately adding logic to allow serving metadata other than signaturedefs is not on the roadmap right now and I'm not sure we have a good understanding of the general use case for which supporting this would make sense.
Regarding how to serve the metadata stored in the saved model, presumably, you'd add a constant to your graph holding the tensor value of interest (the input/output shape), create a new signature using link below and do inference with that signature -- I've never seen this done but I can't imagine why it wouldn't work.
https://www.tensorflow.org/guide/saved_model#manually_build_a_savedmodel
While I still did not find a solution for TensorFlow Serving, it may be of interest to other readers that it can be achieved when using NVidia's Triton Inference Server.
While evaluating it as a TFS alternative (mainly for its built-in support for other model formats such as pytorch and ONNX), I found out that in Triton, it is possible to serve custom meta information via the Model Configuration Extension using the 'parameters' property. After adding
parameters: {
key: "inference_properties"
value: {
string_value: "<my-custom-inference-property-info>"
}
}
to the model's config.pbtxt file, I could retrieve the information on the client side. It's not extremely convenient, as one can only provide a flat map with string values, but still.
Is it possible to use different feature extractor with SSD meta-architecture in Tensorflow's Object Detection API? I know that .config files for mobilenets and inception are provided but is it possible to use a different architecture like AlexNet or VGG?
It's possible but with a little bit of work, as explained here, you should read this page for detailed explanation and links to examples.
In short, you'll need to create a custom FasterRCNNFeatureExtractor class, corresponding to VGG or AlexNet (it may require a bit of knowledge about these, for instance the amount of subsampling invovled). In this class, you'll code how your data should be preprocessed, how to retrieve the 1st and 2nd stage features in it (typically how is the last convolutional layer called), and how to load it.
Then you'll need to register your feaure extractor (tell the object detection API that it exists) by modifying the file object_detection/builders/model_builder.py.
Finally you should be able to make a config file with your custom feature extractor, et voilà !
If I am defining a custom Op in Tensorflow, is it possible to provide two Kernels for the top that are polymorphic on whether the shape for the inputs are fully defined? For example, I can construct certain structures once at Kernel construction if the shape is fully known / defined.
It's not currently possible to do this. The kernel dispatch mechanism is implemented in a low-level part of the TensorFlow code where information about tensor shapes is not (generally) available.
However, the ability to specialize a graph based on known shapes does seem like a useful ability, and it might be worth raising this as a feature request on the GitHub issues page. One possible workaround would be to try registering an optimization pass that makes use of shape information and rewrites the names of ops with known input shapes to a different op that relies on static shape information (e.g. via an additional attr). However, doing this in TensorFlow currently requires you to rebuild from source.