I have an image classification deep learning CNN model (.h5 file) trained using Keras and Tensorflow 2 that I want to use online for predictions. I want an API that takes the single input image over HTTP and responds with the predicted class labels using the trained model. Is there an API provided by Keras or Tensorflow to do the same?
There's two basic options:
Use TensorFlow Serving - it provides ready-to-go REST API server, the only thing that you need to do is to convert your model to .pb format.
Write your own simple REST server (on Flask, for example) which will call model.predict() on the inputs (that approach may be easier to start with, but it will be hard to scale/optimize for heavy load.
Related
Generative Language Models like GPT, seq2seq or chatbots based on transformers have their decoder layers and its inputs.
The model generates words until they meet [EOS] tokens by repeating feeding outputs fro m the decoder as inputs to the decoder back.
However, it seems there is no mechanism for this in TensorFlow Serving.
If I want to use Generative Language Models using TensorFlow Serving, Should I call gRPC prediction request every time generate a word until [EOS] token?
If I should, how can I reduce overheads from gRPC API call?
Are there other recommendations for this kind of models?
I am using TF-Agents for a custom reinforcement learning problem, where I train a DQN (constructed using DqnAgents from the TF-Agents framework) on some features from my custom environment, and separately use a keras convolutional model to extract these features from images. Now I want to combine these two models into a single model and use transfer learning, where I want to initialize the weights of the first part of the network (images-to-features) as well as the second part which would have been the DQN layers in the previous case.
I am trying to build this combined model using keras.layers and compiling it with the Tf-Agents tf.networks.sequential class to bring it to the necessary form required when passing it to the DqnAgent() class. (Let's call this statement (a)).
I am able to initialize the image feature extractor network's layers with the weights since I saved it as a .h5 file and am able to obtain numpy arrays of the same. So I am able to do the transfer learning for this part.
The problem is with the DQN layers, where I saved the policy from the previous example using the prescribed Tensorflow Saved Model Format (pb) which gives me a folder containing model attributes. However, I am unable to view/extract the weights of my DQN in this way, and the recommended tf.saved_model.load('policy_directory') is not really transparent with respect to what data I can see regarding the policy. If I have to follow the transfer learning as I do in statement (a), I need to extract the weights of my DQN and assign them to the new network. The documentation seems to be quite sparse for this case where transfer learning needs to be applied.
Can anyone help me in this, by explaining how I can extract weights from the Saved Model method (from the pb file)? Or is there a better way to go about this problem?
I have trained an image classification model using pytorch.
Now, I want to move it from research to production pipeline.
I am thinking of using TensorFlow extended. I have a very noob doubt that will I'll be able to use my PyTorch trained model in the TensorFlow extended pipeline(I can convert the trained model to ONNX and then to Tensorflow compatible format).
I don't want to rewrite and retrain the training part to TensorFlow as it'll be a great overhead.
Is it possible or Is there any better way to productionize the PyTorch trained models?
You should be able to convert your PyTorch image classification model to Tensorflow format using ONNX, as long as you are using standard layers. I would recommend doing the conversion and then look at both model summaries to make sure they are relatively similar. Also, do some tests to make sure your converted model handles any particular edge cases you have. Once you have confirmed that the converted model works, save your model as a TF SavedModel format and then you should be able to use it in Tensorflow Extended (TFX).
For more info on the conversion process, see this tutorial: https://learnopencv.com/pytorch-to-tensorflow-model-conversion/
You could considering using the torchX library. I haven't use it yet, but it seems to make it easier to deploy models by creating and running model pipelines. I don't think it has the same data validation functionality that Tensorflow Extended has, but maybe that will be added in the future.
I have been trying to visualize and see layers/parameters/Flops for this Tensorflow model of Text-to-speech synthesis https://github.com/Rayhane-mamah/Tacotron-2
Is there a way I can visualize or see graph of Tacotron-2 with all its RNN/LSTM layers using tensorflow?
Do I need train the model first before being able to print the model, or is there a way to simply see what ops are in each layer for the model without training?
I'm having a hard time figuring this out as I'm new to TF/pytorch frameworks. It seems to me one should be able to just print the model as the github has .py source, but I just don't know how this simple/basic things work with python and how to do it.
I have an untrained Keras model. I want to somehow host this untrained model and then be able to train it with different data sets and save the differet trained models via an API. Then I would like to make predictions with the different trained models via an API.
This is the first time I try to develop a backend service that involves machine learning (not only prediction but also training). Normally I work with express but I'm wondering if the best way is to create a Django API that receives the data to train, loads the model from AWS S3 and the somehow runs a routine that trains the model with the data and sabes the result in S3.Finally when I want to make a prediction I can load the model and make a prediction in the Django app when I receive an API call.
Im not sure how would the whole pipeline work. I appreciate any comments.
In reality the idea is to have multiple models but I would like to know how it would work with 1 before thinking about multiple.