I've already managed to create a lambda function that loads a model.pb from S3 and apply object detection to an input image (installed tensorflow 1.12)
Is it possible to load a Sagemaker model/endpoint-configuration inside a lambda function ? I mean install all packages needed inside the lambda, without deploying an endpoint/ec2-like instance.
I guess inference performance would drop, but the solution seems to be more cost effective and scalable ready.
SageMaker Endpoints only run on EC2 instances which are configured as part of the EndpointConfig. You can't use SageMaker models and deploy them onto Lambda.
Related
How to re train a model with new data that is currently served in production using tensorflow serving?
Do we have to train the model manually and serve it again? Or is there any automated way of doing this.
I am using tensorflow serving with docker.
Basically the idea is that:
Considering there is already a model served using tensorflow serving, and in the future I get some bunch of additional data and I want the model to be fitted with this data then, how can we do this training to the same model?
Question 1: I do have a script to train the model, but does the training have to be done locally/manually?
Answer: As far as i understand you are talking it should be done locally or in some remote server, you can do wherever as per convenience the main important step for tensorflow serving is to save model in the respective format that could be used by the server. Please refer to the link on how to save as well as how to load it in the serving docker container.
serving tensorflow model
Question 2: Suppose I create a entirely new model (apart from modelA currently server), how can I load it to tensorflow serving again? Do I have to manually load it to the docker target path?
Answer: Yes if you are loading it without using serving config, you will have to manually shut down container, remap the path in the command and then load it in the docker container. That is where the serving config helps you to load models in runtime only.
Question 3: TFX document says to update the model.config file for adding new models, but how can I update it when the serving is running.
Answer: A basic configuration file would look like this
config {
name: 'my_first_model'
base_path: '/tmp/my_first_model/'
model_platform: 'tensorflow'
}
config {
name: 'my_second_model'
base_path: '/tmp/my_second_model/'
model_platform: 'tensorflow'
}
}
This file would be needed to be mapped before starting your docker container and of course the path as well where different models will be located. This config file when changed will load new models accordingly in the serving docker container. You can also maintain different versions of the same model as well. For more info please refer to this link serving config. This file is automatically looked up by the serving periodically and as soon as it detects some change it will automatically load new models without the need to restart the docker container.
I used TF Hub to retrain a model for image classification. Now I would like to serve it in the cloud. For that i need a SavedModel. The retrain.py script from TF Hub uses tf.saved_model.simple_save to generate the SavedModel after the training is done.
What confuses me is the .pb file inside the SavedModel folder that I get from that method is much smaller than the final .pb saved after the training.
simple_save is also now deprecated and I tried to get my SavedModel after the training is done following this SO issue.
But my variables folder is empty. How can I incorporate that building of SavedModel inside the retrain.py to replace the simple_save method ? Tips would be much appreciated.
To deploy your model to Google Cloud ML, you need a SavedModel which can be produced from tf.saved_model api.
Below are the steps for hosting your trained models in cloud with Cloud ML Engine.
Upload your saved model to a Cloud Storage bucket by setting up a cloud storage bucket using BUCKET_NAME="your_bucket_name"
Select a region for your bucket and set a REGION environment variable.
EGION=us-central1
Create a new bucket gsutil mb -l $REGION gs://$BUCKET_NAME
Upload using
SAVED_MODEL_DIR=$(ls ./your-export-dir-base | tail -1)
gsutil cp -r $SAVED_MODEL_DIR gs://your-bucket
Create a Cloud ML Engine model resource and model version.
Also for your question on incorporating savedmodel inside retrain.py, you need to pass saved model as an argument to the tfhub_module line as below.
python retrain.py --image_dir C: ...\\code\\give the path here --tfhub_module C:
...give the path to saved model directory
I'm trying to add TensorBoard functionality to this SageMaker example: https://github.com/awslabs/amazon-sagemaker-examples/blob/master/hyperparameter_tuning/keras_bring_your_own/hpo_bring_your_own_keras_container.ipynb
The issue is that SageMaker's Estimator.fit() does not seem to support Keras models compiled with callbacks.
Now from this git issue post it was described that what I need to do for TensorBoard functionality is,
"You need your code inside the container to save checkpoints to S3,
and you need to periodically sync your local Tensorboard log directory
with your S3 checkpoints."
So to sum it all up, to enable TensorBoard in SageMaker with this custom Keras docker image, it looks like I need a way of periodically uploading a file to an S3 bucket during training without using callbacks. Is this possible to do? I was considering trying to shove this code into a custom loss function, but I'm not sure if this would be the way to go about it. Any help is greatly appreciated!
From Tensorflow docs, we can use transfer learning for object detection when you run from cloud. Also, can we using transfer-learning for running locally ? i see that we have a doc about running on local but i can not find any documents that write about transfer learning when run on local machine . I expected it in pipeline but i can not find it. So how can we config transfer learning on running locally ?
You are correct. You can use transfer learning when running locally.
The steps are similar to the instruction on running pets on google cloud, but your training config should reference your local file system instead of a path on GCS. In particular, you need to configure the train_config.fine_tune_checkpoint field to point to the unzipped checkpoint.
See the sample resnet101 config for more details. You should only have to change the PATH_TO_BE_CONFIGURED strings to point to the correct value.
The Python Tensorflow package is huge and AWS Lambda allows only 250 MB in which you have to bring all resources in a zip file, including all dependencies.
Is it possible to have a "minified" Tensorflow / Keras?
Yes it is possible by using serverless framework with serverless-ephemeral
A specifc tensorflow packager is included