I have an OD Model trained on a custom dataset. I would like to deploy the model as an API. The model will be used in real-time inference and I am planning on monetizing this API on one of the API marketplaces such as AWS, Rakuten's Rapid API, etc.
My concern is if the OD Model is provided as an API, performing predictions in real-time on a video stream (surveillance camera feed) will bring network latency that will make the app slower. Are there any other alternatives to solve the latency issues?
For instance, If I package the code and artifacts to be executed on the client's system, network latency can be eliminated but at the risk of exposing the model, code, etc. So API seems to be the ideal solution for my use case.
What would be the best approach to execute such a scenario?
Moreover, if pre-processing and post-processing are involved for the images. Are there any platforms that aid to package our application and converting it as a black box that takes image inputs and provides image outputs?
For AWS Marketplace, you can sell a Amazon SageMaker "model package" product, a pre-trained model for making predictions that does not require any further training by the buyer.
This should address your concerns on intellectual-property protection and somewhat address your concerns on latency.
Regarding intellectual-property protection, you as the seller package your model inside a Docker container. When it is deployed in the buyer's Amazon SageMaker service in their AWS account, they have no direct access to the container. They can only interact with your model via the SageMaker APIs. More info here: https://docs.aws.amazon.com/marketplace/latest/userguide/ml-security-and-intellectual-property.html
Regarding latency, the model package is deployed in the buyer's AWS account in the region of their choosing. Although the model package cannot be deployed onto edge devices, it brings it one step closer instead of where you as the seller hosts the API.
For more information on publishing Amazon SageMaker products on AWS Marketplace, see the seller guide: https://docs.aws.amazon.com/marketplace/latest/userguide/machine-learning-products.html
Related
with strategy.scope():
model = transformer(vocab_size=VOCAB_SIZE,
num_layers=NUM_LAYERS,
units=UNITS,
d_model=D_MODEL,
num_heads=NUM_HEADS,
is_encoder=True,
dropout=DROPOUT)
model.load_weights("path")
I get error:
InvalidArgumentError: Unsuccessful TensorSliceReader constructor: Failed to get matching files on path: UNIMPLEMENTED: File system scheme '[local]' not implemented (file: 'path')
TL;DR: You need to use Cloud Storage (GCS), which is a Google Cloud Platform (GCP) service.
As stated in the Cloud TPU documentation (https://cloud.google.com/tpu/docs/troubleshooting/trouble-tf#cannot_use_local_filesystem), TPU servers do not have access to your local storage; they can only see files in GCS buckets.
You need to place all the data used for training a model (or inference, depending on your intent) in a GCS bucket, including pretrained weights and the dataset. Note that GCS is a paid service, albeit not very pricey (and first-time users get a trial period).
Links to GCP officlal docs below might help you get started:
Create storage buckets
Connecting to Cloud Storage Buckets
I've uploaded my trained model to the Google Cloud Platform that I trained and exported on lobe.ai. Now I want to send a test request with an image to it so I can use it on my web application. How do I do this?
With your tensorflow (I deduce this from your tags) model, you have 2 solutions
Either your test locally
Or you can deploy your model on AI Platform in online prediction mode.
In both cases, you have to submit a binary + your features in a JSON instance according with your model inputs
I'm trying to build a data collection web endpoint.The Use case is similar to Google Analytics collect API. I want to add this endpoint(GET method) to all pages on the website and on-page load collect page info through this API.
Actually I'm thinking of doing this by using Google Cloud services like Endpoints, BQ(for storing the data).. I don't want to host it in any dedicated servers. Otherwise, I will be end up doing a lot for managing/monitoring the service.
Please suggest me how do I achieve this with Google Cloud Service? OR direct me to right direction if my idea is wrong
I suggest focussing on deciding where you want to code to run. There are several GCP options that don't require dedicated servers:
Google App Engine
Cloud Functions/Firebase Functions
Cloud Run (new!)
Look here to see which support Cloud Endpoints.
All of these products can support running code that takes the data from the request and sends it to the BigQuery API.
There are various ways of achieving what you want. David's answer is absolutely valid, but I would like to introduce Stackdriver Custom Metrics to the discussion.
Custom metrics are similar to regular Stackdriver Monitoring metrics, but you create your own time series (Stackdriver lingo described here) to keep track of whatever you want and clients can sent in their data through an API.
You could achieve the same thing with a compute solution (Google Cloud Functions for example) and a database (Google BigTable for example) and writing your own logic.. but Custom Metrics is an already built solution that includes dashboards and alerting policies while being a more managed solution.
I need some guidance on how to support having access to digital assets and being able to transfer those between other user on the hyperledger via composer?
I suggest you launch the online Playground:
https://composer-playground.mybluemix.net
From there you can import a range of sample (using the Import/Replace button) that illustrate how to code asset transfer. E.g. Marbles, Car Auction or Animal Tracking.
To quote from the tag information (https://stackoverflow.com/tags/hyperledger-composer/info)
Hyperledger Composer provides a layer of abstraction on top of the Hyperledger Fabric blockchain. Developers can design in terms of business networks, participants, identity, assets, registries, transactions, events and links, in contrast to the low level abstractions of chaincode and key-value stores.
So if you wish to do something with a digital asset, say an MP3, you need to create a model - similar to an entity-relationship model or an ObjectOriented hierachy. With that you can then define the operations on it.
An actual asset maybe the MP3 wouldn't necessarily be stored - but would be linked to. Composer then handles how this is put onto the underlying fabric - to give your applications the quality of service of Fabric's blockchain but without the requirement to write chain-code etc.
Hope that puts Composer into context, if you need more please include a broader description of your use case/problem you're solving.
I tried to search for algorithms about predicting future trends and I found Google Prediction API a good fit for me. I wanted to use it in my project but I can't find any useful information about their RESTful API and their examples are for Google App Engine only.
Can the Google Prediction API be used outside App Engine so that I can use it in my project?
Its not clear what resources you're looking at, but https://developers.google.com/prediction/ has details about the prediction service in general, and https://developers.google.com/prediction/docs/reference/v1.5/ has the more specific api details you're seeking.
The prediction API does not require the use of app engine.