I want to do this without using recognition on server or on cloud. I already trained tensorflow-lite model. I've seen tflite-react-native but it works only with images not real time camera streaming. I'm wandering if it's even possible to make capturing and recognition custom object in real time without streaming the video on backend. Any advices and thoughts are very appreciated.
Related
I have an FYP project (a social media app like Instagram) that requires me to create a simple recommendation system. I've trained my dataset on cosine similarity using Python, but I'm at a loss on what to do next. How can I integrate my trained ml model to React native or if there is a better and easier way to make a recommendation system?
I tried reading documentation and watching videos. But I still don't seem to be able to grasp some difficult concepts. I would greatly appreciate it if you could give me instructions or steps on what to learn after training my model. Or if I have to use some library or packages etc. [not sure if this is the appropriate forum for this inquiry]
I am trying to serve a deep learning model from mobiles. My react native app using tensorflow takes about a minute just to load. My model is about 175mb (about 30 million params). This is my first time trying to run a model on mobiles and I couldn't find any good performance data for tensorflow js on react native.
Is my model too large to expect a reasonably quick loading and inference time on react native? Is this because of hardware limitation or framework? I read that tfjs react native uses web-gl which would be slower than direct access the mobile's gpu so could I use core ml or something like that and expect a better time?
As an extra question (I'll also make a separate post), another route I'm considering is moving inference to a web browser for laptops/desktops. Could I expect a browser web-gl to perform as well as directly using the computer gpu?
Yes, you ML Kit and Core ML are much faster than TF.js.
Also you can create your model, convert it to TF lite then deploy it to firebase and use your model on ios and android in the native side, (currently #react-native-firebase/ml. does not support custom models).
I'm putting my first steps in Machine Learning, I went through many TensorFlow.js tutorials already and I'm trying to achieve this: "Realtime Single Object Tracking/Detection"
Something like this -> input: webcam/video -> output: object bounding box
I know there are SSD and YOLO, and other libraries to predict & locate the objects. But the predicted time is very slow (in browser), I guessed it's because the Neural Network have to predict between so many objects.
https://github.com/ModelDepot/tfjs-yolo-tiny
https://github.com/tensorflow/models/tree/master/research/object_detection
What if I just want to track a single object? Would it be possible? Will the performance be better? Where should I start?
I've been thinking about extract the pre-trained class (object) from a SavedModel, then start training more from it. But there don't seems to be any instructions around Google.
I found some fantastic code by IBM, which I used in the video in this tweet: https://twitter.com/GantLaborde/status/1125735283343921152?s=20
I extracted that code to make a ReactJS component for detecting Rock/Paper/Scissors here: https://github.com/GantMan/rps_tfjs_demo/blob/master/src/AdvancedModel.js
If you'd like to play with the demo, it's at the bottom of this page: https://rps-tfjs.netlify.com/
All of this is open source and seems to work perfectly fast for detecting a single object in realtime.
I have just started experimenting with Deep Learning and Computer Vision technologies. I came across this awesome tutorial. I have setup the TensorFlow environment using docker and trained my own sets of objects and it provided greater accuracy when I tested it out.
Now I want to make the same more real-time. For example, instead of giving an image of an object as the input, I want to utilize a webcam and make it recognize the object with the help of TensorFlow. Can you guys guide me with the right place to start with this work?
You may want to look at TensorFlow Serving so that you can decouple compute from sensors (and distribute the computation), or our C++ api. Beyond that, tensorflow was written emphasizing throughput rather than latency, so batch samples as much as you can. You don't need to run tensorflow at every frame, so input from a webcam should definitely be in the realm of possibilities. Making the network smaller, and buying better hardware are popular options.
I'm trying to send image input over http to classify using tensorflow. I have looked in detail in the c++ code for https://www.tensorflow.org/versions/r0.9/tutorials/image_recognition/index.html
I have implemented the inception-v3 example model using C++ API. It takes image input in the following form:
bazel-bin/tensorflow/examples/label_image/label_image --image=my_image.png
However, I want to add the case of:
bazel-bin/tensorflow/examples/label_image/label_image --image=http://www.somewebsite.com/my_image.png
This is due to the fact that it only accepts local image files. I want to add the functionality to take file pointers from online images and classify it in memory. I'm currently working on this, but so far no luck. Can anyone offer some insight how I would go about implementing this?