I am trying to create a mobile app that uses object detection to detect a specific type of object. To do this I am starting with the Tensorflow object detection example Android app, which uses TF2 and ssd_mobilenet_v1.
I'd like to try Few-Shot training (Colab link) so I started by replacing the example app's SSD Mobilenet v1 download with the Colab's output file model.tflite, however this causes the the app to crash with following error:
java.lang.IllegalStateException: This model does not contain associated files, and is not a Zip file.
at org.tensorflow.lite.support.metadata.MetadataExtractor.assertZipFile(MetadataExtractor.java:313)
at org.tensorflow.lite.support.metadata.MetadataExtractor.getAssociatedFile(MetadataExtractor.java:164)
at org.tensorflow.lite.examples.detection.tflite.TFLiteObjectDetectionAPIModel.create(TFLiteObjectDetectionAPIModel.java:126)
at org.tensorflow.lite.examples.detection.DetectorActivity.onPreviewSizeChosen(DetectorActivity.java:99)
I realize the Colab uses ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz - does this mean there are changes needed in the app code - or is there something more fundamentally wrong with my approach?
Update: I also tried the Lite output of the Colab tf2_image_retraining and got the same error.
The fix apparently was https://github.com/tensorflow/examples/compare/master...cachvico:darren/fix-od - .tflite files can now be zip files including the labels, but the example app doesn't work with the old format.
This doesn't throw error when using the Few Shots colab output. Although I'm not getting results yet - pointing the app at pictures of rubber ducks not yet work.
Related
I am currently trying to train a custom model for use in Unity (Barracuda) for object detection and I am struggling near what I believe to be the last part of the pipeline. Following various tutorials and git-repos I have done the following...
Using Darknet, I have trained a custom-model using the Tiny-Yolov2 model. (model tested successfully on a webcam python script)
I have taken the final weights from that training and converted them
to a Keras (h5) file. (model tested successfully on a webcam python
script)
From Keras, I then use tf.save_model to turn it into a
save_model.pd.
From save_model.pd I then convert it using tf2onnx.convert to change
it to an onnx file.
Supposedly from there it can then work in one of a few Unity sample
projects...
...however, this project fails to read in the Unity Sample projects I've tried to use. From various posts it seems that I may need to use a 'frozen' save_model.pd before converting it to ONNX. However all the guides and python functions that seem to be used for freezing save_models require a lot more arguments than I have awareness of or data for after going through so many systems. https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py - for example, after converting into Keras, I only get left with a h5 file, with no knowledge of what an input_graph_def, or output_node_names might refer to.
Additionally, for whatever reason, I cannot find any TF version (1 or 2) that can successfully run this python script using 'from tensorflow.python.checkpoint import checkpoint_management' it genuinely seems like it not longer exists.
I am not sure why I am going through all of these conversions and steps but every attempt to find a cleaner process between training and unity seemed to lead only to dead ends.
Any help or guidance on this topic would be sincerely appreciated, thank you.
I'm trying to run a converted RepNet model in TFLite on mobile (iOS and Android) using Flutter, and the tf_lite_flutter package.
I have successfully converted the model to TFLite by adapting the colab provided by the authors. The notebook I used can be found in this repository, along with the converted model.
To make sure everything was working before attempting to run on an edge device, I checked everything with the Python TFLite API (this notebook). Everything indeed worked well - the output for the TFLite model matches the output of the Google colab provided by the authors.
I created a Flutter project to get the model running on mobile. I've tried passing in the default input and output, resulting from calls to interpreter.getInputTensors() and interpreter.getOutputTensors() respectively. When using this project to try to run the model, I encounter the following error:
E/tflite (26540): tensorflow/lite/kernels/reshape.cc:69 num_input_elements != num_output_elements (1 != 0)
E/tflite (26540): Node number 1 (RESHAPE) failed to prepare.
I'm admittedly pretty new to Tensorflow and Tensorflow Lite, so my debugging ability is somewhat limited. It does seem strange to me that the expected output shape is 0. Considering it is working with the Python API, I'm not sure why it isn't working on-device. The only thing I might suspect it could be is the batch_size not being configured properly. Using the shape_signature field, as in interpreter.get_input_details()[0]['shape_signature'], I can see that the batch size is dynamic (value -1).
The model was converted using Tensorflow==2.5 in Python, and is being run using the standard TFLite 2.5 binaries (no GPUDelegate).
Any suggestions for fixing this error would be appreciated!
Im following a guide on using TFLite model maker using my own training data loaded from my google drive and keep getting the error "NotFoundError: /content/TFData/Train/img/img (73).jpg; No such file or directory" when trying to train with my data (please see below screenshot). Think i'm missing something obvious but cant seem to figure it out, apologies if this has been asked before, i'm somewhat new to working in this environment.
I have tried renaming all the images and maps and reshuffled the directory format to no avail.
I am trying to train custom data set that consists of currency. i followed a youtube tutorial, made the same folder structure.
I am using google colab for free gpu and darknet. everytime i run data for training it finishes within seconds without any error and the final output says "608 x 608 create 6 permanent cpu threads"
the tutorial i followed shows the training of dataset but mine is keep getting stuck at this message.
I'm using yolov3 to train my dataset, followed every step of changing things in makefile. Also the train.txt and test.txt files stays empty too. (sorry for my bad english)
Below attached screenshot of the message i get when i try to train my model.
SOLVED : the issue was my train.txt file was empty because it wasn’t getting any image paths, soo i changed absolute path of my images folder to relative path and it saved all the images paths in train.txt file which resulted in activation of data training(sorry for my bad english)
If I use the Pre-Trained TFLite Object detection model in MLKit, I get the following error:
CalculatorGraph::Run() failed in Run:
Calculator::Open() for node "BoxClassifierCalculator" failed: #vk Unexpected number of dimensions for output index 0: got 3D, expected either 2D (BxN with B=1) or 4D (BxHxWxN with B=1, W=1, H=1).
Any Idea what I might be doing wrong?
ML Kit does not currently support custom object detection model yet. ML Kit currently only allows developers to use custom image classification models. All TFLite models that are compatible with ML Kit are listed here:
https://tfhub.dev/ml-kit/collections/image-classification/1
If you want to do object detection, you can try out ML Kit's Object Detection API: https://developers.google.com/ml-kit/vision/object-detection
If you want to use a custom object detection model, you can try TFLite task library:
https://www.tensorflow.org/lite/inference_with_metadata/task_library/overview.
Well, since I found out through this post here on SO, I went on to look for other options.
And the tutorial posted here: Tensor Flow Lite OBJ detection did the trick just nicely.
First of all added the
implementation 'org.tensorflow:tensorflow-lite-task-vision:0.2.0'
To build.gradle, and this simple code worked like a charm. Obviously I have to make some changes to reflect my specific needs, but it's detecting without errors.
val image = TensorImage.fromBitmap(bitmap)
val options = ObjectDetector.ObjectDetectorOptions.builder()
.setMaxResults(5)
.setScoreThreshold(0.5f)
.build()
val detector = ObjectDetector.createFromFileAndOptions(
this, // the application context
"model.tflite", // must be same as the filename in assets folder
options
)
A more in depth explanation is given in that link, too much to be thrown here.
Hope this helps somebody else.
You can now create MLKit Custom Models with Google's Vertex AI/AutoML Cloud service and it works well with the CameraX API using an ImageAnalysis Analyzer. Follow the guide here to make and train the TFLite model with your own images via browser (or use the Google Cloud CLI if you have many to upload). After that, this guide walks you through pretty much everything you need to add the model to your project, and how detect and label objects with MLKit.