I am trying to train my model, ssd_mobilenet_v2_fpnlite_320x320_coco17_tpu-8.tar.gz, unfortunately, this is what I got.
Does anyone have a solution for this?
InvalidArgumentError: Graph execution error:
image_size must contain 3 elements[4]
[[{{node RandomCropImage/sample_distorted_bounding_box/SampleDistortedBoundingBoxV2}}]]
[[MultiDeviceIteratorGetNextFromShard]]
[[RemoteCall]]
[[while/body/_1/IteratorGetNext]] [Op:__inference__dist_train_step_51958]
The issue is most likely related to images that are not RGB. Before loading any images, make sure to convert them to RGB (3 channels).
Please use the following code to see which image is not in RGB mode and delete it.
from PIL import Image
import os
path = 'PATH TO THE IMAGES'
for file in os.listdir(path):
extension = file.split('.')[-1]
# image extension could be png, jpg, etc
if extension == 'jpg':
filepath = path+file
img = Image.open(filepath)
if img.mode != 'RGB':
print(file+', '+img.mode)
Related
Does anyone know how to predict Tensorflow base64 encoded models from Python directly for object detection?
If I were using a model exported to expect an input_tensor, I would use:
def convert_image_to_tensor_batch(image):
input_tensor = tf.convert_to_tensor(image)
input_tensor = input_tensor[tf.newaxis, ...]
return input_tensor
to convert my image (once loaded with cv2.imread(image_path))
When it comes to models that are exported to expect base64 encoding though, I have just wasted a whole day failing to work out the magic setup.
I've tried:
def load_image(image_path: str):
image = cv2.imread(image_path)
encoded = base64.b64encode(cv2.imencode(".png", image)[1].tobytes())
return image, encoded.decode("utf-8")
def convert_to_tensor_batch(input_string):
input_tensor = tf.convert_to_tensor(input_string, dtype="string")
input_tensor = input_tensor[tf.newaxis, ...]
return input_tensor
But the output is not recognised, and I get:
INVALID_ARGUMENT: Unknown image file format. One of JPEG, PNG, GIF, BMP required.
Which basically means it doesn't recognise the base64 encoded PNG I'm sending it.
What's frustrating is it works perfectly in TFserving if I serve it the base 64 encoded string with the exact same code shown there, but in a JSON request.
As I don't know how TFserving exactly converts JSON To tensors, my next step will otherwise have to be to make a dummy model that outputs whatever it is converting them too.
I'm trying to run the GAN in the link with my own dataset. First of all, I wanted to try with MNIST dataset and see the results. I am running it on COLAB. When I use the existing versions of tensorflow and keras in Colab, the outputs are noisy and have bad results. An example from 1400.epoch:
But when I downgrade tensorflow to 2.2.0 and keras to 2.3.1 the results are very good. An example from 1350.epoch:
Then, when I ran it with my own dataset without changing the existing library versions in COLAB, I still got noisy and bad results. So I just updated the versions as before. However, as a result, I get the following error:
FailedPreconditionError: Error while reading resource variable
_AnonymousVar45 from Container: localhost. This could mean that the variable was uninitialized. Not found: Resource
localhost/_AnonymousVar45/N10tensorflow3VarE does not exist. [[node
mul_1/ReadVariableOp (defined at
/usr/local/lib/python3.7/dist-packages/keras/backend/tensorflow_backend.py:3009)
]] [Op:__inference_keras_scratch_graph_5103]
Function call stack: keras_scratch_graph
If this error was caused by tensorflow and keras versions, I think I would get the same error when I tried with MNIST. So I couldn't find the source of the error. Maybe it has to do with the way I load my data. However, existing library versions had no problems with this. Anyway I'm adding the way I load the data here:
import zipfile # unziping
import glob # finding image paths
import numpy as np # creating numpy arrays
from skimage.io import imread # reading images
from skimage.transform import resize # resizing images
# 1. Unzip images
path = '/content/gdrive/My Drive/gan/RealImages.zip'
with zipfile.ZipFile(path, 'r') as zip_ref:
zip_ref.extractall('/content/gdrive/My Drive/gan/extracted')
# 2. Obtain paths of images (.png used for example)
img_list = sorted(glob.glob('/content/gdrive/My Drive/gan/extracted/RealImages/RealImages/*.jpg'))
print(img_list)
# 3. Read images & convert to numpy arrays
## create placeholding numpy arrays
IMG_SIZE = 28
x_data = np.empty((len(img_list), IMG_SIZE, IMG_SIZE, 3), dtype=np.float32)
## read and convert to arrays
for i, img_path in enumerate(img_list):
# read image
img = imread(img_path)
print(img_path)
# resize image (1 channel used for example; 1 for gray-scale, 3 for RGB-scale)
img = resize(img, output_shape=(IMG_SIZE, IMG_SIZE,3), preserve_range=True)
# save to numpy array
x_data[i] = img
Then, I changed the old line:
(X_train, _), (_, _) = mnist.load_data()
to:
X_train=x_data
I couldn't find what I did wrong. I would be very happy if you help.
I need to export a custom object detection model, fine-tuned on a custom dataset, to TensorFlow Lite, so that it can run on Android devices.
I'm using TensorFlow 2.4.1 on Ubuntu 18.04, and so far this is what I did:
fine-tuned an 'ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8' model, using a dataset of new images. I used the 'model_main_tf2.py' script from the repository;
I exported the model using 'exporter_main_v2.py'
python exporter_main_v2.py --input_type image_tensor --pipeline_config_path .\models\custom_model\pipeline.config --trained_checkpoint_dir .\models\custom_model\ --output_directory .\exported-models\custom_model
which produced a Saved Model (.pb file);
3. I tested the exported model for inference, and everything works fine. In the detection routine, I used:
def get_model_detection_function(model):
##Get a tf.function for detection
#tf.function
def detect_fn(image):
"""Detect objects in image."""
image, shapes = model.preprocess(image)
prediction_dict = model.predict(image, shapes)
detections = model.postprocess(prediction_dict, shapes)
return detections, prediction_dict, tf.reshape(shapes, [-1])
return detect_fn
and the shape of the produced image object is 640x640, as expected.
Then, I tried to convert this .pb model to tflite.
After updating to the nightly version of tensorflow (with the normal version, I got an error), I was actually able to produce a .tflite file by using this code:
import tensorflow as tf
from tflite_support import metadata as _metadata
saved_model_dir = 'exported-models/custom_model/'
## Convert the model
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.experimental_new_converter = True
converter.target_spec.supported_ops = [
tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()
# Save the model.
with open('tflite/custom_model.tflite', 'wb') as f:
f.write(tflite_model)
I tried to use this model in AndroidStudio, following the instructions given here.
However, I'm getting a couple of errors:
something regarding 'Not a valid Tensorflow lite model' (have to check better on this);
the error:
java.lang.IllegalArgumentException: Cannot copy to a TensorFlowLite tensor (serving_default_input_tensor:0) with 3 bytes from a Java Buffer with 270000 bytes.
The second error seems to indicate there's something weird with the input expected from the tflite model.
I examined the file with Netron, and this is what I got:
the input is expected to have...1x1x1x3 shape, or am I misinterpreting the graph?
Should I somehow set the tensor input size when using the tflite exporter?
Anyway, what is the right way to export my custom model so that it can run on Android?
TF Ops are supported via the Flex delegate. I bet that is the problem. If you want to check if it is that, you can do:
Download benchmark app with flex delegate support for TF Ops. You can find it here, in the section Native benchmark binary: https://www.tensorflow.org/lite/performance/measurement. For example, for android is https://storage.googleapis.com/tensorflow-nightly-public/prod/tensorflow/release/lite/tools/nightly/latest/android_aarch64_benchmark_model_plus_flex
Connect your phone to your computer and where you have downloaded the apk, do adb push <apk_name> /data/local/tmp
Push your model adb push <tflite_model> /data/local/tmp
Open shell adb shell and go to folder cd /data/local/tmp. Then run the app with ./<apk_name> --graph=<tflite_model>
Info from:
https://www.tensorflow.org/lite/guide/ops_select
https://www.tensorflow.org/lite/performance/measurement
I build a simple model using tf.keras and a tf.data.Dataset for efficient loading as the dataset is a couple of GBs big.
The images are in tiff format and therefore need to be loaded directly as numpy.array.
I do have a dataset of labels and file paths and want to map a function for loading on that dataset. Therefore, I somehow have to get the python string representation out of the tensor.
I tried using the usual tf.Tensor.eval() and then joining the chars to a full string but getting the err: ValueError: Cannot evaluate tensor using eval(): No default session is registered. which does make sense as there is no session before the keras model is being executed
Then I tried putting tf.enable_eager_execution() right below my tensorflow import in the dataset file (and changing to from .eval() to .numpy()) but am getting err: AttributeError: 'Tensor' object has not attribute 'numpy' hinting tf.enable_eager_execution() did not work
Basically I'm trying to read a string contained in a tensor a below:
path = tf.decode_raw(path, tf.uint8)
path = ''.join(map(chr, path.eval())) # with session
path = ''.join(map(chr, path.numpy())) # with eager execution
image = PIL.Image.open(path)
image = numpy.array(image)
Both work fine when 'prototyping' in a single file without the rest but do not work when having e.g. my model in a model.py and dataset in dataset.py but having tf.enable_eager_execution() in both
This issue is quite old but as it could be useful for someone esle, that would be my approach:
You can use tf.io.read_file to read the file and then convert it to an image with tf.image.decode_png if your image is a png. Several other formats are available.
with tf_filepath the path to your RGB image contained in a tensor
image = tf.io.read_file(tf_filepath)
image = tf.image.decode_png(image, channels=3, dtype=tf.uint8)
I get the pre-trained .pb file of MobileNet and find it's not quantized while the fully quantized model should be converted into .tflite format. Since I'm not familiar with tools for mobile app developing, how can I get the fully quantized weights of MobileNet from .tflite file. More precisely, how can I extract quantized parameters and view its numerical values ?
The Netron model viewer has nice view and export of data, as well as a nice network diagram view.
https://github.com/lutzroeder/netron
I'm also in the process of studying how TFLite works. What I found may not be the best approach and I would appreciate any expert opinions. Here's what I found so far using flatbuffer python API.
First you'll need to compile the schema with flatbuffer. The output will be a folder called tflite.
flatc --python tensorflow/contrib/lite/schema/schema.fbs
Then you can load the model and get the tensor you want. Tensor has a method called Buffer() which is, according to the schema,
An index that refers to the buffers table at the root of the model.
So it points you to the location of the data.
from tflite import Model
buf = open('/path/to/mode.tflite', 'rb').read()
model = Model.Model.GetRootAsModel(buf, 0)
subgraph = model.Subgraphs(0)
# Check tensor.Name() to find the tensor_idx you want
tensor = subgraph.Tensors(tensor_idx)
buffer_idx = tensor.Buffer()
buffer = model.Buffers(buffer_idx)
After that you'll be able to read the data by calling buffer.Data()
Reference:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/lite/schema/schema.fbs
https://github.com/google/flatbuffers/tree/master/samples
Using TensorFlow 2.0, you can extract the weights and some information regarding the tensor (shape, dtype, name, quantization) with the following script - inspired from TensorFlow documentation
import tensorflow as tf
import h5py
# Load TFLite model and allocate tensors.
interpreter = tf.lite.Interpreter(model_path="v3-large_224_1.0_uint8.tflite")
interpreter.allocate_tensors()
# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# get details for each layer
all_layers_details = interpreter.get_tensor_details()
f = h5py.File("mobilenet_v3_weights_infos.hdf5", "w")
for layer in all_layers_details:
# to create a group in an hdf5 file
grp = f.create_group(str(layer['index']))
# to store layer's metadata in group's metadata
grp.attrs["name"] = layer['name']
grp.attrs["shape"] = layer['shape']
# grp.attrs["dtype"] = all_layers_details[i]['dtype']
grp.attrs["quantization"] = layer['quantization']
# to store the weights in a dataset
grp.create_dataset("weights", data=interpreter.get_tensor(layer['index']))
f.close()
You can view it using Netron app
macOS: Download the .dmg file or run brew install netron
Linux: Download the .AppImage file or run snap install netron
Windows: Download the .exe installer or run winget install netron
Browser: Start the browser version.
Python Server: Run pip install netron and netron [FILE] or netron.start('[FILE]').