I was developing an Image Classifier app in Android Studio with MNIST-fashion database, but I have a little problem. When I try to run the app, I have this common error:
java.lang.IllegalArgumentException: Cannot copy to a TensorFlowLite tensor (serving_default_conv2d_input:0) with 3136 bytes from a Java Buffer with 9408 bytes.
I know this might be the mismatch of input tensor from the model and the buffer that I have in my code. But It's too confusing because my code automatically fits the size of the image from the model and all the info it needs. So I was wondering where is the error...
// Reads type and shape of input and output tensors, respectively.
int imageTensorIndex = 0;
int[] imageShape = tflite.getInputTensor(imageTensorIndex).shape(); // {1, height, width, 1}
imageSizeY = imageShape[1];
imageSizeX = imageShape[2];
DataType imageDataType = tflite.getInputTensor(imageTensorIndex).dataType();
int probabilityTensorIndex = 0;
int[] probabilityShape =
tflite.getOutputTensor(probabilityTensorIndex).shape(); // {1, 10}
DataType probabilityDataType = tflite.getOutputTensor(probabilityTensorIndex).dataType();
// Creates the input tensor.
inputImageBuffer = new TensorImage(imageDataType);
Maybe this is the problem... I'm creating imageShape like this {1, height, width, 1}, and the data type is FLOAT32, so it is supposed to be {1, height, width, 4}, right?
Another reason could be the metadata. But I populate the model with metadata and I have a .json and I don't know how to use it.
Note: If u want the note book to do the .tflite, there u go.
The tensor buffer size is determined by datasize (float32: 4bytes) * flat size of the tensor shape (1 * height * width * 1).
So the above code snippet needs to prepare an float input tensor data with the shape (1, height, width, 1) instead of the shape (1, height, width, 4).
Related
I'm currently working on a Tensorflow Lite image classifier app that can recognice UNO cards. But when I'm running the float model in the class ImageClassifier, something is wrong.
The error is the next one:
java.lang.IllegalArgumentException: Cannot copy from a TensorFlowLite tensor (Identity) with shape [1, 10647, 4] to a Java object with shape [1, 15].
Here's the code that throw that error:
tflite.run(imgData, labelProbArray);
And this is how I have created imgData and labelProbArray:
private static final int DIM_BATCH_SIZE = 1;
private static final int DIM_PIXEL_SIZE = 3; //r+g+b = 1+1+1
static final int DIM_IMG_SIZE_X = 416;
static final int DIM_IMG_SIZE_Y = 416;
imgData = ByteBuffer.allocateDirect(DIM_BATCH_SIZE * DIM_IMG_SIZE_X * DIM_IMG_SIZE_Y * DIM_PIXEL_SIZE * 4); //The last value because size of float is 4
labelProbArray = new float[1][labelList.size()]; // {1, 15}
You can chech the inputs and ouputs of the .tflite file. Source.
I know you should create a buffer for the output values, but I tried to import this and didn't work:
import org.tensorflow.lite.support.tensorbuffer.TensorBuffer;
Any ideas? Thank you so much for read me^^
Edit v2:
Thanks to yyoon I realise that I didn't populate my model with metadata, so I run this line in my cmd:
python ./metadata_writer_for_image_classifier_uno.py \ --model_file=./model_without_metadata/custom.tflite \ --label_file=./model_without_metadata/labels.txt \ --export_directory=model_with_metadata
Before that, I modified this file with my data:
_MODEL_INFO = {
"custom.tflite":
ModelSpecificInfo(
name="UNO image classifier",
version="v1",
image_width=416,
image_height=416,
image_min=0,
image_max=255,
mean=[127.5],
std=[127.5],
num_classes=15)
}
And another error appeared:
ValueError: The number of output tensors (2) should match the number of output tensor metadata (1)
Idk why my model have 2 tensors outputs...
I have tried to migrate a custom model to Android platform. The tensorflow version is 1.12. I used the command line recommended shown as below:
tflite_convert \
--output_file=test.tflite \
--graph_def_file=./models/test_model.pb \
--input_arrays=input_image \
--output_arrays=generated_image
to convert .pb file into tflite format.
I have checked input tensor shape of my .pb file in tensorboard:
dtype
{"type":"DT_FLOAT"}
shape
{"shape":{"dim":[{"size":474},{"size":712},{"size":3}]}}
Then I deploy tflite file upon Android, and allocate input ByteBuffer that planed to feed the model as:
imgData = ByteBuffer.allocateDirect(
4 * 1 * 712 * 474 * 3);
When I run the model on Android device the app crashed and then logcat prints like:
2019-03-04 10:31:46.822 17884-17884/android.example.com.tflitecamerademo E/AndroidRuntime: FATAL EXCEPTION: main
Process: android.example.com.tflitecamerademo, PID: 17884
java.lang.RuntimeException: Unable to start activity ComponentInfo{android.example.com.tflitecamerademo/com.example.android.tflitecamerademo.CameraActivity}: java.lang.IllegalArgumentException: Cannot convert between a TensorFlowLite buffer with 786432 bytes and a ByteBuffer with 4049856 bytes.
It's so weird since allocated ByteBuffer is exactly the product of 4 * 3 * 474 * 712 whereas tensorflow lite buffer is not the multiple of 474 or 712. I don't figure out why tflite model got a wrong shape.
Thanks in advance if anyone can give a solution.
You could visualize the TFLite model to debug what buffer sizes are actually allocated to the input tensors.
TensorFlow Lite models can be visualized using the
visualize.py
script.
If the input tensor's buffer size isn't what you expect it to be, then there might be a bug with the conversion (or with the arguments provided to tflite_convert)
Hello guys,
I also had the similar problem yesterday. I would like to mention solution which works for me.
Seems like TSLite only support exact square bitmap inputs
Like
Size 256* 256 detection working
Size 256* 255 detection not working throwing exception
And max size which is supported
257*257 should be max width and height for any bitmap input
Here is the sample code to crop and resize bitmap
private var MODEL_HEIGHT = 257
private var MODEL_WIDTH = 257
Crop bitmap
val croppedBitmap = cropBitmap(bitmap)
Created scaled version of bitmap for model input
val scaledBitmap = Bitmap.createScaledBitmap(croppedBitmap, MODEL_WIDTH, MODEL_HEIGHT, true)
https://github.com/tensorflow/examples/blob/master/lite/examples/posenet/android/app/src/main/java/org/tensorflow/lite/examples/posenet/PosenetActivity.kt#L578
Crop Bitmap to maintain aspect ratio of model input.
private fun cropBitmap(bitmap: Bitmap): Bitmap {
val bitmapRatio = bitmap.height.toFloat() / bitmap.width
val modelInputRatio = MODEL_HEIGHT.toFloat() / MODEL_WIDTH
var croppedBitmap = bitmap
// Acceptable difference between the modelInputRatio and bitmapRatio to skip cropping.
val maxDifference = 1e-5
// Checks if the bitmap has similar aspect ratio as the required model input.
when {
abs(modelInputRatio - bitmapRatio) < maxDifference -> return croppedBitmap
modelInputRatio < bitmapRatio -> {
// New image is taller so we are height constrained.
val cropHeight = bitmap.height - (bitmap.width.toFloat() / modelInputRatio)
croppedBitmap = Bitmap.createBitmap(
bitmap,
0,
(cropHeight / 2).toInt(),
bitmap.width,
(bitmap.height - cropHeight).toInt()
)
}
else -> {
val cropWidth = bitmap.width - (bitmap.height.toFloat() * modelInputRatio)
croppedBitmap = Bitmap.createBitmap(
bitmap,
(cropWidth / 2).toInt(),
0,
(bitmap.width - cropWidth).toInt(),
bitmap.height
)
}
}
return croppedBitmap
}
https://github.com/tensorflow/examples/blob/master/lite/examples/posenet/android/app/src/main/java/org/tensorflow/lite/examples/posenet/PosenetActivity.kt#L451
Thanks and Regards
Pankaj
I had changed the image dimensions from the standard 224 earlier in the model creation process to 299 for other reasons, so I just searched my Android Studio project for 224 and updated the two final references in ImageClassifier.java to 299, and I was back in business.
The following code uses a tf.while_loop(...) for computations of a dynamic length.
outputs_tensor_array = tf.TensorArray(tf.float32,
size=0,
clear_after_read=False,
infer_shape=False,
dynamic_size = True,
element_shape[self.batch_size, self.size])
initial_args = [outputs_tensor_array, 0]
outputs, *_ = tf.while_loop(lambda out, idx, *_ : idx < max_len,
func,
initial_args + additional_args,
parallel_iterations = 32,
swap_memory = True)
outputs = outputs.stack()
I'm wondering if its possible to enforce a size, or atleast make that size be None in order to enforce a size constraint and enable further computations down the graph. The current shape is [?, batch, hidden_size]
tensor.set_shape will refine the static shape information and throw an error if it is incompatible with current static shape information (in the TensorArray.stack() case it will let you set any value for the zeroth dimension's static shape information).
tf.reshape can also be useful for asserting/filling in shape information, although it's not perfect. It will only throw an error if the size of the Tensor is wrong when the graph is executed (and may otherwise hide a shape error downstream).
More complicated, but you can also set_shape for the static shape information and then use tf.Assert with tf.shape to check the Tensor's shape when the graph is executed.
I'm writing a custom Tensorflow op using the tutorial and I'm having trouble understanding how to read and write to/from Tensors.
let's say I have a Tensor in my OpKernel that I get from
const Tensor& values_tensor = context->input(0); (where context = OpKernelConstruction*)
if that Tensor has shape, say, [2, 10, 20], how can I index into it (e.g. auto x = values_tensor[1, 4, 12], etc.)?
equivalently, if I have
Tensor *output_tensor = NULL;
OP_REQUIRES_OK(context, context->allocate_output(
0,
{batch_size, value_len - window_size, window_size},
&output_tensor
));
how can I assign to output_tensor, like output_tensor[1, 2, 3] = 11, etc.?
sorry for the dumb question, but the docs are really tripping me up here and the examples in the Tensorflow kernel code for built-in ops somehow obfuscate this to the point that I get very confused :)
thank you!
The easiest way to read from and write to tensorflow::Tensor objects is to convert them to an Eigen tensor, using the tensorflow::Tensor::tensor<T, NDIMS>() method. Note that you have to specify the (C++) type of elements in tensor as template parameter T.
For example, to read a particular value from a DT_FLOAT32 tensor:
const Tensor& values_tensor = context->input(0);
auto x = value_tensor.tensor<float, 3>()(1, 4, 12);
To write a particular value to a DT_FLOAT32 tensor:
Tensor* output_tensor = ...;
output_tensor->tensor<float, 3>()(1, 2, 3) = 11.0;
There are also convenience methods for accessing a scalar, vector, or matrix.
I’m running into a number of issues relating to dynamic axes. I am trying to implement a convolutional rnn similar to the of the LSTM() function but handles sequential image input and outputs an image.
I’m able to build the network and pass dummy data through it to produce output, but when I try to compute the error with an input_variable label I consistently see the following error:
RuntimeError: Node '__v2libuid__Input471__v2libname__img_label' (InputValue operation): DataFor: FrameRange's dynamic axis is inconsistent with matrix: {numTimeSteps:1, numParallelSequences:2, sequences:[{seqId:0, s:0, begin:0, end:1}, {seqId:1, s:1, begin:0, end:1}]} vs. {numTimeSteps:2, numParallelSequences:1, sequences:[{seqId:0, s:0, begin:0, end:2}]}`
If I understand this error message correctly, it claims that the value I passed in as the label has inconsistent axes to what is expected with 2 time steps and 1 parallel sequence, when what is desired is 1 time-step and 2 sequences. This makes sense to me, but I’m not sure how the data I’m passing in is not conforming to this. Here are (roughly) the variable declarations and eval statements:
…
img_input = input_variable(shape=img_shape, dtype=np.float32, name="img_input")
convlstm = Recurrence(conv_lstm_cell, initial_state=initial_state)(img_input)
out = select_last(convlstm)
img_label = input_variable(shape=img_shape, dynamic_axes=out.dynamic_axes, dtype=np.float32, name="img_label”)
error = squared_error(out, img_label)
…
dummy_input = np.ones(shape=(2, 3, 3, 32, 32)) # (batch, seq_len, channels, height, width)
dummy_label = np.ones(shape=(2, 3, 32, 32)) # (batch, channels, height, width)
out = error.eval({img_input:dummy_input, img_label:dummy_label})
I believe part of the issue is with the dynamic_axes set when creating the img_label input_variable, I’ve also tried setting it to [Axis.default_batch_axis()] and not setting it at all and either squared error complains about inconsistent axes between out and img_label or I see the same error as above.
The only issue I see with the above setup is that your dummy label should have an explicit dynamic axis so it should be declared as
dummy_label = np.ones(shape=(2, 1, 3, 32, 32))
Assuming your convlstm works similar to an lstm, then the following works without issues for me and it evaluates the loss for two input/output pairs.
x = C.input_variable((3,32,32))
cx = convlstm(x)
lx = C.sequence.last(cx)
y = C.input_variable(lx.shape, dynamic_axes=lx.dynamic_axes)
loss = C.squared_error(y, lx)
x0 = np.arange(2*3*3*32*32,dtype=np.float32).reshape(2,3,3,32,32)
y0 = np.arange(2*1*3*32*32,dtype=np.float32).reshape(2,1,3,32,32)
loss.eval({x:x0, y:y0})