Any hint about debugging this one? I am using model.compile with tensor flow keras. Is it because of the output and the labels are different or is it due to the intermediate output?
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [1024,5] vs. [1024,4] [[node binary_crossentropy/mul (defined at abc/models.py:102) ]] [Op:__inference_train_function_33509]
Related
I have converted a tf model to tflite, and applied quantization in the process, but I cannot load it. The error was raised when I try to do interpreter = tf.lite.Interpreter(tflite_model_path), the error message was:
ValueError: Did not get operators or tensors in subgraph 1.
Also during quantization, I got lots of these INFO messages for every dense layer in my model:
2021-09-06 04:38:40.879693: I tensorflow/lite/tools/optimize/quantize_weights.cc:217] Skipping quantization of tensor bert_token_clssfification/classifier/Tensordot/Shape that is not type float.
These messages confuse me greatly, because I'm sure those weights are of type float32. Any ideas what I'm doing wrong? Thanks!
I figured out the cause in my case. It was because I have dropout layers in my model, and I'm using an input tf.bool tensor to explicitly control the training/inference mode of the dropouts layers. Dropout is not currently supported in tflite, and because I'm explicitly controlling the dropout behaviour, the tflite conversion cannot remove the dropout operations.
The correct way to use dropout is to pass the training kwarg during model invocation: out = model(input_batch, training=True).
I have the following (part of) network architecture:
Obtained by
...
pool = GlobalAvgPool()(gc_2)
predictions = Dense(units=32, activation='relu', use_bias=False)(pool)
predictions = BatchNormalization()(predictions)
...
I am trying to insert a batch normalization layer, but I get the following error:
ValueError: Input 0 of layer batch_normalization_1 is incompatible with the layer: expected ndim=2, found ndim=3. Full shape received: [None, 1, 32]
I am guessing the second dimension is causing this mishap. Is there any way I can get rid of it?
If your model is complied successfully, there is no problem with your model definition.
This is more likely to happen because of the input data shape and dimensions are incompatible with your model's desired input shape.
expected ndim=2, found ndim=3. means that the model requires a 2D tensor with
I'm using tf.data.experimental.make_csv_dataset to create a dataset from a .csv file. I'm also using tf.keras.layers.DenseFeatures as an input layer of my model.
I'm struggling to create a DenseFeatures layer properly so that it is compatible with my dataset in the case when batch_size parameter of make_csv_dataset is not equal to 1 (in case if batch_size=1 my setup works as expected).
I create DenseFeatures layer using a list of tf.feature_column.numeric_column elements with shape=(my_batch_size,), but it seems like in this case for some reason the input layer expects [my_batch_size,my_batch_size] shape instead of [my_batch_size,1].
With my_batch_size=19 I'm getting the following error when trying to fit the model:
ValueError: Cannot reshape a tensor with 19 elements to shape [19,19] (361 elements) for 'MyModel/Input/MyColumn1/Reshape' (op: 'Reshape') with input shapes: [19,1], [2] and with input
tensors computed as partial shapes: input[1] = [19,19].
If I don't specify shape when creating numeric_column it doesn't work either. I'm getting the following error:
tensorflow.python.framework.errors_impl.InvalidArgumentError: The second input must be a scalar, but it has shape [19]
which assumes that numeric_column expects a scalar but recieves the whole batch in one Tensor.
How do I create an input layer of DenseFeatures so that it accepts the dataset produced by make_csv_dataset(batch_size=my_batch_size)?
From the tf.feature_column.numeric_column documentation:
shape: An iterable of integers specifies the shape of the Tensor. An integer can be given which means a single dimension Tensor with given width. The Tensor representing the column will have the shape of [batch_size] + shape.
This means that you must not pass the batch size to the shape argument: shape=().
Currently, with a batch size of 1, you get shape=(1,) that TF can handle thanks to broadcasting or something like that (dimensions of size 1 are easily added by TF if necessary), that's why it works.
Hope this can help. Provide more code if you want more help.
my keras encoder-decoder code works fine on cpu. but on gpu I cannot launch it with batch_size other than 1 ! This is strange. Even when I do not use batch size argument in this fitting I get:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [1568] vs. [32,49]
(by default keras takes batch_size=32)
I can run many other programs with whatever batch_size I want. Wht module is comparing a two dimensional object with one dimensional ?
My problem is similar, but I get:
tensorflow.python.framework.errors_impl.InvalidArgumentError:
Incompatible shapes: [64] vs. [128].
I found that it may be because the loss function does not perform dimensionality reduction operations.
So, I tried tf.reduce_mean(), it shows ok.
There occurs no error in tensorflow graph construction, but I get a shape mismatch error during graph computation in tf.gradients (I guess that the error is in back propagation).
This is the error I get:
InvalidArgumentError (see above for traceback):
Input to reshape is a tensor with 16777216 values, but the requested shape has 4096
[[Node: gradients/truediv_grad/Reshape = Reshape[T=DT_FLOAT, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0
/device:GPU:0"](gradients/truediv_grad/Sum, gradients/truediv_grad/Shape)]]
I solved the issue , using two techniques:
1.Apparently if you are creating custom ops and gradients , you need to be very explicit in providing the shape information to tensorflow using set_shape or tf.reshape
2.Also when you are registering your gradient using tf.register_gradient which takes op and grad as inputs, you need to be careful while chaining the gradients i.e dy/dx = dy/dz*dz/dx.
Say dy/dz is the custom gradient we have created and dz/dxis the gradient of the previous ops as per the chain rule of differentiation.
tf.register_gradient(Mygrad)
def Mygrad(op,grad):
*****do stuff with op.inputs and calculate custom grads say cust_grad or dy/dz****
return cust_grad*grad
I changed this to following:
tf.register_gradient(Mygrad)
def Mygrad(op,grad):
*****do stuff with op.inputs and calculate custom grads say cust_grad or dy/dz****
return tf.matmul(tf.reshape(cust_grad,[calculated_shape]),tf.reshape(grad,expeced_shape))