Incompatible shapes in tensorflow and question regarding the way batching is done by the Keras model fit function (newbie) - numpy

I'm trying to build an audio super-resolution model. The inputs to the model are audio clips of length 256, so the actual shape of one input sample is (256,1), where 256 is the length of the audio clip and 1 is the number of channels. If we also take in consideration the size of a batch (in my case it's 8), the overall shape of the input should be (8, 256, 1).
My issue is that I don't understand what happens when the input and target data are passed as parameters to the model.fit(...) function. If I specify a batch size, does this mean that model.fit() will create the batches automatically? If so, why is there any incompatibility between my input data and the input shape of the model?
This is the error that I get:
WARNING:tensorflow:Model was constructed with shape (8, 256, 1) for input KerasTensor(type_spec=TensorSpec(shape=(8, 256, 1), dtype=tf.float32, name='input_1'), name='input_1', description="created by layer 'input_1'"), but it was called on an input with incompatible shape (None,).
Traceback (most recent call last):
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/training.py", line 31, in <module>
model.fit(input_data, target_data,
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 1129, in autograph_handler
raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1366, in test_function *
return step_function(self, iterator)
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1356, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1349, in run_step **
outputs = model.test_step(data)
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/training.py", line 1303, in test_step
y_pred = self(x, training=False)
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/calandrinon/Documents/Repos/Audio-Super-Resolution/Project/venv/lib/python3.8/site-packages/keras/engine/input_spec.py", line 227, in assert_input_compatibility
raise ValueError(f'Input {input_index} of layer "{layer_name}" '
ValueError: Exception encountered when calling layer "model" (type Functional).
Input 0 of layer "conv1d" is incompatible with the layer: expected min_ndim=3, found ndim=1. Full shape received: (None,)
Call arguments received:
• inputs=tf.Tensor(shape=(None,), dtype=string)
• training=False
• mask=None
I do understand that the model needs inputs with 3 dimensions, but why does the error mention that my inputs have 1 dimension when in fact they have 2 dimensions (i.e (256,1)).
Here is the code from the training script:
...
for index in range(0, BATCH_SIZE*NUMBER_OF_BATCHES):
input_data.append(np.load("preprocessed_dataset/low_res/" + input_data_files[index]))
target_data.append(np.load("preprocessed_dataset/high_res/" + target_data_files[index]))
print("Loaded sample {}".format(index))
input_data = np.array(input_data)
target_data = np.array(target_data)
print("Some input tensor shape: {}".format(input_data[0].shape))
print("Some target tensor shape: {}".format(target_data[0].shape))
print("Input data: {}".format(input_data.shape))
print("Target data: {}".format(target_data.shape))
print("Training started...")
model.fit(input_data, target_data,
batch_size=BATCH_SIZE,
epochs=1,
validation_data=(input_validation_files, target_validation_files),
verbose=True)
...and here is the model input shape
...
def create_model(number_of_blocks, batch_size=BATCH_SIZE, input_size=SAMPLE_SIZE):
x = Input((input_size, 1), batch_size=batch_size)
...
Do I actually have to arrange the input samples in batches? Or is that job done by model.fit(...) if I pass a value to the batch_size parameter?
Here is the Github repo, if that helps: https://github.com/Calandrinon/Audio-Super-Resolution/tree/master/Project

Related

Input 0 of layer "global_average_pooling1d" is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 16)

I just started to learn Tensorflow and got an error. I watched a Neural Network Tutorial of Tech with Tim. I finished this episode and got a problem to the end. I couldn't predict the value. when i try it this error shows up:
Traceback (most recent call last):
File "C:/Users/havil/Documents/GitHub/Python/Machine-Learning-Tutorial/Neural Network Tutorial/Tutorial 2.py", line 56, in <module>
predict = model.predict([test_review])
File "C:\Users\havil\anaconda3\envs\tf_3.7\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\havil\anaconda3\envs\tf_3.7\lib\site-packages\tensorflow\python\framework\func_graph.py", line 1147, in autograph_handler
raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:
File "C:\Users\havil\anaconda3\envs\tf_3.7\lib\site-packages\keras\engine\training.py", line 1801, in predict_function *
return step_function(self, iterator)
File "C:\Users\havil\anaconda3\envs\tf_3.7\lib\site-packages\keras\engine\training.py", line 1790, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "C:\Users\havil\anaconda3\envs\tf_3.7\lib\site-packages\keras\engine\training.py", line 1783, in run_step **
outputs = model.predict_step(data)
File "C:\Users\havil\anaconda3\envs\tf_3.7\lib\site-packages\keras\engine\training.py", line 1751, in predict_step
return self(x, training=False)
File "C:\Users\havil\anaconda3\envs\tf_3.7\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\havil\anaconda3\envs\tf_3.7\lib\site-packages\keras\engine\input_spec.py", line 214, in assert_input_compatibility
raise ValueError(f'Input {input_index} of layer "{layer_name}" '
ValueError: Exception encountered when calling layer "sequential" (type Sequential).
Input 0 of layer "global_average_pooling1d" is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 16)
Call arguments received:
• inputs=('tf.Tensor(shape=(None,), dtype=int32)',)
• training=False
• mask=None
My whole code looks like this:
import tensorflow as tf
from tensorflow import keras
import numpy as np
data = keras.datasets.imdb
(train_data, train_labels), (test_data, test_labels) = data.load_data(num_words=10000)
print(train_data[0])
# decode Data
word_index = data.get_word_index()
word_index = {key: (value+3) for key, value in word_index.items()}
word_index["<PAD>"] = 0
word_index["<START>"] = 1
word_index["<UNK>"] = 2
word_index["<UNUSED>"] = 3
reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])
train_data = keras.preprocessing.sequence.pad_sequences(train_data, value=word_index["<PAD>"], padding="post", maxlen=250)
test_data = keras.preprocessing.sequence.pad_sequences(test_data, value=word_index["<PAD>"], padding="post", maxlen=250)
def decode_review(text):
return " ".join([reverse_word_index.get(i, "?") for i in text])
print(decode_review(test_data[0]))
model = keras.Sequential()
model.add(keras.layers.Embedding(10000, 16))
model.add(keras.layers.GlobalAveragePooling1D())
model.add(keras.layers.Dense(16, activation="relu"))
model.add(keras.layers.Dense(1, activation="sigmoid"))
model.summary()
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
x_val = train_data[:10000]
x_train = train_data[10000:]
y_val = train_labels[:10000]
y_train = train_labels[10000:]
fitModel = model.fit(x_train, y_train, epochs=40, batch_size=512, validation_data=(x_val, y_val), verbose=1)
results = model.evaluate(test_data, test_labels)
print(results)
test_review = test_data[0]
predict = model.predict([test_review])
print("Review: ")
print(decode_review(test_review))
print("Prediction: " + str(predict[0]))
print("Actual: " + str(test_labels[0]))
print(results)
Does anyone have an idea how to fix this error?
I could solve this problem. model.predict doesn´t accept normal Lists. I just replaced it with an Numpy array.
test_review = test_data[0]
prediction = model.predict(np.array([test_review]))
print("Review: " + decode_review(test_review))
print("Prediction: " + str(prediction[0]))
print("Actual: " + str(test_labels[0]))
I have same issue, but i noticed that prediction does not work just for a single input, if you do the same with with all of the test data it works fine
prediction = model.predict(test_data)
for i in range(10):
print("Review", test_data[i])
print("Decoded: ", decode_review(test_data[i]))
print("Actual: ", str(test_labels[i]))
print("Predicted: ", str(prediction[i]))
It probably won't help you but i hope it does because i'm struggling with this all day long and really want to know how to fix it
I had the same issue but I solved it by appending the test sample to an empty list then predicting it from there and it worked

Multiple inputs and one output in TensorFlow

I am very new in using TF. I would like build a model with 6 input and 1 output. My code is available below.
import tensorflow as tf
import numpy as np
import pandas as pd
from tensorflow.keras import layers
import matplotlib.pyplot as plt
abalone_train = pd.read_csv("sipm2.csv",
names=["Hit1", "Time1", "Hit2", "Time2", "Hit3",
"Time3", "PosX"])
abalone_train.head()
abalone_features = abalone_train.copy()
abalone_labels = abalone_features.pop('PosX')
abalone_features = np.array(abalone_features)
print(abalone_features)
abalone_model = tf.keras.Sequential([
layers.Dense(64),
layers.Dense(64)
])
abalone_model.compile(loss='mean_squared_error', optimizer=tf.keras.optimizers.Adam(0.1), metrics=['mean_squared_error'])
history = abalone_model.fit(abalone_features, abalone_labels, epochs=100)
abalone_model.summary()
print("Finished training the model")
#plt.xlabel('Epoch Number')
#plt.ylabel("Loss Magnitude")
#plt.plot(history.history['loss'])
#plt.show()
posx_prediction = abalone_model.predict([75., 1., 75. ,1.3 ,66. ,2.])
print(posx_prediction)
The error I got is:
WARNING:tensorflow:Model was constructed with shape (None, 6) for input KerasTensor(type_spec=TensorSpec(shape=(None, 6), dtype=tf.float32, name='dense_input'), name='dense_input', description="created by layer 'dense_input'"), but it was called on an input with incompatible shape (None,).
Traceback (most recent call last):
File "/Users/sertac/Work/TensorFlow/test3.py", line 39, in <module>
posx_prediction = abalone_model.predict([75., 1., 75. ,1.3 ,66. ,2.])
File "/Users/sertac/Library/Python/3.9/lib/python/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/Users/sertac/Library/Python/3.9/lib/python/site-packages/tensorflow/python/framework/func_graph.py", line 1147, in autograph_handler
raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:
File "/Users/sertac/Library/Python/3.9/lib/python/site-packages/keras/engine/training.py", line 1801, in predict_function *
return step_function(self, iterator)
File "/Users/sertac/Library/Python/3.9/lib/python/site-packages/keras/engine/training.py", line 1790, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "/Users/sertac/Library/Python/3.9/lib/python/site-packages/keras/engine/training.py", line 1783, in run_step **
outputs = model.predict_step(data)
File "/Users/sertac/Library/Python/3.9/lib/python/site-packages/keras/engine/training.py", line 1751, in predict_step
return self(x, training=False)
File "/Users/sertac/Library/Python/3.9/lib/python/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/Users/sertac/Library/Python/3.9/lib/python/site-packages/keras/engine/input_spec.py", line 228, in assert_input_compatibility
raise ValueError(f'Input {input_index} of layer "{layer_name}" '
ValueError: Exception encountered when calling layer "sequential" (type Sequential).
Input 0 of layer "dense" is incompatible with the layer: expected min_ndim=2, found ndim=1. Full shape received: (None,)
Call arguments received:
• inputs=tf.Tensor(shape=(None,), dtype=float32)
• training=False
• mask=None
I would like appreciate if you could help me how to get prediction?
Thanks in advance.
Sipm2.csv
104.241,1.12209,67.51,1.30428,57.354,1.48099,3.9
104.796,1.12425,67.787,1.3103,58.31,1.47481,3.5
103.203,1.10605,67.039,1.29599,56.825,1.47089,3.0
84.857,1.12028,80.898,1.23234,62.08,1.39148,2.5
86.015,1.16361,80.656,1.20233,62.945,1.37363,1.5
83.987,1.20226,80.211,1.17464,62.002,1.36756,1.0
85.35,1.21761,81.777,1.15021,62.541,1.34058,0.5
For 6 input and 1 output, your model should look something like below.
abalone_model = tf.keras.Sequential([
tf.keras.Input(shape=(6,))
layers.Dense(64),
layers.Dense(1, activation='softmax')
])

Incompatible shapes while using triplet loss and pre-trained resnet

I am trying to use pre-trained resnet and fine-tune it using triplet loss. The following code I came up with is a combination of tutorials I found on the topic:
import pathlib
import tensorflow as tf
import tensorflow_addons as tfa
with tf.device('/cpu:0'):
INPUT_SHAPE = (32, 32, 3)
BATCH_SIZE = 16
data_dir = pathlib.Path('/home/user/dataset/')
base_model = tf.keras.applications.ResNet50V2(
weights='imagenet',
pooling='avg',
include_top=False,
input_shape=INPUT_SHAPE,
)
# following two lines are added after edit, originally it was model = base_model
head_model = tf.keras.layers.Lambda(lambda x: tf.math.l2_normalize(x, axis=1))(base_model.output)
model = tf.keras.Model(inputs=base_model.input, outputs=head_model)
datagen = tf.keras.preprocessing.image.ImageDataGenerator(
rotation_range=10,
zoom_range=0.1,
)
generator = datagen.flow_from_directory(
data_dir,
target_size=INPUT_SHAPE[:2],
batch_size=BATCH_SIZE,
seed=42,
)
model.compile(
optimizer=tf.keras.optimizers.Adam(0.001),
loss=tfa.losses.TripletSemiHardLoss(),
)
model.fit(
generator,
epochs=5,
)
Unfortunately after running the code I get the following error:
Found 4857 images belonging to 83 classes.
Epoch 1/5
Traceback (most recent call last):
File "ReID/external_process.py", line 35, in <module>
model.fit(
File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 108, in _method_wrapper
return method(self, *args, **kwargs)
File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/keras/engine/training.py", line 1098, in fit
tmp_logs = train_function(iterator)
File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 780, in __call__
result = self._call(*args, **kwds)
File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 840, in _call
return self._stateless_fn(*args, **kwds)
File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2829, in __call__
return graph_function._filtered_call(args, kwargs) # pylint: disable=protected-access
File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1843, in _filtered_call
return self._call_flat(
File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1923, in _call_flat
return self._build_call_outputs(self._inference_function.call(
File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 545, in call
outputs = execute.execute(
File "/home/user/videolytics/venv_python/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 1328 values, but the requested shape has 16
[[{{node TripletSemiHardLoss/PartitionedCall/Reshape}}]] [Op:__inference_train_function_13749]
Function call stack:
train_function
2020-10-23 22:07:09.094736: W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Failed precondition: Python interpreter state is not initialized. The process may be terminated.
[[{{node PyFunc}}]]
The dataset directory has 83 subdirectories, one per class and each of this subdirectories contains images of given class. The dimension 1328 in the error output is the batch size (16) times number of classes (83), and the dimension 16 is the batch size (both dimensions change accordingly if I change the BATCH_SIZE.
To be honest I do not really understand the error, so any solution or even any kind of indight where is the problem is deeply appreciated.
The problem is that the TripletSemiHardLoss expects
labels y_true to be provided as 1-D integer Tensor with shape [batch_size] of multi-class integer labels
but the flow_from_directory by default generate categorical labels; using class_mode="sparse" should fix the problem.

Adding dropout in middle of pre-trained network

I'm trying to add a Dropout layer in the middle of RestNet50 network,
here is the code I'm trying to run:
base_model = ResNet50(weights='imagenet', include_top=False)
layers = [l for l in base_model.layers]
conv3_block4_out = base_model.get_layer('conv3_block4_out')
x = conv3_block4_out.output
add = False
for layer in layers:
if layer.name == 'conv3_block4_out':
add = True
x = Dropout(0.5)(x)
continue
if not add:
continue
x = layer(x) # This is where the error is thrown, when the layer name is 'conv4_block1_0_conv'
x = GlobalAveragePooling2D()(x)
preds = Dense(196, activation='softmax')(x)
model = Model(inputs=base_model.input, outputs=preds)
What I'm doing is getting the conv3_block4_out layer, adding the Dropout to it, and then just continue stacking the leftovers.
The error I get is:
Traceback (most recent call last):
File "/machine-learning/test-proj/make_model_year_model.py", line 70, in <module>
x = layer(x)
File "/machine-learning/test-proj/venv/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 925, in __call__
return self._functional_construction_call(inputs, args, kwargs,
File "/machine-learning/test-proj/venv/lib/python3.8/site-packages/tensorflow/python/keras/engine/base_layer.py", line 1092, in _functional_construction_call
input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
File "/machine-learning/test-proj/venv/lib/python3.8/site-packages/tensorflow/python/keras/engine/input_spec.py", line 212, in assert_input_compatibility
raise ValueError(
ValueError: Input 0 of layer conv4_block1_0_conv is incompatible with the layer: expected axis -1 of input shape to have value 512 but received input with shape [None, None, None, 256]
This error is thrown after about 5 times that I successfully stuck the layers.
From what I know, dropout layers don't affect the shape of layers, then why do I get that error?
Can someone explain the reason and maybe suggest a change to make it work?

Rank error in tf.nn.dynamic_rnn

I am trying to build a CNN + RNN model and I am getting the following error.
Any help will be appreciated.
fc2 has shape (?,4096)
cell = tf.contrib.rnn.BasicLSTMCell(self.rnn_hidden_units)
stack = tf.contrib.rnn.MultiRNNCell([cell]*self.rnn_layers)
initial_state = cell.zero_state(self.batch_size, tf.float32)
initial_state = tf.identity(initial_state, name='initial_state')
outputs, _ = tf.nn.dynamic_rnn(stack, fc2,dtype=tf.float32)
File "rcnn.py", line 182, in model
outputs, _ = tf.nn.dynamic_rnn(stack, [fc2],dtype=tf.float32)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/rnn.py", line 574, in dynamic_rnn
dtype=dtype)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/rnn.py", line 637, in _dynamic_rnn_loop
for input_ in flat_input)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/rnn.py", line 637, in
for input_ in flat_input)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_shape.py", line 649, in with_rank_at_least
raise ValueError("Shape %s must have rank at least %d" % (self, rank))
ValueError: Shape (4096, ?) must have rank at least 3
Copying the answer of #jdehesa from his comment for better visibility:
The error seems fairly clear, tf.nn.dynamic_rnn expects a 3-dimensional tensor as input (i.e. rank 3), but fc2 has only two dimensions. The shape of fc2 should be something like (<batch_size>, <max_time>, <num_features>) (or (<max_time>, <batch_size>, <num_features>) if you pass time_major=True)
The default input of tf.nn.dynamic_rnn has a dimension of 3 (Batchsize, sequence_length, num_features). Since your num_features is 1 you can expand your fc_seq with
fc2 = tf.expand_dims(fc2, axis = 2)
and then use the code you have above.