(CRNN OCR) Error while training! Invalid Argument: sequence_length(0) <= 18 node ctc/CTCLoss - tensorflow

I use CRNN (CNN + RNN + CTC Loss) for my model on OCR. I'm using Tensorflow Keras
here's my code [from CTC Loss]:
labels = Input(name='the_labels', shape=[max_label_len], dtype='float32')
input_length = Input(name='input_length', shape=[1], dtype='int64')
label_length = Input(name='label_length', shape=[1], dtype='int64')
def ctc_lambda_func(args):
y_pred, labels, input_length, label_length = args
return K.ctc_batch_cost(labels, y_pred, input_length, label_length)
loss_out = Lambda(ctc_lambda_func, output_shape=(1,), name='ctc')([outputs, labels, input_length, label_length])
#model to be used at training time
model = Model(inputs=[inputs, labels, input_length, label_length], outputs=loss_out)
model.compile(loss={'ctc': lambda y_true, y_pred: y_pred}, optimizer = 'adam')
filepath="best_model.hdf5"
checkpoint = ModelCheckpoint(filepath=filepath, monitor='val_loss', verbose=1, save_best_only=True, mode='auto')
callbacks_list = [checkpoint]
training_img = np.array(training_img)
train_input_length = np.array(train_input_length)
train_label_length = np.array(train_label_length)
valid_img = np.array(valid_img)
valid_input_length = np.array(valid_input_length)
valid_label_length = np.array(valid_label_length)
Error here while training:
batch_size = 256
epochs = 10
model.fit(x=[training_img, train_padded_txt, train_input_length, train_label_length], y=np.zeros(len(training_img)),
batch_size=batch_size, epochs = epochs,
validation_data = ([valid_img, valid_padded_txt, valid_input_length, valid_label_length], [np.zeros(len(valid_img))]),
verbose = 1, callbacks = callbacks_list)
ERROR RESULT:
Train on 448 samples, validate on 49 samples
Epoch 1/10
---------------------------------------------------------------------------
InvalidArgumentError Traceback (most recent call last)
<ipython-input-15-1322212af569> in <module>()
4 batch_size=batch_size, epochs = epochs,
5 validation_data = ([valid_img, valid_padded_txt, valid_input_length, valid_label_length], [np.zeros(len(valid_img))]),
----> 6 verbose = 1, callbacks = callbacks_list)
7 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
58 ctx.ensure_initialized()
59 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
---> 60 inputs, attrs, num_outputs)
61 except core._NotOkStatusException as e:
62 if name is not None:
InvalidArgumentError: sequence_length(0) <= 18
[[node ctc/CTCLoss (defined at /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3009) ]] [Op:__inference_keras_scratch_graph_12073]
Function call stack:
keras_scratch_graph
My CRNN architecture is inspired by VGG-16, I'm using 13 conv layers and 3 bi-directional LSTM Layer. I am using CTC Loss and then I got error.
My data is 1000 text-image contains 4-8 words (700 for training&valid, 300 for testing)
if you want to view my code: here's my code using google colab.
https://colab.research.google.com/drive/1nMRNUsLDNrpgeTxPFQ4mhobnFdpbmwUx

I fixed this error. It's because of this!
Before:
# split the 700 data into validation and training dataset as 10% and 90% respectively
if i%10 == 0:
valid_orig_txt.append(txt)
valid_label_length.append(len(txt))
valid_input_length.append(31)
valid_img.append(img)
valid_txt.append(encode_to_labels(txt))
else:
orig_txt.append(txt)
train_label_length.append(len(txt))
train_input_length.append(31)
training_img.append(img)
training_txt.append(encode_to_labels(txt))
After:
# split the 700 data into validation and training dataset as 10% and 90% respectively
if i%10 == 0:
valid_orig_txt.append(txt)
valid_label_length.append(len(txt))
valid_input_length.append(18)
valid_img.append(img)
valid_txt.append(encode_to_labels(txt))
else:
orig_txt.append(txt)
train_label_length.append(len(txt))
train_input_length.append(18)
training_img.append(img)
training_txt.append(encode_to_labels(txt))

Related

embedding for categorical variable and pass to keras network

I have tabular data with two columns , one is input feature and anther one is target variable.
I have converted the input feature to embeddings and pass on to Keras model layer, getting error while executing fit method.
here is the code which i have used for this.
df is the input dataframe and contains two columns.
X = df.drop(columns = ['data_type']).copy()
y = df['data_type']
X_train, X_rem, y_train, y_rem = train_test_split(X,y, train_size=0.8)
X_valid, X_test, y_valid, y_test = train_test_split(X_rem,y_rem, test_size=0.5)
Y_train=pd.DataFrame({'data_type':y_train})
Y_valid=pd.DataFrame({'data_type':y_valid})
Y_test=pd.DataFrame({'data_type':y_test})
enc = OrdinalEncoder()
X_train[["data"]] = enc.fit_transform(X_train[["data"]])
X_valid[["data"]] = enc.fit_transform(X_valid[["data"]])
X_test[["data"]] = enc.fit_transform(X_test[["data"]])
X_train_actual = []
X_train_actual.append(np.array(X_train["data"]).reshape(-1,1))
X_valid_actual = []
X_valid_actual.append(np.array(X_valid["data"]).reshape(-1,1))
Y_train["data_type"] = LabelEncoder().fit_transform(Y_train["data_type"])
Y_valid["data_type"] = LabelEncoder().fit_transform(Y_valid["data_type"])
Y_test["data_type"] = LabelEncoder().fit_transform(Y_test["data_type"])
Y_train_actual= to_categorical(Y_train.to_numpy())
Y_valid_actual= to_categorical(Y_valid.to_numpy())
embedding_size = 50
categories = 7
inputs = Input(shape=(1,))
embedding_layer = Embedding(categories,
embedding_size,
embeddings_regularizer=regularizers.l2(0.01),
input_length=1)(inputs)
cat_input_layers = Reshape(target_shape=(embedding_size,))(embedding_layer)
building the layers is as follows.
dense_1 = Dense(512, activation='relu', kernel_regularizer=regularizers.l2(0.01))(cat_input_layers)
final_out = Dense(7, activation='softmax')(dense_1)
model = keras.Model(inputs=inputs,outputs=final_out)
optimizer = tf.keras.optimizers.Adam(learning_rate=0.00001)
model.compile(loss='mse', optimizer=optimizer, metrics=['mae', 'mse'])
checkpoint = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_path,
save_weights_only=True,
verbose=1)
early_stop = keras.callbacks.EarlyStopping(monitor='val_loss', patience=5)
history = model.fit(X_train_actual,
Y_train_actual,
batch_size=100,
epochs=500,
validation_data =(X_valid_actual, Y_valid_actual),
callbacks=[checkpoint, early_stop],
verbose=1)
getting the error when i am executing the fit call.
InvalidArgumentError Traceback (most recent call last)
Cell In [336], line 1
----> 1 history = model.fit(X_train_actual,
2 Y_train_actual,
3 batch_size=100,
4 epochs=500,
5 validation_data =(X_valid_actual, Y_valid_actual),
6 callbacks=[checkpoint, early_stop],
7 verbose=1)
File ~\Anaconda3\envs\keras-tf-tcb\lib\site-packages\keras\utils\traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
67 filtered_tb = _process_traceback_frames(e.__traceback__)
68 # To get the full stack trace, call:
69 # `tf.debugging.disable_traceback_filtering()`
---> 70 raise e.with_traceback(filtered_tb) from None
71 finally:
72 del filtered_tb
any idea, what exactly the issue,

tf.data.Dataset.from_generator : value error "not enough values to unpack (expected 2, got 1)"

I am currently trying to do a classification problem using a pre-trained transformer model. I wrote a custom generator using tf.data.Dataset.from_generator method. The model takes two inputs : input_id and attn_mask. While calling model.fit i am getting value error "not enough values to unpack (expected 2, got 1)" The received arguments list shows it got both input_id and attn_mask. Can anyone help me solve this?
import tensorflow.keras as keras
from tensorflow.keras.models import Model
from transformers import TFBertModel,BertConfig
def _input_fn():
x = (train_data.iloc[:,0:512]).to_numpy()
y = (train_data.iloc[:,512:516]).to_numpy()
attn = np.asarray(np.tile(attn_mask,x.shape[0]).reshape(-1,512))
def generator():
for s1, s2, l in zip(x, attn, y):
yield {"input_id": s1, "attn_mask": s2}, l
dataset = tf.data.Dataset.from_generator(generator, output_types=({"input_id": tf.int32, "attn_mask": tf.int32}, tf.int32))
#dataset = dataset.batch(2)
#dataset = dataset.shuffle
return dataset
train_data is the dataframe containing the training data(16000 x 516). Last four columns are one hot encoded labels. Since i am not using the Autotokenizer function, i am passing the attention mask as attn_mask.
my model
bert = 'bert-base-uncased'
config = BertConfig(dropout=0.2, attention_dropout=0.2)
config.output_hidden_states = False
transformer_model = TFBertModel.from_pretrained(bert, config = config)
input_ids_in = tf.keras.layers.Input(shape=(512), name='input_id', dtype='int32')
input_masks_in = tf.keras.layers.Input(shape=(512), name='attn_mask', dtype='int32')
embedding_layer = transformer_model(input_ids_in, attention_mask=input_masks_in)[0]
#cls_token = embedding_layer[:,0,:]
#X = tf.keras.layers.BatchNormalization()(cls_token)
X = tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(50, return_sequences=True, dropout=0.1, recurrent_dropout=0.1))(embedding_layer)
X = tf.keras.layers.GlobalMaxPool1D()(X)
#X = tf.keras.layers.BatchNormalization()(X)
X = tf.keras.layers.Dense(50, activation='relu')(X)
X = tf.keras.layers.Dropout(0.2)(X)
X = tf.keras.layers.Dense(4, activation='softmax')(X)
model = tf.keras.Model(inputs=[input_ids_in, input_masks_in], outputs = X)
for layer in model.layers[:3]:
layer.trainable = False
optimizer = tf.keras.optimizers.Adam(0.001, beta_1=0.9, beta_2=0.98,
epsilon=1e-9)
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['categorical_accuracy'])
epochs = 1
batch_size =2
history = model.fit(_input_fn(), epochs= epochs, batch_size= batch_size, verbose=2)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
/tmp/ipykernel_16908/300834086.py in <module>
2 batch_size =2
3 #history = model.fit(trainDataGenerator(batch_size), epochs= epochs, validation_data=valDataGenerator(batch_size), verbose=2) #
----> 4 history = model.fit(_input_fn(), epochs= epochs, batch_size= batch_size, verbose=2) #validation_data=val_ds,
~/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb
~/.local/lib/python3.8/site-packages/transformers/models/bert/modeling_tf_bert.py in call(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict, training, **kwargs)
1124 kwargs_call=kwargs,
1125 )
-> 1126 outputs = self.bert(
1127 input_ids=inputs["input_ids"],
1128 attention_mask=inputs["attention_mask"],
~/.local/lib/python3.8/site-packages/transformers/models/bert/modeling_tf_bert.py in call(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict, training, **kwargs)
771 raise ValueError("You have to specify either input_ids or inputs_embeds")
772
--> 773 batch_size, seq_length = input_shape
774
775 if inputs["past_key_values"] is None:
ValueError: Exception encountered when calling layer "bert" (type TFBertMainLayer).
not enough values to unpack (expected 2, got 1)
Call arguments received:
• input_ids=tf.Tensor(shape=(512,), dtype=int32)
• attention_mask=tf.Tensor(shape=(512,), dtype=int32)
• token_type_ids=None
• position_ids=None
• head_mask=None
• inputs_embeds=None
• encoder_hidden_states=None
• encoder_attention_mask=None
• past_key_values=None
• use_cache=True
• output_attentions=False
• output_hidden_states=False
• return_dict=True
• training=True
• kwargs=<class 'inspect._empty'>
Edit:
Adding the output of calling _input_fn()
<FlatMapDataset shapes: ({input_id: <unknown>, attn_mask: <unknown>}, <unknown>), types: ({input_id: tf.int32, attn_mask: tf.int32}, tf.int32)>
I solved this error by batching my tf.data.Dataset. This gave the TensorSpec in my dataset a shape that had two values to unpack ->
TensorSpec(shape=(16, 200)...
This is what the error refers to.
Solution
print(train_ds) #Before Batching
new_train_ds = train_ds.batch(16, drop_remainder=True)
print(new_train_ds) #After Batching
# Before Batching
<MapDataset element_spec=({'input_ids': TensorSpec(shape=(200,),
dtype=tf.float64, name=None), 'attention_mask': TensorSpec(shape=
(200,), dtype=tf.float64, name=None)}, TensorSpec(shape=(11,),
dtype=tf.float64, name=None))>
# After Batching
<BatchDataset element_spec=({'input_ids': TensorSpec(shape=(16, 200),
dtype=tf.float64, name=None), 'attention_mask': TensorSpec(shape=(16,
200), dtype=tf.float64, name=None)}, TensorSpec(shape=(16, 11),
dtype=tf.float64, name=None))>

ValueError: The `batch_size` argument must not be specified for the given input type

I have created dummy data for an autoencoder model using tf.data.dataset. I am using tensorflow-1.15.x, keras 2.3.1 and numpy 1.19.5 for running on my GPU.
N = 5000
H = 128
W = 128
C = 2
train_data = np.random.randn(N,H,W,C)
train_data = tf.convert_to_tensor(train_data)
test_data = np.random.randn(500,H,W,C)
test_data = tf.convert_to_tensor(test_data)
batch_size = 1
train_dataset = tf.data.Dataset.from_tensor_slices((train_data, train_data))
train_dataset = train_dataset.batch(batch_size, drop_remainder=True)
test_dataset = tf.data.Dataset.from_tensor_slices((test_data, test_data))
test_dataset = test_dataset.batch(batch_size, drop_remainder=True)
But while fitting this data on the model, I get the following error -
epochs = 5
rms = RMSprop(learning_rate=0.00002,
rho=0.9,
epsilon=1e-07)
model.compile(loss='mean_squared_error', optimizer=rms, metrics=['mean_squared_error'])
callback = tf.keras.callbacks.EarlyStopping(monitor='loss', patience=3)
logs = "logs/"
tboard_callback = tf.keras.callbacks.TensorBoard(log_dir = logs,
histogram_freq = 1)
history = model.fit(train_dataset, epochs=epochs, batch_size=batch_size, callbacks=[callback, tboard_callback],
validation_data=test_dataset)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-25-47134a802115> in <module>()
15
16 history = model.fit(train_dataset, epochs=epochs, batch_size=batch_size, callbacks=[callback, tboard_callback],
---> 17 validation_data=test_dataset)
2 frames
/tensorflow-1.15.2/python3.7/tensorflow_core/python/keras/engine/training.py in _validate_or_infer_batch_size(self, batch_size, steps, x)
1813 'The `batch_size` argument must not be specified for the given '
1814 'input type. Received input: {}, batch_size: {}'.format(
-> 1815 x, batch_size))
1816 return
1817
ValueError: The `batch_size` argument must not be specified for the given input type. Received input: <DatasetV1Adapter shapes: ((1, 128, 128, 2), (1, 128, 128, 2)), types: (tf.float64, tf.float64)>, batch_size: 1
Answers to other questions on this error suggest batching both train and validation datasets, but I have batched both already. What could be causing this error? Thanks.
Remove the batch_size in model.fit()
history = model.fit(train_dataset, epochs=epochs, callbacks=[callback, tboard_callback],validation_data=test_dataset)

TypeError when trying to use EarlyStopping with f1-metric as stopping criterion

I want for training a CNN with Early Stopping and want to use the f1-metric as stopping criterion.
When I compile the code for the CNN model I get the a TypeError as error message.
I'm still using Tensorflow 1.4 would like to avoid an upgrade to 2.0, because I have in mind that my previous code doesn't work anymore.
The error message is as follows:
TypeError Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py in _num_samples(x)
158 try:
--> 159 return len(x)
160 except TypeError:
14 frames
/usr/local/lib/python3.6/dist-
packages/tensorflow_core/python/framework/ops.py in __len__(self)
740 "Please call `x.shape` rather than `len(x)` for "
--> 741 "shape information.".format(self.name))
742
TypeError: len is not well defined for symbolic Tensors. (dense_16_target:0) Please call `x.shape` rather than `len(x)` for shape information.
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
<ipython-input-44-cd3da16e057c> in <module>()
----> 1 model = model_cnn(False,False, False,True,6, 0.2, 0.5)
2 X_train, X_val, y_train, y_val = split_data(X_train, y_train,1)
3 cnn, ep = train_model_es(model, X_train, y_train, X_val, y_val, X_test, y_test, 50, 500,1)
<ipython-input-42-d275d9c69c03> in model_cnn(spat, extra_pool, avg_pool, cw, numb_conv, drop_conv, drop_dense)
36 if cw == True:
37 print("sparse categorical crossentropy")
---> 38 model.compile(loss="sparse_categorical_crossentropy", optimizer=Adam(), metrics=['accuracy', f1_metric])
39 #model.compile(loss="sparse_categorical_crossentropy", optimizer=Adam(), metrics=['accuracy'])
40 print("nothing")
/usr/local/lib/python3.6/dist-packages/keras/engine/training.py in compile(self, optimizer, loss, metrics, loss_weights, sample_weight_mode, weighted_metrics, target_tensors, **kwargs)
452 output_metrics = nested_metrics[i]
453 output_weighted_metrics = nested_weighted_metrics[i]
--> 454 handle_metrics(output_metrics)
455 handle_metrics(output_weighted_metrics, weights=weights)
456
/usr/local/lib/python3.6/dist-packages/keras/engine/training.py in handle_metrics(metrics, weights)
421 metric_result = weighted_metric_fn(y_true, y_pred,
422 weights=weights,
--> 423 mask=masks[i])
424
425 # Append to self.metrics_names, self.metric_tensors,
/usr/local/lib/python3.6/dist-packages/keras/engine/training_utils.py in weighted(y_true, y_pred, weights, mask)
426 """
427 # score_array has ndim >= 2
--> 428 score_array = fn(y_true, y_pred)
429 if mask is not None:
430 # Cast the mask to floatX to avoid float64 upcasting in Theano
<ipython-input-9-b21dc3bd89a6> in f1_metric(y_test, y_pred)
1 def f1_metric(y_test, y_pred):
----> 2 f1 = f1_score(y_test, y_pred, average='macro')
3 return f1
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py in f1_score(y_true, y_pred, labels, pos_label, average, sample_weight, zero_division)
1097 pos_label=pos_label, average=average,
1098 sample_weight=sample_weight,
-> 1099 zero_division=zero_division)
1100
1101
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py in fbeta_score(y_true, y_pred, beta, labels, pos_label, average, sample_weight, zero_division)
1224 warn_for=('f-score',),
1225 sample_weight=sample_weight,
-> 1226 zero_division=zero_division)
1227 return f
1228
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py in precision_recall_fscore_support(y_true, y_pred, beta, labels, pos_label, average, warn_for, sample_weight, zero_division)
1482 raise ValueError("beta should be >=0 in the F-beta score")
1483 labels = _check_set_wise_labels(y_true, y_pred, average, labels,
-> 1484 pos_label)
1485
1486 # Calculate tp_sum, pred_sum, true_sum ###
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py in _check_set_wise_labels(y_true, y_pred, average, labels, pos_label)
1299 str(average_options))
1300
-> 1301 y_type, y_true, y_pred = _check_targets(y_true, y_pred)
1302 present_labels = unique_labels(y_true, y_pred)
1303 if average == 'binary':
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py in _check_targets(y_true, y_pred)
78 y_pred : array or indicator matrix
79 """
---> 80 check_consistent_length(y_true, y_pred)
81 type_true = type_of_target(y_true)
82 type_pred = type_of_target(y_pred)
/usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py in check_consistent_length(*arrays)
206 """
207
--> 208 lengths = [_num_samples(X) for X in arrays if X is not None]
209 uniques = np.unique(lengths)
210 if len(uniques) > 1:
/usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py in <listcomp>(.0)
206 """
207
--> 208 lengths = [_num_samples(X) for X in arrays if X is not None]
209 uniques = np.unique(lengths)
210 if len(uniques) > 1:
/usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py in _num_samples(x)
159 return len(x)
160 except TypeError:
--> 161 raise TypeError(message)
162
163
TypeError: Expected sequence or array-like, got <class 'tensorflow.python.framework.ops.Tensor'>
And here is the relevant code:
def f1_metric(y_test, y_pred):
f1 = f1_score(y_test, y_pred, average='macro')
return f1
def train_model_es(model, X, y, X_val, y_val, X_test, y_test):
es = EarlyStopping(monitor='f1_metric', mode='max', patience=20, restore_best_weights=True)
y = np.argmax(y, axis=1)
y_val = np.argmax(y_val, axis=1)
y_test = np.argmax(y_test, axis=1)
class_weights = class_weight.compute_class_weight('balanced', np.unique(y), y)
class_weights = dict(enumerate(class_weights))
history = model.fit(X, y, class_weight=class_weights, batch_size=32,epochs=20, verbose=1,
validation_data=(X_val, y_val), callbacks=[es])
def model_cnn():
model = Sequential()
model.add(Conv2D(32, kernel_size=(3,3), input_shape=(28,28,1),padding='same'))
model.add(BatchNormalization())
model.add(ELU())
model.add(Conv2D(32, kernel_size=(3,3), padding='same'))
model.add(BatchNormalization())
model.add(ELU())
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(256))
model.add(BatchNormalization())
model.add(ELU())
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))
model.compile(loss="sparse_categorical_crossentroy",optimizer=Adam(),metrics=["accuracy",f1_metric])
return model
Does anyone have a tip on how to fix this error message.
Many thanks for every hint
As the error message suggests,
Error 1) You are doing a len() operation on symbolic tensor. You cannot do that operation on symbolic tensor. You can find difference between a variable tensor and symbolic tensor here.
Error 2) You are using a tensor for the operation which expects array as input. Can you please convert y_true and y_pred from tensor to array and use in the f1_score and other operations.
Example - To convert tensor to array
%tensorflow_version 1.x
print(tf.__version__)
import tensorflow as tf
import numpy as np
x = tf.constant([1,2,3,4,5,6])
print("Type of x:",x)
with tf.Session() as sess:
y = np.array(x.eval())
print("Type of y:",y.shape,y)
Output -
1.15.2
Type of x: Tensor("Const_24:0", shape=(6,), dtype=int32)
Type of y: (6,) [1 2 3 4 5 6]

Keras 'Tensor' object has no attribute 'ndim'

I am trying to implement a siamese network (by using triplet loss method). I just cannot make it train. After many tries, I guess that I have my problem in the generator (where I prepare the input data stream for training) but I could not localize the problem so far. HELP! :)
Here is my model definition (it is based on ResNet50).
model = ResNet50(weights='imagenet')
model.layers.pop()
for layer in model.layers:
layer.trainable = False
x = model.get_layer('flatten_1').output
model_out = Dense(128, activation='sigmoid', name='model_out')(x)
new_model = Model(inputs=model.input, outputs=model_out)
Here I define the model to be trained:
anchor_in = Input(shape=(224, 224, 3))
positive_in = Input(shape=(224, 224, 3))
negative_in = Input(shape=(224, 224, 3))
anchor_out = new_model(anchor_in)
positive_out = new_model(positive_in)
negative_out = new_model(negative_in)
merged_vector = concatenate([anchor_out, positive_out, negative_out], axis=-1)
# Define the model to be trained
siamese_model = Model(inputs=[anchor_in, positive_in, negative_in],
outputs=merged_vector)
siamese_model.compile(optimizer=Adam(lr=.001), loss=triplet_loss)
To be able to train the model. I need to feed it with data with a generator and here is how I define it:
(Note that I intentionally put only 1 picture in each folder just to start with.. I will later increase the # pictures in each folder if I can make it work.)
def generator_three_imgs():
train_path = r'C:\Users\jon\Desktop\AI_anaconda\face_recognition\dataset\train\E'
generator1 = ImageDataGenerator()
generator2 = ImageDataGenerator()
generator3 = ImageDataGenerator()
anchor_train_batches = generator1.flow_from_directory(train_path+'\Ed_A', target_size=(224, 224), batch_size=1)
positive_train_batches = generator2.flow_from_directory(train_path+'\Ed_P', target_size=(224, 224), batch_size=1)
negative_train_batches = generator3.flow_from_directory(train_path+'\Ed_N', target_size=(224, 224), batch_size=1)
while True:
anchor_imgs, anchor_labels = anchor_train_batches.next()
positive_imgs, positive_labels = positive_train_batches.next()
negative_imgs, negative_labels = negative_train_batches.next()
concat_out = concatenate([anchor_out, positive_out, negative_out], axis=-1)
yield ([anchor_imgs, positive_imgs, negative_imgs],
concat_out)
And finally, I try totrain the model as following:
siamese_model.fit_generator(generator_three_imgs(),
steps_per_epoch=1, epochs=15, verbose=2)
which fails right away by giving the following error message:
Epoch 1/15
Found 1 images belonging to 1 classes.
Found 1 images belonging to 1 classes.
Found 1 images belonging to 1 classes.
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-23-7537b4595917> in <module>()
1 siamese_model.fit_generator(generator_three_imgs(),
----> 2 steps_per_epoch=1, epochs=15, verbose=2)
~\Anaconda3\envs\tensorflow\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
89 warnings.warn('Update your `' + object_name +
90 '` call to the Keras 2 API: ' + signature, stacklevel=2)
---> 91 return func(*args, **kwargs)
92 wrapper._original_function = func
93 return wrapper
~\Anaconda3\envs\tensorflow\lib\site-packages\keras\engine\training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
2228 outs = self.train_on_batch(x, y,
2229 sample_weight=sample_weight,
-> 2230 class_weight=class_weight)
2231
2232 if not isinstance(outs, list):
~\Anaconda3\envs\tensorflow\lib\site-packages\keras\engine\training.py in train_on_batch(self, x, y, sample_weight, class_weight)
1875 x, y,
1876 sample_weight=sample_weight,
-> 1877 class_weight=class_weight)
1878 if self.uses_learning_phase and not isinstance(K.learning_phase(), int):
1879 ins = x + y + sample_weights + [1.]
~\Anaconda3\envs\tensorflow\lib\site-packages\keras\engine\training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, check_array_lengths, batch_size)
1478 output_shapes,
1479 check_batch_axis=False,
-> 1480 exception_prefix='target')
1481 sample_weights = _standardize_sample_weights(sample_weight,
1482 self._feed_output_names)
~\Anaconda3\envs\tensorflow\lib\site-packages\keras\engine\training.py in _standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix)
74 data = data.values if data.__class__.__name__ == 'DataFrame' else data
75 data = [data]
---> 76 data = [np.expand_dims(x, 1) if x is not None and x.ndim == 1 else x for x in data]
77
78 if len(data) != len(names):
~\Anaconda3\envs\tensorflow\lib\site-packages\keras\engine\training.py in <listcomp>(.0)
74 data = data.values if data.__class__.__name__ == 'DataFrame' else data
75 data = [data]
---> 76 data = [np.expand_dims(x, 1) if x is not None and x.ndim == 1 else x for x in data]
77
78 if len(data) != len(names):
AttributeError: 'Tensor' object has no attribute 'ndim'
There is maybe someone out there who has more experience on this..?
I recognized that I pasted the wrong data above. But that still would not solve the problem. The solution Daniel Möller suggested below solved the problem.
There was a typo in the content of generator function above. The corrected one (incl. Daniel's suggestion below) looks as folowing:
def generator_three_imgs(batch_size=1):
train_path = r'C:\Users\sinthes\Desktop\AI_anaconda\face_recognition\dataset\train\E'
generator1 = ImageDataGenerator()
generator2 = ImageDataGenerator()
generator3 = ImageDataGenerator()
anchor_train_batches = generator1.flow_from_directory(train_path+'\Ed_A', target_size=(224, 224), batch_size=batch_size)
positive_train_batches = generator2.flow_from_directory(train_path+'\Ed_P', target_size=(224, 224), batch_size=batch_size)
negative_train_batches = generator3.flow_from_directory(train_path+'\Ed_N', target_size=(224, 224), batch_size=batch_size)
while True:
anchor_imgs, anchor_labels = anchor_train_batches.next()
positive_imgs, positive_labels = positive_train_batches.next()
negative_imgs, negative_labels = negative_train_batches.next()
concat_out = np.concatenate([anchor_labels, positive_labels, negative_labels], axis=-1)
yield ([anchor_imgs, positive_imgs, negative_imgs],
concat_out)
Yes, your generator is using a keras function (for tensors) to concatenate numpy data.
Use numpy.concatenate([anchor_labels, positive_labels, negative_labels], axis=-1).