Related
I have two hidden layers of dimension 50 from 2 different autoencoder models. The shape is [None,50] for both of them. But when executing the following code:
concat_layer = Concatenate()([_1.layers[7], _2.layers[11]])
softmax_layer = keras.layers.Dense(2, activation='softmax')(concat_layer)
sum_model = keras.models.Model(inputs=[_1_x_train, _2_x_train], outputs=softmax_layer)
sum_model.compile(optimizer='Adam', loss='mse')
I get the Error: TypeError: 'NoneType' object is not subscriptable for Concatenate()([_1.layers[7], _2.layers[11]])
Edit: Here is the layer structure of the two models.
_1 summary:
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 500)] 0
_________________________________________________________________
dense (Dense) (None, 250) 125250
_________________________________________________________________
dropout (Dropout) (None, 250) 0
_________________________________________________________________
dense_1 (Dense) (None, 100) 25100
_________________________________________________________________
dropout_1 (Dropout) (None, 100) 0
_________________________________________________________________
dense_2 (Dense) (None, 50) 5050
_________________________________________________________________
dropout_2 (Dropout) (None, 50) 0
_________________________________________________________________
dense_3 (Dense) (None, 50) 2550
_________________________________________________________________
dense_4 (Dense) (None, 100) 5100
_________________________________________________________________
dense_5 (Dense) (None, 250) 25250
_________________________________________________________________
dense_6 (Dense) (None, 500) 125500
=================================================================
_2 summary:
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 24765)] 0
_________________________________________________________________
dense (Dense) (None, 5000) 123830000
_________________________________________________________________
dropout (Dropout) (None, 5000) 0
_________________________________________________________________
dense_1 (Dense) (None, 2500) 12502500
_________________________________________________________________
dropout_1 (Dropout) (None, 2500) 0
_________________________________________________________________
dense_2 (Dense) (None, 1000) 2501000
_________________________________________________________________
dropout_2 (Dropout) (None, 1000) 0
_________________________________________________________________
dense_3 (Dense) (None, 500) 500500
_________________________________________________________________
dense_4 (Dense) (None, 250) 125250
_________________________________________________________________
dense_5 (Dense) (None, 100) 25100
_________________________________________________________________
dense_6 (Dense) (None, 50) 5050
_________________________________________________________________
dense_7 (Dense) (None, 50) 2550
_________________________________________________________________
dense_8 (Dense) (None, 100) 5100
_________________________________________________________________
dense_9 (Dense) (None, 250) 25250
_________________________________________________________________
dense_10 (Dense) (None, 500) 125500
_________________________________________________________________
dense_11 (Dense) (None, 1000) 501000
_________________________________________________________________
dense_12 (Dense) (None, 2500) 2502500
_________________________________________________________________
dense_13 (Dense) (None, 5000) 12505000
_________________________________________________________________
dense_14 (Dense) (None, 24765) 123849765
=================================================================
I have to add something here because otherwise the changes will not be accepted because its only 'code' i added.
This error indicates, it is trying to get subscript (object[index]) of an object which is NoneType.
The input to atf.keras.layers.Concatenate() layer should be a tensor. But you have passed layer instances. So instead of passing layers, pass their output like this:
concat_layer = Concatenate()([_1.layers[7].output, _2.layers[11].output])
In addition, your model definition should change, since you have passed input data as inputs, instead of the first layers. Get the models input layer by model.input. So modified code should be like this:
#sum_model = keras.models.Model(inputs=[_1_x_train, _2_x_train], outputs=softmax_layer)
sum_model = keras.models.Model(inputs=[_1.input, _2.input], outputs=softmax_layer)
You should pass input data to model.fit().
When I change my CNN model's activation from ReLU to LeakyReLU, both training and validation losses become nan. How can I resolve this issue?
Here is my model's summary:
Shape of all data: (1889, 10801)
Shape of X_train: (1322, 10800, 1)
Shape of Y_train: (1322, 3)
Shape of X_test: (567, 10800, 1)
Shape of y_test: (567, 3)
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv1d_48 (Conv1D) (None, 10721, 128) 10368
_________________________________________________________________
batch_normalization_48 (Batc (None, 10721, 128) 512
_________________________________________________________________
activation_48 (Activation) (None, 10721, 128) 0
_________________________________________________________________
max_pooling1d_41 (MaxPooling (None, 5360, 128) 0
_________________________________________________________________
dropout_29 (Dropout) (None, 5360, 128) 0
_________________________________________________________________
conv1d_49 (Conv1D) (None, 5357, 128) 65664
_________________________________________________________________
batch_normalization_49 (Batc (None, 5357, 128) 512
_________________________________________________________________
activation_49 (Activation) (None, 5357, 128) 0
_________________________________________________________________
max_pooling1d_42 (MaxPooling (None, 2678, 128) 0
_________________________________________________________________
dropout_30 (Dropout) (None, 2678, 128) 0
_________________________________________________________________
conv1d_50 (Conv1D) (None, 2675, 128) 65664
_________________________________________________________________
batch_normalization_50 (Batc (None, 2675, 128) 512
_________________________________________________________________
activation_50 (Activation) (None, 2675, 128) 0
_________________________________________________________________
max_pooling1d_43 (MaxPooling (None, 1337, 128) 0
_________________________________________________________________
dropout_31 (Dropout) (None, 1337, 128) 0
_________________________________________________________________
conv1d_51 (Conv1D) (None, 1334, 256) 131328
_________________________________________________________________
batch_normalization_51 (Batc (None, 1334, 256) 1024
_________________________________________________________________
activation_51 (Activation) (None, 1334, 256) 0
_________________________________________________________________
max_pooling1d_44 (MaxPooling (None, 667, 256) 0
_________________________________________________________________
global_max_pooling1d_6 (Glob (None, 256) 0
_________________________________________________________________
dense_20 (Dense) (None, 512) 131584
_________________________________________________________________
dense_21 (Dense) (None, 3) 1539
=================================================================
The model was compiled as follows:
n_lr = 1e-5
weight_decay = 1e-4
adam = Adam(lr=n_lr, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=weight_decay)
model.compile(loss='categorical_crossentropy', optimizer=adam, metrics=['acc'])
p.s. I'm aware of that this issue has already been reported to Keras on GitHub; but it has not got any responses as of posting this question.
Hi wish to enquire some help regarding neural networks, i am doing a school project whereby i am required to build deep fake detection neural network. I am unsure on why by adding more layers into the neural. My Accuracy during training goes from 0.7 in the first epoch and jumps to 1.0 in the second to fifth epoch which is overfittin and the loss value goes to a weird number, Wish to seek advice on how i could adjust the neural network to suit deepfake detections.
Thank you all for the time in reading
import tensorflow as tf
from keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D, Dropout
model = Sequential()
model.add(Conv2D(32, (3,3), input_shape = (256,256,3)))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (3,3)))
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(64, (3,3)))
model.add(Activation("relu"))
model.add(Dropout(0.20))
model.add(Conv2D(64, (3,3)))
model.add(Activation("relu"))
model.add(Dropout(0.20))
model.add(Conv2D(64, (3,3)))
model.add(Activation("relu"))
model.add(Dropout(0.20))
model.add(Conv2D(64, (3,3)))
model.add(Activation("relu"))
#flatten the layer conv 2d dense is 1d data set
model.add(Flatten()) #convets 3d feature maps to 1D feature Vectors
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dense(1))
model.add(Activation('sigmoid'))
model.compile(loss="binary_crossentropy", optimizer="adam", metrics=['accuracy'])
model.fit(X, y, batch_size=32, epochs=5)
Model Summary
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 254, 254, 32) 896
_________________________________________________________________
activation (Activation) (None, 254, 254, 32) 0
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 127, 127, 32) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 125, 125, 64) 18496
_________________________________________________________________
activation_1 (Activation) (None, 125, 125, 64) 0
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 62, 62, 64) 0
_________________________________________________________________
conv2d_2 (Conv2D) (None, 60, 60, 64) 36928
_________________________________________________________________
activation_2 (Activation) (None, 60, 60, 64) 0
_________________________________________________________________
dropout (Dropout) (None, 60, 60, 64) 0
_________________________________________________________________
conv2d_3 (Conv2D) (None, 58, 58, 64) 36928
_________________________________________________________________
activation_3 (Activation) (None, 58, 58, 64) 0
_________________________________________________________________
dropout_1 (Dropout) (None, 58, 58, 64) 0
_________________________________________________________________
conv2d_4 (Conv2D) (None, 56, 56, 64) 36928
_________________________________________________________________
activation_4 (Activation) (None, 56, 56, 64) 0
_________________________________________________________________
dropout_2 (Dropout) (None, 56, 56, 64) 0
_________________________________________________________________
conv2d_5 (Conv2D) (None, 54, 54, 64) 36928
_________________________________________________________________
activation_5 (Activation) (None, 54, 54, 64) 0
_________________________________________________________________
flatten (Flatten) (None, 186624) 0
_________________________________________________________________
dense (Dense) (None, 64) 11944000
_________________________________________________________________
activation_6 (Activation) (None, 64) 0
_________________________________________________________________
dense_1 (Dense) (None, 1) 65
_________________________________________________________________
activation_7 (Activation) (None, 1) 0
=================================================================
Total params: 12,111,169
Trainable params: 12,111,169
Non-trainable params: 0
_________________________________________________________________
You have to specify more stuff inside each layer, not only the size and number of filters. This will help you to increase the model performance.
For example, you could use adam from keras_optimizers, which will help to increase the accuracy during training the model. Also, l2 from keras.regularizers will help you to reduce overfitting. Which means you can't increase the accuracy just by increasing the epochs, you must first build a good model before starting the training
I'm trying to plot a confusion matrix between the predicted test labels and the actual ones, but I'm getting this error
ValueError: Found input variables with inconsistent numbers of samples: [1263, 12630]
Dataset: GTSRB
Code used
Image augmentation
train_datagen = ImageDataGenerator(rescale=1./255,
rotation_range=20,
horizontal_flip=True,
width_shift_range=0.1,
height_shift_range=0.1,
shear_range=0.01,
zoom_range=[0.9, 1.25],
brightness_range=[0.5, 1.5])
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator and test_generator
batch_size = 10
train_generator = train_datagen.flow_from_directory(
directory=train_path,
target_size=(224, 224),
color_mode="rgb",
batch_size=batch_size,
class_mode="categorical",
shuffle=True,
seed=42
)
test_generator = test_datagen.flow_from_directory(
directory=test_path,
target_size=(224, 224),
color_mode="rgb",
batch_size=batch_size,
class_mode="categorical",
shuffle=False,
seed=42
)
Output of that code
Found 39209 images belonging to 43 classes.
Found 12630 images belonging to 43 classes.
Then, I used a VGG-16 model and replaced the latest Dense layer with a Dense(43, activation='softmax')
Model summary
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
block1_conv1 (Conv2D) (None, 224, 224, 64) 1792
_________________________________________________________________
block1_conv2 (Conv2D) (None, 224, 224, 64) 36928
_________________________________________________________________
block1_pool (MaxPooling2D) (None, 112, 112, 64) 0
_________________________________________________________________
block2_conv1 (Conv2D) (None, 112, 112, 128) 73856
_________________________________________________________________
block2_conv2 (Conv2D) (None, 112, 112, 128) 147584
_________________________________________________________________
block2_pool (MaxPooling2D) (None, 56, 56, 128) 0
_________________________________________________________________
block3_conv1 (Conv2D) (None, 56, 56, 256) 295168
_________________________________________________________________
block3_conv2 (Conv2D) (None, 56, 56, 256) 590080
_________________________________________________________________
block3_conv3 (Conv2D) (None, 56, 56, 256) 590080
_________________________________________________________________
block3_pool (MaxPooling2D) (None, 28, 28, 256) 0
_________________________________________________________________
block4_conv1 (Conv2D) (None, 28, 28, 512) 1180160
_________________________________________________________________
block4_conv2 (Conv2D) (None, 28, 28, 512) 2359808
_________________________________________________________________
block4_conv3 (Conv2D) (None, 28, 28, 512) 2359808
_________________________________________________________________
block4_pool (MaxPooling2D) (None, 14, 14, 512) 0
_________________________________________________________________
block5_conv1 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_conv2 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_conv3 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_pool (MaxPooling2D) (None, 7, 7, 512) 0
_________________________________________________________________
flatten (Flatten) (None, 25088) 0
_________________________________________________________________
fc1 (Dense) (None, 4096) 102764544
_________________________________________________________________
fc2 (Dense) (None, 4096) 16781312
_________________________________________________________________
predictions (Dense) (None, 1000) 4097000
_________________________________________________________________
dense_1 (Dense) (None, 43) 43043
=================================================================
Total params: 138,400,587
Trainable params: 43,043
Non-trainable params: 138,357,544
_________________________________________________________________
Compile the model
my_sgd = SGD(lr=0.01)
model.compile(
optimizer=my_sgd,
loss='categorical_crossentropy',
metrics=['accuracy']
)
Train the model
STEP_SIZE_TRAIN=train_generator.n//train_generator.batch_size
epochs=10
model.fit_generator(generator=train_generator,
steps_per_epoch=STEP_SIZE_TRAIN,
epochs=epochs,
verbose=1
)
Predictions
STEP_SIZE_TEST=test_generator.n//test_generator.batch_size
test_generator.reset()
predictions = model.predict_generator(test_generator, steps=STEP_SIZE_TEST, verbose=1)
Output
1263/1263 [==============================] - 229s 181ms/step
Predictions shape
print(predictions.shape)
(12630, 43)
Getting the test_data and test_labels
test_data = []
test_labels = []
batch_index = 0
while batch_index <= test_generator.batch_index:
data = next(test_generator)
test_data.append(data[0])
test_labels.append(data[1])
batch_index = batch_index + 1
test_data_array = np.asarray(test_data)
test_labels_array = np.asarray(test_labels)
Shape of test_data_array and test_labels_array
test_data_array.shape
(1263, 10, 224, 224, 3)
test_labels_array.shape
(1263, 10, 43)
Confusion Matrix
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(test_labels_array, predictions)
I get the output
ValueError: Found input variables with inconsistent numbers of samples: [1263, 12630]
I understand that this error is because the test_labels_array size isn't equal to the predictions; 1263 and 12630 respectively, but I don't really know what I'm doing wrong.
Any help would be much appreciated.
PS: If anyone has any tips on how to increase the training accuracy while we're at it, that would be brilliant.
Thanks!
You should reshape test_data_array and test_labels_array as follows:
data_count, batch_count, w, h, c = test_data_array.shape
test_data_array=np.reshape(test_data_array, (data_count*batch_count, w, h, c))
test_labels_array = np.reshape(test_labels_array , (data_count*batch_count, -1))
the way you are appending the results of test_generator is the reason. In fact the first call of your test_generator will generate 10 data with shape of (224, 224, 3). For the next call again your test_generator will generate 10 data with shape of (224, 224, 3). So now you should have 20 data of shape (224, 224, 3) while the way you are appending the results would cause that you came up with 2 data of shape (10, 224, 224, 3). which is not what you are expecting.
I am taking a CNN model that is pretrained, and then trying to implement a CNN-LSTM with parallel CNNs all with the same weights from the pretraining.
# load in CNN
weightsfile = 'final_weights.h5'
modelfile = '2dcnn_model.json'
# load model from json
json_file = open(modelfile, 'r')
loaded_model_json = json_file.read()
json_file.close()
fixed_cnn_model = keras.models.model_from_json(loaded_model_json)
fixed_cnn_model.load_weights(weightsfile)
# remove the last 2 dense FC layers and freeze it
fixed_cnn_model.pop()
fixed_cnn_model.pop()
fixed_cnn_model.trainable = False
print(fixed_cnn_model.summary())
This will produce the summary:
_
________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 32, 32, 4) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 30, 30, 32) 1184
_________________________________________________________________
conv2d_2 (Conv2D) (None, 28, 28, 32) 9248
_________________________________________________________________
conv2d_3 (Conv2D) (None, 26, 26, 32) 9248
_________________________________________________________________
conv2d_4 (Conv2D) (None, 24, 24, 32) 9248
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 12, 12, 32) 0
_________________________________________________________________
conv2d_5 (Conv2D) (None, 10, 10, 64) 18496
_________________________________________________________________
conv2d_6 (Conv2D) (None, 8, 8, 64) 36928
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 4, 4, 64) 0
_________________________________________________________________
conv2d_7 (Conv2D) (None, 2, 2, 128) 73856
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 1, 1, 128) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 128) 0
_________________________________________________________________
dropout_1 (Dropout) (None, 128) 0
_________________________________________________________________
dense_1 (Dense) (None, 512) 66048
=================================================================
Total params: 224,256
Trainable params: 0
Non-trainable params: 224,256
_________________________________________________________________
Now, I will add to it and compile and show that the non-trainable all become trainable.
# create sequential model to get this all before the LSTM
# initialize loss function, SGD optimizer and metrics
loss = 'binary_crossentropy'
optimizer = keras.optimizers.Adam(lr=1e-4,
beta_1=0.9,
beta_2=0.999,
epsilon=1e-08,
decay=0.0)
metrics = ['accuracy']
currmodel = Sequential()
currmodel.add(TimeDistributed(fixed_cnn_model, input_shape=(num_timewins, imsize, imsize, n_colors)))
currmodel.add(LSTM(units=size_mem,
activation='relu',
return_sequences=False))
currmodel.add(Dense(1024, activation='relu')
currmodel.add(Dense(2, activation='softmax')
currmodel = Model(inputs=currmodel.input, outputs = currmodel.output)
config = currmodel.compile(optimizer=optimizer, loss=loss, metrics=metrics)
print(currmodel.summary())
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
time_distributed_3_input (In (None, 5, 32, 32, 4) 0
_________________________________________________________________
time_distributed_3 (TimeDist (None, 5, 512) 224256
_________________________________________________________________
lstm_3 (LSTM) (None, 50) 112600
_________________________________________________________________
dropout_1 (Dropout) (None, 50) 0
_________________________________________________________________
dense_1 (Dense) (None, 1024) 52224
_________________________________________________________________
dropout_2 (Dropout) (None, 1024) 0
_________________________________________________________________
dense_2 (Dense) (None, 2) 2050
=================================================================
Total params: 391,130
Trainable params: 391,130
Non-trainable params: 0
_________________________________________________________________
How am I supposed to freeze the layers in this case? I am almost 100% positive that I had working code in this format in an earlier keras version. It seems like this is the right direction, since you define a model and declare certain layers trainable, or not.
Then you add layers, which are by default trainable. However, this seems to convert all the layers to trainable.
try adding
for layer in currmodel.layers[:5]:
layer.trainable = False
First print the layer numbers in you network
for i,layer in enumerate(currmodel.layers):
print(i,layer.name)
Now check which layers are trainable and which are not
for i,layer in enumerate(model.layers):
print(i,layer.name,layer.trainable)
Now you can set the parameter 'trainable' for the layers which you want. Let us say you want to train only last 2 layers out of total 6 (the numbering starts from 0) then you can write something like this
for layer in model.layers[:5]:
layer.trainable=False
for layer in model.layers[5:]:
layer.trainable=True
To cross check try to print again and you will get the desired settings.