1st layer tf.keras output shape set at multiple - tensorflow

The output shape of my first layer when calling model.summary() comes out as "multiple". I'm pretty sure this means that I have multiple inputs acting on it but I can not figure out which parts of my code are acting on it in this way.
So I am asking if anyone can help point out my mistakes in my code and offer any alternatives?
Code is as follows:
import tensorflow as tf
from tensorflow.keras.layers import Input, Lambda, Dense, Flatten,Dropout
from tensorflow.keras.models import Model
from tensorflow.keras import Model, layers, utils
from tensorflow.keras.applications.vgg16 import VGG16
from tensorflow.keras.applications.vgg16 import preprocess_input
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.preprocessing import image
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import GlobalAveragePooling2D
import numpy as np
import pandas as pd
import os
import cv2
import matplotlib.pyplot as plt
PATH = '../img_class/images/british_carribae/'
test_path= os.path.join(PATH, 'test')
train_path=os.path.join(PATH,'train')
val_path=os.path.join(PATH,'val')
IMAGE_SIZE = (224, 224)
BATCH_SIZE = 32
x_train = tf.keras.utils.image_dataset_from_directory(train_path,
shuffle=True,
batch_size=BATCH_SIZE,
image_size=IMAGE_SIZE)
x_test = tf.keras.utils.image_dataset_from_directory(test_path,
shuffle=True,
batch_size=BATCH_SIZE,
image_size=IMAGE_SIZE)
x_val = tf.keras.utils.image_dataset_from_directory(val_path,
shuffle=True,
batch_size=BATCH_SIZE,
image_size=IMAGE_SIZE)
AUTOTUNE = tf.data.AUTOTUNE
x_train = x_train.prefetch(buffer_size=AUTOTUNE)
x_val= x_val.prefetch(buffer_size=AUTOTUNE)
x_test = x_test.prefetch(buffer_size=AUTOTUNE)
preprocess_input = tf.keras.applications.vgg16.preprocess_input
IMG_SHAPE = IMAGE_SIZE +(3,)
vgg = tf.keras.applications.VGG16(input_shape=IMG_SHAPE, weights='imagenet', include_top=False, pooling='max')
image_batch, label_batch = next(iter(x_train))
feature_batch = vgg(image_batch)
print(feature_batch.shape)
for layer in vgg.layers:
layer.trainable = False
inp = layers.Input((224,224,3))
cnn = vgg(inp)
x = layers.BatchNormalization()(cnn)
x = layers.Dropout(0.2)(x)
x = layers.Dense(256, activation='softmax')(x)
x = layers.BatchNormalization()(x)
x = layers.Dropout(0.2)(x)
out = layers.Dense(291, activation='softmax')(x)
model = Model(inp, out)
#Flattening nested model
def flatten_model(model_nested):
layers_flat = []
for layer in model_nested.layers:
try:
layers_flat.extend(layer.layers)
except AttributeError:
layers_flat.append(layer)
model_flat = tf.keras.models.Sequential(layers_flat)
return model_flat
model_flat = flatten_model(model)
model_flat.summary()
this is the results of the summary printout for reference:
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) multiple 0
block1_conv1 (Conv2D) (None, 224, 224, 64) 1792
block1_conv2 (Conv2D) (None, 224, 224, 64) 36928
block1_pool (MaxPooling2D) (None, 112, 112, 64) 0
block2_conv1 (Conv2D) (None, 112, 112, 128) 73856
block2_conv2 (Conv2D) (None, 112, 112, 128) 147584
block2_pool (MaxPooling2D) (None, 56, 56, 128) 0
block3_conv1 (Conv2D) (None, 56, 56, 256) 295168
block3_conv2 (Conv2D) (None, 56, 56, 256) 590080
block3_conv3 (Conv2D) (None, 56, 56, 256) 590080
block3_pool (MaxPooling2D) (None, 28, 28, 256) 0
block4_conv1 (Conv2D) (None, 28, 28, 512) 1180160
block4_conv2 (Conv2D) (None, 28, 28, 512) 2359808
block4_conv3 (Conv2D) (None, 28, 28, 512) 2359808
block4_pool (MaxPooling2D) (None, 14, 14, 512) 0
block5_conv1 (Conv2D) (None, 14, 14, 512) 2359808
block5_conv2 (Conv2D) (None, 14, 14, 512) 2359808
block5_conv3 (Conv2D) (None, 14, 14, 512) 2359808
block5_pool (MaxPooling2D) (None, 7, 7, 512) 0
global_max_pooling2d (Globa (None, 512) 0
lMaxPooling2D)
batch_normalization (BatchN (None, 512) 2048
ormalization)
dropout (Dropout) (None, 512) 0
dense (Dense) (None, 256) 131328
batch_normalization_1 (Batc (None, 256) 1024
hNormalization)
dropout_1 (Dropout) (None, 256) 0
dense_1 (Dense) (None, 291) 74787
=================================================================
Total params: 14,923,875
Trainable params: 207,651
Non-trainable params: 14,716,224
Thanks for any help!

Update:
Try this code for the input layer (or any shape with multiple output shapes):
layer._inbound_nodes = []
This fixed my issue. Hope it helps you as well.
I have the same problem in a different way. My first layer has the normal output shape but the rest of my layers have multiple output shapes which doesn't make sense. The only thing is that I am recreating my model using a list of the layers I have.
Here is the code:
model_1 = tf.keras.applications.VGG16(weights = 'imagenet',include_top = False)
model_1.trainable = False
model_1.summary()
layers = [l for l in model_1.layers]
for layer in layers:
layer._outbound_nodes = []
input = keras.Input(shape=(224, 224, 3))
x = layers[1](input)
x = layers[2](x)
x = layers[3](x)
x = layers[4](x)
x = layers[5](x)
x = layers[6](x)
x = layers[7](x)
x = layers[8](x)
x = layers[9](x)
x = layers[10](x)
x = layers[11](x)
x = layers[12](x)
x = layers[13](x)
out = Dense(38, activation='softmax')(x)
result_model = Model(inputs=input, outputs=out)
result_model.summary()
Layer (type) Output Shape Param #
=================================================================
input_59 (InputLayer) [(None, 224, 224, 3)] 0
block1_conv1 (Conv2D) multiple 1792
block1_conv2 (Conv2D) multiple 36928
block1_pool (MaxPooling2D) multiple 0
block2_conv1 (Conv2D) multiple 73856
block2_conv2 (Conv2D) multiple 147584
block2_pool (MaxPooling2D) multiple 0
block3_conv1 (Conv2D) multiple 295168
block3_conv2 (Conv2D) multiple 590080
block3_conv3 (Conv2D) multiple 590080
block3_pool (MaxPooling2D) multiple 0
block4_conv1 (Conv2D) multiple 1180160
block4_conv2 (Conv2D) multiple 2359808
block4_conv3 (Conv2D) multiple 2359808
dense_54 (Dense) (None, 28, 28, 38) 19494
=================================================================
Total params: 7,654,758
Trainable params: 19,494
Non-trainable params: 7,635,264
I wrote it here because maybe we can figure out the solution together.

Related

Run time connection loss while trainig VGG 16 model on Google Collaboratory

I'm Training the VGG16 model in colab while running it some time disconnects and reconnect again and sometimes while reaching 20, 21/35 epochs all connection loss and when I reconnect drive mounting restart due to this I lost all outputs, so I have to re-run all code. how can this problem be solved?
Even I'm using only 3000 images dataset for this which is divided into valid, train and test dataset
the code which I Run is
vgg16_model = tf.keras.applications.vgg16.VGG16()
vgg16_model.summary()
model = Sequential()
for layer in vgg16_model.layers[:-1]:
model.add(layer)
for layer in model.layers:
layer.trainable = False
model.add(Dense(units=2, activation='softmax'))
model.summary()
model.compile(optimizer = Adam(learning_rate=0.0001), loss = 'categorical_crossentropy', metrics = ['accuracy'])
model.fit(x=train_batches,
steps_per_epoch=len(train_batches),
validation_data=valid_batches,
validation_steps=len(valid_batches),
epochs=35,
verbose=2
)
I was able to execute sample code using VGG16 without any issues. Please refer working code as shown below
import os
import numpy as np
import tensorflow as tf
from tensorflow.keras.applications import vgg16
from keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.layers import Dense, Dropout, Flatten
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.models import Sequential
from google.colab import drive
drive.mount('/content/drive')
train_dir = '/content/drive/My Drive/Dogs_Vs_Cats/train'
valid_dir = '/content/drive/My Drive/Dogs_Vs_Cats/test'
img_width, img_height = 224, 224
input_shape = (img_width, img_height, 3)
batch_size = 32
train_datagen = ImageDataGenerator(
rescale = 1. /255,
horizontal_flip = True)
valid_datagen = ImageDataGenerator(
rescale = 1. /255)
train_batches = train_datagen.flow_from_directory(
train_dir,
target_size = (img_width, img_height),
batch_size = batch_size,
class_mode = 'binary')
valid_batches = valid_datagen.flow_from_directory(
valid_dir,
target_size = (img_width, img_height),
class_mode = 'binary')
vgg16_model = vgg16.VGG16()
model = Sequential()
for layer in vgg16_model.layers[:-1]:
model.add(layer)
for layer in model.layers:
layer.trainable = False
model.add(Dense(units=1, activation='softmax'))
model.summary()
model.compile(optimizer = Adam(learning_rate=0.0001), loss = 'binary_crossentropy', metrics = ['accuracy'])
model.fit(
train_batches,
steps_per_epoch = 10,
epochs = 2,
validation_data = valid_batches,
verbose = 1,
validation_steps = 32)
Output:
Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount("/content/drive", force_remount=True).
Found 2000 images belonging to 2 classes.
Found 1018 images belonging to 2 classes.
Model: "sequential_2"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
block1_conv1 (Conv2D) (None, 224, 224, 64) 1792
_________________________________________________________________
block1_conv2 (Conv2D) (None, 224, 224, 64) 36928
_________________________________________________________________
block1_pool (MaxPooling2D) (None, 112, 112, 64) 0
_________________________________________________________________
block2_conv1 (Conv2D) (None, 112, 112, 128) 73856
_________________________________________________________________
block2_conv2 (Conv2D) (None, 112, 112, 128) 147584
_________________________________________________________________
block2_pool (MaxPooling2D) (None, 56, 56, 128) 0
_________________________________________________________________
block3_conv1 (Conv2D) (None, 56, 56, 256) 295168
_________________________________________________________________
block3_conv2 (Conv2D) (None, 56, 56, 256) 590080
_________________________________________________________________
block3_conv3 (Conv2D) (None, 56, 56, 256) 590080
_________________________________________________________________
block3_pool (MaxPooling2D) (None, 28, 28, 256) 0
_________________________________________________________________
block4_conv1 (Conv2D) (None, 28, 28, 512) 1180160
_________________________________________________________________
block4_conv2 (Conv2D) (None, 28, 28, 512) 2359808
_________________________________________________________________
block4_conv3 (Conv2D) (None, 28, 28, 512) 2359808
_________________________________________________________________
block4_pool (MaxPooling2D) (None, 14, 14, 512) 0
_________________________________________________________________
block5_conv1 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_conv2 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_conv3 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_pool (MaxPooling2D) (None, 7, 7, 512) 0
_________________________________________________________________
flatten (Flatten) (None, 25088) 0
_________________________________________________________________
fc1 (Dense) (None, 4096) 102764544
_________________________________________________________________
fc2 (Dense) (None, 4096) 16781312
_________________________________________________________________
dense_2 (Dense) (None, 1) 4097
=================================================================
Total params: 134,264,641
Trainable params: 4,097
Non-trainable params: 134,260,544
_________________________________________________________________
Epoch 1/2
10/10 [==============================] - 503s 54s/step - loss: 0.6935 - accuracy: 0.4292 - val_loss: 0.6861 - val_accuracy: 0.4912
Epoch 2/2
10/10 [==============================] - 506s 55s/step - loss: 0.6856 - accuracy: 0.4669 - val_loss: 0.6748 - val_accuracy: 0.4912
Note: To debug your code try with 5 epoch first.
If you are facing session timeout, you can refer solutions discussed in Google Colab session timeout.

How to feed data into Multiple dense(2,)?

Model :
import tensorflow as tf
from tensorflow.keras.layers import Dense
from tensorflow.keras import Model
vgg_base =tf.keras.applications.VGG16(include_top=False, weights='imagenet')
x = vgg_base.output
x = tf.keras.layers.GlobalAveragePooling2D()(x)
cl1 = Dense(2, activation = 'softmax',name='cl1')(x)
cl2 = Dense(2, activation = 'softmax',name='cl2')(x)
model = Model(inputs=vgg_base.input, outputs= [cl1,cl2])
model.compile(optimizer='adam', loss='categorical_crossentropy' , metrics=['categorical_accuracy'])
model.summary()
Here i have three Dense(2,) layers. I tried to feed data into model.fit() by using following custom data function
def func(img_batch, lb_batch):
lbs = tf.one_hot(lb_batch,depth=2)
return img_batch, lbs
train_Data = train_ds.map(func)
model.fit(train_Data,steps_per_epoch=400,validation_steps=40,
epochs=50,verbose=1))
But getting the following format
ValueError: Error when checking model target: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), for inputs ['cl2', 'cl2'] but instead got the following list of 1 arrays: [<tf.Tensor 'args_1:0' shape=(None, 2, 2) dtype=float32>]...
Here the model is expecting a 2 separate arrays , but the custom funtion return single array with shape (2,2). So have to convert the single tf.Tensor array with shape(2,2) into 3 seperate tf.Tensor array .. How to solve this !
Model.summary() output :
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, None, None, 0
__________________________________________________________________________________________________
block1_conv1 (Conv2D) (None, None, None, 6 1792 input_1[0][0]
__________________________________________________________________________________________________
block1_conv2 (Conv2D) (None, None, None, 6 36928 block1_conv1[0][0]
__________________________________________________________________________________________________
block1_pool (MaxPooling2D) (None, None, None, 6 0 block1_conv2[0][0]
__________________________________________________________________________________________________
block2_conv1 (Conv2D) (None, None, None, 1 73856 block1_pool[0][0]
__________________________________________________________________________________________________
block2_conv2 (Conv2D) (None, None, None, 1 147584 block2_conv1[0][0]
__________________________________________________________________________________________________
block2_pool (MaxPooling2D) (None, None, None, 1 0 block2_conv2[0][0]
__________________________________________________________________________________________________
block3_conv1 (Conv2D) (None, None, None, 2 295168 block2_pool[0][0]
__________________________________________________________________________________________________
block3_conv2 (Conv2D) (None, None, None, 2 590080 block3_conv1[0][0]
__________________________________________________________________________________________________
block3_conv3 (Conv2D) (None, None, None, 2 590080 block3_conv2[0][0]
__________________________________________________________________________________________________
block3_pool (MaxPooling2D) (None, None, None, 2 0 block3_conv3[0][0]
__________________________________________________________________________________________________
block4_conv1 (Conv2D) (None, None, None, 5 1180160 block3_pool[0][0]
__________________________________________________________________________________________________
block4_conv2 (Conv2D) (None, None, None, 5 2359808 block4_conv1[0][0]
__________________________________________________________________________________________________
block4_conv3 (Conv2D) (None, None, None, 5 2359808 block4_conv2[0][0]
__________________________________________________________________________________________________
block4_pool (MaxPooling2D) (None, None, None, 5 0 block4_conv3[0][0]
__________________________________________________________________________________________________
block5_conv1 (Conv2D) (None, None, None, 5 2359808 block4_pool[0][0]
__________________________________________________________________________________________________
block5_conv2 (Conv2D) (None, None, None, 5 2359808 block5_conv1[0][0]
__________________________________________________________________________________________________
block5_conv3 (Conv2D) (None, None, None, 5 2359808 block5_conv2[0][0]
__________________________________________________________________________________________________
block5_pool (MaxPooling2D) (None, None, None, 5 0 block5_conv3[0][0]
__________________________________________________________________________________________________
global_average_pooling2d (Globa (None, 512) 0 block5_pool[0][0]
__________________________________________________________________________________________________
cl1 (Dense) (None, 2) 1026 global_average_pooling2d[0][0]
__________________________________________________________________________________________________
cl2 (Dense) (None, 2) 1026 global_average_pooling2d[0][0]
==================================================================================================
Total params: 14,716,740
Trainable params: 14,716,740
Non-trainable params: 0
__________________________________________________________________________
You can use the tf.split function to split the one tensor into three separate tensors.
img_batch_split = tf.split(img_batch, 3, axis = 1)
This will return a list containing three tensors of shape (None, 1, 2). To get rid of the extra 1 dimension, you can use a tf.squeeze.
img_batch_split_squeezed = [tf.squeeze(x, axis = 1) for x in img_batch_split]
After this, each element in the list will be of dimension (None, 2). Your entire function may look like this:
def func(img_batch, lb_batch):
img_batch_split = tf.split(img_batch, 3, axis = 1)
img_batch_split_squeezed = [tf.squeeze(x, axis = 1) for x in img_batch_split]
lbs = tf.one_hot(lb_batch,depth=2)
lbs_split = tf.split(lbs, 3, axis = 1)
lbs_split_squeezed = [tf.squeeze(x, axis = 1) for x in lbs_split]
return img_batch_split_squeezed, lbs_split_squeezed

Make an image after flatted fully connected layer in CNN

I am using this reference project/paper https://feedforward.github.io/blog/depthmap-prediction-from-a-single/ for predicting the depth estimation from 2D images. I cannot understand how the coarse depth image (Coarse7) is formed after the fully connected layer.
I am using an color input image of size (576,172)
import pandas as pd
import matplotlib.pyplot as plt
import tensorflow as tf
import keras
from keras.models import Sequential, Model
from keras.applications import vgg16
from keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D, InputLayer,UpSampling2D
from keras.layers.normalization import BatchNormalization
import cv2
model=Sequential()
model.add(Conv2D(96,(11,11),strides=(4,4),input_shape=new_arr.shape,padding='same'))
# model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(256,(5,5),padding='same'))
# model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(384,(3,3),padding='same'))
# model.add(BatchNormalization())
model.add(Activation("relu"))
# model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Conv2D(384,(3,3),padding='same'))
# model.add(BatchNormalization())
model.add(Activation("relu"))
# model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dense(256))
# model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.4))
model.add(Flatten())
model.add(Dense(4096))
# model.add(BatchNormalization())
model.add(Activation("relu"))
model.add(Dropout(0.4))
# model.add(UpSampling2D(size=(2, 2)))
# model.add(Conv2D(128, 5, padding='same',activation='relu', kernel_initializer='glorot_normal'))
model.summary()
This the model summary
```Layer (type) Output Shape Param #
=================================================================
conv2d_94 (Conv2D) (None, 43, 144, 96) 34944
_________________________________________________________________
activation_137 (Activation) (None, 43, 144, 96) 0
_________________________________________________________________
max_pooling2d_79 (MaxPooling (None, 21, 72, 96) 0
_________________________________________________________________
conv2d_95 (Conv2D) (None, 21, 72, 256) 614656
_________________________________________________________________
activation_138 (Activation) (None, 21, 72, 256) 0
_________________________________________________________________
max_pooling2d_80 (MaxPooling (None, 10, 36, 256) 0
_________________________________________________________________
conv2d_96 (Conv2D) (None, 10, 36, 384) 885120
_________________________________________________________________
activation_139 (Activation) (None, 10, 36, 384) 0
_________________________________________________________________
conv2d_97 (Conv2D) (None, 10, 36, 384) 1327488
_________________________________________________________________
activation_140 (Activation) (None, 10, 36, 384) 0
_________________________________________________________________
dense_44 (Dense) (None, 10, 36, 256) 98560
_________________________________________________________________
activation_141 (Activation) (None, 10, 36, 256) 0
_________________________________________________________________
max_pooling2d_81 (MaxPooling (None, 5, 18, 256) 0
_________________________________________________________________
dropout_44 (Dropout) (None, 5, 18, 256) 0
_________________________________________________________________
flatten_14 (Flatten) (None, 23040) 0
_________________________________________________________________
dense_45 (Dense) (None, 4096) 94375936
_________________________________________________________________
activation_142 (Activation) (None, 4096) 0
_________________________________________________________________
dropout_45 (Dropout) (None, 4096) 0
=================================================================
Total params: 97,336,704
Trainable params: 97,336,704
Non-trainable params: 0
After you have an flattend image or a dense layer you can reshape it to an image size with keras.layers.Reshape.
Than just upsample it to your output size.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Reshape
I hope that helped

Sci-kit Learn Confusion Matrix: Found input variables with inconsistent numbers of samples

I'm trying to plot a confusion matrix between the predicted test labels and the actual ones, but I'm getting this error
ValueError: Found input variables with inconsistent numbers of samples: [1263, 12630]
Dataset: GTSRB
Code used
Image augmentation
train_datagen = ImageDataGenerator(rescale=1./255,
rotation_range=20,
horizontal_flip=True,
width_shift_range=0.1,
height_shift_range=0.1,
shear_range=0.01,
zoom_range=[0.9, 1.25],
brightness_range=[0.5, 1.5])
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator and test_generator
batch_size = 10
train_generator = train_datagen.flow_from_directory(
directory=train_path,
target_size=(224, 224),
color_mode="rgb",
batch_size=batch_size,
class_mode="categorical",
shuffle=True,
seed=42
)
test_generator = test_datagen.flow_from_directory(
directory=test_path,
target_size=(224, 224),
color_mode="rgb",
batch_size=batch_size,
class_mode="categorical",
shuffle=False,
seed=42
)
Output of that code
Found 39209 images belonging to 43 classes.
Found 12630 images belonging to 43 classes.
Then, I used a VGG-16 model and replaced the latest Dense layer with a Dense(43, activation='softmax')
Model summary
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
block1_conv1 (Conv2D) (None, 224, 224, 64) 1792
_________________________________________________________________
block1_conv2 (Conv2D) (None, 224, 224, 64) 36928
_________________________________________________________________
block1_pool (MaxPooling2D) (None, 112, 112, 64) 0
_________________________________________________________________
block2_conv1 (Conv2D) (None, 112, 112, 128) 73856
_________________________________________________________________
block2_conv2 (Conv2D) (None, 112, 112, 128) 147584
_________________________________________________________________
block2_pool (MaxPooling2D) (None, 56, 56, 128) 0
_________________________________________________________________
block3_conv1 (Conv2D) (None, 56, 56, 256) 295168
_________________________________________________________________
block3_conv2 (Conv2D) (None, 56, 56, 256) 590080
_________________________________________________________________
block3_conv3 (Conv2D) (None, 56, 56, 256) 590080
_________________________________________________________________
block3_pool (MaxPooling2D) (None, 28, 28, 256) 0
_________________________________________________________________
block4_conv1 (Conv2D) (None, 28, 28, 512) 1180160
_________________________________________________________________
block4_conv2 (Conv2D) (None, 28, 28, 512) 2359808
_________________________________________________________________
block4_conv3 (Conv2D) (None, 28, 28, 512) 2359808
_________________________________________________________________
block4_pool (MaxPooling2D) (None, 14, 14, 512) 0
_________________________________________________________________
block5_conv1 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_conv2 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_conv3 (Conv2D) (None, 14, 14, 512) 2359808
_________________________________________________________________
block5_pool (MaxPooling2D) (None, 7, 7, 512) 0
_________________________________________________________________
flatten (Flatten) (None, 25088) 0
_________________________________________________________________
fc1 (Dense) (None, 4096) 102764544
_________________________________________________________________
fc2 (Dense) (None, 4096) 16781312
_________________________________________________________________
predictions (Dense) (None, 1000) 4097000
_________________________________________________________________
dense_1 (Dense) (None, 43) 43043
=================================================================
Total params: 138,400,587
Trainable params: 43,043
Non-trainable params: 138,357,544
_________________________________________________________________
Compile the model
my_sgd = SGD(lr=0.01)
model.compile(
optimizer=my_sgd,
loss='categorical_crossentropy',
metrics=['accuracy']
)
Train the model
STEP_SIZE_TRAIN=train_generator.n//train_generator.batch_size
epochs=10
model.fit_generator(generator=train_generator,
steps_per_epoch=STEP_SIZE_TRAIN,
epochs=epochs,
verbose=1
)
Predictions
STEP_SIZE_TEST=test_generator.n//test_generator.batch_size
test_generator.reset()
predictions = model.predict_generator(test_generator, steps=STEP_SIZE_TEST, verbose=1)
Output
1263/1263 [==============================] - 229s 181ms/step
Predictions shape
print(predictions.shape)
(12630, 43)
Getting the test_data and test_labels
test_data = []
test_labels = []
batch_index = 0
while batch_index <= test_generator.batch_index:
data = next(test_generator)
test_data.append(data[0])
test_labels.append(data[1])
batch_index = batch_index + 1
test_data_array = np.asarray(test_data)
test_labels_array = np.asarray(test_labels)
Shape of test_data_array and test_labels_array
test_data_array.shape
(1263, 10, 224, 224, 3)
test_labels_array.shape
(1263, 10, 43)
Confusion Matrix
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(test_labels_array, predictions)
I get the output
ValueError: Found input variables with inconsistent numbers of samples: [1263, 12630]
I understand that this error is because the test_labels_array size isn't equal to the predictions; 1263 and 12630 respectively, but I don't really know what I'm doing wrong.
Any help would be much appreciated.
PS: If anyone has any tips on how to increase the training accuracy while we're at it, that would be brilliant.
Thanks!
You should reshape test_data_array and test_labels_array as follows:
data_count, batch_count, w, h, c = test_data_array.shape
test_data_array=np.reshape(test_data_array, (data_count*batch_count, w, h, c))
test_labels_array = np.reshape(test_labels_array , (data_count*batch_count, -1))
the way you are appending the results of test_generator is the reason. In fact the first call of your test_generator will generate 10 data with shape of (224, 224, 3). For the next call again your test_generator will generate 10 data with shape of (224, 224, 3). So now you should have 20 data of shape (224, 224, 3) while the way you are appending the results would cause that you came up with 2 data of shape (10, 224, 224, 3). which is not what you are expecting.

Keras - Freezing A Model And Then Adding Trainable Layers

I am taking a CNN model that is pretrained, and then trying to implement a CNN-LSTM with parallel CNNs all with the same weights from the pretraining.
# load in CNN
weightsfile = 'final_weights.h5'
modelfile = '2dcnn_model.json'
# load model from json
json_file = open(modelfile, 'r')
loaded_model_json = json_file.read()
json_file.close()
fixed_cnn_model = keras.models.model_from_json(loaded_model_json)
fixed_cnn_model.load_weights(weightsfile)
# remove the last 2 dense FC layers and freeze it
fixed_cnn_model.pop()
fixed_cnn_model.pop()
fixed_cnn_model.trainable = False
print(fixed_cnn_model.summary())
This will produce the summary:
_
________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) (None, 32, 32, 4) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 30, 30, 32) 1184
_________________________________________________________________
conv2d_2 (Conv2D) (None, 28, 28, 32) 9248
_________________________________________________________________
conv2d_3 (Conv2D) (None, 26, 26, 32) 9248
_________________________________________________________________
conv2d_4 (Conv2D) (None, 24, 24, 32) 9248
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 12, 12, 32) 0
_________________________________________________________________
conv2d_5 (Conv2D) (None, 10, 10, 64) 18496
_________________________________________________________________
conv2d_6 (Conv2D) (None, 8, 8, 64) 36928
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 4, 4, 64) 0
_________________________________________________________________
conv2d_7 (Conv2D) (None, 2, 2, 128) 73856
_________________________________________________________________
max_pooling2d_3 (MaxPooling2 (None, 1, 1, 128) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 128) 0
_________________________________________________________________
dropout_1 (Dropout) (None, 128) 0
_________________________________________________________________
dense_1 (Dense) (None, 512) 66048
=================================================================
Total params: 224,256
Trainable params: 0
Non-trainable params: 224,256
_________________________________________________________________
Now, I will add to it and compile and show that the non-trainable all become trainable.
# create sequential model to get this all before the LSTM
# initialize loss function, SGD optimizer and metrics
loss = 'binary_crossentropy'
optimizer = keras.optimizers.Adam(lr=1e-4,
beta_1=0.9,
beta_2=0.999,
epsilon=1e-08,
decay=0.0)
metrics = ['accuracy']
currmodel = Sequential()
currmodel.add(TimeDistributed(fixed_cnn_model, input_shape=(num_timewins, imsize, imsize, n_colors)))
currmodel.add(LSTM(units=size_mem,
activation='relu',
return_sequences=False))
currmodel.add(Dense(1024, activation='relu')
currmodel.add(Dense(2, activation='softmax')
currmodel = Model(inputs=currmodel.input, outputs = currmodel.output)
config = currmodel.compile(optimizer=optimizer, loss=loss, metrics=metrics)
print(currmodel.summary())
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
time_distributed_3_input (In (None, 5, 32, 32, 4) 0
_________________________________________________________________
time_distributed_3 (TimeDist (None, 5, 512) 224256
_________________________________________________________________
lstm_3 (LSTM) (None, 50) 112600
_________________________________________________________________
dropout_1 (Dropout) (None, 50) 0
_________________________________________________________________
dense_1 (Dense) (None, 1024) 52224
_________________________________________________________________
dropout_2 (Dropout) (None, 1024) 0
_________________________________________________________________
dense_2 (Dense) (None, 2) 2050
=================================================================
Total params: 391,130
Trainable params: 391,130
Non-trainable params: 0
_________________________________________________________________
How am I supposed to freeze the layers in this case? I am almost 100% positive that I had working code in this format in an earlier keras version. It seems like this is the right direction, since you define a model and declare certain layers trainable, or not.
Then you add layers, which are by default trainable. However, this seems to convert all the layers to trainable.
try adding
for layer in currmodel.layers[:5]:
layer.trainable = False
First print the layer numbers in you network
for i,layer in enumerate(currmodel.layers):
print(i,layer.name)
Now check which layers are trainable and which are not
for i,layer in enumerate(model.layers):
print(i,layer.name,layer.trainable)
Now you can set the parameter 'trainable' for the layers which you want. Let us say you want to train only last 2 layers out of total 6 (the numbering starts from 0) then you can write something like this
for layer in model.layers[:5]:
layer.trainable=False
for layer in model.layers[5:]:
layer.trainable=True
To cross check try to print again and you will get the desired settings.