I'm trying to get VGG16 model to acceptable accuracy but I can't get it above .3
here's the model
def VGG16():
model = Sequential()
model.add(Conv2D(input_shape=(224,224,3),filters=64,kernel_size=(3,3),padding='same', activation='relu'))
model.add(Conv2D(filters=64,kernel_size=(3,3),padding='same', activation='relu'))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
model.add(Conv2D(filters=128, kernel_size=(3,3), padding='same', activation='relu'))
model.add(Conv2D(filters=128, kernel_size=(3,3), padding='same', activation='relu'))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
model.add(Conv2D(filters=256, kernel_size=(3,3), padding='same', activation='relu'))
model.add(Conv2D(filters=256, kernel_size=(3,3), padding='same', activation='relu'))
model.add(Conv2D(filters=256, kernel_size=(3,3), padding='same', activation='relu'))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
model.add(Conv2D(filters=512, kernel_size=(3,3), padding='same', activation='relu'))
model.add(Conv2D(filters=512, kernel_size=(3,3), padding='same', activation='relu'))
model.add(Conv2D(filters=512, kernel_size=(3,3), padding='same', activation='relu'))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
model.add(Conv2D(filters=512, kernel_size=(3,3), padding='same', activation='relu'))
model.add(Conv2D(filters=512, kernel_size=(3,3), padding='same', activation='relu'))
model.add(Conv2D(filters=512, kernel_size=(3,3), padding='same', activation='relu'))
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2),name='vgg16'))
model.add(Flatten(name='flatten'))
model.add(Dense(4096, activation='relu', name='fc1'))
model.add(Dense(4096, activation='relu', name='fc2'))
model.add(Dense(9, activation='softmax', name='output'))
return model
opt = SGD(learning_rate=1e-6, momentum=0.9)
model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])
some answers here suggested changing the number of neurons in the fully connected layers to 4096 (originally used 256 and 128), using SGD instead of Adam, increasing the number of epochs (tried 50, 100 and 200) and the batch size (tried 64 and 128) but I can't get it above .3 and usually it's .2.
parameters I used in the best result are
fully connected neurons 4096
optimizer SGD
learning rate e-6
epochs 100
batch size 128
edit dataset used https://www.kaggle.com/datasets/nodoubttome/skin-cancer9-classesisic
you did not show the data for the model training but I suspect your model will be very prone to over fitting. You need to add some dropout layers and some regularization.
After your flatten layer type the following
model.add(BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001 ))
model.add(Dense(256, kernel_regularizer = regularizers.l2(l = 0.016),activity_regularizer=regularizers.l1(0.006),
bias_regularizer=regularizers.l1(0.006) ,activation='relu') )
model.add(Dropout(rate=.4, seed=123), name='dropout'))
model.add(Dense(9, activation='softmax', name='output'))
it would be helpful if you provide the model training data as well
Related
I am running a 3 class classification problem with around 7200 total images using CNN. I have used 80:20 split and the accuracy and loss curves are attached along with the code.
Can someone explain why the validation accuracy is higher then the training and similarly training loss is higher than the validation loss. Although the difference is marginal (around 1%), I am not sure if its acceptable or I need to improve the model.
Here is the model I am running with batch_size = 64
# define cnn model
def define_model():
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same', input_shape=(64, 64, 3)))
model.add(BatchNormalization())
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.2))
model.add(Conv2D(64, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(BatchNormalization())
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.3))
model.add(Conv2D(128, (3, 3), activation='relu', kernel_initializer='he_uniform', padding='same'))
model.add(BatchNormalization())
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.4))
model.add(Flatten())
model.add(Dense(128, activation='relu', kernel_initializer='he_uniform'))
model.add(BatchNormalization())
model.add(Dropout(0.5))
model.add(Dense(3, activation='softmax'))
# compile model
opt = SGD(lr=0.001, momentum=0.9)
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
return model
I tried various other models but the accuracy and loss curves didn't seem good. This one seems good but I was expecting the training accuracy to be higher than the validation accuracy
I am training a CNN model where,
Training data=687 , validation data=102 , testing data=79
The validation accuracy is higher than training accuracy
The test accuracy is very low compared to both validation accuracy and training accuracy.
Validation loss is lower than training loss.
code snippet:
train_datagen = ImageDataGenerator(
rescale=1./255,
# rotation_range=30,
zoom_range=0.1,
horizontal_flip=True,
# vertical_flip=True,
# fill_mode='nearest',
validation_split=.15) # set validation split
val_datagen = ImageDataGenerator(rescale=1./255, validation_split=0.15)
train_generator = train_datagen.flow_from_directory(
train_dir,
target_size=(height, width),
batch_size=batch_size,
class_mode='categorical',
seed=13
)
validation_generator = val_datagen.flow_from_directory(
train_dir, # same directory as training data
target_size=(height, width),
batch_size=batch_size,
class_mode='categorical',
seed=13,
subset='validation'
)
model = Sequential()
model.add(Conv2D(16,3,padding="same", activation="relu", input_shape=(height, width, 3)))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))
model.add(Conv2D(32,3,padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))
model.add(Conv2D(32, 3, padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))
# model.add(Conv2D(32, 3, padding="same", activation="relu", kernel_regularizer=l2(0.0001)))
# model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.4))
model.add(Conv2D(64, 3, padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same",))
# model.add(Dropout(0.2))
model.add(Conv2D(64, 3, padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))
model.add(Conv2D(128, 3, padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))
model.add(Conv2D(128, 3, padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))
model.add(Conv2D(256, 3, padding="same", activation="relu"))
model.add(AveragePooling2D(strides=(2,2), padding="same"))
# model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(256,activation="relu"))
model.add(Dropout(.5))
# model.add(Dense(256,activation="relu"))
# model.add(Dropout(.5))
model.add(Dense(4, activation="softmax"))
model.summary()
Adam(learning_rate=0.0001, name='Adam')
model.compile(optimizer = 'Adam',loss = 'categorical_crossentropy',metrics = ['accuracy'])
I've done few things to solve this problem:
I didn't used dropout layers between the conv layers
I decreased the range of the data augmentation.
I trained for longer period(the testing accuracy drops to 62% and the val_acc eventually reaches 100%).
What could be the cause of this issue and how can it be resolved?
How can I display test images in Python that have high levels of inaccuracy?
You need to add subset='training' in train_generator. Right now, you are training on both training and validation data.
train_generator = train_datagen.flow_from_directory(
train_dir,
target_size=(height, width),
batch_size=batch_size,
class_mode='categorical',
seed=13,
subset='training'
)
i'm pretty new to machine learning. I followed a tutorial to classify if the user is similing or not. I created this code:
def get_model(input_size, classes=7):
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), padding='same', activation='relu', input_shape =input_size))
model.add(Conv2D(64, kernel_size=(3, 3), activation='relu', padding='same'))
model.add(BatchNormalization())
model.add(MaxPooling2D(2, 2))
model.add(Dropout(0.25))
model.add(Conv2D(128, kernel_size=(3, 3), activation='relu', padding='same', kernel_regularizer=regularizers.l2(0.01)))
model.add(Conv2D(256, kernel_size=(3, 3), activation='relu', kernel_regularizer=regularizers.l2(0.01)))
model.add(BatchNormalization())
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(1024, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(classes, activation='softmax'))
#Compliling the model
model.compile(optimizer=Adam(lr=0.0001, decay=1e-6),
loss='categorical_crossentropy',
metrics=['accuracy'])
return model
if i try to predict an array from flow_from_directory its working fine but i would like to predict it using the following code:
final_image = cv2.imread('./tesimg.jpeg')
final_image = np.expand_dims(final_image, axis=0)
final_image = final_image/255.0
The problem is that i'm getting this error:
UnimplementedError: Graph execution error:
I am developing a GAN using the Mnist dataset. I have developed the Generator and Discriminator. However, when I combine them together I get this error: Input 0 of layer "conv2d" is incompatible with the layer: expected axis -1 of input shape to have value 1, but received input with shape (None, 57, 57, 1024). Does anyone know why this happens? Do I have to add something else?
The preprocessing:
(x_train, _), (x_test, _) = mnist.load_data()
x_train = x_train.reshape(60000, 28, 28, 1)
x_test = x_test.reshape(10000, 28, 28, 1)
x_train = x_train.astype('float32')/255
x_test = x_test.astype('float32')/255
img_rows, img_cols = 28, 28
channels = 1
img_shape = (img_rows, img_cols, channels)
The Generator:
def generator():
model = Sequential()
model.add(Conv2DTranspose(32, (3,3), strides=(2, 2), activation='relu', use_bias=False,
input_shape=img_shape))
model.add(BatchNormalization(momentum=0.3))
model.add(Conv2DTranspose(64,(3,3),strides=(2,2), activation='relu', padding='same',
use_bias=False))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(LeakyReLU(alpha=0.2))
model.add(Conv2DTranspose(64,(3,3),strides=(2,2), activation='relu', padding='same',
use_bias=False))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
model.add(BatchNormalization(momentum=0.3))
model.add(LeakyReLU(alpha=0.2))
model.add(Dense(512, activation=LeakyReLU(alpha=0.2)))
model.add(BatchNormalization(momentum=0.7))
model.add(Dense(1024, activation='tanh'))
model.summary()
model.compile(loss=keras.losses.binary_crossentropy, optimizer=Adam(learning_rate=0.02))
return model
generator = generator()
The Discriminator:
def discriminator():
model = Sequential()
model.add(Conv2D(32, (5,5), strides=(2, 2), activation='relu', use_bias=False,
input_shape=img_shape))
model.add(BatchNormalization(momentum=0.3))
model.add(Conv2D(64,(5,5),strides=(2,2), activation='relu', padding='same',
use_bias=False))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(LeakyReLU(alpha=0.2))
model.add(Conv2D(64,(5,5),strides=(2,2), activation='relu', padding='same',
use_bias=False))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.5))
model.add(BatchNormalization(momentum=0.3))
model.add(LeakyReLU(alpha=0.2))
model.add(Dense(512, activation=LeakyReLU(alpha=0.2)))
model.add(BatchNormalization(momentum=0.7))
model.add(Dense(1024, activation='tanh'))
model.summary()
model.compile(loss=keras.losses.binary_crossentropy, optimizer=Adam(learning_rate=0.02))
return model
discriminator = discriminator()
Both models combined (Where I get the error):
def GAN(generator, discriminator):
model = Sequential()
model.add(generator)
discriminator.trainable = False
model.add(discriminator)
model.summary()
model.compile()
return model
gan = GAN(generator, discriminator)
Your generator needs to produce images, thus the output shape of the generator must be the same shape as the images. The activation also must be compatible with the range in the images. I don't think your images go from -1 to +1, so you should not use "tanh". You must choose an activation compatible with the images.
Last generator layer:
Dense(img_shape[-1], ...)
Your discriminator needs to say whether the images are true or false, thus its output must have one value only, 0 or 1.
Last discriminator layer:
Dense(1, activation="sigmoid")
I have a cnn network, which I'm trying to test.
I'm getting errors about the input, and I can't figure why
model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(99,13,1)))
model.add(Conv2D(64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy',optimizer='adam', metrics=['accuracy'])
print(model.summary())
model.fit(x_train, y_train, batch_size=4, epochs=10, verbose=1, validation_data=(x_test, y_test))
where:
x_train / test .shape = {tuple: 3}(30, 99, 13)
y_train / test shape = {tuple: 1}30
Error:
ValueError: Input 0 of layer sequential is incompatible with the layer: : expected min_ndim=4, found ndim=3. Full shape received: [None, 99, 13]
What's wrong and how can I fix it ?
model.add(Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(99,13,1)))
Model waiting for shape [batch, (99,13,1)], you are trying to feed [batch, 99, 13].
I think you need change your input to input_shape=(99,13) in order to match train data
Conv2D requires 4D+ shape
Expand dimension of your data to batch,99,13,1