CNN Model With Low Accuracy - tensorflow

Question

I'm currently working on a CNN model that classifies food images. So far, I have managed to build a functioning CNN but I would like to improve the accurracy. For the dataset, I have used some images from Kaggle and few from my own collection.
Here is some information about the dataset:
There are 91 classes of food images.
Each class has around 500 to 650 images.
The dataset has been manually cleaned and checked for unrelated or bad quality images (the photos are of different sizes).
Here is my CNN model:
classifier = Sequential()
def cnn_layer_creation(classifier):
classifier.add(InputLayer(input_shape=[224,224,3]))
classifier.add(Conv2D(filters=32,kernel_size=5,strides=1,padding='same',activation='relu',data_format='channels_first'))
classifier.add(MaxPooling2D(pool_size=5,padding='same'))
classifier.add(Conv2D(filters=50,kernel_size=5,strides=1,padding='same',activation='relu'))
classifier.add(MaxPooling2D(pool_size=5,padding='same'))
classifier.add(Conv2D(filters=80,kernel_size=5,strides=1,padding='same',activation='relu',data_format='channels_last'))
classifier.add(MaxPooling2D(pool_size=5,padding='same'))
classifier.add(Dropout(0.25))
classifier.add(Flatten())
classifier.add(Dense(64,activation='relu'))
classifier.add(Dropout(rate=0.5))
classifier.add(Dense(91,activation='softmax'))
# Compiling the CNN
classifier.compile(optimizer="RMSprop", loss = 'categorical_crossentropy', metrics = ['accuracy'])
data_initialization(classifier)
def data_initialization(classifier):
# Part 2 - Fitting the CNN to the images
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
training_set = train_datagen.flow_from_directory('food_image/train',
target_size = (224, 224),
batch_size = 100,
class_mode = 'categorical')
test_set = test_datagen.flow_from_directory('food_image/test',
target_size = (224, 224),
batch_size = 100,
class_mode = 'categorical')
classifier.fit_generator(training_set,
steps_per_epoch = 100,
epochs = 100,
validation_data = test_set,
validation_steps = 100)
classifier.save("brynModelGPULite.h5")
classifier.summary()
def main():
cnn_layer_creation(classifier)
Training is done on GPU (nVidia 980M)
Unfortunately, the accuracy has not exceeded 10%. Things I've tried are:
Increase the number of epochs.
Change the optimizer (ADAM, RMSPROP).
Change the activation function.
Reduce the image input size.
Increase the batch size.
Change the filter size to 32, 64, 128.
None of these have improved the accuracy.
Could anyone shine some light on how I might improve my model accuracy?

Answer 1

You can augment only the training data.
The following code
test_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
should be
test_datagen = ImageDataGenerator(rescale = 1./255)

Answer 2

Firstly, I assume you are building your model from scratch. As such training on fewer epochs(I assume you would not have trained your model for more than 1000 epochs), will not help as the network would not evolve completely because the representations would not have been completely learnt in so few epochs when you train a model from scratch. You can try increasing the number of epochs to around 10000 and see. Rather why not try and use transfer learning for the same, additionally you can also using feature extraction and fine tuning or both using a pre trained convnet. For reference you can have a look at chapter 5 in the book by Francois Chollet titled Deep Learning with Python.

Answer 3

I had same problem with another dataset and I replaced the flatten layer with globalAveragepooling and it solved the problem.
I'm not sure this is going to work for you but as my model has a structure similar to yours, I think this can help you. But the difference is that I trained my model for 3 classes.