Keras dimension mismatch with ImageDataGenerator - tensorflow

Question

I am attempting to 'flow' my data into a neural network with Keras. I am using the .flow_from_directory method and the process is giving me fits. I am using the basic example from the keras documentation (I am using tensorflow):
ROWS = 64
COLS = 64
CHANNELS = 3
from keras.preprocessing.image import ImageDataGenerator
train_datagen = ImageDataGenerator(
rescale=1./255)
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
'train',
target_size=(64, 64),
batch_size=32,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
'../tutorial/l1/kaggle_solutions/dogs_vs_cats/valid',
target_size=(64, 64),
batch_size=1,
class_mode='binary')
from keras.models import Sequential
from keras.layers import Convolution2D
from keras.layers import Dense, Activation, Flatten, Dropout, MaxPooling2D
from keras.regularizers import l2
model = Sequential()
model.add(Convolution2D(4, 4, 4, border_mode='same', input_shape=(64, 64,3), activation='relu'))
from keras.utils.np_utils import to_categorical
from keras.optimizers import SGD, RMSprop
model.compile(loss='binary_crossentropy', optimizer=RMSprop(lr=1e-4), metrics=['accuracy'])
model.fit_generator(
train_generator,
samples_per_epoch=2500,
nb_epoch=20,
validation_data=validation_generator,
nb_val_samples=3100)
Running this i get the following error:
Exception: Error when checking model target: expected convolution2d_84 to have 4 dimensions, but got array with shape (32, 1)
I have been tinkering around for a long time and found the following--switching the 'model.add' to grayscale input
model.add(Convolution2D(4, 4, 4, border_mode='same', input_shape=(64, 64,3), activation='relu'))
gives me the following error (as expected--but appears to confirm my original input was correct):
Error when checking model input: expected convolution2d_input_49 to have shape (None, 64, 64, 1) but got array with shape (32, 64, 64, 3)
So I am passing (in the original) a 4-d array of 32,64,64,3 with the original, but I am getting the error that I THINK means
Expected (1,64,64,3) and got (32,64,64,3)
As I am sending data in batches of 32. Curiously enough if I set the batch to zero (to give a 0,64,64,3 input) I get:
Exception: Error when checking model target: expected convolution2d_87 to have 4 dimensions, but got array with shape (0, 1)
Based on the documentation, I cannot figure out the proper way to flow the data into the model--i cannot pass the batch size to the model when using fit_generator, and it appears that the batch_size (num of samples) is the problem.
Any help would be greatly appreciated.

Answer 1

There is no problem with your ImageDataGenerator. As stated in the error message there is a mismatch between the shape of your model output and the shape of its targets. You use class_mode = 'binary', so expected output of your model is a single value, but instead it yields output of shape (batch_size, 64, 64, 4) since you have one convolutional layer and nothing else in your model.
Try something like this:
model.add(Convolution2D(4, 4, 4, border_mode='same', input_shape=(64, 64,3), activation='relu'))
model.add(Flatten())
model.add(Dense(1))
model.add(Activation('sigmoid'))