Predicting images Jupyter notebook - tensorflow

Question

For my first Machine Learning experience i have a basic classification to do.
I have 3 different folders :
train_path = './dataset/pneumonia/train/'
test_path = './dataset/pneumonia/test/'
val_path = './dataset/pneumonia/val/
each folders :
os.listdir(train_path)
returns
['NORMAL', 'PNEUMONIA']
In each sets :
Training set:
Normal: 949
Pneumonia: 949
Test set:
Normal: 317
Pneumonia: 317
Validation set:
Normal: 317
Pneumonia: 317
I use tensorflow :
from tensorflow.keras.preprocessing.image import ImageDataGenerator
image_gen = ImageDataGenerator(
rotation_range=10, # rotate the image 10 degrees
width_shift_range=0.10, # Shift the pic width by a max of 5%
height_shift_range=0.10, # Shift the pic height by a max of 5%
rescale=1/255, # Rescale the image by normalzing it.
shear_range=0.1, # Shear means cutting away part of the image (max 10%)
zoom_range=0.1, # Zoom in by 10% max
horizontal_flip=True, # Allow horizontal flipping
fill_mode='nearest' # Fill in missing pixels with the nearest filled value
)
image_gen.flow_from_directory(train_path)
image_gen.flow_from_directory(test_path)
I create a model (basic model) :
model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=(image_width, image_height, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(64, (3, 3), input_shape=(image_width, image_height, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(128, (3, 3), input_shape=(image_width, image_height, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(256, (3, 3), input_shape=(image_width, image_height, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(512, (3, 3), input_shape=(image_width, image_height, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(512, activation='relu'))
# Dropouts help reduce overfitting by randomly turning neurons off during training.
# Here we say randomly turn off 50% of neurons.
model.add(Dropout(0.5))
# Last layer, remember its binary so we use sigmoid
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
Then I train it :
train_image_gen = image_gen.flow_from_directory(
train_path,
target_size=image_shape[:2],
color_mode='rgb',
batch_size=batch_size,
class_mode='binary'
)
results = model.fit_generator(train_image_gen,epochs=20,
validation_data=test_image_gen,
callbacks=[early_stop, board])
So far so good results are correct :
pred_probabilities = model.predict_generator(test_image_gen)
predictions = pred_probabilities > 0.5
confusion_matrix(test_image_gen.classes,predictions)
I obtain rather good results :
My issue is when I want to predict images it returns results which are far from being correct:
val_image_gen = image_gen.flow_from_directory(
val_path,
target_size=image_shape[:2],
color_mode='rgb',
class_mode='binary',
)
pred_probabilities = model.predict_generator(val_image_gen)
predictions = pred_probabilities > 0.5
Here are some output I obtain :
precision recall f1-score support
0 0.51 0.57 0.53 317
1 0.51 0.44 0.47 317
accuracy 0.51 634
macro avg 0.51 0.51 0.50 634
weighted avg 0.51 0.51 0.50 634
THe confusion matrix on this data set is the following :
[[180 137]
[176 141]]

Answer 1

A few issues with your code:
You are using test set for validation and validation set for testing.
This may be a problem or not, depending on your data and how it was
split.
Augmentation should be applied only to training set. Use separate
instance of ImageDataGenerator(rescale=1/255) for testing and
validation.
Your test results look like they were got from untrained model. Check if the model object you are running test on is the same one you were training. You may want to use model.save() and load_model() functions to preserve model weights after training.

Answer 2

I replaced :
val_image_gen = image_gen.flow_from_directory(
val_path,
target_size=image_shape[:2],
color_mode='rgb',
class_mode='binary',
)
by:
val_image_gen = image_gen.flow_from_directory(
val_path,
target_size=image_shape[:2],
color_mode='rgb',
batch_size=batch_size,
class_mode='binary',
shuffle=False
)
I obtain nice results :
[[269 48]
[ 3 314]]