CoreMLtools and Keras ValueError: need more than 1 value to unpack - tensorflow

I'm fine-tuning the Inception V3 model with Keras, in order to convert it with coremltools into a .mlmodel file.
However, when converting the model coremltools throws an error saying the following when the converter reaches the last layer of the model:
coremltools/models/neural_network.py", line 2501, in set_pre_processing_parameters
channels, height, width = array_shape
ValueError: need more than 1 value to unpack
I used the code from the Keras documentation on applications found here: https://keras.io/applications/#fine-tune-inceptionv3-on-a-new-set-of-classes
And added a piece of code loading my dataset from the VGG example found here: https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html
My final script looks like this, using TesorFlow as backend:
LOAD THE DATA
from keras.preprocessing.image import ImageDataGenerator
img_width, img_height = 299, 299
train_data_dir = 'data/train'
validation_data_dir = 'data/validation'
nb_train_samples = 358
nb_validation_samples = 21
epochs = 1
batch_size = 15
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='categorical')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='categorical')
TRAIN THE MODEL
base_model = InceptionV3(weights='imagenet', include_top=False)
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(1024, activation='relu')(x)
predictions = Dense(7, activation='softmax')(x)
model = Model(inputs=base_model.input, outputs=predictions)
for layer in base_model.layers:
layer.trainable = False
model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size)
for i, layer in enumerate(base_model.layers):
print(i, layer.name)
for layer in model.layers[:249]:
layer.trainable = False
for layer in model.layers[249:]:
layer.trainable = True
from keras.optimizers import SGD
model.compile(optimizer=SGD(lr=0.0001, momentum=0.9), loss='categorical_crossentropy', metrics=['accuracy'])
model.fit_generator(
train_generator,
steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs,
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size)
model.save('finetuned_inception.h5')

I'm writing here in response to #SwimBikeRun's request (as I need a bit more space)
I was converting YOLO to Keras and then Keras to CoreML. For conversion I was using this script https://github.com/qqwweee/keras-yolo3/blob/master/convert.py
In the conversion-process the model was eventually created like that:
input_layer = Input(shape=(None, None, 3))
...
model = Model(inputs=input_layer, outputs=[all_layers[i] for i in out_index])
And those "None"-inputs was what made CoreML conversion fail. For CoreML the input-size to your model must be known. So I changed it to this:
input_layer = Input(shape=(416, 416, 3)
Your input-size will probably vary.
For your original question:
Maybe check your base_model.input size for the same problem.

Related

Ways to decrease validation loss % and increase validation accuracy %?

I'm trying to work with a image classification model for gravity waves detection.
So I want to check if there is something I could do to lower validation loss %, or more importantly, increase validation accuracy %.
The dataset is about a total of 460 images, split into
300 images that belong to 2 classes
60 images belonging to 2 classes
100 images belonging to 2 classes
For context, this is the code for pre processing:
batch_size = 32
Data Augmentation:
train_datagen = ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
horizontal_flip=True,
)
test_datagen = ImageDataGenerator(
rescale=1./255)
The generator that reads images to generate batches of augmented image data
train_generator = train_datagen.flow_from_directory(
'./train', target directory
target_size=(256, 256), images resized to 150x150
batch_size=batch_size,
# batch_size=40,
class_mode='binary')
validation_generator = train_datagen.flow_from_directory(
'./validation',
target_size=(256, 256),
batch_size=batch_size,
# batch_size=20,
class_mode='binary')
test_generator = test_datagen.flow_from_directory(
'./test',
target_size=(256, 256),
# batch_size=batch_size,
batch_size=batch_size,
#class_mode=None,
class_mode= None,
shuffle=False)
And this is the model used:
from tensorflow.keras.applications.inception_v3 import InceptionV3
import tensorflow as tf
from keras import regularizers
base_model = InceptionV3(input_shape = (256,256,3), include_top = False, weights = 'imagenet')
x = layers.Flatten()(base_model.output)
x = layers.Dense(1024, activation='relu')(x)
x = layers.Dropout(0.2)(x)
x = layers.Dense(1, activation='sigmoid')(x)
model = Model(base_model.input, x)
model.compile(optimizer = tf.keras.optimizers.SGD(learning_rate=0.001), loss = 'binary_crossentropy', metrics = ['accuracy'])
fitness = model.fit(
train_generator,
steps_per_epoch= 120,
epochs = 100,
validation_data=validation_generator,
validation_steps= 64)
So far the accuracy and loss % have been around:
Average training accuracy: 0.9237500047683715
Average training loss: 0.17309745135484264
Average validation accuracy: 0.6489999979734421
Average validation loss: 0.9121886402368545
The predicitons have been around:
validation predictions:
(24, 36)
test predictions:
(45, 55)
And the confusion matrix:
Confusion Matrix:
array([[12, 18],
[12, 18]])

Keras: "ValueError: Error when checking target"

I am trying to build a model, which will classify video to certain category.
For this I used pretrained model - InceptionV3 and trained it on my own data. Training process was completed successfully, but when I tried to classify video I got the error:
ValueError: Error when checking : expected input_1 to have shape (None, None, None, 3) but got array with shape (1, 1, 104, 2048)
However for prediction I used the same video as for training process.
Defined model:
train_datagen = ImageDataGenerator(
rescale=1./255,
shear_range=0.2,
horizontal_flip=True,
rotation_range=10.,
width_shift_range=0.2,
height_shift_range=0.2)
test_datagen = ImageDataGenerator(rescale=1./255)
train_generator = train_datagen.flow_from_directory(
os.path.join('data', 'train'),
target_size=(299, 299),
batch_size=32,
classes=data.classes,
class_mode='categorical')
validation_generator = test_datagen.flow_from_directory(
os.path.join('data', 'test'),
target_size=(299, 299),
batch_size=32,
classes=data.classes,
class_mode='categorical')
base_model = InceptionV3(weights=weights, include_top=False)
# add a global spatial average pooling layer
x = base_model.output
x = GlobalAveragePooling2D()(x)
# let's add a fully-connected layer
x = Dense(1024, activation='relu')(x)
# and a logistic layer
predictions = Dense(len(data.classes), activation='softmax')(x)
# this is the model we will train
model = Model(inputs=base_model.input, outputs=predictions)
model.fit_generator(
train_generator,
steps_per_epoch=100,
validation_data=validation_generator,
validation_steps=10,
epochs=nb_epoch,
callbacks=callbacks)
Predictions:
#extract features from frames of video
files = [f for f in os.listdir('.') if os.path.isfile(f)]
for f in files:
features = extractor_model.extract(f)
sequence.append(features)
np.save(sequence_path, sequence)
sequences = np.load("data_final.npy")
#convert numpy array tp 4 dimensions
sequences = np.expand_dims(sequences, axis=0)
sequences = np.expand_dims(sequences, axis=0)
prediction = model.predict(sequences)
Features extractor:
def extract(self, image_path):
#print(image_path)
img = image.load_img(image_path, target_size=(299, 299))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
# Get the prediction.
features = self.model.predict(x)
if self.weights is None:
# For imagenet/default network:
features = features[0]
else:
# For loaded network:
features = features[0]
return features
Keras complains about the shape is not None...
However I expect to receive some predictions of the model, but got this error. Please help. Thanks

Validation Accuracy stuck to 50% using keras VGG16 model

I am working on food classification project using keras (Tensorflow backend) VGG16 model with food-101 image dataset. However, I have faced some problems related to validation accuracy. (I believe the problem is about overfitting).
My validation accuracy is not increasing and always stick to around 48-51%
I have 40 classes (40 different food) with 700 images for train and 300 images for validation for each food. And I evaluated my model with a bunch of random food images. I have tried:
Decreasing Learning Rate
Changing Dropout Layer to 0.75
Image Augmentation
Although it helped me a little bit but it does not increased the validation accuracy drastically. I heard that someone used preprocess_input() function to increase validation accuracy, but I am not sure about it.
Here's my code:
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Dropout, Flatten, Dense
from keras import applications
from keras.utils import to_categorical
from keras import optimizers
# Dimensions of images
img_width, img_height = 150, 150
top_model_weights_path = 'test2_classes.h5'
train_data_dir = 'D:\intallation\dataset\dataset-101/food/train'
validation_data_dir = 'D:\intallation\dataset\dataset-101/food/validation'
nb_train_samples = 28000
nb_validation_samples = 12000
epochs = 80
batch_size = 32
def save_bottlebeck_features():
datagen = ImageDataGenerator(rescale=1. / 255)
# build the VGG16 network
model = applications.VGG16(include_top=False, weights='imagenet')
generator = datagen.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode=None,
shuffle=False)
bottleneck_features_train = model.predict_generator(
generator, nb_train_samples // batch_size)
np.save('test_trained.npy', bottleneck_features_train)
generator = datagen.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode=None,
shuffle=False)
bottleneck_features_validation = model.predict_generator(
generator, nb_validation_samples // batch_size)
np.save('test_validation.npy', bottleneck_features_validation)
def train_top_model():
# Class Labels for Training Data
datagen_top = ImageDataGenerator(rescale=1./255,
width_shift_range=0.05,
height_shift_range=0.05,
shear_range=0.05,
zoom_range=0.05,
fill_mode='nearest',
channel_shift_range=0.2*255)
datagen_top_val = ImageDataGenerator(rescale=1./255)
generator_top = datagen_top_val.flow_from_directory(
train_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode='categorical',
shuffle=False)
np.save('test_class_indices.npy', generator_top.class_indices)
num_classes = len(generator_top.class_indices)
train_data = np.load('test_trained.npy')
train_labels = generator_top.classes # Get Class Labels
train_labels = to_categorical(train_labels, num_classes=num_classes)
# Class Labels for Validation Data
generator_top = datagen_top.flow_from_directory(
validation_data_dir,
target_size=(img_width, img_height),
batch_size=batch_size,
class_mode=None,
shuffle=False)
validation_data = np.load('test_validation.npy')
validation_labels = generator_top.classes
validation_labels = to_categorical(validation_labels, num_classes=num_classes)
model = Sequential()
model.add(Flatten(input_shape=train_data.shape[1:]))
model.add(Dense(4096, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(num_classes, activation='softmax'))
sgd = optimizers.SGD(lr=1e-4, momentum=0.9, nesterov=True)
model.compile(optimizer=sgd,
loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(train_data, train_labels,
epochs=epochs,
batch_size=batch_size,
validation_data=(validation_data, validation_labels))
model.save_weights(top_model_weights_path)
save_bottlebeck_features()
train_top_model()

Dimension mismatch error in Keras with Tensorflow Backend

I'm fine-tuning the last convolutional block of the VGG16 model alongside the top-level classifier by starting from a trained network, then re-training it on a new dataset using very small weight updates. I instantiated the convolutional base of VGG16 and loaded its weights, added the previously defined fully-connected model on top, and loaded its weights and freeze the layers of the VGG16 model up to the last convolutional block. Here is the code:
from keras import applications
from keras.preprocessing.image import ImageDataGenerator
from keras import optimizers
from keras.models import Sequential
from keras.layers import Dropout, Flatten, Dense
# path to the model weights files.
weights_path = '../keras/examples/vgg16_weights.h5'
top_model_weights_path = 'modelvgg16.h5'
# dimensions of our images.
img_width, img_height = 32, 32
train_data_dir = 'data7/train'
validation_data_dir = 'data7/validation'
nb_train_samples = 1600
nb_validation_samples = 400
epochs = 100
batch_size = 16
# build the VGG16 network
model = applications.VGG16(weights='imagenet', include_top=False, input_shape=(224,224,3))
print('Model loaded.')
# build a classifier model to put on top of the convolutional model
top_model = Sequential()
top_model.add(Flatten(input_shape=model.output_shape[1:]))
top_model.add(Dense(256, activation='relu'))
top_model.add(Dropout(0.5))
top_model.add(Dense(1, activation='sigmoid'))
# note that it is necessary to start with a fully-trained
# classifier, including the top classifier,
# in order to successfully do fine-tuning
top_model.load_weights(top_model_weights_path)
# add the model on top of the convolutional base
model.add(top_model)
# set the first 25 layers (up to the last conv block)
# to non-trainable (weights will not be updated)
for layer in model.layers[:15]:
layer.trainable = False
# compile the model with a SGD/momentum optimizer
# and a very slow learning rate.
model.compile(loss='binary_crossentropy',
optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
metrics=['accuracy'])
# prepare data augmentation configuration
train_datagen = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
test_datagen = ImageDataGenerator(rescale=1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='binary')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir,
target_size=(img_height, img_width),
batch_size=batch_size,
class_mode='binary')
# fine-tune the model
model.fit_generator(
train_generator,
samples_per_epoch=nb_train_samples,
epochs=epochs,
validation_data=validation_generator,
nb_val_samples=nb_validation_samples)
On executing the script I get:
ValueError: Dimension O in both shapes must be equal, but are 25088 and 512 for 'Assign_26' (op: 'Assign') with input shapes: [25088, 256], [512, 256].
The error pops up with this instruction:
top_model.load_weights(top_model_weights_path)
Kindly help with the modification of the code. Thanks in advance.

Problems with input shapes at finetuning VGG with Keras

I'm trying to finetune the last layer of the VGG-16. Here is the part of the code where i make the new model:
def train2false(model):
for layer in model.layers:
layer.trainable = False
return model
def define_training_layers(model):
model.layers = model.layers[0:21]
model = train2false(model)
last_layer = model.get_layer('fc7')
out = Dense(n_classes, activation='softmax', name='fc8')(last_layer)
model = Model(input=model.input, output=out)
return model
def compile_model(epochs, lrate, model):
decay = lrate / epochs
sgd = SGD(lr=lrate, momentum=0, decay=0.0002, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
print (model.summary())
return model
def train_evaluate(model, X_train, y_train, X_test, y_test, epochs):
model.fit(X_train, y_train, validation_data=(X_test, y_test), nb_epoch=epochs, batch_size=32)
# Final evaluation of the model
scores = model.evaluate(X_test, y_test, verbose=0)
print("Accuracy: %.2f%%" % (scores[1]*100))
return model
X_train, X_test, labels_test, labels_train, n_classes = load_dataset()
image_input = Input(shape=(3, 224, 224))
vgg_model = VGGFace(input_tensor= image_input, include_top=True)
custom_vgg_model = define_training_layers(vgg_model)
custom_vgg_model = compile_model(epochs=50, lrate=0.001, model=custom_vgg_model)
custom_vgg_model = train_evaluate(custom_vgg_model, X_train=X_train, y_train=labels_train, X_test=X_test, y_test=labels_test, epochs=50)
I get the following error:
tensorflow.python.framework.errors_impl.InvalidArgumentError:
Dimension 1 in both shapes must be equal, but are 1000 and 2622 for
'Assign_30' (op: 'Assign') with input shapes: [4096,1000],
[4096,2622].
It works for me if i try to finetune all the fully connected part with include_top=False instead of just the softmax activation.
Is there something that i'm missing?
Solved!!! I've take the pre-trained weights from https://github.com/rcmalli/keras-vggface/releases/download/v1.0/rcmalli_vggface_th_weights_th_ordering.h5 which has 2622 number of outputs and i had 1000 outputs. So just change the number of outputs for the last layer in VGG.py