I'm trying to resize some images to use them with Inception. I want to do it as a separate preprocessing step to speed things up later. Running tf.image.resize on all of the images at once crashes, as does a loop.
I'd like to do it in batches, but I don't know how to do that with making it part of my neural network model - I want to keep it outside as a separate preprocessing step.
I made this:
inception = tf.keras.applications.inception_v3.InceptionV3(include_top=True, input_shape=(299, 299, 3))
inception = tf.keras.Model([inception.input], [inception.layers[-2].output])
for layer in inception.layers:
layer.trainable = False
resize_incept = tf.keras.Sequential([
tf.keras.layers.Resizing(299, 299),
inception])
resize_incept.compile()
So can I just call it on my images? But then how do I batch it? When I call it like resize_incept(images) it crashes (too big), but if I call resize_incept(images, batch_size = 25), it doesn't work (TypeError: call() got an unexpected keyword argument 'batch_size').
EDIT: I'm trying to figure out if I can use tf.data.dataset for this
EDIT2: I've put my data (which was an array of (batch, 32, 32, 3) into tf.data.dataset so I can try this:
train_dataset = train_dataset.map(lambda x, y: (resize_incept(x), y))
But when I try it, it give me this error:
ValueError: Input 0 is incompatible with layer model_1: expected shape=(None, 299, 299, 3), found shape=(299, 299, 3)
The problem seems to be that whatever is coming out of the resize layer is somehow wrong for going into the inception layer (because what I'm putting in at first is (32,32,3) and there are no complaints about those dimensions)? But the inception layer, already has input_shape=(299, 299, 3) so I would think that's what shape it would take?
If you want it as a seperate preprocessing step. I would recommend to use the image_dataset_from_directory() function included in keras preprocesssing.
tf.keras.preprocessing.image_dataset_from_directory(
directory,
labels="inferred",
label_mode="int",
class_names=None,
color_mode="rgb",
batch_size=32,
image_size=(256, 256),
shuffle=True,
seed=None,
validation_split=None,
subset=None,
interpolation="bilinear",
follow_links=False,
crop_to_aspect_ratio=False,
**kwargs)
You can manipulate the batch size and the wanted image size and choose wether to split the data or not. Also, be carefull with crop_to_aspect_ratio as you may lose important features from the cropped areas.
If you want to know more about it's parameters and a code example you could check the
image_dataset_from_directory documentation.
Good luck!
Related
Hello I am trying to get an output of an array of 7 classes. But when I run my code it says that it expects my data output labels to have some other shape. Here is my code -
def make_model(self):
self.model.add(InceptionV3(include_top=False,
input_shape=(self.WIDTH, self.HEIGHT, 3),
weights="imagenet"))
self.model.add(Dense(7, activation='softmax'))
self.model.layers[0].trainable = False
My model compilation and fitment part
def train(self):
self.model.compile(optimizer=self.optimizer, loss='mse', metrics=['accuracy'])
self.model.fit(x=x, y=y, batch_size=64,
validation_split=0.15, shuffle=True, epochs=self.epochs,
callbacks=[self.tensorboard, self.reducelr])
I get the error -
File "model.py", line 60, in train
callbacks=[self.tensorboard, self.reducelr])
ValueError: A target array with shape (23639, 7) was passed for an output of shape (None, 6, 13, 7) while using as loss `mean_squared_error`. This loss expects targets to have the same shape as the output.
Now here it is saying that it expected (None, 6, 13, 7) however i gave it labels - (23639, 7)
Now we can clearly see that in the self.model.add(Dense(7, activation='softmax')) I have specified 7 as the number of output categories
Here is the model summary -
So can someone tell me what is wrong here
By the way i did try using categorical_crossentropy to see if it makes a difference but it didn't.
In case you wanted the full code -
Full Code
The problem is in the output of the InceptionV3... it returns 4D sequences, you need to reduce the dimensionality before the final dense layer in order to match the target dimensionality (2D). you can do this using Flatten or GlobalPooling layers.
If yours is a classification problem I also recommend you use categorical_crossentropy (if you have one-hot encoded label) or sparse_categorical_crossentropy (if u have integer encoded labels). mse is suited for regression problems
I trained a simple MLP model using new tf.keras version 2.2.4-tf. Here is how the model look like:
input_layer = Input(batch_shape=(138, 28))
first_layer = Dense(30, activation=activation, name="first_dense_layer_1")(input_layer)
first_layer = Dropout(0.1)(first_layer, training=True)
second_layer = Dense(15, activation=activation, name="second_dense_layer")(first_layer)
out = Dense(1, name='output_layer')(second_layer)
model = Model(input_layer, out)
I'm getting an error when I try to do prediction prediction_result = model.predict(test_data, batch_size=138). The test_data has shape of (69, 28), so it is smaller than the batch_size which is 138. Here is the error, it seems like the issue comes from first dropout layer:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [138,30] vs. [69,30]
[[node model/dropout/dropout/mul_1 (defined at ./mlp_new_tf.py:471) ]] [Op:__inference_distributed_function_1700]
The same solution works with no issues in older version of keras (2.2.4) and tensorflow (1.12.0). How can I fix the issue? I don't have more data for test, so I can't change the test_data set to have more data points!
Since you are seeing the issue at prediction time, one way of getting around this would be to pad the test data to be a multiple of your batch size. It shouldn't slow down the prediction since the number of batches doesn't change. numpy.pad should do the trick.
I currently have a simple LSTM model implemented in Keras, with a training set x_train of dimensions (1,18227, 98) and a test set x_test of dimensions (1,3217, 98) timesteps/features, respectively. Currently the model is training without hitch, but when I attempt to evaluate using my test set, I receive this error:
File "keras_LSTM.py", line 170, in <module>
loss, f1, precision = model.evaluate(x_test, y_test,
batch_size=batch_size)
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-
packages/keras/engine/training.py", line 1102, in evaluate
batch_size=batch_size)
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/keras/engine/training.py", line 751, in _standardize_user_data
exception_prefix='input')
File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-
packages/keras/engine/training_utils.py", line 138, in
standardize_input_data
str(data_shape))
ValueError: Error when checking input: expected lstm_1_input to have
shape (18227, 98) but got array with shape (3217, 98)
Any help would be greatly appreciated - will provide code if needed. It should also be noted that my input shapes are 3 dimensional - however the error report omits the batch_size dimension and output a tuple of (sequence_length, feature_number).
Keras LSTM layer expects the input to be 3 dims as (batch_size, seq_length, input_dims), but you have assigned it wrongly. Try this
First reshape your data as:
(1,18227, 98) and a test set x_test of dimensions (1,3217, 98)
X_train = x_train.reshape(-1,98)
X_test = x_test.reshape(-1,98)
Now, use choose seq_length, i chose 10.
seq_length = 10
X_train1 = []
X_test1 = []
for i in range(0, X_train.shape[0] - seq_length, 1):
X_train1.append(X_train[i:i+seq_length])
X_test1.append(X_test[i:i+seq_length])
# labels.append(labels[i+seq_length-1])
import numpy as np
X_train1 = np.reshape(X_train1, (-1, seq_length, 98))
X_test1 = np.reshape(X_test1, (-1, seq_length, 98))
Now, you are good to go
input_dims = 98 # an integer
seq_length = 10 # an integer
model = Sequential()
model.add(LSTM(128, activation='relu', input_shape=(seq_length, input_dims), return_sequences=True))
You were using single sequence, for your model, which is ineffective way.
Ankish Bansal's idea of splitting the long sequence in small windows may be a good approach. But you might want to keep the entire sequence connected for some reason.
In that case, you should set your input_shape=(None,98), this way, your model accepts any sequence length. (Provided you don't use any Flatten layer or others that require fixed dimensions.)
But, if there is "more than one sequence" in your data, you should probably review everything, because the number of sequences usually should be the batch size.
It is is a common practice in convolutional neural networks to oversample a given image during inference,
I.e to create a batch from different transformation of the same image (most common - different crops and mirroring), transfer the entire batch through the network and average (or another kind of reducing function) over the results to get a single prediction (caffe example),
How can this approach be implemented in tensorflow?
You can take a look at the TF cnn tutorial. In particular, the function distorted_inputs does the image preprocessing step.
In short, there are a couple of TF functions in the tf.image package that help with distorting the images. You can use either them or regular numpy functions to create an extra dimension for the output, for which you can average the results:
Before:
input_place = tf.placeholder(tf.float32, [None, 256, 256, 3])
prediction = some_model(input_place) # size: [None]
sess.run(prediction, feed_dict={input_place: batch_of_images})
After:
input_place = tf.placeholder(tf.float32, [None, NUM_OF_DISTORTIONS, 256, 256, 3])
prediction = some_model(input_place) # make sure it is of size [None, NUM_DISTORTIONS]
new_prediction = tf.reduce_mean(prediction, axis=1)
new_batch = np.zeros(batch_size, NUM_OF_DISTORTIONS, 256, 256, 3)
for i in xrange(len(batch_of_images)):
for f in xrange(len(distortion_functions)):
new_batch[i, f, :, :, :] = distortion_functions[f](batch_of_images[i])
sess.run(new_prediction, feed_dict={input_place: new_batch})
Take a look at TF's image-related functions. You could apply those transformations at test time to some input image, and stack all of them together to make a batch.
I imagine you could also do this using OpenCV or some other image processing tool. I don't see a need to do it in the computation graph. You could create the batches beforehand, and pass it through in feed_dict.
I would like to do a transfer learning from the given inceptionV3 in tensorflow example. Following the classify image example and the operator and tensor names given here https://github.com/AKSHAYUBHAT/VisualSearchServer/blob/master/notebooks/notebook_network.ipynb I can create my graph. But when, I put a batch of images of size (100, 299, 299, 3) in the pre-computed inception graph, I get the following shape error at the pool_3 layer :
ValueError: Cannot reshape a tensor with 204800 elements to shape [1, 2048] (2048 elements)
It seems that this inceptionV3 graph doesn't accept image batch as input. am I wrong ?
Actually it works for transfer learning if you extract the right thing. There is no problem feeding a batch of images in the shape of [N, 299, 299, 3] as ResizeBilinear:0 and then using the pool_3:0 tensor. It's the reshaping afterwards that breaks, but you can reshape yourself (you'll have your own layers afterwards anyway). If you wanted to use the original classifier with a batch, you could add your own reshaping on top of pool_3:0 and then add the softmax layer, reusing the weights/biases tensors of the original softmax.
TLDR: With double_img being a stack of two images with shape (2, 299, 299, 3) this works:
pooled_2 = sess.graph.get_tensor_by_name("pool_3:0").eval(session=sess, feed_dict={'ResizeBilinear:0':double_img})
pooled_2.shape
# => (2, 1, 1, 2048)
You're not wrong. This seems like a very reasonable feature request, so I've opened a ticket for it on github. Follow that for updates.
Something like this should do it:
with g.as_default():
inputs = tf.placeholder(tf.float32, shape=[batch_size, 299, 299, 3],
name='input')
with slim.arg_scope(inception.inception_v3_arg_scope()):
logits, end_points = inception.inception_v3( inputs,
num_classes=FLAGS.num_classes, is_training=False)
variables_to_restore = lim.get_variables_to_restore(exclude=exclude)
sess = tf.Session()
saver = tf_saver.Saver(variables_to_restore)
Then you should be able to call the operation:
sess.run("pool_3:0",feed_dict={'ResizeBilinear:0':images})
etarion made a very good point. However, we don't have to reshape it ourselves; instead, we could change the value of shape that reshape takes as input. I.e.,
input_tensor_name = 'import/input:0'
shape_tensor_name = 'import/InceptionV3/Predictions/Shape:0'
output_tensor_name= 'import/InceptionV3/Predictions/Reshape_1:0'
output_tensor = tf.import_graph_def(
graph.as_graph_def(),
input_map={input_tensor_name: image_batch,
shape_tensor_name: [batch_size, num_class]},
return_elements=[output_tensor_name])
These tensor names are based on inception_v3_2016_08_28_frozen.pb.