keras blog autoencoder code
I am trying to run the code for Convolutional Autoencode from
https://blog.keras.io/building-autoencoders-in-keras.html
from keras.layers import Input, Dense, Convolution2D, MaxPooling2D, UpSampling2D
from keras.models import Model
input_img = Input(shape=(1, 28, 28))
x = Convolution2D(16, 3, 3, activation='relu', border_mode='same')(input_img)
x = MaxPooling2D((2, 2), border_mode='same')(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
x = MaxPooling2D((2, 2), border_mode='same')(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
encoded = MaxPooling2D((2, 2), border_mode='same')(x)
# at this point the representation is (8, 4, 4) i.e. 128-dimensional
Convolution2D(8, 3, 3, activation='relu', border_mode='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Convolution2D(8, 3, 3, activation='relu', border_mode='same')(x)
x = UpSampling2D((2, 2))(x)
x = Convolution2D(16, 3, 3, activation='relu')(x)
x = UpSampling2D((2, 2))(x)
decoded = Convolution2D(1, 3, 3, activation='sigmoid', border_mode='same')(x)
autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
after running it I run this code for training :
from keras.datasets import mnist
import numpy as np
(x_train, _), (x_test, _) = mnist.load_data()
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_train = np.reshape(x_train, (len(x_train), 1, 28, 28))
x_test = np.reshape(x_test, (len(x_test), 1, 28, 28))
now I want to plot the result I using callback ! I type this
tensorboard --logdir=/tmp/autoencoder
in my terminal and it successfully switch back to theano but when I run
from keras.callbacks import TensorBoard
autoencoder.fit(x_train, x_train,
nb_epoch=50,
batch_size=128,
shuffle=True,
validation_data=(x_test, x_test),
callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])
it still imply that not switch back to tensorflow. Does anyone know how to fix it?
RuntimeError Traceback (most recent call last)
<ipython-input-4-fc8458b2c2ba> in <module>()
6 shuffle=True,
7 validation_data=(x_test, x_test),
----> 8 callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])
/home/hoda/anaconda2/lib/python2.7/site-packages/keras/callbacks.pyc in __init__(self, log_dir, histogram_freq, write_graph, write_images)
487 super(TensorBoard, self).__init__()
488 if K._BACKEND != 'tensorflow':
--> 489 raise RuntimeError('TensorBoard callback only works '
490 'with the TensorFlow backend.')
491 self.log_dir = log_dir
RuntimeError: TensorBoard callback only works with the TensorFlow backend.
To switch to the Tensorflow backend you have to edit the keras.json file located in ~/.keras.
You should see a line "backend": "theano", change "theano" to "tensorflow" and if Tensorflow is properly installed it should work and the line "Using TensorFlow backend." should appear when you import Keras.
Related
I need to add a Cropping2D layer where the left and right crop arguments are determined dynamically by the output of previous layers. I.E., the left_crop and right_crop arguments are not known at code-time. However, I seem unable to access the value of a previous tensor in the model. Here is my code:
input1 = Input(name='dirty', shape=(IMG_HEIGHT, None, 1), dtype='float32')
input2 = Input(name='x0', shape=(), dtype='int32')
input3 = Input(name='x1', shape=(), dtype='int32')
# Encoder
conv1 = Conv2D(48, kernel_size=(3, 3), activation='relu', padding='same', name='conv1')(input1)
pool1 = MaxPooling2D(pool_size=(2, 2), strides=(2, 2), name='pool1')(conv1)
conv2 = Conv2D(64, kernel_size=(3, 3), activation='relu', padding='same', name='conv2')(pool1)
# Decoder
deconv2 = Conv2DTranspose(48, kernel_size=(3, 3), activation='relu', padding='same', name='deconv2')(conv2)
depool1 = UpSampling2D(size=(2, 2), name='depool1')(deconv2)
output1 = Conv2DTranspose(1, kernel_size=(3, 3), activation='relu', padding='same', name='clean')(depool1)
_, _, width, _ = K.int_shape(output1)
left = K.eval(input2)
right = width - K.eval(input3)
output2 = Cropping2D(name='clean_snippet', cropping=((0, 0), (left, right)))(output1)
That produces the following error:
Traceback (most recent call last):
File "test.py", line 81, in <module>
left = K.eval(input2)
File "/Users/garnet/Library/Python/3.8/lib/python/site-packages/keras/backend.py", line 1632, in eval
return get_value(to_dense(x))
File "/Users/garnet/Library/Python/3.8/lib/python/site-packages/keras/backend.py", line 4208, in get_value
return x.numpy()
AttributeError: 'KerasTensor' object has no attribute 'numpy'
I'm using TF 2.10.0 with Keras 2.10.0. I've tried both with and without eager mode enabled. My question is specifically about the four lines after the "HERE'S THE AREA IN QUESTION..." comment in my code above. How can I access previous layer values to use them as an argument (not the input layer) to Cropping2D(). Any ideas?
For context, here's my entire code:
import tensorflow as tf
import cv2
import random
import os
import numpy as np
from tensorflow.keras import backend as K
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import SGD
from tensorflow.keras.layers import Conv2D, Conv2DTranspose, MaxPooling2D, Cropping2D, UpSampling2D, Input
from tensorflow.keras import losses
SNIPPET_WIDTH = 40
IMG_HEIGHT = 60
def get_data(paths):
for path in paths:
clean = cv2.imread(path.decode('utf-8'), cv2.IMREAD_GRAYSCALE)
h, w = clean.shape
dirty = cv2.blur(clean, (random.randint(1, 5), random.randint(1, 5)))
x0 = random.randint(0, w - SNIPPET_WIDTH)
x1 = x0 + SNIPPET_WIDTH
y0 = 0
y1 = h - 1
clean_snippet = clean[y0:y1, x0:x1]
dirty[y0:y1, x0:x1] = 0 # masked out region
dirty = (256. - dirty.astype(np.float32)) / 255.
dirty = tf.convert_to_tensor(np.expand_dims(dirty, axis=2))
x0 = tf.convert_to_tensor(x0)
x1 = tf.convert_to_tensor(x1)
clean = (256. - clean.astype(np.float32)) / 255.
clean = tf.convert_to_tensor(np.expand_dims(clean, axis=2))
clean_snippet = (256. - clean_snippet.astype(np.float32)) / 255.
clean_snippet = tf.convert_to_tensor(np.expand_dims(clean_snippet, axis=2))
yield {'dirty': dirty, 'x0': x0, 'x1': x1}, {'clean': clean, 'clean_snippet': clean_snippet}
train_directory = 'data/training/'
files = os.listdir(train_directory)
paths = []
for f in files:
filename = os.fsdecode(f)
paths.append(train_directory + filename)
train_ds = tf.data.Dataset.from_generator(get_data, args=[paths], output_signature=(
{
'dirty': tf.TensorSpec(shape=(IMG_HEIGHT, None, 1), dtype=tf.float32),
'x0': tf.TensorSpec(shape=(), dtype=tf.int32),
'x1': tf.TensorSpec(shape=(), dtype=tf.int32)
},
{
'clean': tf.TensorSpec(shape=(IMG_HEIGHT, None, 1), dtype=tf.float32),
'clean_snippet': tf.TensorSpec(shape=(IMG_HEIGHT, None, 1), dtype=tf.float32)
}
))
bucket_sizes = [400, 500, 600, 700, 800]
bucket_batch_sizes = [16, 16, 16, 16, 16, 16]
train_ds = train_ds.bucket_by_sequence_length(element_length_func=lambda x, y: tf.shape(y['clean'])[1],
bucket_boundaries=bucket_sizes, bucket_batch_sizes=bucket_batch_sizes)
input1 = Input(name='dirty', shape=(IMG_HEIGHT, None, 1), dtype='float32')
input2 = Input(name='x0', shape=(), dtype='int32')
input3 = Input(name='x1', shape=(), dtype='int32')
# Encoder
conv1 = Conv2D(48, kernel_size=(3, 3), activation='relu', padding='same', name='conv1')(input1)
pool1 = MaxPooling2D(pool_size=(2, 2), strides=(2, 2), name='pool1')(conv1)
conv2 = Conv2D(64, kernel_size=(3, 3), activation='relu', padding='same', name='conv2')(pool1)
# Decoder
deconv2 = Conv2DTranspose(48, kernel_size=(3, 3), activation='relu', padding='same', name='deconv2')(conv2)
depool1 = UpSampling2D(size=(2, 2), name='depool1')(deconv2)
output1 = Conv2DTranspose(1, kernel_size=(3, 3), activation='relu', padding='same', name='clean')(depool1)
# HERE'S THE AREA IN QUESTION...
_, _, width, _ = K.int_shape(output1)
left = K.eval(input2)
right = width - K.eval(input3)
output2 = Cropping2D(name='clean_snippet', cropping=((0, 0), (left, right)))(output1)
# ...END AREA IN QUESTION
model = Model(inputs=[input1, input2, input3], outputs=[output1, output2])
optimizer = SGD(lr=0.02, decay=1e-6, momentum=0.9, nesterov=True, clipnorm=5)
loss_fcns = {'clean': losses.MeanAbsoluteError(), 'clean_snippet': losses.MeanAbsoluteError()}
model.compile(loss=losses.MeanAbsoluteError(), optimizer=optimizer, metrics=['acc'])
model.fit(x=train_ds, y=None, epochs=1000, shuffle=True, verbose=1)
Here's the working solution inspired by #Yaoshiang's comment:
import tensorflow as tf
import cv2
import random
import os
import numpy as np
from tensorflow.keras.models import Model
from tensorflow.keras.optimizers import SGD
from tensorflow.keras.layers import Conv2D, Conv2DTranspose, MaxPooling2D, Cropping2D, UpSampling2D, Input, Multiply
from tensorflow.keras import losses
SNIPPET_WIDTH = 40
IMG_HEIGHT = 60
def normalize(img):
return np.expand_dims((256. - img.astype(np.float32)) / 255., axis=2)
def get_data(paths):
for path in paths:
clean = cv2.imread(path.decode('utf-8'), cv2.IMREAD_GRAYSCALE)
h, w = clean.shape
dirty = cv2.blur(clean, (random.randint(1, 5), random.randint(1, 5)))
x0 = random.randint(0, w - SNIPPET_WIDTH)
x1 = x0 + SNIPPET_WIDTH
y0 = 0
y1 = h - 1
dirty[y0:y1, x0:x1] = 0 # masked out region
dirty = normalize(dirty)
clean = normalize(clean)
mask = np.zeros((h, w, 1), dtype=np.float32)
mask[:, x0:x1, :] = 1.0
clean_snippet = clean * mask
clean = tf.convert_to_tensor(clean)
dirty = tf.convert_to_tensor(dirty)
mask = tf.convert_to_tensor(mask)
clean_snippet = tf.convert_to_tensor(clean_snippet)
yield {'dirty': dirty, 'mask': mask}, {'clean': clean, 'clean_snippet': clean_snippet}
train_directory = 'data/training/'
files = os.listdir(train_directory)
paths = []
for f in files:
filename = os.fsdecode(f)
paths.append(train_directory + filename)
train_ds = tf.data.Dataset.from_generator(get_data, args=[paths], output_signature=(
{
'dirty': tf.TensorSpec(shape=(IMG_HEIGHT, None, 1), dtype=tf.float32),
'mask': tf.TensorSpec(shape=(IMG_HEIGHT, None, 1), dtype=tf.float32)
},
{
'clean': tf.TensorSpec(shape=(IMG_HEIGHT, None, 1), dtype=tf.float32),
'clean_snippet': tf.TensorSpec(shape=(IMG_HEIGHT, None, 1), dtype=tf.float32)
}
))
bucket_sizes = [400, 500, 600, 700, 800]
bucket_batch_sizes = [16, 16, 16, 16, 16, 16]
train_ds = train_ds.bucket_by_sequence_length(element_length_func=lambda x, y: tf.shape(y['clean'])[1],
bucket_boundaries=bucket_sizes, bucket_batch_sizes=bucket_batch_sizes)
input1 = Input(name='dirty', shape=(IMG_HEIGHT, None, 1), dtype='float32')
input2 = Input(name='mask', shape=(IMG_HEIGHT, None, 1), dtype='float32')
# Encoder
conv1 = Conv2D(48, kernel_size=(3, 3), activation='relu', padding='same', name='conv1')(input1)
pool1 = MaxPooling2D(pool_size=(2, 2), strides=(2, 2), name='pool1')(conv1)
conv2 = Conv2D(64, kernel_size=(3, 3), activation='relu', padding='same', name='conv2')(pool1)
# Decoder
deconv2 = Conv2DTranspose(48, kernel_size=(3, 3), activation='relu', padding='same', name='deconv2')(conv2)
depool1 = UpSampling2D(size=(2, 2), name='depool1')(deconv2)
output1 = Conv2DTranspose(1, kernel_size=(3, 3), activation='relu', padding='same', name='clean')(depool1)
output2 = Multiply(name='clean_snippet')([output1, input2])
model = Model(inputs=[input1, input2], outputs=[output1, output2])
optimizer = SGD(lr=0.02, decay=1e-6, momentum=0.9, nesterov=True, clipnorm=5)
loss_fcns = {'clean': losses.MeanAbsoluteError(), 'clean_snippet': losses.MeanAbsoluteError()}
model.compile(loss=loss_fcns, optimizer=optimizer, metrics=['acc'])
model.fit(x=train_ds, y=None, epochs=1000, shuffle=True, verbose=1)
This is a classic bug that pops up because of graph mode. When you run this code, it's not really running the code, but Tensorflow introspects the python code and compiles it to a graph that runs well on GPU. Some of the things you think you can do in Python, you can't do when it's compiled.
In this case, tensor shapes must be fixed during execution, so you can't have dynamic output shapes during training.
Instead of cropping in the model, I'd just zero out the pixels you would have cropped. And in your dataset of training images, instead of dynamically adjusting the image sizes, dynamically adjust then pad with zeros to match the image size (and exception location). The MAE of those zero pixels in the ground truth and the hard coded zeros will be zero.
And drop the k.eval. You won't need it anymore - you can build masks with input2 and input3 directly using tf ops. Note that tf ops take the full batch, unlike Keras layers, and you can't loop, so you'll need to do it vectorized. You can do it with tf.sequence_mask.
I was studying different CNN architectures to predict the CIFAR10 dataset, and I found this interesting Github repository:
https://gist.github.com/wielandbrendel/ccf1ff6f8f92139439be
I tried to run the model, but it was created in 6 years ago and the following Keras command is no longer valid:
model.add(Convolution2D(32, 3, 3, 3, border_mode='full'))
How is this command translated into the modern Keras syntax for Conv2D?
I get an error in Keras when I try to input the sequence of integers in Convolution2D(32, 3, 3, 3, ...)?
I guess 32 is the number of channels, and then we specify a 3x3 kernel size, but I am not sure about the meaning of the last 3 mentioned (4th position).
PS. Changing border_mode into padding = 'valid' or 'same' returns the following error:
model.add(Convolution2D(32, 3, 3, 3, padding='valid'))
TypeError: __init__() got multiple values for argument 'padding'
The gist there you're following is backdated and also has some issues. You don't need to follow this now. Here is the updated version of it. Try this.
Imports and DataSet
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import (Dense, Dropout, Activation,
Flatten, Conv2D, MaxPooling2D)
from tensorflow.keras.optimizers import SGD, Adadelta, Adagrad
import tensorflow as tf
# parameters
batch_size = 32
nb_classes = 10
nb_epoch = 5
# the data, shuffled and split between tran and test sets
(X_train, y_train), (X_test, y_test) = tf.keras.datasets.cifar10.load_data()
# convert class vectors to binary class matrices
Y_train = tf.keras.utils.to_categorical(y_train, nb_classes)
Y_test = tf.keras.utils.to_categorical(y_test, nb_classes)
# train model
X_train = X_train.astype("float32") / 255
X_test = X_test.astype("float32") / 255
X_train.shape, y_train.shape, X_test.shape, y_test.shape
((50000, 32, 32, 3), (50000, 1), (10000, 32, 32, 3), (10000, 1))
Modeling
model = Sequential()
model.add(Conv2D(filters=32, kernel_size=(3, 3),
strides=(1, 1), activation='relu', padding="same"))
model.add(Activation('relu'))
model.add(Conv2D(filters=32, kernel_size=(3, 3),
strides=(1, 1), activation='relu', padding="same"))
model.add(Activation('relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.25))
model.add(Conv2D(filters=32, kernel_size=(3, 3),
strides=(1, 1), activation='relu', padding="same"))
model.add(Activation('relu'))
model.add(Conv2D(filters=32, kernel_size=(3, 3),
strides=(1, 1), activation='relu', padding="same"))
model.add(Activation('relu'))
model.add(MaxPooling2D((2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(nb_classes))
model.add(Activation('softmax'))
# let's train the model using SGD + momentum (how original).
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
Compile and Run
model.fit(X_train, Y_train, batch_size=batch_size, epochs=nb_epoch)
# test score & top 1 performance
score = model.evaluate(X_test, Y_test, batch_size=batch_size)
y_hat = model.predict(X_test)
yhat = np.argmax(y_hat, 1)
top1 = np.mean(yhat == np.squeeze(y_test))
print('Test score/Top1', score, top1)
The Convolutional2D is now named Conv2D, but there is still an alias for Convolutional2D, so that's not a problem.
The border_mode argument is not available anymore, the equivalent is padding, with options valid or same.
Try both to see if any of those fits the shapes of the outputs and allows to code to work.
I am building a multi-class CNN model but I am unable to compile the model due to loss shape error.
Both output layer and labels should have correct shapes; labels being (m, 1, 3) and final dense layer containing 3 perceptions with softmax activation
loss='categorical_crossentropy'
import numpy as np
import pandas as pd
from preprocess import DataLoader
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten, Conv3D, Dropout, MaxPooling3D
from tensorflow.keras.metrics import categorical_crossentropy
from tensorflow.keras import optimizers
target_width = 160
target_height = 192
target_depth = 192
num_classes = 3
batch_size = 4
data_loader = DataLoader(target_shape=(target_width, target_height, target_depth))
train, test = data_loader.Get_Data_List()
print("Train size: " + str(len(train)))
print("Test size: " + str(len(test)))
def custom_one_hot(labels):
label_dict = {"stableAD":np.array([0,0,1]),
"stableMCI":np.array([0,1,0]),
"stableNL":np.array([1,0,0])}
encoded_labels = []
for label in labels:
encoded_labels.append(label_dict[label].reshape(1,3))
return np.asarray(encoded_labels)
def additional_data_prep(train, test):
# Extract data from tuples
train_labels, train_data = zip(*train)
test_labels, test_data = zip(*test)
X_train = np.asarray(train_data)
X_test = np.asarray(test_data)
y_train = custom_one_hot(train_labels)
y_test = custom_one_hot(test_labels)
return X_train, y_train, X_test, y_test
X, y, X_test, y_test = additional_data_prep(train, test)
X = np.expand_dims(X, axis=-1).reshape((X.shape[0],target_width,target_height,target_depth,1))
X_test = np.expand_dims(X_test, axis=-1).reshape((X_test.shape[0],target_width,target_height,target_depth,1))
model = Sequential()
model.add(Conv3D(24, kernel_size=(13, 11, 11), activation='relu', input_shape=(target_width,target_height,target_depth,1), padding='same', strides=4))
model.add(MaxPooling3D(pool_size=(3, 3, 3), strides=2))
model.add(Dropout(0.1))
model.add(Conv3D(48, kernel_size=(6, 5, 5), activation='relu', padding='same'))
model.add(MaxPooling3D(pool_size=(3, 3, 3), strides=2))
model.add(Dropout(0.1))
model.add(Conv3D(24, kernel_size=(4, 3, 3), activation='relu'))
model.add(MaxPooling3D(pool_size=(3, 3, 3), strides=2))
model.add(Dropout(0.1))
model.add(Conv3D(8, kernel_size=(2, 2, 2), activation='relu'))
model.add(MaxPooling3D(pool_size=(1, 1, 1), strides=2))
model.add(Dropout(0.1))
model.add(Flatten())
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer=optimizers.Adam(learning_rate=0.0015),
metrics=['accuracy','categorical_crossentropy'])
model.fit(X, y, batch_size=batch_size, epochs=10, verbose=2, use_multiprocessing=True)
model.evaluate(X_test, y_test, verbose=2, use_multiprocessing=True)
Results in this error message:
Traceback (most recent call last):
File "train.py", line 70, in <module>
model.fit(X, y, batch_size=batch_size, epochs=10, verbose=2, use_multiprocessing=True)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 819, in fit
use_multiprocessing=use_multiprocessing)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 235, in fit
use_multiprocessing=use_multiprocessing)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 593, in _process_training_inputs
use_multiprocessing=use_multiprocessing)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 646, in _process_inputs
x, y, sample_weight=sample_weights)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 2383, in _standardize_user_data
batch_size=batch_size)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 2489, in _standardize_tensors
y, self._feed_loss_fns, feed_output_shapes)
File "/home/554282/.local/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_utils.py", line 810, in check_loss_and_target_compatibility
' while using as loss `' + loss_name + '`. '
ValueError: A target array with shape (8, 1, 3) was passed for an output of shape (None, 3) while using as loss `categorical_crossentropy`. This loss expects targets to have the same shape as the output.
The custom_one_hot function returns a [M, 1, 3] array. You should reshape that to [M, 3] since the output of the CNN is [M, 3]. M here is the batch size.
I am trying to train a model on Google Colab, in order to play around with training on TPU. However, I am running into the following error:
AttributeError Traceback (most recent call last)
<ipython-input-82-e74efc36d872> in <module>()
----> 1 tpu_model.fit_generator(training_set, steps_per_epoch = 8000, epochs = 25)
2 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/callbacks.py in configure_callbacks(callbacks, model, do_validation, batch_size, epochs, steps_per_epoch, samples, verbose, count_mode, mode)
118 callback_list.model.stop_training = False
119 # pylint: disable=protected-access
--> 120 if callback_list.model._ckpt_saved_epoch is not None:
121 # The attribute `_ckpt_saved_epoch` is supposed to be None at the start of
122 # training (it should be made None at the end of successful multi-worker
AttributeError: 'KerasTPUModel' object has no attribute '_ckpt_saved_epoch'
While trying to run the following code.
import tensorflow as tf
from tensorflow.keras import layers
from tensorflow.keras.preprocessing.image import ImageDataGenerator
import os
import zipfile
print(tf.VERSION)
local_zip = '/home/cats_and_dogs_filtered.zip'
zip_ref = zipfile.ZipFile(local_zip, 'r')
zip_ref.extractall('/home')
zip_ref.close()
def create_model():
classifier = tf.keras.models.Sequential()
classifier.add(layers.Conv2D(32, (3, 3), input_shape=(64, 64, 3), activation='relu'))
classifier.add(layers.MaxPooling2D(pool_size=(2, 2)))
classifier.add(layers.Conv2D(32, (3, 3), activation= 'relu'))
classifier.add(layers.MaxPooling2D(pool_size=(2, 2)))
classifier.add(layers.Flatten())
classifier.add(layers.Dense(units=128, activation= 'relu'))
classifier.add(layers.Dense(units=1, activation= 'sigmoid'))
return classifier
train_datagen = ImageDataGenerator(rescale = 1./255, shear_range = 0.2, zoom_range = 0.2, horizontal_flip = True)
training_set = train_datagen.flow_from_directory('/home/cats_and_dogs_filtered/train', target_size = (64, 64), batch_size = 32, class_mode = 'binary')
model = create_model()
TPU_WORKER = 'grpc://' + os.environ['COLAB_TPU_ADDR']
tpu_model = tf.contrib.tpu.keras_to_tpu_model( model, strategy=tf.contrib.tpu.TPUDistributionStrategy(tf.contrib.cluster_resolver.TPUClusterResolver(TPU_WORKER)))
tpu_model.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
tpu_model.save_weights('./tpu_model.h5', overwrite=True)
tpu_model.fit_generator(training_set, steps_per_epoch = 8000, epochs = 25)
I am not sure what is going on. I used similar code to train it on CPU (takes a long time to train).
I have four variables train_X, train_Y, test_X, test_Y, where train_X, train_Yare the training set, and test_X, test_Yare the test set. I have the following Keras neural network
from keras.models import Sequential, Model
from keras.optimizers import RMSprop
from keras.layers import Input, Dense, Convolution2D, LSTM, MaxPooling2D, \
UpSampling2D, RepeatVector, Flatten, Dropout, Activation
from keras.callbacks import TensorBoard
from keras.preprocessing.image import ImageDataGenerator
idg = ImageDataGenerator()
nb_epoch = 25
idg.fit(train_X)
input_data = Input(shape=(100, 100, 1))
conv1 = Convolution2D(32, 3, 3, activation='relu', border_mode='same')(input_data)
pool1 = MaxPooling2D((2, 2), border_mode='same')(conv1)
conv2 = Convolution2D(32, 3, 3, activation='relu', border_mode='same')(pool1)
pool2 = MaxPooling2D((2, 2), border_mode='same')(conv2)
conv3 = Convolution2D(64, 3, 3, activation='relu', border_mode='same')(pool2)
pool3 = MaxPooling2D((2, 2), border_mode='same')(conv3)
flatten = Flatten()(pool3)
dense1 = Dense(64)(flatten)
activation = Activation('relu')(dense1)
dropout = Dropout(0.5)(activation)
dense2 = Dense(1)(dropout)
output_data = Activation('sigmoid')(dense2)
model = Model(input_data, output_data)
model.compile(optimizer='adadelta', loss='mean_squared_error')
model.fit_generator(idg.flow(train_X, train_Y, batch_size=32, seed = 0),
samples_per_epoch = len(train_X), nb_epoch = nb_epoch,
validation_data = (test_X, test_Y), callbacks = [TensorBoard(log_dir='log_dir')])
However, the following line is giving me zeros for everything:
predictions = model.predict(test_X)
I checked obvious things, such as test_X being zero. My guess is that the problem is some kind of vanishing gradient issue. Any help is appreciated; thanks!