I am trying to set up a ConvLSTM2D architecture that takes as input and output the data with the following shapes:
Input shape: (1000, 10, 100, 100, 1)
Output shape: (1000, 5, 100, 100, 1)
As you can notice, the difference between these input and output shapes are timesteps. Is there any way I can set up such an architecture?
The following is the one that I am using right now, which doesn't allow me to specify input / output timesteps separately.
inputs = layers.Input(shape=(None, x_train.shape[2], x_train.shape[2], 1 ))
x = layers.ConvLSTM2D(filters=64, kernel_size=(5,5), padding="same", return_sequences=True, activation="relu")(inputs)
x = layers.BatchNormalization()(x)
x = layers.ConvLSTM2D(filters=64, kernel_size=(3,3), padding="same", return_sequences=True, activation="relu")(x)
x = layers.BatchNormalization()(x)
x = layers.ConvLSTM2D(filters=64, kernel_size=(1,1), padding="same", return_sequences=True, activation="relu")(x)
outputs = layers.Conv3D(filters=1, kernel_size=(3,3,3), activation="sigmoid", padding="same")(x)
model = keras.models.Model(inputs, outputs)
model.compile(loss=keras.losses.binary_crossentropy, optimizer=keras.optimizers.Adam())
model.summary()
Appreciate any help!
Related
I defined my model starting with inputs = tf.keras.Input(shape=(512, 512, 3), batch_size=BATCH_SIZE). Then I use model.fit with data of shape (1, 720, 1280, 3). The model still trains normally and the loss decreases. Why is that?
Thanks.
I tried to train with same shape. But the result did not turns out to be good.
So here is the code:
inputs = tf.keras.Input(shape=(512, 512, 3), batch_size=BATCH_SIZE) # REVISE,
x = tf.keras.layers.Cropping2D(cropping=((256, 0), (0, 0)))(inputs)
x = tf.keras.layers.Resizing(66,200)(x)
# x = x / 255
x = tf.keras.layers.Conv2D(24, (5,5), (2,2), activation="elu")(x)
x = tf.keras.layers.Conv2D(36, (5,5), (2,2), activation="elu")(x)
x = tf.keras.layers.Conv2D(48, (5,5), (2,2), activation="elu")(x)
x = tf.keras.layers.Conv2D(64, (3,3), activation="elu")(x)
x = tf.keras.layers.Dropout(0.2)(x)
x = tf.keras.layers.Conv2D(64, (3,3), activation="elu")(x)
x = tf.keras.layers.Flatten()(x)
x = tf.keras.layers.Dropout(0.2)(x)
x = tf.keras.layers.Dense(100, activation='elu')(x)
x = tf.keras.layers.Dense(50, activation='elu')(x)
x = tf.keras.layers.Dense(10, activation='elu')(x)
outputs = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(inputs=inputs, outputs=outputs)
print(model.summary())
optimizer = Adam(learning_rate=1e-3)
model.compile(loss='mse', optimizer=optimizer)
img_gen = image_data_generator(X_train, y_train, batch_size=1)
X = next(img_gen)[0]
print(X.shape)
print(model(X).shape)
# saves the model weights after each epoch if the validation loss decreased
checkpoint_callback = keras.callbacks.ModelCheckpoint(filepath=os.path.join(model_output_dir,'lane_navigation_new_nvmodel_big2'), verbose=1, save_best_only=True) # revise
with tf.device('/device:GPU:0'):
history = model.fit(image_data_generator( X_train, y_train, batch_size=BATCH_SIZE), # 重要！batch_size大了好训练，1个batch_size训不出东西！
steps_per_epoch=10, # 300
epochs=300,
validation_data = image_data_generator( X_valid, y_valid, batch_size=BATCH_SIZE),
validation_steps=10, # 200
verbose=1,
shuffle=1,
callbacks=[checkpoint_callback])
I used the CNN model with Keras to make an image binary classification, during the final prediction part, I defined such function below to output the prediction result:
model = keras.Sequential()
model.add(Conv2D(filters = 64, kernel_size = (3, 3), activation = 'relu', input_shape = ((256,256,3))))
model.add(MaxPooling2D(pool_size = (2, 2), strides=(2, 2)))
model.add(Conv2D(filters = 128, kernel_size = (3, 3), activation = 'relu'))
model.add(MaxPooling2D(pool_size = (2, 2), strides=(2, 2)))
model.add(Conv2D(filters = 256, kernel_size = (3, 3), activation = 'relu'))
model.add(MaxPooling2D(pool_size = (2, 2), strides=(2, 2)))
model.add(Flatten())
model.add(Dense(units = 512, activation = 'relu'))
model.add(Dense(units = 1,activation='sigmoid'))
model.compile(optimizer='adam',
loss=tf.keras.losses.BinaryCrossentropy(),
metrics=['accuracy'])
history = model.fit(
train_ds,
validation_data=valid_ds,
epochs=10)
def testing_image(image_directory):
test_image = image.load_img(image_directory, target_size = (256, 256))
test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis = 0)
result = model.predict(test_image)
print(result)
testing_image('/content/drive/MyDrive/testing/01.jpg')
The output is:
[[0.4733843]]
The output is always a decimal number, but I want the output the result as only
0or 1 and without the array representation.
Any help is appreciated.
Sigmoid activation function returns the values between 0 to 1 where the values <0.5 implies to category zero(0) and >0.5 implies to category one(1) in binary classification.
To get these binary numbers, you need to add one more line of code in testing_image() as below:
Fixed code:
def testing_image(image_directory):
test_image = image.load_img(image_directory, target_size = (256, 256))
test_image = image.img_to_array(test_image)
test_image = np.expand_dims(test_image, axis = 0)
#Changes in code
pred = model.predict(test_image)
result = np.where(pred > 0.5, 1, 0) #<--to get the binary category
print(result)
testing_image('/content/drive/MyDrive/testing/01.jpg')
BATCH_SIZE = 32 # ADVISED NOT TO CHANGE THIS
N_PAST = 10 # DO NOT CHANGE THIS
N_FUTURE = 10 # DO NOT CHANGE THIS
SHIFT = 1
model = tf.keras.models.Sequential([
tf.keras.layers.Conv1D(filters=32, kernel_size=5,
strides=1, padding="causal",
activation="relu",
input_shape=[None, 1]),
tf.keras.layers.LSTM(64, return_sequences=True),
tf.keras.layers.LSTM(64, return_sequences=True),
tf.keras.layers.Dense(30, activation="relu"),
tf.keras.layers.Dense(10, activation="relu"),
tf.keras.layers.Dense(N_FEATURES)
])
I build a time-siries forcasting model but i can't understand how to handle this.
which layer should i use to make this available?
Model input shape must be (BATCH_SIZE, N_PAST = 10, N_FEATURES = 1)
Model output shape must be (BATCH_SIZE, N_FUTURE = 10, N_FEATURES = 1)
the batch_size is not specified in the model.
model = tf.keras.models.Sequential([
tf.keras.layers.Conv1D(filters=32, kernel_size=5,
strides=1, padding="causal",
activation="relu",
input_shape=[N_PAST,1]),
tf.keras.layers.LSTM(64, return_sequences=True),
tf.keras.layers.Dense(30, activation="relu"),
tf.keras.layers.Dense(10, activation="relu"),
tf.keras.layers.Dense(N_FUTURE)
])
You don't need to specify batch_size in input layer. Just change input shape as follows:
input_shape=[N_PAST,1]
I am trying to create a UNET using VGG16 as first layers.
def BuildUNet2():
keras.backend.set_learning_phase(1)
inputs = keras.layers.Input(shape=(PATCH_SIZE, PATCH_SIZE, 3), name="inputs")
vggModel=keras.applications.VGG16(include_top=False, input_tensor=inputs)
layers = dict([(layer.name, layer) for layer in vggModel.layers])
print("Layers", len(layers), layers)
block1_conv2 = layers["block1_conv2"].output
block2_conv2 = layers["block2_conv2"].output
block3_conv3 = layers["block3_conv3"].output
block4_conv3 = layers["block4_conv3"].output
vggTop = layers["block5_conv3"].output
up6=keras.layers.concatenate([keras.layers.Conv2DTranspose(256, (2,2), strides=(2,2), padding="same")(vggTop), block4_conv3], axis=3)
conv61=keras.layers.Conv2D(256, 3, activation="relu", padding="same", kernel_initializer="he_normal")(up6)
conv62=keras.layers.Conv2D(256, 3, activation="relu", padding="same", kernel_initializer="he_normal")(conv61)
up7 = keras.layers.concatenate([keras.layers.Conv2DTranspose(128, (2, 2), strides=(2, 2), padding="same")(conv62), block3_conv3], axis=3)
conv71=keras.layers.Conv2D(128, 3, activation="relu", padding="same", kernel_initializer="he_normal")(up7)
conv72=keras.layers.Conv2D(128, 3, activation="relu", padding="same", kernel_initializer="he_normal")(conv71)
up8 = keras.layers.concatenate([keras.layers.Conv2DTranspose(64, (2, 2), strides=(2, 2), padding="same")(conv72), block2_conv2], axis=3)
conv81=keras.layers.Conv2D(64, 3, activation="relu", padding="same", kernel_initializer="he_normal")(up8)
conv82=keras.layers.Conv2D(64, 3, activation="relu", padding="same", kernel_initializer="he_normal")(conv81)
up9 = keras.layers.concatenate([keras.layers.Conv2DTranspose(32, (2, 2), strides=(2, 2), padding="same")(conv82), block1_conv2], axis=3)
conv91=keras.layers.Conv2D(32, 3, activation="relu", padding="same", kernel_initializer="he_normal")(up9)
conv92=keras.layers.Conv2D(32, 3, activation="relu", padding="same", kernel_initializer="he_normal")(conv91)
conv93=keras.layers.Conv2D(1, (1, 1), activation="sigmoid")(conv92)
model = keras.models.Model(input=[inputs], output=[conv93])
for layer in model.layers[:19]:
layer.trainable = False
model.compile(optimizer=keras.optimizers.Adam(lr=1e-5), loss=metric.dice_coef_loss,
metrics=[metric.dice_coef, "accuracy"])
model.summary()
return model
I am training with:
with h5py.File(parms.training, "r") as trainingsFile:
wrk=trainingsFile["work"].value
np.random.seed(42)
np.random.shuffle(wrk)
limit=int(wrk.shape[0]*0.8)
trainData=wrk[:limit]
valData=wrk[limit:]
trainGen=DataGenerator(trainData, parms.batchSize)
valGen=DataGenerator(valData, parms.batchSize)
bestCheckpoint = keras.callbacks.ModelCheckpoint("best.h5",
monitor="val_loss",
save_best_only=True,
save_weights_only=False)
regCheckpoint = keras.callbacks.ModelCheckpoint("checkpoint-{epoch:04d}.h5", period=10)
csvLog = keras.callbacks.CSVLogger("log.csv", append=True)
runName = datetime.datetime.now().isoformat("#")[:19].replace(":", "-")
tensorBoard = keras.callbacks.TensorBoard(log_dir="./logs/%s/" % runName)
lrPlateau = keras.callbacks.ReduceLROnPlateau(monitor="val_loss", factor=0.2, patience=10, cooldown=5)
model.fit_generator(trainGen,
epochs=parms.epochs,
steps_per_epoch=trainGen.__len__(),
validation_data=valGen,
validation_steps=valGen.__len__(),
callbacks=[bestCheckpoint, regCheckpoint, csvLog, tensorBoard, lrPlateau],
use_multiprocessing=False,
)
The DataGenerator is defined as:
class DataGenerator(keras.utils.Sequence):
def __init__(self, data, batchSize):
self.data=data
self.batchSize=batchSize
def __len__(self):
return int((self.data.shape[0]+self.batchSize-1)/(self.batchSize))
def __getitem__(self, item):
X=np.zeros((self.batchSize, self.data.shape[1], self.data.shape[2], 3), dtype=np.float32)
Y=np.zeros((self.batchSize, self.data.shape[1], self.data.shape[2]), dtype=np.float32)
j=0
wrk=np.zeros((self.data.shape[1], self.data.shape[2], self.data.shape[3]), dtype=np.float32)
for i in range(item*self.batchSize, min((item+1)*self.batchSize,self.data.shape[0])):
wrk=self.data[i, :, :, :]
if random.random() < 0.5:
wrk=wrk[:, ::-1, :]
if random.random() < 0.5:
wrk = wrk[::-1, :, :]
direction = int(random.random() * 4) * 90
if direction:
wrk = imutils.rotate(wrk, direction)
X[j, :, :, :]=wrk[:, :, 0: 3]
Y[j, :, :]=wrk[:, :, 3]
j+=1
X=X.resize((j, X.shape[1], X.shape[2], X.shape[3]))
Y=Y.resize((j, Y.shape[1], Y.shape[2]))
return X, Y
Trying to train the model results in
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'conv2d_9_sample_weights' with dtype float and shape [?]
Even explicitly returning a sample_weight (an addtional np.ones((j), dtype=np.float32) from the DataGenerator does not solve the problem.
What's wrong?
How do I correct it?
The problem was with DataGenerator.getitem():
resize does not return a new numpy array. It changes the original array and returns nothing. Therefore the getitem method returned None, None.
The keras error messages is misleading.
I am trying to use a function to add layers to a very deep CNN using keras. Here is my function:
def add_layer(input_shape, kernel_size, filters, count):
x = Conv2D(filters, (kernel_size, kernel_size), padding = 'same', activation= None)(Input(input_shape))
x = BatchNormalization()(x)
x = Activation('relu')(x)
x = Conv2D(filters, (kernel_size, kernel_size), padding = 'same', activation= None)(x)
x = BatchNormalization()(x)
x = Activation('relu')(x)
return keras.layers.add([x,Input(input_shape)])
When I call this function from:
x = Input(shape = (6,264,264))
y = Conv2D(64, (7, 7), padding='same', activation='relu')(x)
y = MaxPooling2D((2,2))(y)
y = add_layer(y.shape, 3, 64, 3)
It gives following error:
ValueError: Input 0 is incompatible with layer conv2d_2: expected ndim=4, found ndim=5
When I remove the add_layer function and simply terminate the maxpooling to a dense layer, I get:
AttributeError: 'Tensor' object has no attribute 'ndim'
What could be the problem ? (Additionally my input has 50 np arrays of size (6,264,264)) i.e (50,6,264,264)
Pretty sure that your line,
x = Conv2D(filters, (kernel_size, kernel_size), padding = 'same', activation= None)(Input(input_shape))
should be
x = Conv2D(filters, (kernel_size, kernel_size), padding = 'same', activation= None)(Input(batch_shape=input_shape))
By default the ny_layer.shape would add a batch size in the shape. However, for Input(input_shape) the first argument assumes the shape is without is batch size and adds another dimension to its output. This would explain the origin of the extra dimension.