I have a shared network models to make a model that has multiple inputs. my code is :
f1 = 128
f2 = 256
f3 = 1
shared_conv1 = L.Conv2D(filters=f1, kernel_size=(5, 5), strides=1, padding='same',name='shared_conv1')
shared_conv2 = L.Conv2D(filters=f1, kernel_size=(3, 3), strides=1, padding='same',name='shared_conv2')
shared_conv3 = L.Conv2D(filters=f1, kernel_size=(3, 3), strides=1, padding='same',name='shared_conv3')
shared_batch1 = L.BatchNormalization(name='shared_batch1')
shared_batch2 = L.BatchNormalization(name='shared_batch2')
shared_batch3 = L.BatchNormalization(name='shared_batch3')
shared_relu1 = L.ReLU(name='shared_relu1')
shared_relu2 = L.ReLU(name='shared_relu2')
shared_relu3 = L.ReLU(name='shared_relu3')
for i in range(length):
x_64 = shared_conv1(input[i])
x_64 = shared_batch1(x_64)
x_64 = shared_relu1(x_64)
x_64 = shared_conv2(x_64)
x_64 = shared_batch2(x_64)
x_64 = shared_relu2(x_64)
x_64 = shared_conv3(x_64)
x_64 = shared_batch3(x_64)
x_64 = shared_relu3(x_64)
print(x_64)
I want to use 8 input, and want x_64 to be output 8 times, and I saw print result like this :
Tensor("shared_relu1_3/Relu:0", shape=(None, 64, 64, 128), dtype=float32)
Tensor("shared_relu1_3_1/Relu:0", shape=(None, 64, 64, 128), dtype=float32)
Tensor("shared_relu1_3_2/Relu:0", shape=(None, 64, 64, 128), dtype=float32)
Since I expected "shared_relu1_3/Relu:0" to be printed 3 times, I could not judge that my network is worked as shared network correctly.
Am I doint right ?
Yes, it is working correctly, what it might confuse you is that the output tensors actually represent computation, so each of them are different but it does not mean that the weights are different, they are being shared.
Sharing works by passing several tensors through the same layer instance, since layers are the ones that have the weights inside them. So it should be working as expected.
Also there is no need to share ReLU's, since they do not have weights. Only layers that have weights should be shared if needed.
Related
I am very much a beginner and currently trying to get started with CNNs. I wanted to try out lane detection.
Using the tusimple dataset, I have images of roads and create masks with the lanes. It looks like this:
I was following this blog where something similar is done. However, my results look nothing like in the blog. For simplicity reasons, I only use one image as dataset. This way, the network should very easily be able to detect the lane in this one image. However, this cnn basically just adds a red filter on the input image. The output looks somewhat like this:
Maybe you can point me into the right direction / tell me what I am doing wrong.
I posted the whole notebook here: https://colab.research.google.com/drive/1igOulIU-1HA-Ecf4diQTLXM-mrnAeFXz?usp=sharing
Or the most relevant code included:
def convolutional_block(inputs=None, n_filters=32, dropout_prob=0, max_pooling=True):
conv = Conv2D(n_filters,
kernel_size = 3,
activation='relu',
padding='same',
kernel_initializer=tf.keras.initializers.HeNormal())(inputs)
conv = Conv2D(n_filters,
kernel_size = 3,
activation='relu',
padding='same',
kernel_initializer=tf.keras.initializers.HeNormal())(conv)
if dropout_prob > 0:
conv = Dropout(dropout_prob)(conv)
if max_pooling:
next_layer = MaxPooling2D(pool_size=(2,2))(conv)
else:
next_layer = conv
#conv = BatchNormalization()(conv)
skip_connection = conv
return next_layer, skip_connection
def upsampling_block(expansive_input, contractive_input, n_filters=32):
up = Conv2DTranspose(
n_filters,
kernel_size = 3,
strides=(2,2),
padding='same')(expansive_input)
merge = concatenate([up, contractive_input], axis=3)
conv = Conv2D(n_filters,
kernel_size = 3,
activation='relu',
padding='same',
kernel_initializer=tf.keras.initializers.HeNormal())(merge)
conv = Conv2D(n_filters,
kernel_size = 3,
activation='relu',
padding='same',
kernel_initializer=tf.keras.initializers.HeNormal())(conv)
return conv
def unet_model(input_size=(720, 1280,3), n_filters=32, n_classes=3):
inputs = Input(input_size)
#contracting path
cblock1 = convolutional_block(inputs, n_filters)
cblock2 = convolutional_block(cblock1[0], 2*n_filters)
cblock3 = convolutional_block(cblock2[0], 4*n_filters)
cblock4 = convolutional_block(cblock3[0], 8*n_filters, dropout_prob=0.2)
cblock5 = convolutional_block(cblock4[0],16*n_filters, dropout_prob=0.2, max_pooling=None)
#expanding path
ublock6 = upsampling_block(cblock5[0], cblock4[1], 8 * n_filters)
ublock7 = upsampling_block(ublock6, cblock3[1], n_filters*4)
ublock8 = upsampling_block(ublock7,cblock2[1] , n_filters*2)
ublock9 = upsampling_block(ublock8,cblock1[1], n_filters)
conv9 = Conv2D(n_classes,
1,
activation='relu',
padding='same',
kernel_initializer='he_normal')(ublock9)
conv10 = Activation('softmax')(conv9)
model = tf.keras.Model(inputs=inputs, outputs=conv10)
return model
For my project I need to initialize the CNN 1st Layer kernel with Gammatone filters according to papers ( https://www.mdpi.com/1099-4300/20/12/990/htm ) ,( https://www.groundai.com/project/end-to-end-environmental-sound-classification-using-a-1d-convolutional-neural-network/1 ) and a few others. What does it exactly mean to initialize the cnn kernel with Gammatone filter (Or any filter). How does one implement it? Is it a custom layer? Any tips and guidance would be much appreciated!
for instance
conv_1 = Conv1D(filters = 64, kernel_size = 3, kernel_initializer = *insert Gammatone Filter*, padding = 'same', activation='relu', input_shape = (timesteps, features))(decoder_outputs3)
TIA
You could use TensorFlows constant initializer:
gammatone_filter_kernel = np.array([...])
init_kernel = tf.constant_initializer(gammatone_filter_kernel)
# ...
conv_1 = Conv1D(filters = 64, kernel_size = 3, kernel_initializer = init_kernel, padding = 'same', activation='relu', input_shape = (timesteps, features))(decoder_outputs3)
# ...
If your filter is some kind of preprocessing step to your signal you could set the trainable attribute of the conv laver to False and the weights will be fixed.
I want to classification for pictures with different input sizes. I would like to use the following paper ideas.
'Fully Convolutional Networks for Semantic Segmentation'
https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Long_Fully_Convolutional_Networks_2015_CVPR_paper.pdf
I did change the dense layer to conv2D layer like this.
def FullyCNN(input_shape, n_classes):
inputs = Input(shape=(None, None, 1))
first_layer = Conv2D(filters=16, kernel_size=(12,16), strides=1, activation='relu', kernel_initializer='he_normal', name='conv1')(inputs)
first_layer = BatchNormalization()(first_layer)
first_layer = MaxPooling2D(pool_size=2)(first_layer)
second_layer = Conv2D(filters=24, kernel_size=(8,12), strides=1, activation='relu', kernel_initializer='he_normal', name='conv2')(first_layer)
second_layer = BatchNormalization()(second_layer)
second_layer = MaxPooling2D(pool_size=2)(second_layer)
third_layer = Conv2D(filters=32, kernel_size=(5,7), strides=1, activation='relu', kernel_initializer='he_normal', name='conv3')(first_layer)
third_layer = BatchNormalization()(third_layer)
third_layer = MaxPooling2D(pool_size=2)(third_layer)
fully_layer = Conv2D(64, kernel_size=8, activation='relu', kernel_initializer='he_normal')(third_layer)
fully_layer = BatchNormalization()(fully_layer)
fully_layer = Dropout(0.5)(fully_layer)
fully_layer = Conv2D(n_classes, kernel_size=1)(fully_layer)
output = Conv2DTranspose(n_classes, kernel_size=1, activation='softmax')(fully_layer)
model = Model(inputs=inputs, outputs=output)
return model
and I made generator for using fit_generator().
def data_generator(x_train, y_train):
while True:
index = np.asscalar(np.random.choice(len(x_train),1))
feature = np.expand_dims(x_train[index],-1)
feature = np.resize(feature,(-1,feature.shape))
feature = np.expand_dims(feature,0) # make (1,input_height,input_width,1)
label = y_train[index]
yield (feature,label)
and These are images about my data.
However, there is some problems about dimension.
Since the output layer must have 4 dimensions unlike the original CNN model, dimensions do not fit in the label.
Model summary:
Original CNN model summary:
How can I handle this problem? I tried to change dimension about label by expanding the dimension.
label = np.expand_dims(label,0)
label = np.expand_dims(label,0)
label = np.expand_dims(label,0)
I think there is a better way and I wonderd that is it necessary to have conv2DTranspose? And should batch size be 1?
I am working with the OpenAI gym environment (using policy gradient). My network is outputting an action which is higher than the possible action range.
n_outputs = 9
learning_rate = 0.01
initializer = tf.variance_scaling_initializer()
X = tf.placeholder(tf.float32, shape=[None, 50, 70, 1])
network = tflearn.conv_2d(X, 32, 5, strides=2, activation='relu')
network = tflearn.max_pool_2d(network, 2)
network = tflearn.conv_2d(network, 32, 5, strides=2, activation='relu')
network = tflearn.max_pool_2d(network, 2)
network = tflearn.fully_connected(network, 256, activation='relu')
hidden = tf.layers.dense(network, 64, activation=tf.nn.relu, kernel_initializer=initializer)
logits = tf.layers.dense(hidden, n_outputs)
outputs = tf.nn.softmax(logits)
action = tf.multinomial(outputs, num_samples=1)
It outputs 9, which creates an error in the gym environment.
The full code.
tf.multinomial will sample outside of the range if it encounters numerical error, so in other words - you have NaNs in your graph.
As the weights are not explicitly defined, how can I pass them to a summary writer?
For exemple:
conv1 = tf.layers.conv2d(
tf.reshape(X,[FLAGS.batch,3,160,320]),
filters = 16,
kernel_size = (8,8),
strides=(4, 4),
padding='same',
kernel_initializer=tf.contrib.layers.xavier_initializer(),
bias_initializer=tf.zeros_initializer(),
kernel_regularizer=None,
name = 'conv1',
activation = tf.nn.elu
)
=>
summarize_tensor(
??????
)
Thanks!
While Da Tong's answer is complete, it took me a while to realize how to use it. To save time for another beginner, you need to add the following to you code to add all trainable variables to the tensorboard summary:
for var in tf.trainable_variables():
tf.summary.histogram(var.name, var)
merged_summary = tf.summary.merge_all()
That depends on what you are going to record in TensorBoard. If you want to put every variables into TensorBoard, call tf.all_variables() or tf.trainable_variables() will give you all the variables. Note that the tf.layers.conv2d is just a wrapper of creating a Conv2D instance and call apply method of it. You can unwrap it like this:
conv1_layer = tf.layers.Conv2D(
filters = 16,
kernel_size = (8,8),
strides=(4, 4),
padding='same',
kernel_initializer=tf.contrib.layers.xavier_initializer(),
bias_initializer=tf.zeros_initializer(),
kernel_regularizer=None,
name = 'conv1',
activation = tf.nn.elu
)
conv1 = conv1_layer.apply(tf.reshape(X,[FLAGS.batch,3,160,320]))
Then you can use conv1_layer.kernel to access the kernel weights.