For each convolution activation map, I want to concatenate a layer of constants -- more particularly, I want to concatenate a meshgrid. (This is to reproduce a paper from Uber.)
For example, say I have an activation map of (?, 256, 256, 32); then I want to concatenate a constants layer of shape (?, 256, 256, 1).
This is how I am doing this:
from keras import layers
import tensorflow as tf
import numpy as np
input_layer = layers.Input((256, 256, 3))
conv = layers.Conv2D(32, 3, padding='same')(input_layer)
print('conv:', conv.shape)
xx, yy = np.mgrid[:256, :256] # [(256, 256), (256, 256)]
xx = tf.constant(xx, np.float32)
yy = tf.constant(yy, np.float32)
xx = tf.reshape(xx, (-1, 256, 256, -1))
yy = tf.reshape(yy, (-1, 256, 256, -1))
print('xx:', xx.shape, 'yy:', yy.shape)
concat = layers.Concatenate()([conv, xx, yy])
print('concat:', concat.shape)
conv2 = layers.Conv2D(32, 3, padding='same')(concat)
print('conv2:', conv2.shape)
But I get the error:
conv: (?, 256, 256, 32)
xx: (?, 256, 256, ?) yy: (?, 256, 256, ?)
concat: (?, 256, 256, ?)
Traceback (most recent call last):
File "temp.py", line 21, in <module>
conv2 = layers.Conv2D(32, 3, padding='same')(concat)
[...]
raise ValueError('The channel dimension of the inputs '
ValueError: The channel dimension of the inputs should be defined. Found `None`.
The problem is that my constants layer is (?, 256, 256, ?), instead of (?, 256, 256, 1), and then the next convolution layers errors-out.
I have tried other things without success.
PS: The paper I was trying to implement is already implemented here.
The problem is that tf.reshape can't infer the shape of more than one dimension (i.e. using -1 for more than one dimension results in undefined dimensions ?). Since you want the shapes of xx and yy to be (?, 256, 256, 1), you could reshape these tensors as follows:
xx = tf.reshape(xx, (-1, 256, 256, 1))
yy = tf.reshape(yy, (-1, 256, 256, 1))
The resulting shapes will be (1, 256, 256, 1). Now, conv is (?, 256, 256, 32), and keras.layers.Concatenate requires the shapes of all the inputs to match except for the concat axis. You could then use tf.tile to repeat tensors xx and yy along the first dimension in order to match the batch size:
xx = tf.tile(xx, [tf.shape(conv)[0], 1, 1, 1])
yy = tf.tile(yy, [tf.shape(conv)[0], 1, 1, 1])
The shapes of xx and yy are now (?, 256, 256, 1), and the tensors can be concatenated because their first dimension matches the batch size.
Related
I have a numpy array having the 512 data points, I want to consider this as the input to my 1d convolutional model.
After training the model in unsupervised way I want to predict the same input data from the model and want to save to a file
inp_data can be calculated as
import numpy as np
inp_data = np.random.normal(0, .1, 512)
So I did something like
inp_data = np.expand_dims(inp_data, axis=1) #512,1
img_width = inp_data.shape[1]
img_height = inp_data.shape[0]
input_img = Input((img_height,img_width))
print(input_img.shape) #(None, 512, 1)
x = Conv1D(256, 16, activation='relu', padding='same', strides=2)(input_img)
x = Conv1D(128, 16, activation='relu', padding='same',strides=2)(x)
x = BatchNormalization(name='bn1')(x)
x = Conv1D(64, 16, activation='relu', padding='same',strides=2 )(x)
encoded =Conv1D(32, 16, activation='relu', padding='same',strides=2 )(x)
x = Conv1DTranspose(32, 16,activation='relu', padding='same',strides=2 )(encoded)
x = Conv1DTranspose(64, 16, activation='relu', padding='same',strides=2 )(x)
x = BatchNormalization(name='bn3')(x)
x = Conv1DTranspose(128, 16, activation='relu', padding='same',strides=2)(x)
x = Conv1DTranspose(256, kernel_size=16, activation='relu', padding='same')(x)
decoded = Conv1DTranspose(1, 16, activation='sigmoid', padding='same',strides=2)(x)
autoencoder = Model(input_img, decoded)
sgd = tf.keras.optimizers.Adam(lr=0.001)
autoencoder.compile(optimizer=sgd, loss='mse')
autoencoder.summary()
batch = 1
history = autoencoder.fit(inp_data,inp_data, epochs=50, batch_size=batch, shuffle=True)
out = autoencoder.predict(inp_data)
print(out.shape)
np.savetxt('outfile',out)
However I am getting Errors:
WARNING:tensorflow:Model was constructed with shape (None, 512, 1) for input KerasTensor(type_spec=TensorSpec(shape=(None, 512, 1), dtype=tf.float32, name='input_1'), name='input_1', description="created by layer 'input_1'"), but it was called on an input with incompatible shape (32, 1, 1).
(512, 16, 1)
and while trying to save the predicted output to a file I am getting errors like
np.savetxt('outfile',out)
File "<__array_function__ internals>", line 6, in savetxt
File "/home/yuanj/SFTR/anaconda3/envs/tfenv/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1383, in savetxt
"Expected 1D or 2D array, got %dD array instead" % X.ndim)
ValueError: Expected 1D or 2D array, got 3D array instead
I hope experts may help me overcoming this problem.Thanks.
I have made image patches from an image classifier. When I print it's shape it is (81, 256) which means 81 patches each with size of 256. but it lost the information of the image like these 81 patches belong to a specific image. I want to get it's shape like (1, 81, 256) so if I have batch of 20 images then it will show (20, 81, 256).
class Patches(layers.Layer):
def __init__(self, patch_size):
super(Patches, self).__init__()
self.patch_size = patch_size
def call(self, images):
batch_size = tf.shape(images)[0]
patches = tf.image.extract_patches(
images=images,
sizes=[1, self.patch_size, self.patch_size, 1],
strides=[1, self.patch_size, self.patch_size, 1],
rates=[1, 1, 1, 1],
padding="VALID",
)
patch_dims = patches.shape[-1]
patches = tf.reshape(patches, [batch_size, -1, patch_dims])
return patches
Model code
xception = keras.applications.Xception(
include_top=False, weights="imagenet", pooling="avg"
)
for layer in xception.layers:
layer.trainable = trainable
inputs = layers.Input(shape=(300, 300, 3), name="image_input")
patch_size = 72
print("inputs shape is ", inputs.shape)
xception_input = tf.keras.applications.xception.preprocess_input(inputs)
print("xception shape is", xception_input.shape)
patches = Patches(patch_size)(xception_input)
print("patches shape is", patches.shape)
patches = tf.reshape(patches, [-1, 32, 32, 3])
print("patches shape is", patches.shape)
embeddings = xception(patches)
output = embeddings(
embeddings, projection_layers, dims, dropout_rate
)
print("output shape is ", output.shape)
return keras.Model(inputs, output)
Ouput
inputs shape is (None, 300, 300, 3)
xception shape is (None, 300, 300, 3)
patches shape is (None, None, 15552)
patches shape is (None, 32, 32, 3)
output shape is (81, 256)
Below code converts each patch to size of 256 lenght
def embeddings(
embeddings, projection_layers, dims, dropout_rate
):
projected_embeddings = layers.Dense(units=projection_dims)(embeddings)
for _ in range(num_projection_layers):
x = tf.nn.gelu(projected_embeddings)
x = layers.Dense(projection_dims)(x)
x = layers.Dropout(dropout_rate)(x)
x = layers.Add()([projected_embeddings, x])
projected_embeddings = layers.LayerNormalization()(x)
return projected_embeddings
If it's just a matter of adding a dimension to your numpy array, you can use expand_dims like this:
import numpy as np
array = np.zeros((81, 256)) # example array of same shape like yours
print("Old shape: {}".format(array.shape))
new_array = np.expand_dims(array, axis=0)
print("New shape: {}".format(new_array.shape))
The output:
Old shape: (81, 256)
New shape: (1, 81, 256)
Recently, i was in an interview and i was asked to build a neural netowrk using tensorflow which meets the following requirements:
The input layer of the model must have an input shape of (32, 10, 1)
The model must have an output shape of (32, 10, 1)
and in response, i provided the following solution:
import tensorflow as tf
from tensorflow.keras.layers import Dense, Conv1D
model = tf.keras.models.Sequential([
Conv1D(filters=32, kernel_size=1, activation='relu', input_shape=(32, 10, 1)),
Dense(30, activation='relu'),
Dense(10, activation='relu'),
tf.keras.layers.Dense(1)
])
and in order to prove that my model can pass the requeirment, i printed the input-shape and out-put shape of each model using the below code:
for layer in model.layers:
print('input shape: ',layer.input_shape, 'output shape: ',layer.output_shape)
and here is what i got in the output:
input shape: (None, 32, 10, 1) output shape: (None, 32, 10, 32)
input shape: (None, 32, 10, 32) output shape: (None, 32, 10, 30)
input shape: (None, 32, 10, 30) output shape: (None, 32, 10, 10)
input shape: (None, 32, 10, 10) output shape: (None, 32, 10, 1)
Sadly and apparently my answer to this question was not correct and i don't know how to build such model ?
As you can see, my model has 4 dimentions and the input and output layer start by None.
Is it the problem ?
I am not 100% sure but for me really seems like you did not explicitly declared the input layer, I really think at the shape's command response we should not see a 'None' on it.
Two possible solutions I found at this source, which the best one seems to be the following (not tested):
inputs = Input(shape=(32, 10, 1))
x = Conv1D(filters=32, kernel_size=1)(inputs)
x = Dense(30, "relu")(x)
outputs = Dense(10, "relu")(x)
model = Model(inputs=inputs, outputs=outputs, name="my_model_name")
Let's see if that makes any sense.
Thanks to #Pedro Silva and #AloneTogether i came out with a possible solution as below. So, in the Input or Conv1D layer the input_shape does not include the Batch_size of the input data. The input_shape only specifies the shape of each Data point or (entry of data) and if we need to specify the Batch_size then we cn use the batch_size parameter in the layer. So, if we develop the mode as :
import tensorflow as tf
from tensorflow.keras.layers import Dense, Conv1D,Input
from tensorflow.keras.models import Model
model = tf.keras.models.Sequential([
Conv1D(filters=32, kernel_size=1, activation='relu', input_shape=(10, 1),batch_size=32),
Dense(30, activation='relu'),
Dense(10, activation='relu'),
tf.keras.layers.Dense(1)
])
for layer in model.layers:
print('input shape: ',layer.input_shape, 'output shape: ',layer.output_shape)
or this:
inputs = Input(shape=(10, 1),batch_size=32)
x = Conv1D(filters=32, kernel_size=1)(inputs)
x = Dense(30, "relu")(x)
outputs = Dense(10, "relu")(x)
model = Model(inputs=inputs, outputs=outputs, name="my_model_name")
for layer in model.layers:
print('input shape: ',layer.input_shape, 'output shape: ',layer.output_shape)
Then in both cases, the model has the following shape of input and output:
input shape: (32, 10, 1) output shape: (32, 10, 1)
input shape: (32, 10, 1) output shape: (32, 10, 32)
input shape: (32, 10, 32) output shape: (32, 10, 30)
input shape: (32, 10, 30) output shape: (32, 10, 10)
I'm trying to implement a layer GlobalMaxPooling2D layer. I have a 10x10x128 input and want it reduced to a 3D tensor of shape 1x1x128. I tried using keepdims=True, but it throws a
TypeError: ('Keyword argument not understood:', 'keepdims')
I have tried adding data_format too but with no avail (which is the default "channel_last").
Here is the code for GlobalMaxPooling2D
ug = layers.GlobalMaxPooling2D(data_format='channel_last',keepdims=True)(inputs)
The inputs variable is the output of a 2D conv operation:
conv4 = layers.Conv2D(filters=128, kernel_size=3, strides=1, padding='valid', activation='relu', name='conv4')(conv3)
Am i messing up somewhere due to this Conv layer or while calling GlobalMaxPooling2D layer ?
Is there a way to get a 1x1x128 output from the GlobalMaxPooling2D layer ?
For tf < 2.6, you can do
import tensorflow as tf; print(tf.__version__)
input_shape = (1, 10, 10, 128)
x = tf.random.normal(input_shape)
y = tf.keras.layers.GlobalMaxPool2D()(x)
z = tf.keras.layers.Reshape((1, 1, input_shape[-1]))(y)
print(x.shape)
print(y.shape)
print(z.shape)
2.5.0
(1, 10, 10, 128)
(1, 128)
(1, 1, 1, 128)
And from tf > = 2.6, you can use keepdims arguments.
!pip install tensorflow==2.6.0rc0 -q
import tensorflow as tf; print(tf.__version__)
input_shape = (1, 10, 10, 128)
x = tf.random.normal(input_shape)
y = tf.keras.layers.GlobalMaxPool2D(keepdims=True)(x)
print(x.shape)
print(y.shape)
2.6.0-rc0
(1, 10, 10, 128)
(1, 1, 1, 128)
I am writing a Python code, visualizing an output of a ConvNet layer. Everything seems t be fine, but I get a dimension conversion error as below, at sess.run function call.
ValueError: Cannot feed value of shape (128, 128, 3) for Tensor 'image-in:0', which has shape '(?, 128, 128, 3)'
loaded_graph = tf.Graph()
image_to_use = train_images[0]
print(image_to_use.shape) # (128, 128, 3)
with tf.Session(graph=loaded_graph) as sess:
# Load model
loader = tf.train.import_meta_graph(save_model_path + ".meta")
loader.restore(sess, save_model_path)
# Get Tensors from loaded model
hidden_layer_1 = loaded_graph.get_tensor_by_name("hidden-layer-1:0")
keep_prob_tf = tf.placeholder(tf.float32, name="keep-prob-in")
image_in_tf = tf.placeholder(tf.float32, [None, image_to_use.shape[0], image_to_use.shape[1], image_to_use.shape[2]], name="image-in")
units = sess.run(hidden_layer_1, feed_dict={image_in_tf:image_to_use, keep_prob_tf:1.0})
ValueError: Cannot feed value of shape (128, 128, 3) for Tensor 'image-in:0', which has shape '(?, 128, 128, 3)'
You have a problem in this line
image_to_use = train_images[0]
where image_to_use has dimension (128, 128, 3).
Change it to this:
image_to_use = np.asarray([train_images[0]])
Now image_to_use has shape (1, 128, 128, 3) and it will be compatible with the expected shape (?, 128, 128, 3).
Also, define the shape of your image_in_tf placeholder like this:
image_in_tf = tf.placeholder(tf.float32, (None, ) + image_to_use[0].shape)
Otherwise it will accept wrong dimensions in current format of your code (we changed the first dimensions).