I'm trying to create a CNN to solve a problem. The input_shape for first layer I provided was (20, 196, 1).
However, when I do model.summary() I get dimensions as (None, 20, 196, 1) and my X is a list of features of dimensions (20, 196, 1). While performing model.fit() I get this error -
Error when checking input: expected input_1 to have 4 dimensions, but
got array with shape (20, 196, 1).
Can anyone point out what I'm doing wrong? Also, if I wanted to increase the dimension from (20, 196, 1) to (None, 20, 196, 1), what do I do?
The first axis should always correspond to the batch size.
For example, consider the case where you can want N elements in your batch. Each element consists of input features with dimension (20, 196, 1). Now, your batch would have a size of (N, 20, 196, 1).
An option would be stacking the samples on the first axis: first, create a list of samples, then assign this to the input data. For example:
# list of samples with size (20, 196, 1)
list_of_samples = [x1, x2, x3, .. xn]
# your input data would be:
input_batch = np.array(list_of_samples)
Otherwise, if your samples xi are already tensors, another possibility is to stack them on the first axis:
# given xi = tensor with shape (20, 196, 1), for i = 1, 2,..., N
input_batch = tf.stack([x1, x2, x3, ..., xn], axis=0)
# input_batch has now shape (N, 20, 196, 1)
Related
I have a keras transformer model trained with tensorflow 2.7.0 and python 3.7 with input shape: (None, 250, 3) and a 2D array input with shape: (250, 3)(not an image)
When making a prediction with:
prediction = model.predict(state)
I get ValueError: Input 0 of layer "model" is incompatible with the layer: expected shape=(None, 250, 3), found shape=(None, 3)
project code: https://github.com/MikeSifanele/TT
This is how state looks like:
state = np.array([[-0.07714844,-0.06640625,-0.140625],[-0.140625,-0.1650391,-0.2265625]...[0.6376953,0.6005859,0.6083984],[0.7714844,0.7441406,0.7578125]], np.float32)
Some explanation:
For input shape to the model i.e. (None, 250, 3), the first axis (represented by None) is the "sample" axis, while the rest i.e. 250,3 denotes the input dimension. Thus, when the input shape is (250, 3) it assumes the first axis as the "sample" axis and the rest as the input dimension i.e. just 3. So, to make it consistent we need to add a dimension at the beginning described in the following:
state = np.expand_dims(state, axis=0)
The shape of state then becomes (1, 250, 3) ~(None, 250, 3).
With TFF 0.18, I found this problem :
images, labels = next(img_gen.flow_from_directory(path0,target_size=(180, 180), batch_size = 2,class_mode=None))
sample_batch = (images,labels) # assumes images and labels are np.ndarray
input_spec = tf.nest.map_structure(tensor_spec_from_ndarray, sample_batch)
here is the output of input_spec
(TensorSpec(shape=(180, 180, 3), dtype=tf.float32, name=None), TensorSpec(shape=(180, 180, 3), dtype=tf.float32, name=None))
And here is my model:
model = tf.keras.applications.ResNet50(include_top=False, weights=None, input_tensor=tf.keras.Input(shape=(180, 180, 3)), pooling=None)
At a high level, the error message is saying the tensors are not the same rank (4 vs 3).
expected shape=(None, 180, 180, 3)
The expected shape has a leading None dimension, which is the batch dimension.
found shape=(180, 180, 3)
The found shape only has rank 3, with no batch dimension.
This is somewhat surprising from the code in the question which has the line batch_size = 2. I would dig into how that parameter is used by the img_gen.flow_from_directory() function to see if getting a batch dimension is possible.
I have implemented and trained the model from the following website, and using the author's source code:
I am now running an image through the trained network and want to get the network output (feature maps etc.) at every stage.
My Approach
To that end, I have tried making sub-models from groups of layers from the full model (called sizedModel in my code) and checking their output.
I have done that for the first L1(Conv2D)
L1_inp = sizedModel.layers[1].input
L1_out = sizedModel.layers[1].output
Layer1 = Model(L1_inp,L1_out)
L1_result = Layer1.predict(
tf.expand_dims(tf.expand_dims(lrImage, axis=0), axis=3)
)
# Save feature maps to greyscale images
Xresult = tf.squeeze(L1_result)
for i in range(Xresult.shape[2]):
data = (Xresult[:,:,i].numpy() * 255).astype(np.uint8)
filename = 'tmp_test/result'+str(i).zfill(2)+'.png'
Image.fromarray(data).save(filename)`
the result of .predict() on that model is tensor shaped (1,360,640,12) which is as expected and the images look fine.
I am now trying to feed that tensor into the layers marked as L2 in the diagram (sizedModel.layers[2-8]).
The only way I know how to do it is isolated in a new model. To that end I am doing:
# Pick up input shape from output of L1 (1,360,640,12)
_input = tf.keras.Input(shape=L1_result.shape, name="input_layer")
# First "tf_op_layer_strided_slice" from large model
_split = largeModel.layers[2](_input)
# First "L2-0 (Conv 3x3)" from large model
_conv = largeModel.layers[5](_split)
This results in
# ValueError: Input 0 of layer L2-0 is incompatible with the layer: expected axis -1 of input shape
# to have value 4 but received input with shape [None, 1, 360, 4, 12]
# Build sub-model from these two layers of the large model
Layer2 = Model(_input,_conv)
# Pass first sub-model (L1) output to this model
Result_L2 = Layer2.predict(L1_result)
Why is the Input 0 of Layer L2-0 incompatible?
Is there an easier way to debug the output of each layer individually?
If I understand your question properly, you want to get output feature maps of each layer of a model. Normally, as we mentioned in the comment box, a model with one (or multiple) inputs and one (or multiple) outputs. But in order to inspect the activation feature maps of inside layers, we can adopt some strategies. Some possible scenarios: (1). Want to get output feature maps of each layer in run-time or training time. (2). Want to get output feature maps of each layer in the inference time or after training. And as you quoted:
I am now running an image through the trained network and want to get
the network output (feature maps etc.) at every stage.
That goes to number 2, get the feature maps in inference time. Below is a simple possible workaround to do this. First, we build a model, and then after training, we will modify the trained model to get feature maps of each layer within it (technically creating the same model with some modification).
# The model we trained
input
|
conv1
|
conv2
|
conv3
|
output
|
# The modified model we use for inference
input
|
conv1 --> outputA [Get Feature Mpas]
|
conv2 --> outputB [G. F. M.]
|
conv3 --> outputC [G. F. M.]
|
output
|
I'm using the model definition from here that you reference in your post.
def buildModel(inputSize=(9,9), networkWidth=[48, 24], parBlockSize=12):
....
....
return tf.keras.models.Model(inputs=input, outputs=output)
Now, we iterate over the layers and get each layer output and build a new model.
model = buildModel(...)
features_list = [layer.output for layer in model.layers]
activations_model = tf.keras.Model(inputs=model.input, outputs=features_list)
Now, if we pass some input to activations_model, we will get more than one output, unlike with model where we get only the last layer output.
img = np.random.random((1, 128, 128, 1)).astype("float32")
activations = activations_model.predict(img)
for i, _ in enumerate(activations):
print(activations[i].shape)
(1, 128, 128, 1)
(1, 124, 124, 48)
(1, 124, 124, 12)
(1, 124, 124, 12)
(1, 124, 124, 12)
(1, 124, 124, 12)
(1, 122, 122, 8)
(1, 122, 122, 8)
(1, 122, 122, 8)
(1, 122, 122, 8)
(1, 122, 122, 32)
(1, 122, 122, 24)
(1, 122, 122, 12)
(1, 122, 122, 12)
(1, 120, 120, 8)
(1, 120, 120, 8)
(1, 120, 120, 16)
(1, 120, 120, 4)
(1, 240, 240, 1)
So, we have total 1 + 48 + 12*6 + 8*6 + 32 + 24 + 16 + 4 + 1 = 246 feature maps from a single instance. Now, we can save these feature maps as we wanted.
layer_names = []
for layer in model.layers:
layer_names.append(layer)
for i, (layer_name, layer_activation) in enumerate(zip(layer_names, activations)):
n_features = layer_activation.shape[-1]
feat_maps = tf.squeeze(layer_activation)
print(feat_maps.shape)
if n_features == 1:
tf.keras.preprocessing.image.save_img(
f'tmp/{layer_name.name}_{n_features}.png',
tf.expand_dims(feat_maps, axis=-1))
else:
for n_feature in range(n_features):
feat_map = feat_maps[:, :, n_feature]
tf.keras.preprocessing.image.save_img(
f'tmp/{layer_name.name}_{n_feature}.png',
tf.expand_dims(feat_map, axis=-1))
I have tensor (None, 196) and after reshaping, it becomes (None, 14, 14).
And now, I want to copy it to channel axis, so that the shape should be (None, 14, 14, 512). Lastly, I want to copy to timestep axis, so it becomes (None, 10, 14, 14, 512). I accomplish those steps using this snippet code:
def replicate(tensor, input_target):
batch_size = K.shape(tensor)[0]
nf, h, w, c = input_target
x = K.reshape(tensor, [batch_size, 1, h, w, 1])
# Replicate to channel dimension
x = K.tile(x, [batch_size, 1, 1, 1, c])
# Replicate to timesteps dimension
x = K.tile(x, [batch_size, nf, 1, 1, 1])
return x
x = ...
x = Lambda(replicate, arguments={'input_target':input_shape})(x)
another_x = Input(shape=input_shape) # shape (10, 14, 14, 512)
x = layers.multiply([x, another_x])
x = ...
I plot the model and the output shape is just like I want it to be. But, the problem arises in model training. I set the batch size to 2. This the the error message:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [8,10,14,14,512] vs. [2,10,14,14,512]
[[{{node multiply_1/mul}} = Mul[T=DT_FLOAT, _class=["loc:#training/Adam/gradients/multiply_1/mul_grad/Sum"], _device="/job:localhost/replica:0/task:0/device:GPU:0"](Lambda_2/Tile_1, _arg_another_x_0_0/_189)]]
[[{{node metrics/top_k_categorical_accuracy/Mean_1/_265}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_6346_metrics/top_k_categorical_accuracy/Mean_1", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
Looks like, K.tile() increases the batch size from 2 to 8. When I set the batch size to 10, it becomes 1000.
So, my question is how to achieve the result as I want? Is it good way to use tile()? Or, should I use repeat_elements()? Thanks!
I am using Tensorflow 1.12.0 and Keras 2.2.4.
As a rule of thumb, try to avoid bringing batch size to the transformations happening in the Lambda layer.
When you use tile operation, you only set only the dimension that needs to change (for example you had batch_size value in your tile operation which is wrong). Also I am using tf.tile instead of K.tile (TF 1.12 doesn't have tile in the Keras backend it seems).
def replicate(tensor, input_target):
_, nf, h, w, c = input_target
x = K.reshape(tensor, [-1, 1, h, w, 1])
# Replicate to channel dimension
# You can combine below lines to tf.tile(x, [1, nf, 1, 1, c]) as well
x = tf.tile(x, [1, 1, 1, 1, c])
# Replicate to timesteps dimension
x = tf.tile(x, [1, nf, 1, 1, 1])
return x
Simple example
input_shape= [None, 10, 14, 14, 512]
x = Input(shape=(196,))
x = Lambda(replicate, arguments={'input_target':input_shape})(x)
print(x.shape)
Which gives
>>> (?, 10, 14, 14, 512)
I have a 5-d numpy array, the shape is (5, 1000, 32, 32, 3), which means there are 3 channels of 32*32 pixels, and 1000 samples, 5 different timestamps. How do I print specific 32*32 data, for example, I want to print the 32*32 data from 16th sample, 2nd timestamp, 1st channel?
With a mix of inedexing and slicing this can be done like this:
arr = np.random.randint(1000, size=(5, 1000, 32, 32, 3))
result = arr[1, 15, :, :, 0]
print(result.shape)
This will output the shape of the result:
(32, 32)