I want to embed sentences that all contain 5 words and a my training-set has a total vocabulary of 10000 words. I use this code:
import tensorflow as tf
vocab_size = 10000
inputs = tf.keras.layers.Input(shape=(5,vocab_size), name="input", )
embedding = tf.keras.layers.Embedding(10000, 64)(inputs)
conv2d_1 = Conv2D( filters = 32, kernel_size = (3,3),
strides =(1), padding = 'SAME',)(embedding)
model = tf.keras.models.Model(inputs=inputs, outputs=conv2d_1)
model.summary()
After running I get:
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input (InputLayer) [(None, 5, 10000)] 0
_________________________________________________________________
embedding_105 (Embedding) (None, 5, 10000, 64) 640000
_________________________________________________________________
conv2d_102 (Conv2D) (None, 5, 10000, 32) 18464
=================================================================
I want to do the embedding to convert the sparse 10000x5 tensor to a dense 64x5 tensor. Apparently that doesn't work as intended, so my question is: Why is the shape of the next layer not (None, 5, 64, 32) instead of (None, 5, 10000, 32)? How can I achieve the compactization?
Related
I am trying to train a lstm model for music generation, but something seems to cause an error when calling model.fit().
# Compile the model
lstm.compile(loss='categorical_crossentropy', optimizer='rmsprop')
tf.keras.utils.plot_model(lstm, show_shapes=True)
# Train the model
lstm.fit([trainChords, trainDurations], [targetChords, targetDurations], epochs=500)
Model: "model_6"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_19 (InputLayer) [(None, 32)] 0 []
input_20 (InputLayer) [(None, 32)] 0 []
embedding_18 (Embedding) (None, 32, 64) 2048 ['input_19[0][0]']
embedding_19 (Embedding) (None, 32, 64) 2048 ['input_20[0][0]']
concatenate_9 (Concatenate) (None, 64, 64) 0 ['embedding_18[0][0]',
'embedding_19[0][0]']
lstm_8 (LSTM) (None, 64, 512) 1181696 ['concatenate_9[0][0]']
dense_18 (Dense) (None, 64, 256) 131328 ['lstm_8[0][0]']
dense_19 (Dense) (None, 64, 32) 8224 ['dense_18[0][0]']
dense_20 (Dense) (None, 64, 32) 8224 ['dense_18[0][0]']
==================================================================================================
Total params: 1,333,568
Trainable params: 1,333,568
Non-trainable params: 0
I get the error that some shapes are not compatible, but I don't know how to fix this.
ValueError Traceback (most recent call last)
<ipython-input-63-8785c106bc4b> in <module>
5
6 # Train the model
----> 7 lstm.fit([trainChords, trainDurations], [targetChords, targetDurations], epochs=500)
ValueError: Shapes (None,) and (None, 64, 32) are incompatible
Any help would be appreciated.
Update on building the model. Maybe the error is with the output layers(?)
# Define input layers
chordInput = tf.keras.layers.Input(shape = (nChords))
durationInput = tf.keras.layers.Input(shape = (nDurations))
# Define embedding layers
chordEmbedding = tf.keras.layers.Embedding(nChords, embedDim, input_length = sequenceLength)(chordInput)
durationEmbedding = tf.keras.layers.Embedding(nDurations, embedDim, input_length = sequenceLength)(durationInput)
# Merge embedding layers using a concatenation layer
mergeLayer = tf.keras.layers.Concatenate(axis=1)([chordEmbedding, durationEmbedding])
# Define LSTM layer
lstmLayer = tf.keras.layers.LSTM(512, return_sequences=True)(mergeLayer)
# Define dense layer
denseLayer = tf.keras.layers.Dense(256)(lstmLayer)
# Define output layers
chordOutput = tf.keras.layers.Dense(nChords, activation = 'softmax')(denseLayer)
durationOutput = tf.keras.layers.Dense(nDurations, activation = 'softmax')(denseLayer)
# nChords and nDurations are both 32
# Define model
lstm = tf.keras.Model(inputs = [chordInput, durationInput], outputs = [chordOutput, durationOutput])
I would like to draw a class activation map for a model built upon EfficeintNet B3. But when I follow different tutorials and codes from different sources, it simply fails....
#load images
img = tf.keras.preprocessing.image.load_img(
base, target_size=(img_height, img_width))
img_array = tf.keras.preprocessing.image.img_to_array(img)
img_array = tf.expand_dims(img_array, 0) # Create a batch
predictions = model.predict(img_array)
score = tf.nn.softmax(predictions[0])
last_conv = model.layers[2].layers[-3]
grad_model = tf.keras.models.Model(
[model.inputs], [last_conv.output, model.output])
Can't build a grad_model
ValueError: Graph disconnected: cannot obtain value for tensor
KerasTensor(type_spec=TensorSpec(shape=(None, 300, 300, 3),
dtype=tf.float32, name='input_1'), name='input_1',
description="created by layer 'input_1'") at layer "stem_conv". The
following previous layers were accessed without issue: []
This is the model:
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
sequential (Sequential) (None, 300, 300, 3) 0
_________________________________________________________________
rescaling (Rescaling) (None, 300, 300, 3) 0
_________________________________________________________________
efficientnet-b3 (Functional) (None, 10, 10, 1536) 10783528
_________________________________________________________________
global_average_pooling2d (Gl (None, 1536) 0
_________________________________________________________________
dropout (Dropout) (None, 1536) 0
_________________________________________________________________
dense (Dense) (None, 128) 196736
_________________________________________________________________
dense_1 (Dense) (None, 5) 645
=================================================================
To address the graph disconnected value error, you need to build the grad cam model properly. Here is one of the ways to build a model for grad-cam.
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
inputs = tf.keras.Input(shape=(300, 300, 3))
x = keras.applications.EfficientNetB3(
input_tensor=inputs, # pass input to input_tensor
include_top=False,
weights=None
)
# flat the base model with x.output
x = layers.GlobalAveragePooling2D()(x.output)
# others
x = layers.Dense(128)(x)
x = layers.Dropout(0.5)(x)
x = layers.Dense(5)(x)
model = keras.Model(inputs, x)
for i, layer in enumerate(model.layers[-10:]):
print(i, layer.name, layer.output_shape, layer.trainable)
0 block7b_project_bn (None, 10, 10, 384) True
1 block7b_drop (None, 10, 10, 384) True
2 block7b_add (None, 10, 10, 384) True
3 top_conv (None, 10, 10, 1536) True
4 top_bn (None, 10, 10, 1536) True
5 top_activation (None, 10, 10, 1536) True # < - We will pick this 2D maps
6 global_average_pooling2d_2 (None, 1536) True
7 dense_2 (None, 128) True
8 dropout (None, 128) True
9 dense_3 (None, 5) True
Build Grad-CAM Model
grad_model = keras.models.Model(
[model.inputs],
[
model.get_layer('top_activation').output,
model.output
]
)
Check
With your setup, you would get a disconnected error with the following code. But now, it wouldn't happen.
import numpy as np
image = np.random.rand(1, 300, 300, 3).astype(np.float32)
with tf.GradientTape() as tape:
convOutputs, predictions = grad_model(tf.cast(image, tf.float32))
loss = predictions[:, tf.argmax(predictions[0])]
grads = tape.gradient(loss, convOutputs)
print(grads.shape)
(1, 10, 10, 1536) # NO DISCONNECTED ERROR
To get the heatmaps from your grad-cam model, check the following answers and sources as references.
Grad-CAM
Guided-GradCAM
Grad-CAM mult-output Model
GradCAM-Callback
Swin-Transformer : GradCAM
I'm trying to train my keras model but shapes are incompatible.
The error says
ValueError: Shapes (None, 3) and (None, 3, 3) are incompatible
My train set's shape is (2000, 3, 768) and lable's shape is (2000, 3).
What is the wrong the point?
Model define & fit code
input_shape = x_train.shape[1:]
model = my_dnn(input_shape, 3)
model.fit(x_train, y_train, epochs=25, verbose=1)
Model code
def my_dnn(input, num_classes):
model = Sequential()
model.add(tf.keras.Input(input))
model.add(Dense(1024))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dense(225))
model.add(Activation('relu'))
model.add(Dense(100))
model.add(Activation('relu'))
model.add(Dense(num_classes))
model.add(Activation('sigmoid'))
model.compile( loss='categorical_crossentropy',
optimizer='adam',
metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])
return model
In addition to what's said, it seems you are carrying the second dimension of the input data until the end of the model. So your model summary is something like this:
Layer (type) Output Shape Param #
=================================================================
dense_1 (Dense) (None, 3, 1024) 787456
_________________________________________________________________
activation_1 (Activation) (None, 3, 1024) 0
_________________________________________________________________
dropout_1 (Dropout) (None, 3, 1024) 0
_________________________________________________________________
dense_2 (Dense) (None, 3, 512) 524800
_________________________________________________________________
activation_2 (Activation) (None, 3, 512) 0
_________________________________________________________________
dense_3 (Dense) (None, 3, 225) 115425
_________________________________________________________________
activation_3 (Activation) (None, 3, 225) 0
_________________________________________________________________
dense_4 (Dense) (None, 3, 100) 22600
_________________________________________________________________
activation_4 (Activation) (None, 3, 100) 0
_________________________________________________________________
dense_5 (Dense) (None, 3, 3) 303
_________________________________________________________________
activation_5 (Activation) (None, 3, 3) 0
=================================================================
Total params: 1,450,584
Trainable params: 1,450,584
Non-trainable params: 0
As you can see, the output shape of the model (None, 3, 3) is not compatible with the label's shape (None, 3), and at some point, you need to use a Flatten layer.
There are two possible reasons:
Your problem is multi-class classification, hence you need softmax instead of sigmoid + accuracy or CategoricalAccuracy() as a metric.
Your problem is multi-label classification, hence you need binary_crossentropy and tf.keras.metrics.BinaryAccuracy()
Depending on how your dataset is built/the task you are trying to solve, you need to opt for one of those.
For case 1, ensure your data is OHE(one-hot encoded).
Also, Marco Cerliani and Amir (in the comment below) point out that the data output needs to be in a 2D format rather than 3D : you should either preprocess the data accordingly before feeding it to the network or use, as suggested in the comment below, a Flatten() at a point (probably before the final Dense())
I'm trying to apply Conv1D layers for a classification model which has a numeric dataset. The neural network of my model is as follows:
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Conv1D(8,kernel_size = 3, strides = 1,padding = 'valid', activation = 'relu',input_shape = (14999,7)))
model.add(tf.keras.layers.Conv1D(16,kernel_size = 3, strides = 1,padding = 'valid', activation = 'relu'))
model.add(tf.keras.layers.MaxPooling1D(2))
model.add(tf.keras.layers.Dropout(0.2))
model.add(tf.keras.layers.Conv1D(32,kernel_size = 3, strides = 1,padding = 'valid', activation = 'relu'))
model.add(tf.keras.layers.Conv1D(64,kernel_size = 3, strides = 1,padding = 'valid', activation = 'relu'))
model.add(tf.keras.layers.MaxPooling1D(2))
model.add(tf.keras.layers.Dropout(0.2))
model.add(tf.keras.layers.Conv1D(128,kernel_size = 3, strides = 1,padding = 'valid', activation = 'relu'))
model.add(tf.keras.layers.Conv1D(256,kernel_size = 3, strides = 1,padding = 'valid', activation = 'relu'))
model.add(tf.keras.layers.MaxPooling1D(2))
model.add(tf.keras.layers.Dropout(0.2))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(512,activation = 'relu'))
model.add(tf.keras.layers.Dense(128,activation = 'relu'))
model.add(tf.keras.layers.Dense(32,activation = 'relu'))
model.add(tf.keras.layers.Dense(3, activation = 'softmax'))
And the model's input shape is (14999, 7).
model.summary() gives the following output
Model: "sequential_8"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv1d_24 (Conv1D) (None, 14997, 8) 176
_________________________________________________________________
conv1d_25 (Conv1D) (None, 14995, 16) 400
_________________________________________________________________
max_pooling1d_10 (MaxPooling (None, 7497, 16) 0
_________________________________________________________________
dropout_9 (Dropout) (None, 7497, 16) 0
_________________________________________________________________
conv1d_26 (Conv1D) (None, 7495, 32) 1568
_________________________________________________________________
conv1d_27 (Conv1D) (None, 7493, 64) 6208
_________________________________________________________________
max_pooling1d_11 (MaxPooling (None, 3746, 64) 0
_________________________________________________________________
dropout_10 (Dropout) (None, 3746, 64) 0
_________________________________________________________________
conv1d_28 (Conv1D) (None, 3744, 128) 24704
_________________________________________________________________
conv1d_29 (Conv1D) (None, 3742, 256) 98560
_________________________________________________________________
max_pooling1d_12 (MaxPooling (None, 1871, 256) 0
_________________________________________________________________
dropout_11 (Dropout) (None, 1871, 256) 0
_________________________________________________________________
flatten_3 (Flatten) (None, 478976) 0
_________________________________________________________________
dense_14 (Dense) (None, 512) 245236224
_________________________________________________________________
dense_15 (Dense) (None, 128) 65664
_________________________________________________________________
dense_16 (Dense) (None, 32) 4128
_________________________________________________________________
dense_17 (Dense) (None, 3) 99
=================================================================
Total params: 245,437,731
Trainable params: 245,437,731
Non-trainable params: 0
The code for model fitting is:
model.compile(loss = 'sparse_categorical_crossentropy', optimizer = 'adam', metrics = ['accuracy'])
history = model.fit(xtrain_scaled, ytrain_scaled, epochs = 30, batch_size = 5, validation_data = (xval_scaled, yval_scaled))
While executing, I'm facing the following error:
ValueError: Input 0 is incompatible with layer model: expected shape=(None, 14999, 7), found shape=(None, 7)
Could anyone help to sort out this issue?
TL;DR:
Change
model.add(tf.keras.layers.Conv1D(8,kernel_size = 3, strides = 1,padding = 'valid', activation = 'relu',input_shape = (14999,7)))
to
model.add(tf.keras.layers.Conv1D(8,kernel_size = 3, strides = 1,padding = 'valid', activation = 'relu',input_shape = (7)))
Full answer:
Assumption: I am guessing the reason you chose to write 14999 in the input shape is because that's your batch size / total size of training data
Problem with this:
Tensorflow assumes the input shape does not include the batch size
By specifying a 2D input_shape you're making Tensorflow expect a 3D input of shape (Batch_size, 14999, 7). But your input is clearly of size (Batch_size, 7)
Solution:
Change the shape from (14999, 7) to just (7)
TF will now be expecting the same shape that you are providing
PS: Don't be worried about informing your model of how many training examples you have in the dataset. TF Keras works with the assumption it can be shown unlimited training examples.
I am trying to train a neural network on Semantic Role Labeling task (text classification task). The dataset consist of sentences on which the neural network has to be trained to predict a class for each word. Apart from using the embedding matrix, I am also using other features (meta_data_features). The number of classes in Y_train are 61. The number 3306 represents the number of sentences in my dataset (size of my dataset). MAX_LEN = 67. The code for the architecture is:
embedding_layer = Embedding(67,
300,
embeddings_initializer=Constant(embedding_matrix),
input_length=MAX_LEN,
trainable=False)
sentence_input = Input(shape=(67,), dtype='int32')
meta_input = Input(shape=(67,), name='meta_input')
embedded_sequences = embedding_layer(sentence_input)
x_1 = (SimpleRNN(256))(embedded_sequences)
x = concatenate([x_1, meta_input], axis=1)
x = Dropout(0.3)(x)
x = Dense(32, activation='relu')(x)
predictions = Dense(61, activation='softmax')(x)
model = Model([sentence_input,meta_input], predictions)
model.compile(loss='sparse_categorical_crossentropy',
optimizer='adam',
metrics=['sparse_categorical_accuracy'])
print(model.summary())
The snapshot of model summary is:
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) (None, 67) 0
__________________________________________________________________________________________________
embedding_1 (Embedding) (None, 67, 300) 1176000 input_1[0][0]
__________________________________________________________________________________________________
simple_rnn_1 (SimpleRNN) (None, 256) 142592 embedding_1[0][0]
__________________________________________________________________________________________________
meta_input (InputLayer) (None, 67) 0
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, 323) 0 simple_rnn_1[0][0]
meta_input[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout) (None, 323) 0 concatenate_1[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 32) 10368 dropout_1[0][0]
__________________________________________________________________________________________________
dense_2 (Dense) (None, 61) 2013 dense_1[0][0]
==================================================================================================
Total params: 1,330,973
Trainable params: 154,973
Non-trainable params: 1,176,000
__________________________________________________________________________________________________
The function call is:
simple_RNN_model_trainable.fit([padded_sentences, meta_data_features], padded_verbs,batch_size=32,epochs=1)
X_train constitutes [padded_sentences, meta_data_features] and Y_train is padded_verbs. Their shapes are:
padded_sentences - (3306, 67)
meta_data_features - (3306, 67)
padded_verbs - (3306, 67, 1)
When I try to fit the model, I get the error, "ValueError: Error when checking target: expected dense_2 to have 2 dimensions, but got array with shape (3306, 67, 1)"
It would be great if somebody can help me in resolving the error. Thanks!