I tryied to test direct keras model call with a demo, the code is simple as below.
from tensorflow import keras
from tensorflow.keras import layers
model = keras.models.Sequential()
model.add(layers.Embedding(input_dim=100, output_dim=32, input_length=5))
model.add(layers.Flatten())
model.add(layers.Dense(units=5, activation='sigmoid'))
f = np.random.randint(0,100, 5)
print(model(f))
It raises error:
ValueError: Input 0 of layer dense is incompatible with the layer: expected axis -1 of input shape to have value 160 but received input with shape (5, 32)
Is the data I mocked format wrong or keras model doesn't support this kind of debugging/testing.
It turns out I made a foolish mistake.
The input should be batched called.
So change f with f = np.random.randint(0,100, (1,5)) will work
Related
I am trying to do a transfer learning; for that purpose I want to remove the last two layers of the neural network and add another two layers. This is an example code which also output the same error.
from keras.models import Sequential
from keras.layers import Input,Flatten
from keras.layers.convolutional import Convolution2D, MaxPooling2D
from keras.layers.core import Dropout, Activation
from keras.layers.pooling import GlobalAveragePooling2D
from keras.models import Model
in_img = Input(shape=(3, 32, 32))
x = Convolution2D(12, 3, 3, subsample=(2, 2), border_mode='valid', name='conv1')(in_img)
x = Activation('relu', name='relu_conv1')(x)
x = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), name='pool1')(x)
x = Convolution2D(3, 1, 1, border_mode='valid', name='conv2')(x)
x = Activation('relu', name='relu_conv2')(x)
x = GlobalAveragePooling2D()(x)
o = Activation('softmax', name='loss')(x)
model = Model(input=in_img, output=[o])
model.compile(loss="categorical_crossentropy", optimizer="adam")
#model.load_weights('model_weights.h5', by_name=True)
model.summary()
model.layers.pop()
model.layers.pop()
model.summary()
model.add(MaxPooling2D())
model.add(Activation('sigmoid', name='loss'))
I removed the layer using pop() but when I tried to add its outputting this error
AttributeError: 'Model' object has no attribute 'add'
I know the most probable reason for the error is improper use of model.add(). what other syntax should I use?
EDIT:
I tried to remove/add layers in keras but its not allowing it to be added after loading external weights.
from keras.models import Sequential
from keras.layers import Input,Flatten
from keras.layers.convolutional import Convolution2D, MaxPooling2D
from keras.layers.core import Dropout, Activation
from keras.layers.pooling import GlobalAveragePooling2D
from keras.models import Model
in_img = Input(shape=(3, 32, 32))
def gen_model():
in_img = Input(shape=(3, 32, 32))
x = Convolution2D(12, 3, 3, subsample=(2, 2), border_mode='valid', name='conv1')(in_img)
x = Activation('relu', name='relu_conv1')(x)
x = MaxPooling2D(pool_size=(3, 3), strides=(2, 2), name='pool1')(x)
x = Convolution2D(3, 1, 1, border_mode='valid', name='conv2')(x)
x = Activation('relu', name='relu_conv2')(x)
x = GlobalAveragePooling2D()(x)
o = Activation('softmax', name='loss')(x)
model = Model(input=in_img, output=[o])
return model
#parent model
model=gen_model()
model.compile(loss="categorical_crossentropy", optimizer="adam")
model.summary()
#saving model weights
model.save('model_weights.h5')
#loading weights to second model
model2=gen_model()
model2.compile(loss="categorical_crossentropy", optimizer="adam")
model2.load_weights('model_weights.h5', by_name=True)
model2.layers.pop()
model2.layers.pop()
model2.summary()
#editing layers in the second model and saving as third model
x = MaxPooling2D()(model2.layers[-1].output)
o = Activation('sigmoid', name='loss')(x)
model3 = Model(input=in_img, output=[o])
its showing this error
RuntimeError: Graph disconnected: cannot obtain value for tensor input_4 at layer "input_4". The following previous layers were accessed without issue: []
You can take the output of the last model and create a new model. The lower layers remains the same.
model.summary()
model.layers.pop()
model.layers.pop()
model.summary()
x = MaxPooling2D()(model.layers[-1].output)
o = Activation('sigmoid', name='loss')(x)
model2 = Model(inputs=in_img, outputs=[o])
model2.summary()
Check How to use models from keras.applications for transfer learnig?
Update on Edit:
The new error is because you are trying to create the new model on global in_img which is actually not used in the previous model creation.. there you are actually defining a local in_img. So the global in_img is obviously not connected to the upper layers in the symbolic graph. And it has nothing to do with loading weights.
To better resolve this problem you should instead use model.input to reference to the input.
model3 = Model(input=model2.input, output=[o])
Another way to do it
from keras.models import Model
layer_name = 'relu_conv2'
model2= Model(inputs=model1.input, outputs=model1.get_layer(layer_name).output)
As of Keras 2.3.1 and TensorFlow 2.0, model.layers.pop() is not working as intended (see issue here). They suggested two options to do this.
One option is to recreate the model and copy the layers. For instance, if you want to remove the last layer and add another one, you can do:
model = Sequential()
for layer in source_model.layers[:-1]: # go through until last layer
model.add(layer)
model.add(Dense(3, activation='softmax'))
model.summary()
model.compile(optimizer='adam', loss='categorical_crossentropy')
Another option is to use the functional model:
predictions = Dense(3, activation='softmax')(source_model.layers[-2].output)
model = Model(inputs=inputs, outputs=predictions)
model.compile(optimizer='adam', loss='categorical_crossentropy')
model.layers[-1].output means the last layer's output which is the final output, so in your code, you actually didn't remove any layers, you added another head/path.
An alternative to Wesam Na's answer, if you don't know the layer names you can simply cut off the last layer via:
from keras.models import Model
model2= Model(inputs=model1.input, outputs=model1.layers[-2].output)
I'm trying to build a Sequential model with tensorflow.
import tensorflow as tf
import keras
from tensorflow.keras import layers
from keras import optimizers
import numpy as np
model = keras.Sequential (name="model")
model.add(keras.Input(shape=(786,)))
model.add(layers.Dense(2048, activation="relu", name="layer1"))
model.add(layers.Dense(786, activation="relu", name="layer2"))
model.add(layers.Dense(786, activation="relu", name="layer3"))
output = model.add(layers.Dense(786, activation="relu", name="output"))
model.summary()
model.compile(
optimizer=tf.optimizers.Adam(), # Optimizer
loss=keras.losses.CategoricalCrossentropy(),
metrics=[keras.metrics.SparseCategoricalAccuracy()],
)
history = model.fit(
x_train,
y_train,
batch_size=1,
epochs=5,
)
The input shape is a vector with length of 768 (so the input shape is (768,) right?), representing a chess board:
def get_dataset():
container = np.load('/content/drive/MyDrive/test_data_vector.npz')
b, v = container['arr_0'], container['arr_1']
v = np.asarray(v / abs(v).max() / 2 + 0.5, dtype=np.float32) # normalization (0 - 1)
return b, v
xtrain, ytrain = get_dataset()
print(xtrain.shape)
print(ytrain.shape)
>> (37, 786) #there are 37 samples
>> (37, 786)
But I always get the error:
ValueError: Input 0 of layer model is incompatible with the layer: expected axis -1 of input shape to have value 786 but received input with shape (1, 1, 768)
I tried with np.expand_dims(), which ended in the same Error.
The error is just a typo, as the user mentioned the issue is resolved by changing the output shape from 786 to 768 and the issue is resolved.
One suggestion based on the model structure.
The number of units are not related to your input shape, you don't have to match that number.
The number of units like 2048 and 786 in dense layer is too large and this may not help the model to learn better.
Try with smaller numbers like 32,64 etc, you can refer some of the examples in the tensorflow document.
I'm trying to classify mnist's handwritten digits but I keep getting the same error from keras.
import tensorflow as tf
import numpy as np
import tensorflow.keras as keras
from tensorflow.keras.datasets import mnist
model = keras.Sequential()
model.add(keras.layers.Dense(15, input_shape=(784,), activation='relu'))
model.add(keras.layers.Dense(10, activation='softmax'))
(data, label), (val_data, val_label) = mnist.load_data()
data = data.reshape(data.shape[0],data.shape[1]*data.shape[2])
val_data = val_data.reshape(val_data.shape[0],val_data.shape[1]*val_data.shape[2])
model.compile(optimizer=tf.train.GradientDescentOptimizer(0.01),
loss='mse',
metrics=['acc'])
model.fit(data,label,batch_size=30,epochs=10,validation_data=(val_data,val_label))
The softmax layer expects a tensor of size (None, 10). So, you have to encode your label data using one hot encoder. It can be done in the following way:
label = keras.utils.to_categorical(label, num_classes = 10)
val_label = keras.utils.to_categorical(val_label, num_classes = 10)
If you are not familiar with one hot encoding, you can refer it here: One hot encoding in python
I created a custom activation function with keras, which reduce the channel size by half (max-feature map activation).
Here's what part of the code looks like :
import tensorflow as tf
import keras
from keras.utils.generic_utils import get_custom_objects
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers import Conv2D, MaxPooling2D, Activation
def MyMFM (x):
Leng = int(x.shape[-1])
ind1=int(Leng/2)
X1=x[:,:,:,0:ind1]
X2=x[:,:,:,ind1:Leng]
MfmOut=tf.maximum(X1,X2)
return MfmOut
get_custom_objects().update({'MyMFM ': Activation(MyMFM)})
model = Sequential()
model.add(Conv2D(32, kernel_size=(5, 5),strides=(1, 1), padding = 'same',input_shape = (513,211,1)))
model.add(Activation(MyMFM))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Conv2D(48, kernel_size=(1, 1),strides=(1, 1 ), padding = 'same'))
When I compile this code, I get the following error :
number of input channels does not match corresponding dimension of filter, 16 != 32
This error is from the last line of code. After activation, the channel length is reduced to 16 from 32. But the next layer automatically considers the channel length as 32 (No of filters in the first layer) not 16. I tried adding input_shape argument in the second convolution layer to define the input shape as (513,211,16). But that also gave me the same error. What should I do to pass the shape of the tensor to the next layer after activation?
Thank you
So - based on this documentation, you may see that keras engine automatically sets the output shape from a layer to be the same as its input shape.
Use Lambda layer instead.
I get the following error for the code snippet below:
You must feed a value for placeholder tensor
'bidirectional_1/keras_learning_phase' with dtype bool
If I add the dropout layer model.add(Dropout(dropout)), it works. Anyone knows why? The back-end is Tensorflow, Keras 2.0.1
def prep_model1(embedding_layer1, embedding_layer2, dropout=0.5):
model0 = Sequential()
model0.add(embedding_layer1)
model0.add(Bidirectional(LSTM(128, return_sequences=False, dropout=dropout)))
model1 = Sequential()
model1.add(embedding_layer2)
model1.add(Bidirectional(LSTM(128, return_sequences=False, dropout=dropout)))
model = Sequential()
model.add(Merge([model0, model1], mode='concat', concat_axis=1))
#model.add(Dropout(dropout))
model.add(Dense(1, activation='sigmoid'))
return model
Try to import K and set learning phase before your model.
from keras import backend as K
K.set_learning_phase(1) #set learning phase
From this issue