Trying to use TF2 and the Universal Sentence Encoder. I use the same code below and it works for a few other Hub models (nnlm-20,50,128), but not for the USE v4. I think it has to do with the output shape (due to the error and what model.summary() shows), because the output is a dictionary of {'outputs':(None,512)}:
embed_layer = hub.KerasLayer("https://tfhub.dev/google/universal-sentence-encoder-large/4",input_shape=[],dtype=tf.string,trainable=True)
model = tf.keras.Sequential()
model.add(embed_layer)
model.add(tf.keras.layers.Dense(16, activation='relu'))
model.add(tf.keras.layers.Dense(1, activation='sigmoid'))
model.summary()
model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'])
history = model.fit(train_dataset,
validation_data=validation_dataset,
epochs=30,
verbose=1
)
ValueError: Error when checking input: expected keras_layer_input to have 1 dimensions, but got array with shape ()
Related
I am using one of the above methods to save my model. But I get this below error , can some one help me . I am very new to this tensor flows.
My Code is
Model Creation
model = Sequential()
model.add(Embedding(vocab_size, 8, input_length=max_length))
model.add(Flatten())
model.add(Dense(512, activation='relu'))
model.add(Dense(256, activation='relu'))
model.add(Dropout(.2))
model.add(Dense(128, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(number_of_classes, activation='softmax'))
# compile the model
##only use if pre-trained
sgd = optimizers.SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc'])
input_shape = X.shape
model.build(input_shape)
# summarize the model
model.summary()
checkpoint = ModelCheckpoint('Files5.hdf5', monitor='val_acc', verbose=1, save_best_only=True, mode='max')
history = model.fit(X_train, Y_train, validation_data=(X_val, Y_val), epochs=10, batch_size=20, callbacks=[checkpoint_callback], verbose=2)
return model, history
NotImplementedError: Layer ModuleWrapper has arguments in __init__ and therefore must override get_config
I referred these ones - but still no luck
Layer ModuleWrapper has arguments in `__init__` and therefore must override `get_config`. in Colab
How to save final model using keras?
Thank you
I also encountered a similar problem. Later I found out that the error was caused by mixing keras and tensorflow.keras. By removing tensorflow.keras, I successfully solved the error.
The above answer worked for me, except I kept getting deep copy errors on updating, so commented out the update resolves this:
def get_config(self):
config = super().get_config().copy()
# config.update({
# 'labels': self.labels_array,
# 'num_labels': self.num_labels,
# })
return config
Now the keras model saves callback checkpoints with the new softmax derivative layer I implemented. [tensorflow 1.15, using tf.keras]
Thanks for the assist!
This can be resolved by converting all your imports to tensorflow.keras.
example :
from tensorflow.python.keras.models import Sequential
from tensorflow.python.keras.layers import Dense, LeakyReLU, ReLU, Dropout
I have a simple MLP built in Keras. The shapes of my inputs are:
X_train.shape - (6, 5)
Y_train.shape - 6
Create the model
model = Sequential()
model.add(Dense(32, input_shape=(X_train.shape[0],), activation='relu'))
model.add(Dense(Y_train.shape[0], activation='softmax'))
# Compile and fit
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X_train, Y_train, epochs=10, batch_size=1, verbose=1, validation_split=0.2)
# Get output vector from softmax
output = model.layers[-1].output
This gives me the error:
ValueError: Error when checking input: expected dense_1_input to have shape (6,) but got array with shape (5,).
I have two questions:
Why do I get the above error and how can I solve it?
Is output = model.layers[-1].output the way to return the softmax vector for a given input vector? I haven't ever done this in Keras.
in the input layer use input_shape=(X_train.shape[1],) while your last layer has to be a dimension equal to the number of classes to predict
the way to return the softmax vector is model.predict(X)
here a complete example
n_sample = 5
n_class = 2
X = np.random.uniform(0,1, (n_sample,6))
y = np.random.randint(0,n_class, n_sample)
model = Sequential()
model.add(Dense(32, input_shape=(X.shape[1],), activation='relu'))
model.add(Dense(n_class, activation='softmax'))
# Compile and fit
model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X, y, epochs=10, batch_size=1, verbose=1)
# Get output vector from softmax
model.predict(X)
I am following this tutorial to try fine-tuning using VGG16 model, I trained the model and saved .h5 file using model.save_weights and
vgg_conv = VGG16(include_top=False, weights='imagenet', input_shape=(image_size, image_size, 3))
# Freeze the layers except the last 4 layers
for layer in vgg_conv.layers[:-4]:
layer.trainable = False
model = Sequential()
model.add(vgg_conv)
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(11, activation='softmax'))
I then tried to rebuild the architecture and load weights using the below
def create_model(self):
model = Sequential()
vgg_model = VGG16(include_top=False, weights='imagenet', input_shape=(150, 150, 3))
model.add(vgg_model)
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(11, activation='softmax'))
model.load_weights(self.top_model_weights_path) # throws error
return model
but it then throws this error
ValueError: Cannot feed value of shape (512, 512, 3, 3) for Tensor 'Placeholder:0', which has shape '(3, 3, 3, 64)'
what am I doing wrong?
I am not sure how to intepret the error but you could try saving the model architecture and the weights together model.save("model.h5") after fine tuning.
To load the model you can type
model = load_model('model.h5')
# summarize model.
model.summary()
I think this has the benefeit of not having to rebuild the model and requires only one line to acomplish the same purpose.
The problem comes from the trainable difference between the two models. If you freeze the 4 last layers in the create_model function, it will work.
But as said by Igna, model.save and model.load_model is simpler.
Y_train = to_categorical(Y_train, num_classes = 10)#
random_seed = 2
X_train,X_val,Y_train,Y_val = train_test_split(X_train, Y_train, test_size = 0.1, random_state=random_seed)
Y_train.shape
model = Sequential()
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dense(128, activation='relu'))
model.add(Dense(10, activation='softmax'))
model.compile(optimizer = 'adam', loss = 'sparse_categorical_crossentropy',metrics=['accuracy'])
model.fit(X_train, Y_train, batch_size = 86, epochs = 3,validation_data = (X_val, Y_val), verbose =2)
I have to classify the MNIST data into 10 classes. I am converting the Y_train into one hot encoded array. I have gone through a number of answers but none have helped. Kindly guide me in this regard as I am a novice in ML and neural network.
It seems there is no need to use model.add(Flatten()) in your first layer. Instead of doing so, you can use a dense layer with a specific input size like: model.add(Dense(64, input_shape=your_input_shape, activation="relu").
To ensure this issue happens because of the layers, you can check whether to_categorical() function works alone with jupyter notebook.
Updated Answer
Before the model, you should reshape your model. In that case 28*28 to 784.
train_images = train_images.reshape((-1, 784))
test_images = test_images.reshape((-1, 784))
I also suggest to normalize the data that could be done by simply dividing the images to 255
After that step you should create your model.
model = Sequential([
Dense(64, activation='relu', input_shape=(784,)),
Dense(64, activation='relu'),
Dense(10, activation='softmax'),
])
Have you noticed input_shape=(784,) That is the shape of your flattened input.
Last step, compiling and fitting.
model.compile(
optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy'],
)
model.fit(
train_images,
train_labels,
epochs=10,
batch_size=16,
)
What you do is you have just flattened the input layer without feeding the network with an input. That's why you experience an issue. The point is you should manually reshape your inputs and feed forward to the Dense() layers with parameter input_shape
I'm implementing this simple neural network, with these inputs data:
x_train = np.asarray(x_train)
y_train = np.asarray(y_train)
x_test = np.asarray(x_test)
y_test = np.asarray(y_test)
After have defined the network's structure:
model = Sequential()
model.add(Dense(20, input_dim=5, init='normal', activation='sigmoid'))
model.add(Dense(1, init='normal', activation='sigmoid'))
I run this to train and evaluate the NN:
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
history = model.fit(x_train, y_train, nb_epoch=10, validation_split=0.2)
and I always get the same result from model.fit( ... ) :
32/143 [=====>........................] - ETA: 0s.
It seems that doesn't work at all, despite I obtain consistent results on training and validation. How does I have to interpreter this stationary result about model. fit output?