I'm getting the error in following code
flatten = Flatten()(drop_5)
aux_rand = Input(shape=(1,))
concat = Concatenate([flatten, aux_input])
fc1 = Dense(512, kernel_regularizer=regularizers.l2(weight_decay))(concat)
ValueError: Layer dense_1 was called with an input that isn't a
symbolic tensor. Received type: . Full input: []. All inputs to the
layer should be tensors.
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
aux_rand = np.random.rand(y_train.shape[0])
model_inst = cifar10vgg()
x_train_input = Input(shape=(32,32,3))
aux_input = Input(shape=(1,))
model = Model(inputs=[x_train_input, aux_input], output=model_inst.build_model())
model.fit(x=[x_train, aux_rand], y=y_train, batch_size=batch_size, steps_per_epoch=x_train.shape[0] // batch_size,
epochs=maxepoches, validation_data=(x_test, y_test),
callbacks=[reduce_lr, tensorboard], verbose=2)
model_inst.build_model() returns output from model's last layer (Activation('softmax')(fc2))
The error happens because concat is not a tensor. The layer keras.layers.Concatenate should be called as follows:
concat = Concatenate()([flatten, aux_input])
Related
I need to run gridsearch CV on a Keras model but keep running into the following error:
TypeError: Only integers, slices (:), ellipsis (...), tf.newaxis (None) and scalar tf.int32/tf.int64 tensors are valid indices, got array([20000, 20001, 20002, ..., 59997, 59998, 59999])
on line grid_result = grid.fit(x_train, y_train)
The code to run the Gridsearch CV is as follows:
batch_size = 128
epochs = 20
model_CV = KerasClassifier(build_fn=create_model,epochs=epochs,batch_size=batch_size, verbose=0)
define the grid search parameters
init_mode = ['uniform', 'normal', 'he_normal','he_uniform']
param_grid = dict(init_mode=init_mode)
grid = GridSearchCV(estimator=model_CV,param_grid=param_grid, cv=3)
grid_result = grid.fit(x_train, y_train)
create_model used above
def create_model(init_mode='uniform'):
model = Sequential()
model.add(Dense(64, kernel_initializer=init_mode,
activation=tf.nn.relu, input_dim=784))
model.add(Dropout(rate=0.5))
model.add(Dense(64, kernel_initializer=init_mode,
activation=tf.nn.relu))
model.add(Dense(10, kernel_initializer=init_mode, activation=tf.nn.softmax))
compile model
model.compile(loss='categorical_crossentropy',
optimizer=RMSprop(),
metrics=['accuracy'])
return model
Data Source
mnist = keras.datasets.mnist
(x_train, y_train),(x_test, y_test) = mnist.load_data()
Data Preprocessing
flatten = tf.keras.layers.Flatten(input_shape=[28,28])
x_train = flatten(x_train)
x_train = x_train / 255
from tensorflow.keras.utils import to_categorical
y_train = to_categorical(y_train, num_classes = num_classes)
I tried changing y_train by flattening it or not running to_categorical on y_train but I still run into the same issue.
Is the problem with x_train or y_train and how can I fix it? Thank you for any help provided.
I'm using this pre-trained model by bert-for-tf2 in my functional model like following:
def build_model(model_dir, batch_size, max_seq_num, max_seq_len):
bert_params = bert.params_from_pretrained_ckpt(model_dir)
l_bert = bert.BertModelLayer.from_params(bert_params, name="bert", trainable=False)
input_ids = tf.keras.layers.Input(shape=(max_seq_num, max_seq_len,), dtype='int32', name='input_ids')
reshaped_input_ids = tf.reshape(input_ids, (batch_size * max_seq_num, max_seq_len))
token_type_ids = tf.keras.layers.Input(shape=(max_seq_num, max_seq_len,), dtype='int32', name='token_type')
reshaped_token_type_ids = tf.reshape(token_type_ids, (batch_size * max_seq_num, max_seq_len))
mask_ids = tf.keras.layers.Input(shape=(max_seq_num, max_seq_len,), dtype='int32', name='mask_ids')
reshaped_mask_ids = tf.reshape(mask_ids, (batch_size * max_seq_num, max_seq_len))
# provide a custom token_type/segment id as a layer input
bert_embedd = l_bert([reshaped_input_ids, reshaped_token_type_ids], mask=reshaped_mask_ids) # [batch_size*max_seq_num, max_seq_len, hidden_size]
model = tf.keras.models.Model(inputs=[input_ids, token_type_ids, mask_ids], outputs=bert_embedd)
model.build(input_shape=[(batch_size, max_seq_num, max_seq_len),
(batch_size, max_seq_num, max_seq_len),
(batch_size, max_seq_num, max_seq_len)])
bert.load_bert_weights(l_bert, os.path.join(model_dir, "bert_model.ckpt")) # should be called after model.build()
model.summary()
tf.keras.utils.plot_model(model, show_shapes=True)
learning_rate = 1e-2
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=learning_rate),
loss=tf.keras.losses.MeanSquaredError(), metrics=['mse'])
return model
I can build the model succeed. But when I fed data to the model:
model = build_model(path, 16, 16, 16)
x_input = np.random.randint(0, 10000, size=[16, 16, 16])
x_token_type = [[[i] * 16 for i in range(16)] for _ in range(16)]
x_mask = np.ones(shape=[16, 16, 16])
y_predict = model(x_input, x_token_type, x_mask)
the error appears:
ValueError: Layer model expects 2 input(s), but it received 1 input tensors. Inputs received: ...
In the last instruction,
y_predict = model(x_input, x_token_type, x_mask)
Two inputs are expected, but you provide only one.
I think you have three inputs—x_input, x_token_type, and x_mask. If you want to train your model, try this:
model.fit([x_input, x_token_type, x_mask])
So, I need to concatenate an input to the flattened layer before going in the dense layer.
I'm using Keras with TF as backend.
model.add(Flatten())
aux_input = Input(shape=(1, ))
model.add(Concatenate([model, aux_input]))
model.add(Dense(512,kernel_regularizer=regularizers.l2(weight_decay)))
I have a scenario like this: X_train, y_train, aux_train. The shape of y_train and aux_train is same (1, ). An image has a ground-truth and an aux_input.
How do I add this aux_input to the model while doing model.fit?
As suggested in answers, I changed my model with functional api. However, now, I get the following error.
ValueError: Layer dense_1 was called with an input that isn't a
symbolic tensor. Received type: . Full input:
[]. All
inputs to the layer should be tensors.
Here's the code for that part.
flatten = Flatten()(drop_5)
aux_rand = Input(shape=self.aux_shape)
concat = Concatenate([flatten, aux_input])
fc1 = Dense(512, kernel_regularizer=regularizers.l2(weight_decay))(concat)
Shape of aux input
aux_shape = (1,)
And then calling the model as follow
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
aux_rand = np.random.rand(y_train.shape[0])
model_inst = cifar10vgg()
x_train_input = Input(shape=(32,32,3))
aux_input = Input(shape=(1,))
model = Model(inputs=[x_train_input, aux_input], output=model_inst.build_model())
model.fit(x=[x_train, aux_rand], y=y_train, batch_size=batch_size, steps_per_epoch=x_train.shape[0] // batch_size,
epochs=maxepoches, validation_data=(x_test, y_test),
callbacks=[reduce_lr, tensorboard], verbose=2)
model_inst.build_model() returns Activation('softmax')(fc2) which is the output to be fed into the Model (as far as I understood)
As I see from your code, you implement the model with sequential API which is not a good option in this case. If you have some auxiliary inputs the best way to implement such a feature is to use functional API.
Here is a example from Keras website:
from keras.layers import Input, Embedding, LSTM, Dense
from keras.models import Model
main_input = Input(shape=(100,), dtype='int32', name='main_input')
x = Embedding(output_dim=512, input_dim=10000, input_length=100)(main_input)
lstm_out = LSTM(32)(x)
auxiliary_output = Dense(1, activation='sigmoid', name='aux_output')(lstm_out)
auxiliary_input = Input(shape=(5,), name='aux_input')
x = keras.layers.concatenate([lstm_out, auxiliary_input])
x = Dense(64, activation='relu')(x)
main_output = Dense(1, activation='sigmoid', name='main_output')(x)
model = Model(inputs=[main_input, auxiliary_input], outputs=[main_output, auxiliary_output])
Based on description, I think following code can give you some intuition:
x1 = Input(shape=(32, 32, 3))
flatten1 = Flatten()(x1)
x2 = Input(shape=(244, 244, 3))
vgg = VGG19(weights='imagenet', include_top=False)(x2)
flatten2 = Flatten()(vgg)
concat = Concatenate()([flatten1, flatten2])
d = Dense(10)(concat)
model = Model(inputs=[x1, x2], outputs=[d])
model.compile('adam', 'categorical_crossentropy')
model.fit(x=[x_train1, x_train2],outputs=y_labels)
I have time series data input 72 value by separate last 6 value for test prediction. I want to use CONV1D with LSTM.
This is my code.
df = pd.read_csv('D://data.csv',
engine='python')
df['DATE_'] = pd.to_datetime(df['DATE_']) + MonthEnd(1)
df = df.set_index('DATE_')
df.head()
split_date = pd.Timestamp('03-01-2015')
train = df.loc[:split_date, ['COLUMN3DATA']]
test = df.loc[split_date:, ['COLUMN3DATA']]
sc = MinMaxScaler()
train_sc = sc.fit_transform(train)
test_sc = sc.transform(test)
X_train = train_sc[:-1]
y_train = train_sc[1:]
X_test = test_sc[:-1]
y_test = test_sc[1:]
################### Convolution #######################
X_train_t = X_train[None,:]
print(X_train_t.shape)
X_test_t = X_test[:, None]
K.clear_session()
model = Sequential()
model.add(Conv1D(6, 3, activation='relu', input_shape=(12,1)))
model.add(LSTM(6, input_shape=(1,3), return_sequences=True))
model.add(LSTM(3))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam' )
model.summary()
model.fit(X_train_t, y_train, epochs=400, batch_size=10, verbose=1)
y_pred = model.predict(X_test_t)
When I run it show error like this
ValueError: Error when checking input: expected conv1d_1_input to have shape (None, 12, 1) but got array with shape (1, 64, 1)
How to use conv1D with lstm
The problem is between your input data and your input shape.
You said in the model that your input shape is (12,1) (= batch_shape=(None,12,1))
But your data X_train_t has shape (1,64,1).
Either you fix the input shape of the model, or you fix your data if this is not the expected shape.
For variable lengths/timesteps, you can use input_shape=(None,1).
You don't need an input_shape in the second layer.
I'm trying to implement this simple neural network by Keras (Tensorflow beckend):
x_train = df_train[["Pclass", "Gender", "Age","SibSp", "Parch"]]
y_train = df_train ["Survived"]
x_test = df_test[["Pclass", "Gender", "Age","SibSp", "Parch"]]
y_test = df_test["Survived"]
y_train = y_train.values
y_test = y_test.values
But when I run this part:
model = Sequential()
model.add(Dense(input_dim=5, output_dim=1))
model.add(Activation("softmax"))
model.compile(loss='sparse_categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])
model.fit(x_train, y_train)
I get this error: IndexError: indices are out-of-bounds. I am supposing that it is about the arguments in model.fit(x_train, y_train). I have tried to pass these as numpy arrays by .values, but I still have the same error.
Keras expects numpy arrays not pandas, so you need to convert all of the data that you are feeding into Keras APIs.. not just y_train and y_test
So:
x_train = x_train.values
y_train = y_train.values
x_test = x_test.values
y_test = y_test.values
Or
x_train = numpy.asarray(x_train)
y_train = numpy.asarray(y_train)
x_test = numpy.asarray(x_test)
y_test = numpy.asarray(y_test)