I want to feed IMDB dataset with multi layer dynamic LSTM network but it seems that next LSTM layers couldn't parse previous layers output.
code:
net = tflearn.input_data([None, 100])
net = tflearn.embedding(net, input_dim=6819, output_dim=256)
net = tflearn.lstm(net, 256, weights_init="xavier", dynamic=True, return_seq=True)
net = tflearn.dropout(net, 0.8)
net = tflearn.lstm(net, 256, weights_init="xavier", dynamic=True)
net = tflearn.dropout(net, 0.8)
net = tflearn.fully_connected(net, 2, activation='softmax')
error:
Traceback (most recent call last):
File "train.py", line 65, in <module>
model.fit(train_x, train_y, validation_set=(test_x, test_y), show_metric=True, batch_size=32)
File ".../tflearn/models/dnn.py", line 216, in fit
callbacks=callbacks)
File ".../tflearn/helpers/trainer.py", line 339, in fit
show_metric)
File ".../tflearn/helpers/trainer.py", line 818, in _train
feed_batch)
File ".../tensorflow/python/client/session.py", line 895, in run
run_metadata_ptr)
File ".../tensorflow/python/client/session.py", line 1100, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (32, 2) for Tensor u'TargetsData/Y:0', which has shape '(100, 2)'
Related
I am trying to build a CNN-LSTM model. My dataset is made by a bunch of single-channel images of size (n_spot, n_meas) = (100,102). In this small instance, I have got just n_sim = 20 images so that:
print(X_train.shape) -> (12, 100, 102, 1)
print(X_val.shape) -> (4, 100, 102, 1)
print(X_test.shape) -> (4, 100, 102, 1)
print(y_train.shape) -> (12, 200)
print(y_val.shape) -> (4, 200)
print(y_test.shape) -> (4,200)
My code is the following:
# create model
model = Sequential()
# add model layers
model.add(TimeDistributed(Conv2D(filters = 8, kernel_size=[3,3], padding = 'same',
activation='relu', input_shape=(None,n_spot,n_meas,1))))
model.add(TimeDistributed(MaxPool2D(pool_size=(3,3))))
model.add(TimeDistributed(Flatten()))
model.add(LSTM(20, return_sequences=False))
model.add(Dense(2*n_spot, activation = 'linear'))
#compile model using accuracy to measure model performance
model.compile(optimizer='adam', loss='mean_squared_error')
model.summary() returns the following error:
'This model has not yet been built. '
ValueError: This model has not yet been built. Build the model first by calling build() or calling fit() with some data. Or specify input_shape or batch_input_shape in the first layer for automatic build.
while I get this one whenever I try to train my model with
model.fit(X_train, y_train, validation_data=(X_val, y_val), epochs=30)
Using TensorFlow backend.
Traceback (most recent call last):
File "<ipython-input-1-1109305aff30>", line 1, in <module>
runfile('C:/Users/nle5266/Documents/positioning/reduced_data.py', wdir='C:/Users/nle5266/Documents/positioning')
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 786, in runfile
execfile(filename, namespace)
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/nle5266/Documents/positioning/reduced_data.py", line 338, in <module>
model.fit(X_train, y_train, validation_data=(X_val, y_val), epochs=30, callbacks=[tensorboard])
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\keras\engine\training.py", line 952, in fit
batch_size=batch_size)
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\keras\engine\training.py", line 677, in _standardize_user_data
self._set_inputs(x)
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\keras\engine\training.py", line 589, in _set_inputs
self.build(input_shape=(None,) + inputs.shape[1:])
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\keras\engine\sequential.py", line 221, in build
x = layer(x)
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\keras\engine\base_layer.py", line 457, in __call__
output = self.call(inputs, **kwargs)
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\keras\layers\wrappers.py", line 248, in call
y = self.layer.call(inputs, **kwargs)
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\keras\layers\convolutional.py", line 171, in call
dilation_rate=self.dilation_rate)
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\keras\backend\tensorflow_backend.py", line 3650, in conv2d
data_format=tf_data_format)
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 779, in convolution
data_format=data_format)
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\tensorflow\python\ops\nn_ops.py", line 828, in __init__
input_channels_dim = input_shape[num_spatial_dims + 1]
File "C:\Users\nle5266\AppData\Local\conda\conda\envs\tensorflow_env\lib\site-packages\tensorflow\python\framework\tensor_shape.py", line 616, in __getitem__
return self._dims[key]
IndexError: list index out of range
Googling around, I realized that this kind of problem is usually due to the input shape of the time distributed conv2D layer which should a 4-tuple. I tried with (None,n_spot,n_meas,1) with no success.
I'm using keras 2.2.4 with tensorflow 1.12.0. I tried to upgrade tensorflow to the last nightly but it didn't solve the problem.
I have a few matrices that pass through multiple layers, the last one is Dense layer to produce a vector for each matrix. Now I wish to feed these matrices to an RNN of keras and that is where I face this error.
I tried stacking the vectors together in order to pass them to the RNN. Here's a piece of code for that idea:
input1 = Dense(20, activation = "relu")(input1)
input2 = Dense(20, activation = "relu")(input2)
out = Lambda(lambda x: tf.stack([x[0], x[1]], axis=1), output_shape=(None, 2, 20))([input1, input2])
out = SimpleRNN(50, activation="relu")(out)
And I receive:
>Traceback (most recent call last):
>>File "model.py", line 106, in <module>
model = make_model()
>>File "model.py", line 60, in make_model
out = SimpleRNN(50, activation="relu")(out)
>>File "/home/yamini/.local/lib/python3.6/site-packages/keras/layers/recurrent.py", line 532, in __call__
return super(RNN, self).__call__(inputs, **kwargs)
>>File "/home/yamini/.local/lib/python3.6/site-packages/keras/engine/base_layer.py", line 440, in __call__
self.assert_input_compatibility(inputs)
>>File "/home/yamini/.local/lib/python3.6/site-packages/keras/engine/base_layer.py", line 368, in assert_input_compatibility
str(x_shape))
>>ValueError: Input 0 is incompatible with layer simple_rnn_1: expected shape=(None, None, 20), found shape=(None, None, 2, 20)
And If I change the output_shape=(None, None, 20) in Lambda layer, I get:
Traceback (most recent call last):
>> File "model.py", line 107, in <module>
model.fit([input1, input2], y_train, epochs = 15, batch_size = 20, verbose = 2)
>>File "/home/yamini/.local/lib/python3.6/site-packages/keras/engine/training.py", line 952, in fit
batch_size=batch_size)
>>File "/home/yamini/.local/lib/python3.6/site-packages/keras/engine/training.py", line 789, in _standardize_user_data
exception_prefix='target')
>>File "/home/yamini/.local/lib/python3.6/site-packages/keras/engine/training_utils.py", line 138, in standardize_input_data
str(data_shape))
>>ValueError: Error when checking target: expected simple_rnn_1 to have shape (50,) but got array with shape (1,)
You can change output_shape,which should not contain batch_size.
from keras.layers import Dense,Lambda,SimpleRNN,Input
import tensorflow as tf
input1 = Input(shape=(20,))
input2 = Input(shape=(20,))
input1 = Dense(20, activation = "relu")(input1)
input2 = Dense(20, activation = "relu")(input2)
out = Lambda(lambda x: tf.stack([x[0], x[1]], axis=1), output_shape=(2, 20))([input1, input2])
out = SimpleRNN(50, activation="relu")(out)
I am trying to export a model that I trained using keras with a tensorflow backend into a .pb file so I can use it for android and I am getting the following error when I try to freeze my graph.
Traceback (most recent call last):
File "youtubeExample.py", line 102, in <module>
export_model(tf.train.Saver(), model, ["conv2d_1_input"], "output")
File "youtubeExample.py", line 49, in export_model
'out/frozen_' + MODEL_NAME + '.pb', True, "")
File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/tools/freeze_graph.py", line 244, in freeze_graph
saved_model_tags.split(","), checkpoint_version=checkpoint_version)
File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/tools/freeze_graph.py", line 153, in freeze_graph_with_def_protos
variable_names_blacklist=variable_names_blacklist)
File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/graph_util_impl.py", line 232, in convert_variables_to_constants
inference_graph = extract_sub_graph(input_graph_def, output_node_names)
File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/graph_util_impl.py", line 174, in extract_sub_graph
_assert_nodes_are_present(name_to_node, dest_nodes)
File "/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/graph_util_impl.py", line 133, in _assert_nodes_are_present
assert d in name_to_node, "%s is not in graph" % d
AssertionError: output is not in graph
The the part of code that I use to define my model is:
model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(224,224,3)))
model.add(Flatten())
model.add(Dense(units = 2, activation='softmax', name = "output"))
model.compile(Adam(lr=.0001), loss='categorical_crossentropy', metrics=['accuracy'])
I would appreciate any help I could get with this issue as I am relatively new to tensorflow/keras.
There is an in-detailed post on keras blog.
But when compiling the code I get the error as follows:
Using TensorFlow backend.
Traceback (most recent call last):
File "visulaize_cifar.py", line 24, in <module>
model.add(MaxPooling2D((2, 2), strides=(2, 2)))
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/keras/models.py", line 332, in add
output_tensor = layer(self.outputs[0])
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 572, in __call__
self.add_inbound_node(inbound_layers, node_indices, tensor_indices)
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 635, in add_inbound_node
Node.create_node(self, inbound_layers, node_indices, tensor_indices)
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 166, in create_node
output_tensors = to_list(outbound_layer.call(input_tensors[0], mask=input_masks[0]))
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/keras/layers/pooling.py", line 160, in call
dim_ordering=self.dim_ordering)
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/keras/layers/pooling.py", line 210, in _pooling_function
pool_mode='max')
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 2866, in pool2d
x = tf.nn.max_pool(x, pool_size, strides, padding=padding)
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/tensorflow/python/ops/nn_ops.py", line 1617, in max_pool
name=name)
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 1598, in _max_pool
data_format=data_format, name=name)
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
op_def=op_def)
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2242, in create_op
set_shapes_for_outputs(ret)
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1617, in set_shapes_for_outputs
shapes = shape_func(op)
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1568, in call_with_requiring
return call_cpp_shape_fn(op, require_shape_fn=True)
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 610, in call_cpp_shape_fn
debug_python_shape_fn, require_shape_fn)
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 675, in _call_cpp_shape_fn_impl
raise ValueError(err.message)
ValueError: Negative dimension size caused by subtracting 2 from 1 for 'MaxPool_1' (op: 'MaxPool') with input shapes: [1,1,64,128].
This error goes when I set dim_ordering='th'.
But as I am using tensorflow backend so dimension ordering should be dim_ordering='tf'.
Even after setting dim_ordering as 'th', I get error while loading weights from vgg16_weights.h5 as follows :
Traceback (most recent call last):
File "visulaize_cifar.py", line 67, in <module>
model.layers[k].set_weights(weights)
File "/home/dude_perf3ct/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 985, in set_weights
'provided weight shape ' + str(w.shape))
ValueError: Layer weight shape (3, 3, 128, 64) not compatible with provided weight shape (64, 3, 3, 3).
As detailed in this post about 'th' and 'tf'. The above error implies layer weights are in 'tf' (but I set it to 'th' to avoid first error) and provided weight shape in 'th' ordering.
What seems to be the error?
Answer to this question was pretty simple. As, I was using tensorflow as backend. So, to convert I inserted the line
if K.backend()=='tensorflow':
K.set_image_dim_ordering("th")
after from keras import backend as K.
This is because the vgg16_weights.h5 has th format and also cifar10.load_data().
I was following the standard cifar 10 keras tutorial here: https://github.com/fchollet/keras/blob/master/examples/cifar10_cnn.py
Which I modified to use my own training images. Each image replicates the dimensions of the cifar set, ie, they are each 32x32 and 3 channels.
Shape of each image:
(32,32,3)
However, I run into a ValueError as shown in the full output below.
X_train shape: (7200, 32, 32, 3)
7200 train samples
800 test samples
Using real-time data augmentation.
Epoch 1/200
Traceback (most recent call last):
File "<ipython-input-16-70ca8831e139>", line 162, in <module>
validation_data=(X_test, Y_test))
File "/storage/programfiles/anaconda3/lib/python3.5/site-packages/keras/models.py", line 651, in fit_generator
max_q_size=max_q_size)
File "/storage/programfiles/anaconda3/lib/python3.5/site-packages/keras/engine/training.py", line 1383, in fit_generator
class_weight=class_weight)
File "/storage/programfiles/anaconda3/lib/python3.5/site-packages/keras/engine/training.py", line 1167, in train_on_batch
outputs = self.train_function(ins)
File "/storage/programfiles/anaconda3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 659, in __call__
updated = session.run(self.outputs + self.updates, feed_dict=feed_dict)
File "/storage/programfiles/anaconda3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 372, in run
run_metadata_ptr)
File "/storage/programfiles/anaconda3/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 625, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (32, 32, 32, 3) for Tensor 'convolution2d_input_7:0', which has shape '(?, 3, 32, 32)'
Can anyone help me out? :)
EDIT:
I tried reshaping as follows:
X_train = X_train.reshape((7200,3,32,32))
X_test = X_test.reshape((-1,3,32,32))
It crashed instead.
You actually need to transpose your array to the correct ordering, not a reshape:
X_train = np.transpose(X_train, (0, 3, 1, 2))
X_test = np.transpose(X_test, (0, 3, 1, 2))