I have some data which looks like this:
X = [[1,2,3,4],[01010],[-1.6]]
y = [[4,2]]
I am trying to train a neural net on this data using tflearn. I'm using the same example given on the TFlearn github homepage (https://github.com/tflearn/tflearn) except that I have changed the shape of the data.
tflearn.init_graph(num_cores=1)
net = tflearn.input_data(shape=[None, 2,2,1])
net = tflearn.fully_connected(net, 64)
net = tflearn.dropout(net, 0.5)
net = tflearn.fully_connected(net, 10, activation='softmax')
net = tflearn.regression(net, optimizer='adam', loss='categorical_crossentropy')
model = tflearn.DNN(net)
model.fit(X,y)
I keep getting this error:
"IndexError: index 2 is out of bounds for axis 0 with size 1."
I think this is either due to the shape of the data being specified is incorrect or something to do with the fully_connected layer.
What does this error mean? Is it due to the shape being wrong? What do I need to change in the code above to prevent this error?
Any help would be greatly appreciated.
The issue has been discussed in detail in the following thread.
List index out of......
Apparently a simple addition of the following code on top fixed the issue:
tf.reset_default_graph()
tf here means tensorflow, so don't forget to import tensorflow
I hope this helps
Related
I trained a simple MLP model using new tf.keras version 2.2.4-tf. Here is how the model look like:
input_layer = Input(batch_shape=(138, 28))
first_layer = Dense(30, activation=activation, name="first_dense_layer_1")(input_layer)
first_layer = Dropout(0.1)(first_layer, training=True)
second_layer = Dense(15, activation=activation, name="second_dense_layer")(first_layer)
out = Dense(1, name='output_layer')(second_layer)
model = Model(input_layer, out)
I'm getting an error when I try to do prediction prediction_result = model.predict(test_data, batch_size=138). The test_data has shape of (69, 28), so it is smaller than the batch_size which is 138. Here is the error, it seems like the issue comes from first dropout layer:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [138,30] vs. [69,30]
[[node model/dropout/dropout/mul_1 (defined at ./mlp_new_tf.py:471) ]] [Op:__inference_distributed_function_1700]
The same solution works with no issues in older version of keras (2.2.4) and tensorflow (1.12.0). How can I fix the issue? I don't have more data for test, so I can't change the test_data set to have more data points!
Since you are seeing the issue at prediction time, one way of getting around this would be to pad the test data to be a multiple of your batch size. It shouldn't slow down the prediction since the number of batches doesn't change. numpy.pad should do the trick.
I am currently learning Deep learning and Keras. When I am executing this code I am getting weird error: "TypeError: Unable to build Dense layer with non-floating point dtype " and I can't figure out what is the problem. What am I missing? How to fix this weird error?
The error show at the model.fit(...
def create_nerual_network():
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu)) # Simple Dense Layer
model.add(tf.keras.layers.Dense(128, activation=tf.nn.relu)) # Simple Dense Layer
model.add(tf.keras.layers.Dense(2, activation=tf.nn.softmax)) # Output layer
return model
train_images, train_labels = load_dataset() #this function works fine
model = create_nerual_network()
model.compile(optimizer='adam',loss='sparse_categorical_crossentropy',metrics=['accuracy'])
model.fit(train_images, train_labels, epochs = 15, verbose=2)
train_loss, train_acc = model.evaluate(train_images, train_labels)
It is interesting that you do not specify your input shape anywhere before the model compilation but maybe newer versions of Keras can figure this out from provided input.
In which case I am quite certain that the problem is with train_images, look at what dtype is this array, it's probably int8 which is usual format for images as they use 8 bit integers for each color channel.
It is common practice to at least normalize your data before training and always convert it to float.
Try putting this before calling model.fit:
train_images = train_images / 256.
This will normalize your images into range [0, 1) and convert it to float array. It is possible that you have to convert to floats also your labels.
I am trying to create a CNN with keras to process 20x20 patches from a larger image of 600x600.
When I attempt the run the code below I receive an error AttributeError: 'Tensor' object has no attribute '_keras_history'
The below code is only intended to look at the first 20 x 20 patch out of an total of 900, I am trying to get this functioning before attempting to loop through the entire input image.
I don't understand why it is returning the error as each layer is generated with an keras layer and I haven't applied any other operations to the tensor?
I am using tensorflow 1.3 and keras 2.0.6.
nb_filters=16
input_image=Input(shape=(600,600,3))
Input_1R=Reshape((900,20,20,3))(input_image)
conv1=Convolution2D(nb_filters,(5,5),activation='relu',padding='valid')(Input_1R[:,0])
conv4=Convolution2D(1,(6,6),activation='hard_sigmoid',padding='same')(conv1)
dense6=Dense(1)(conv4)
output_dense=dense6
model = Model(inputs=input_image, outputs=output_dense)
The error occurs because the slicing operation Input_1R[:,0] is not performed in a Keras layer.
You can wrap it into a Lambda layer:
sliced = Lambda(lambda x: x[:, 0])(Input_1R)
conv1 = Convolution2D(nb_filters, (5,5), activation='relu', padding='valid')(sliced)
This it the code:
X = tf.placeholder(tf.float32, [batch_size, seq_len_1, 1], name='X')
labels = tf.placeholder(tf.float32, [None, alpha_size], name='labels')
rnn_cell = tf.contrib.rnn.BasicLSTMCell(512)
m_rnn_cell = tf.contrib.rnn.MultiRNNCell([rnn_cell] * 3, state_is_tuple=True)
pre_prediction, state = tf.nn.dynamic_rnn(m_rnn_cell, X, dtype=tf.float32)
This is full error:
ValueError: Trying to share variable rnn/multi_rnn_cell/cell_0/basic_lstm_cell/kernel, but specified shape (1024, 2048) and found shape (513, 2048).
I'm using a GPU version of tensorflow.
I encountered a similar problem when I upgraded to v1.2 (tensorflow-gpu).
Instead of using [rnn_cell]*3, I created 3 rnn_cells (stacked_rnn) by a loop (so that they don't share variables) and fed MultiRNNCell with stacked_rnn and the problem goes away. I'm not sure it is the right way to do it.
stacked_rnn = []
for iiLyr in range(3):
stacked_rnn.append(tf.nn.rnn_cell.LSTMCell(num_units=512, state_is_tuple=True))
MultiLyr_cell = tf.nn.rnn_cell.MultiRNNCell(cells=stacked_rnn, state_is_tuple=True)
An official TensorFlow tutorial recommends this way of multiple LSTM network definition:
def lstm_cell():
return tf.contrib.rnn.BasicLSTMCell(lstm_size)
stacked_lstm = tf.contrib.rnn.MultiRNNCell(
[lstm_cell() for _ in range(number_of_layers)])
You can find it here: https://www.tensorflow.org/tutorials/recurrent
Actually it it almost the same approach that Wasi Ahmad and Maosi Chen suggested above but maybe in a little bit more elegant form.
I guess it's because your RNN cells on each of your 3 layers share the same input and output shape.
On layer 1, the input dimension is 513 = 1(your x dimension) + 512(dimension of the hidden layer) for each timestamp per batch.
On layer 2 and 3, the input dimension is 1024 = 512(output from previous layer) + 512 (output from previous timestamp).
The way you stack up your MultiRNNCell probably implies that 3 cells share the same input and output shape.
I stack up MultiRNNCell by declaring two separate types of cells in order to prevent them from sharing input shape
rnn_cell1 = tf.contrib.rnn.BasicLSTMCell(512)
run_cell2 = tf.contrib.rnn.BasicLSTMCell(512)
stack_rnn = [rnn_cell1]
for i in range(1, 3):
stack_rnn.append(rnn_cell2)
m_rnn_cell = tf.contrib.rnn.MultiRNNCell(stack_rnn, state_is_tuple = True)
Then I am able to train my data without this bug.
I'm not sure whether my guess is correct, but it works for me. Hope it works for you.
I encountered the same issue using Google Colab's Jupiter notebook. I resolved the issue by restarting the kernel and then rerunning the code.
I recently have started using Keras to build neural networks. I built a simple CNN to classify MNIST dataset. Before learning the model I used K.set_image_dim_ordering('th') in order to plot a convolutional layer weights. Right now I am trying to visualize convolutional layer output with K.function method, but I keep getting error.
Here is what I want to do for now:
input_image = X_train[2:3,:,:,:]
output_layer = model.layers[1].output
input_layer = model.layers[0].input
output_fn = K.function(input_layer, output_layer)
output_image = output_fn.predict(input_image)
print(output_image.shape)
output_image = np.rollaxis(np.rollaxis(output_image, 3, 1), 3, 1)
print(output_image.shape)
fig = plt.figure()
for i in range(32):
ax = fig.add_subplot(4,8,i+1)
im = ax.imshow(output_image[0,:,:,i], cmap="Greys")
plt.xticks(np.array([]))
plt.yticks(np.array([]))
fig.subplots_adjust(right=0.8)
cbar_ax = fig.add_axes([1, 0.1, 0.05 ,0.8])
fig.colorbar(im, cax = cbar_ax)
plt.tight_layout()
plt.show()
And this is what I get:
File "/home/kinshiryuu/anaconda3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 1621, in function
return Function(inputs, outputs, updates=updates)
File "/home/kinshiryuu/anaconda3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 1569, in __init__
raise TypeError('`inputs` to a TensorFlow backend function '
TypeError: `inputs` to a TensorFlow backend function should be a list or tuple.
You should do the following changes:
output_fn = K.function([input_layer], [output_layer])
output_image = output_fn([input_image])
K.function takes the input and output tensors as list so that you can create a function from many input to many output. In your case one input to one output.. but you need to pass them as a list none the less.
Next K.function returns a tensor function and not a model object where you can use predict(). The correct way of using is just to call as a function
I think you can also use K.function to get gradients.
self.action_gradients = K.gradients(Q_values, actions)
self.get_action_gradients=K.function[*self.model.input, K.learning_phase()], outputs=action_gradients)
which basically runs the graph to obtain the Q-value to calculate the gradient of the Q-value w.r.t. action vector in DDPG. Source code here (lines 64 to 70): https://github.com/nyck33/autonomous_quadcopter/blob/master/criticSolution.py#L65
In light of the accepted answer and this usage here (originally from project 5 autonomous quadcopter in the Udacity Deep Learning nanodegree), a question remains in my mind, ie. is K.function() something that can be used fairly flexibly to run the graph and to designate as outputs of K.function() for example outputs of a particular layer, gradients or even weights themselves?
Lines 64 to 67 here: https://github.com/nyck33/autonomous_quadcopter/blob/master/actorSolution.py
It is being used as a custom training function for the actor network in DDPG:
#caller
self.actor_local.train_fn([states, action_gradients, 1])
#called
self.train_fn = K.function(inputs=[self.model.input, action_gradients, K.learning_phase()], \
outputs=[], updates=updates_op)
outputs is given a value of an empty list because we merely want to train the actor network with the action_gradients from the critic network.