I have a problem with linear regression and 3d matrices.
They are all floating point numbers, with labels.
I got started from this code but I changed the matrix:
https://aqibsaeed.github.io/2016-07-07-TensorflowLR/
With 2 dimensions, it is working well but, with 3, I can not get it running.
this is the shape
(387, 7, 10) shape train_x
(387, 1) shape train_x
(43, 7, 10) test_x.shape
(43, 1) test_y.shape
n_dim = f.shape[1]
train_x, test_x, train_y, test_y = train_test_split(f,l,test_size=0.1, shuffle =False)
print(train_x.shape)
print(train_y.shape)
print(test_x.shape)
print(test_y.shape)
learning_rate = 0.01
training_epochs = 1000
cost_history = np.empty(shape=[1],dtype=float)
X = tf.placeholder(tf.float32,[None,n_dim])
Y = tf.placeholder(tf.float32,[None,1])
W = tf.Variable(tf.ones([n_dim,1]))
#init = tf.initialize_all_variables()
init = tf.global_variables_initializer()
y_ = tf.matmul(X, W)
cost = tf.reduce_mean(tf.square(y_ - Y))
training_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
sess = tf.Session()
sess.run(init)
for epoch in range(training_epochs):
sess.run(training_step,feed_dict={X:train_x,Y:train_y})
cost_history = np.append(cost_history,sess.run(cost,feed_dict={X: train_x,Y: train_y}))
plt.plot(range(len(cost_history)),cost_history)
plt.axis([0,training_epochs,0,np.max(cost_history)])
plt.show()
pred_y = sess.run(y_, feed_dict={X: test_x})
mse = tf.reduce_mean(tf.square(pred_y - test_y))
print("MSE: %.4f" % sess.run(mse))
fig, ax = plt.subplots()
ax.scatter(test_y, pred_y)
ax.plot([test_y.min(), test_y.max()], [test_y.min(), test_y.max()], 'k--', lw=3)
ax.set_xlabel('Measured')
ax.set_ylabel('Predicted')
plt.show()
</ blink>
this is the mistake
\session.py", line 1100, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (387, 7, 10) for Tensor 'Placeholder_12:0', which has shape '(?, 7)'
Your error message shows exact reason why it is raised.
The dimension between placeholder and train_x doesn't fit.
train_x has a (387, 7, 10) shape. In usual convention, you have 387 datapoint which has (7, 10) dimension.
But, X (placeholder, the bucket you will put train_x in) has a [None, n_dim] (I guess n_dim is 7) shape.
Using [None, ~] in the first element is only accepted as the number of datapoints, not dimension of your data.
So you need to change [None, n_dim] to [None, 7, 10] in this case.
edited)
In this case, X is not exacty 3D data. just a bunch of 2D data. For direct weight multiplication of 2D data, you need convolution step. That is CNN. But you only have very small dimension data matrix, you just need to reshape (7,10) matrix shape data to (7*10) vector shape data.
Using tf.reshape function.tf.reshape(X, shape=[387, 7*10]) will be works, and also change your W to right dimension to multiply. like, tf.Variable(tf.ones([7*10,1])).
Related
I built a neural network with tensorflow, here the code :
class DQNetwork:
def __init__(self, state_size, action_size, learning_rate, name='DQNetwork'):
self.state_size = state_size
self.action_size = action_size
self.learning_rate = learning_rate
with tf.variable_scope(name):
# We create the placeholders
self.inputs_ = tf.placeholder(tf.float32, shape=[state_size[1], state_size[0]], name="inputs")
self.actions_ = tf.placeholder(tf.float32, [None, self.action_size], name="actions_")
# Remember that target_Q is the R(s,a) + ymax Qhat(s', a')
self.target_Q = tf.placeholder(tf.float32, [None], name="target")
self.fc = tf.layers.dense(inputs = self.inputs_,
units = 50,
kernel_initializer=tf.contrib.layers.xavier_initializer(),
activation = tf.nn.elu)
self.output = tf.layers.dense(inputs = self.fc,
units = self.action_size,
kernel_initializer=tf.contrib.layers.xavier_initializer(),
activation=None)
# Q is our predicted Q value.
self.Q = tf.reduce_sum(tf.multiply(self.output, self.actions_))
# The loss is the difference between our predicted Q_values and the Q_target
# Sum(Qtarget - Q)^2
self.loss = tf.reduce_mean(tf.square(self.target_Q - self.Q))
self.optimizer = tf.train.AdamOptimizer(self.learning_rate).minimize(self.loss)
But i have an issue with the output,
the output should normaly be at the same size than "action_size", and action_size value is 3
but i got an output like [[5][3]] instead of just [[3]] and i realy don't understand why...
This network got 2 dense layers, one with 50 perceptrons and the other with 3 perceptrons (= action_size).
state_size is format : [[9][5]]
If someone know why my output is two dimensions i will be very thankful
Your self.inputs_ placeholder has shape (5, 9). You perform the matmul(self.inputs_, fc1.w) operation in dense layer fc1 which has shape (9, 50) and it results in shape (5, 50). You then apply another dense layer with shape (50, 3) which results in output shape (5, 3).
The same schematically:
matmul(shape(5, 9), shape(9, 50)) ---> shape(5, 50) # output of 1st dense layer
matmul(shape(5, 50), shape(50, 3)) ---> shape(5, 3) # output of 2nd dense layer
Usually, the first dimension of the input placeholder represents batch size and the second dimension is the dimension of inputs feature vector. So for each sample in a batch you (batch size is 5 in your case) you get the output shape 3.
To get probabilities, use this:
import tensorflow as tf
import numpy as np
inputs_ = tf.placeholder(tf.float32, shape=(None, 9))
actions_ = tf.placeholder(tf.float32, shape=(None, 3))
fc = tf.layers.dense(inputs=inputs_, units=2)
output = tf.layers.dense(inputs=fc, units=3)
reduced = tf.reduce_mean(output, axis=0)
probs = tf.nn.softmax(reduced) # <--probabilities
inputs_vals = np.ones((5, 9))
actions_vals = np.ones((1, 3))
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print(probs.eval({inputs_:inputs_vals,
actions_:actions_vals}))
# [0.01858923 0.01566187 0.9657489 ]
I'm using tensorboard to visualize my train images(cifar10 dataset). But the TensorBoard shows me some very strange images. Below is the screenshot.
the strange images
Here is some relative code. Note that DISPLAY_STEP is 10, BATCH_SIZE is 64.
x = tf.placeholder(tf.float32, shape=[None, N_FEATURES], name='x')
x_image = tf.reshape(x, [-1, 32, 32, 3])
tf.summary.image('input', x_image, max_outputs=BATCH_SIZE)
y = tf.placeholder(tf.float32, [None, N_CLASSES], name='labels')
'''There is other code.'''
with tf.Session() as sess:
sess.run(init)
summary_writer = tf.summary.FileWriter('./cifar10_model/6', graph=tf.get_default_graph())
for i in range(TRAINING_EPOCHS):
batch_x, batch_y = cifar10.train.next_batch(BATCH_SIZE)
if i % DISPLAY_STEP == 0:
s = sess.run(merged_summary, feed_dict={x: batch_x, y: batch_y})
summary_writer.add_summary(s, i)
sess.run(train_step, feed_dict={x: batch_x, y: batch_y})
Could anyone tell me what's going on? Thanks in advance.
It looks like the cifar images are not reshaped properly. According to the dataset website:
data -- a 10000x3072 numpy array of uint8s. Each row of the array stores a 32x32 colour image. The first 1024 entries contain the red channel values, the next 1024 the green, and the final 1024 the blue. The image is stored in row-major order, so that the first 32 entries of the array are the red channel values of the first row of the image.
You should make sure this 3072 long array is reshaped properly.
I am training PTB dataset for predicting characters (i.e. character-level LSTM).
The dimension for training batches is [len(dataset) x vocabulary_size]. Here, vocabulary_size = 27 (26+1[for unk tokens and spaces or fullstops.]).
This is the code for converting to one_hot for both batches input(arrX) and labels(arrY).
arrX = np.zeros((len(train_data), vocabulary_size), dtype=np.float32)
arrY = np.zeros((len(train_data)-1, vocabulary_size), dtype=np.float32)
for i, x in enumerate(train_data):
arrX[i, x] = 1
arrY = arrX[1, :]
I am making a placeholder of input(X) and labels(Y) in Graph to pass it to tflearn LSTM.Following is the code for the graph and session.
batch_size = 256
with tf.Graph().as_default():
X = tf.placeholder(shape=(None, vocabulary_size), dtype=tf.float32)
Y = tf.placeholder(shape=(None, vocabulary_size), dtype=tf.float32)
print (utils.get_incoming_shape(tf.concat(0, Y)))
print (utils.get_incoming_shape(X))
net = tflearn.lstm(X, 512, return_seq=True)
print (utils.get_incoming_shape(net))
net = tflearn.dropout(net, 0.5)
print (utils.get_incoming_shape(net))
net = tflearn.lstm(net, 256)
net = tflearn.fully_connected(net, vocabulary_size, activation='softmax')
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(net, Y))
optimizer = tf.train.AdamOptimizer(learning_rate=0.01).minimize(loss)
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
offset=0
avg_cost = 0
total_batch = (train_length-1) / 256
print ("No. of batches:", '%d' %total_batch)
for i in range(total_batch) :
batch_xs, batch_ys = trainX[offset : batch_size + offset], trainY[offset : batch_size + offset]
sess.run(optimizer, feed_dict={X: batch_xs, Y: batch_ys})
cost = sess.run(loss, feed_dict={X: batch_xs, Y: batch_ys})
avg_cost += cost/total_batch
if i % 20 == 0:
print("Step:", '%03d' % i, "Loss:", str(cost))
offset += batch_size
SO, I get the following error assert ndim >= 3, "Input dim should be at least 3."
AssertionError: Input dim should be at least 3.
How can I resolve this error? Is there any alternate solution?
Should I write separate LSTM definition?
I'm not used to these kind of datasets but have you tried using the tflearn.input_data(shape) with the tflearn.embedding layer ? If you use the embedding I suppose that you won't have to reshape your data in 3 dimension.
lstm layer takes input of shape 3-D Tensor [samples, timesteps, input dim]. You can reshape your input data to 3D. In your problem shape of trainX is [len(dataset) x vocabulary_size]. Using trainX = trainX.reshape( trainX.shape+ (1,)) shape will be changed to [len(dataset), vocabulary_size, 1]. This data can be pass to lstm by simple change in input placeholder X. Add one more dimention to placeholder by X = tf.placeholder(shape=(None, vocabulary_size, 1), dtype=tf.float32).
I am building a 1d convolutional neural net for my own data (spectra) and am having an issue with tf.reshape. First I load in the data with pandas, and convert these to numpy arrays, composed of 708 training example spectra, each of length 2151,
import pandas as pd
import numpy as np
data = pd.read_csv('test.csv',header=None)
yTrue = data.ix[:,0].as_matrix()
data = data - data.mean()
data = data.ix[:,1:].as_matrix()
where I subtract the mean value in each column. So data is of dimensions 708 x 2151 here. I then create a network that starts with,
sess = tf.InteractiveSession()
## define inputs
x_ = tf.placeholder(tf.float32, shape=[None, 2151])
x_ = tf.reshape(x_, [-1,1,2151,1])
y_ = tf.placeholder(tf.float32, shape=[None])
which are inputs for my 1d convolutional neural net (with kernels with a width of 10, and 32 feature maps),
W_conv1 = weight_variable([1, 10, 1, 32])
b_conv1 = bias_variable([32])
h_conv1 = tf.nn.relu(conv2d(x_, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)
I then build the rest of the network and then try to run ADAM on it,
cost_function = tf.reduce_mean(tf.pow(y_out - y_, 2))/(2 * samples_number) #L2 loss
train_step = tf.train.AdamOptimizer(1e-4).minimize(cost_function)
correct_prediction = tf.equal(tf.argmax(y_out,1), tf.argmax(y_,1))
sess.run(tf.initialize_all_variables())
for i in range(20000):
print(i)
sess.run(train_step, feed_dict={x_: data, y_: yTrue})
However I get the following error:
ValueError: Cannot feed value of shape (708, 2151) for Tensor u'Reshape_26:0',
which has shape '(?, 1, 2151, 1)'
I have looked at these answers: TensorFlow/TFLearn: ValueError: Cannot feed value of shape (64,) for Tensor u'target/Y:0', which has shape '(?, 10)'; Tensorflow error using my own data which suggest that I need to be doing some reshaping before I pass my data to the network. However, I am not sure what this should be? Particularly since the following works on the first row of the data,
t = tf.constant(data[0])
tf.reshape(t,[1,1,2151,1])
Does anyone have any ideas here?
Best,
Ben
The issue is that feed_dict can replace any Tensor, and since you've changed x_ to reference the reshape op, that's the thing that it's trying to replace. It should work if you just use different Python variables to reference the placeholder and the reshape op:
x_placeholder_ = tf.placeholder(tf.float32, shape=[None, 2151])
x_ = tf.reshape(x_placeholder_, [-1,1,2151,1])
Then when feeding, use x_placeholder_:
sess.run(train_step, feed_dict={x_placeholder_: data, y_: yTrue})
I know basic use for tf.placeholder:
x = tf.placeholder(tf.float32, shape=(1024, 1024))
y = tf.matmul(x, x)
with tf.Session() as sess:
print(sess.run(y)) # ERROR: will fail because x was not fed.
rand_array = np.random.rand(1024, 1024)
print(sess.run(y, feed_dict={x: rand_array})) # Will succeed.
I know the second parameter is about shape. However I don't know what is that mean when the first one is None in the shape. ex:[None,784].
From the tutorial: Deep MNIST for Experts
Here we assign it a shape of [None, 784], where 784 is the dimensionality of a single flattened 28 by 28 pixel MNIST image, and None indicates that the first dimension, corresponding to the batch size, can be of any size.