I know basic use for tf.placeholder:
x = tf.placeholder(tf.float32, shape=(1024, 1024))
y = tf.matmul(x, x)
with tf.Session() as sess:
print(sess.run(y)) # ERROR: will fail because x was not fed.
rand_array = np.random.rand(1024, 1024)
print(sess.run(y, feed_dict={x: rand_array})) # Will succeed.
I know the second parameter is about shape. However I don't know what is that mean when the first one is None in the shape. ex:[None,784].
From the tutorial: Deep MNIST for Experts
Here we assign it a shape of [None, 784], where 784 is the dimensionality of a single flattened 28 by 28 pixel MNIST image, and None indicates that the first dimension, corresponding to the batch size, can be of any size.
Related
I am trying to build multilayer, multiclass, multilabel LSTM in Tensorflow. I have been trying to bend this tutorial to my data.
However, I am getting an error that says I have dimension mismatch when building RNN.
ValueError: Dimensions must be equal, but are 1000 and 923 for 'rnn/while/rnn/multi_rnn_cell/cell_0/lstm_cell/MatMul_1' (op: 'MatMul') with input shapes: [?,1000], [923,2000].
I cannot pinpoint which variable is incorrect in building architecture:
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
def bias_variable(shape):
initial = tf.constant(0.0, shape=shape)
return tf.Variable(initial)
def lstm(x, weight, bias, n_steps, n_classes):
cell = rnn_cell.LSTMCell(cfg.n_hidden_cells_in_layer, state_is_tuple=True)
multi_layer_cell = tf.nn.rnn_cell.MultiRNNCell([cell] * 2)
# FIXME : ERROR binding x to LSTM as it is
output, state = tf.nn.dynamic_rnn(multi_layer_cell, x, dtype=tf.float32)
# FIXME : ERROR
output_flattened = tf.reshape(output, [-1, cfg.n_hidden_cells_in_layer])
output_logits = tf.add(tf.matmul(output_flattened, weight), bias)
output_all = tf.nn.sigmoid(output_logits)
output_reshaped = tf.reshape(output_all, [-1, n_steps, n_classes])
# ??? switch batch size with sequence size. ???
# then gather last time step values
output_last = tf.gather(tf.transpose(output_reshaped, [1, 0, 2]), n_steps - 1)
return output_last, output_all
These are my placeholders, loss function and all that jazz:
x_test, y_test = load_multiple_vector_files(test_filepaths)
x_valid, y_valid = load_multiple_vector_files(valid_filepaths)
n_input, n_steps, n_classes = get_input_target_lengths(check_print=False)
# FIXME n_input should be the problem
x = tf.placeholder("float", [None, n_steps, n_input])
y = tf.placeholder("float", [None, n_classes])
y_steps = tf.placeholder("float", [None, n_classes])
weight = weight_variable([cfg.n_hidden_layers, n_classes])
bias = bias_variable([n_classes])
y_last, y_all = lstm(x, weight, bias, n_steps, n_classes)
#all_steps_cost=tf.reduce_mean(-tf.reduce_mean((y_steps * tf.log(y_all))+(1 - y_steps) * tf.log(1 - y_all),reduction_indices=1))
all_steps_cost = -tf.reduce_mean((y_steps * tf.log(y_all)) + (1 - y_steps) * tf.log(1 - y_all))
last_step_cost = -tf.reduce_mean((y * tf.log(y_last)) + ((1 - y) * tf.log(1 - y_last)))
loss_function = (cfg.alpha * all_steps_cost) + ((1 - cfg.alpha) * last_step_cost)
optimizer = tf.train.AdamOptimizer(learning_rate=cfg.learning_rate).minimize(loss_function)
I am pretty sure it is my X placeholder that is causing the problem, resulting in layers and their matrices dimensions not matching. The constant which the linked example is using is rather tough to see what it actually stands for.
Can anyone help me out here? :)
UPDATE:
I have made an "educated guess" on the mismatching dimensions.
One is 2*hidden_width, so hidden getting new input + its old recurrent input. The mismatching dimension, however, is input_width + hidden_width, like it was trying to set recurrency for width of hidden layer to the input layer.
I figured out I am incorrectly setting the weight variable, using the constant for n_hidden_layers(number of hidden layers) instead of n_hidden_cells_in_layer(number of layers).
I have lately been vexed by the following error message:
ValueError: Cannot feed value of shape (2455040,) for Tensor 'Placeholder:0', which has shape '(2455040, ?)'
Which is being produced from running the following code:
NUMCLASSES=16
NUMPIXELS=959*640*4
# set up to feed an array of images [images, size_of_image]
x = tf.placeholder(tf.float32, [NUMPIXELS,None])
....deletia....
# Define loss and optimizer..why is this 2d?
y_ = tf.placeholder(tf.float32, [None,NUMCLASSES])
sess = tf.InteractiveSession()
tf.global_variables_initializer().run(session=sess)
tl = get_tensor_list()
for f, n in tl:
str = '/users/me/downloads/train/' + f
mm = Image.open(str)
mm = mm.convert('F')
mma=np.array(mm)
i = mma.flatten() #now this is an array of floats of size NUMPIXELS
sess.run(train_step, feed_dict={x: i, y_: n}) # <<DEATH
Somehow, that array is getting a shape that tf does not like [(x,) when it wants (x,?)]. How to satisfy the tensorgods in this case? The tensor must be what it must be for other mathematical reasons not discussed.
reshaping the array might help.
i = mma.flatten().reshape((NUMPIXELS,1))
The error happens because the two tensors have different ranks: tensor with shape (2455040,) has rank 1, while tensor with shape (2455040,?) has rank 2.
You can do this:
x = tf.placeholder(tf.float32, [None])
x = tf.reshape(x, [NUMPIXELS,-1])
I have a problem with linear regression and 3d matrices.
They are all floating point numbers, with labels.
I got started from this code but I changed the matrix:
https://aqibsaeed.github.io/2016-07-07-TensorflowLR/
With 2 dimensions, it is working well but, with 3, I can not get it running.
this is the shape
(387, 7, 10) shape train_x
(387, 1) shape train_x
(43, 7, 10) test_x.shape
(43, 1) test_y.shape
n_dim = f.shape[1]
train_x, test_x, train_y, test_y = train_test_split(f,l,test_size=0.1, shuffle =False)
print(train_x.shape)
print(train_y.shape)
print(test_x.shape)
print(test_y.shape)
learning_rate = 0.01
training_epochs = 1000
cost_history = np.empty(shape=[1],dtype=float)
X = tf.placeholder(tf.float32,[None,n_dim])
Y = tf.placeholder(tf.float32,[None,1])
W = tf.Variable(tf.ones([n_dim,1]))
#init = tf.initialize_all_variables()
init = tf.global_variables_initializer()
y_ = tf.matmul(X, W)
cost = tf.reduce_mean(tf.square(y_ - Y))
training_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
sess = tf.Session()
sess.run(init)
for epoch in range(training_epochs):
sess.run(training_step,feed_dict={X:train_x,Y:train_y})
cost_history = np.append(cost_history,sess.run(cost,feed_dict={X: train_x,Y: train_y}))
plt.plot(range(len(cost_history)),cost_history)
plt.axis([0,training_epochs,0,np.max(cost_history)])
plt.show()
pred_y = sess.run(y_, feed_dict={X: test_x})
mse = tf.reduce_mean(tf.square(pred_y - test_y))
print("MSE: %.4f" % sess.run(mse))
fig, ax = plt.subplots()
ax.scatter(test_y, pred_y)
ax.plot([test_y.min(), test_y.max()], [test_y.min(), test_y.max()], 'k--', lw=3)
ax.set_xlabel('Measured')
ax.set_ylabel('Predicted')
plt.show()
</ blink>
this is the mistake
\session.py", line 1100, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (387, 7, 10) for Tensor 'Placeholder_12:0', which has shape '(?, 7)'
Your error message shows exact reason why it is raised.
The dimension between placeholder and train_x doesn't fit.
train_x has a (387, 7, 10) shape. In usual convention, you have 387 datapoint which has (7, 10) dimension.
But, X (placeholder, the bucket you will put train_x in) has a [None, n_dim] (I guess n_dim is 7) shape.
Using [None, ~] in the first element is only accepted as the number of datapoints, not dimension of your data.
So you need to change [None, n_dim] to [None, 7, 10] in this case.
edited)
In this case, X is not exacty 3D data. just a bunch of 2D data. For direct weight multiplication of 2D data, you need convolution step. That is CNN. But you only have very small dimension data matrix, you just need to reshape (7,10) matrix shape data to (7*10) vector shape data.
Using tf.reshape function.tf.reshape(X, shape=[387, 7*10]) will be works, and also change your W to right dimension to multiply. like, tf.Variable(tf.ones([7*10,1])).
I'm using tensorboard to visualize my train images(cifar10 dataset). But the TensorBoard shows me some very strange images. Below is the screenshot.
the strange images
Here is some relative code. Note that DISPLAY_STEP is 10, BATCH_SIZE is 64.
x = tf.placeholder(tf.float32, shape=[None, N_FEATURES], name='x')
x_image = tf.reshape(x, [-1, 32, 32, 3])
tf.summary.image('input', x_image, max_outputs=BATCH_SIZE)
y = tf.placeholder(tf.float32, [None, N_CLASSES], name='labels')
'''There is other code.'''
with tf.Session() as sess:
sess.run(init)
summary_writer = tf.summary.FileWriter('./cifar10_model/6', graph=tf.get_default_graph())
for i in range(TRAINING_EPOCHS):
batch_x, batch_y = cifar10.train.next_batch(BATCH_SIZE)
if i % DISPLAY_STEP == 0:
s = sess.run(merged_summary, feed_dict={x: batch_x, y: batch_y})
summary_writer.add_summary(s, i)
sess.run(train_step, feed_dict={x: batch_x, y: batch_y})
Could anyone tell me what's going on? Thanks in advance.
It looks like the cifar images are not reshaped properly. According to the dataset website:
data -- a 10000x3072 numpy array of uint8s. Each row of the array stores a 32x32 colour image. The first 1024 entries contain the red channel values, the next 1024 the green, and the final 1024 the blue. The image is stored in row-major order, so that the first 32 entries of the array are the red channel values of the first row of the image.
You should make sure this 3072 long array is reshaped properly.
I am trying to use the softmax regression method discussed in https://www.tensorflow.org/get_started/mnist/beginners to recognize characters.
My code is as follows.
train_data = pd.read_csv('CharDataSet/train.csv')
print(train_data.shape)
x = tf.placeholder(tf.float32, [None, 130])
W = tf.Variable(tf.zeros([130, 26]))
b = tf.Variable(tf.zeros([26]))
y = tf.nn.softmax(tf.matmul(x, W) + b)
y_ = tf.placeholder(tf.float32, [None, 26])
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)
sess = tf.InteractiveSession()
tf.global_variables_initializer().run()
for _ in range(10):
batch_xs = train_data.iloc[:, 2:]
print(batch_xs)
batch_ys = getencodedbatch(train_data.iloc[:, 1])
print(batch_ys)
sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})
However, I am getting an accuracy of 1, which shouldn't be the case.
The reason why I am getting it so is because my y tensor results with an array like
[nan, ..., nan]
Can anyone explain to me what is wrong in my code?
I converted each character to a one-hot encoding using the method below
def getencodedbatch(param):
s = (param.shape[0],26)
y_encode = np.zeros(s)
row=0
# print(y_encode)
for val in param:
col = ord(val)-97
y_encode[row, col] = 1
row += 1
return pd.DataFrame(y_encode)
Here is the problem you are having:
You set your initial weights and biases to 0 (this is wrong, as your
network does not learn).
The result is that y consists of all zeros
You take the log of y.. and a log of 0 is not defined... Hence the NaN.
Good luck!
Edit to tell you how to fix it: look for an example on classifying MNIST characters and see what they do. You probably want to initialise your weights to be random normals ;)