I have a tensor tf.shape(X) == [M, N, N] and a set of indices tf.shape(IDX) == [N, N]. How can I form a tensor tf.shape(Y) = [N, N], which equals to the slice of X using indices IDX in the first dimension? I.e.
Y[i, j] = X[IDX[i, j], i, j] for all i,j = 1..N.
I have tried to play with tf.gather_nd but with no result :(
Update 10-12-2016:
As of tensorflow version 0.11 and up one can index into tensors in the same way as numpy.
a = tf.Variable([9,10,11])
b = tf.constant([[1,2,3,4],[5,6,7,8]])
a = b[0,1:]
Gradients are also supported on the indexing.
What did you try already?
It seems like there's a bug with tf.gather_nd that I reported.
Here's the response
Support for partial indices in gather_nd (fewer indices than dimensions) was added quite recently. You are you using a version of TensorFlow where each index tensor must have exactly the number of tensor dimensions. The code should work at HEAD.
so by version 0.10 or above gather_nd should work like you want.
However this below works
import tensorflow as tf
x = tf.constant([[1,1,1,1],[1,2,3,4]],shape=(2,4))
indices = [[0,0],[0,1]]
y = tf.gather_nd(x,indices)
so it seems like you need the full index description at the moment, not just slice 0. You also try tf.pack.
You can also track the progress of indexing tensors in tensorflow here:
https://github.com/tensorflow/tensorflow/issues/206
Related
I'm new to tensorflow, keras and a bit confused about how tf treats its input and matrix multiplication and all that jazz.
You see, in Linear Algebra (LA) you can treat contravariant vectors as columns matices(math standard)
or as rows matrices
Somewhere I've heard that:
a tensor of shape (n,) e.g. [1,2,3,4,5] is not considered as a "vector" according to tf. Only tensors of shape (n,1) and (1,n) are considered vectors. But in many manuals people use those words without any system. Createing complete confusion in my head.
a tensor of shape (n,1) is considered as a column vector (according to tf)
but sending this column-vector (n,1) to some layer.call() as an input you can see that it is treated as a row-vector, because it's being multiplicated on the right by a self.w, but for column-oriented LA it must have been multiplicated by the self.w the left.
def call(self, inputs):
return tf.matmul(inputs, self.w) + self.b
So the questions are these:
What does it mean being x-oriented according to tf?
Is tensor flow column-vector or row-vector oriented?
What's up with non-vectors (n,1), and why those are not "vectors" according to TF?
What is expected as an input to a layer column or row vectors?
left-right matrix multiplication x*W+b in TF source code. Why it's x on the right and M on the left and not vice-versa? Why if layer expects a column-vector as an input its being multiplied by W on the right?
I see that I'm confused and can't clearly state the question. Please, be patient. Thanks.
It is because the elements of a rank 1 tensor are treated as a scalar.
For example
Example 1.
x = tf.constant(5)
y = tf.constant([1,2,3])
x*y will produce [5,10,15]
Example 2.
x = tf.constant([1,2,3])
y = tf.constant([[1],[2],[3]])
x*y will produce [[1 2 3] [2 4 6] [3 6 9]].
It also uses broadcast, whenever the shapes are incompatible. https://numpy.org/doc/stable/user/basics.broadcasting.html
I have a Tensor tensor of shape (?, 1082) and I want to slice this Tensor into n subparts in a for-loop but I want to keep the original shape, including the unknown dimension ?.
Example:
lst = []
for n in range(15):
sub_tensor = tensor[n] # this will reduce the first dimension
print(sub_tensor.get_shape())
Print output I'm looking for:
(?, 1082)
(?, 1082)
etc.
How can this be achieved in TensorFlow?
Considering that your problem can have many constraints, I can think of at least 3 solutions.
You can use tf.split. I'll use tf.placeholder, but it's applicable to tensors and variables as well.
p = tf.placeholder(shape=[None,10], dtype=tf.int32)
s1, s2 = tf.split(value=p, num_or_size_splits=2, axis=1)
However, this approach can become unfeasible if number of splits required is large. Note that it can split None axis as well.
for n in range(15):
sub_tensor = tensor[n, :]
s = tf.slice(p, [0,2], [-1, 2])
Slice can be used for multidimensional tensors, but it' pretty tricky to use. And you can use tf.Tensor.getitem method, almost as you described in your question. It acts similar to NumPy. So this should do the job:
for n in range(10):
print(p[n, :])
However, usage of these methods heavily depend on your particular application. Hope this helps.
I need to compute the following tensor with three-dimensional tensors x,y
tf.einsum("ijk,ljk->ilj",x,y)
Unfortunately, this is pretty slow. Is there any way to rewrite this using only matmul operations that I didn't think of?
You can (implicitely) broadcast x to the shape iljk and y to the shape ilkj.
Then it is possible to use tf.matmul() to get shape iljj and tf.sum() for eliminating one j.
The resulting shape then is ilj.
x = tf.expand_dims(x, axis=1)
y = tf.transpose(y, [0,2,1])
y = tf.expand_dims(y, axis=0)
tf.sum(tf.matmul(x, y), axis=-2)
However, I dont think that this would be faster since you get a 4D tensor here.
I need to convert a list of tensors of dimensionality N to a new tensor with dimensionality N+1 so that the new dimension would be the right most dimension.
For example if x and y would be tensors of shape (4,3) both then I am trying to create a new tensor z of shape (4,3,2) by forming z and setting tensor x as the 0th element along the third dimension and setting tensor y as the 1st element along the third dimension. In pseudocode:
z = tf.fromList([x,y],3)
What's the best way to do that in Tensorflow. I was unable to figure it out from the documentation of TF 0.7.1.
If I'm reading you correctly, you want to interleave the data of the two tensors.
You want to tf.pack() them together, which would form a tensor of shape [2, 4, 3] and then tf.transpose([1, 2, 0]) that resulting tensor to get to the interleaving you want.
dga's method works, but tf.pack() has been removed from TensorFlow V1.0 onwards.
You can use tf.stack() to achieve the same.
Docs: https://www.tensorflow.org/api_docs/python/tf/stack
I'm trying to write my own cost function in tensor flow, however apparently I cannot 'slice' the tensor object?
import tensorflow as tf
import numpy as np
# Establish variables
x = tf.placeholder("float", [None, 3])
W = tf.Variable(tf.zeros([3,6]))
b = tf.Variable(tf.zeros([6]))
# Establish model
y = tf.nn.softmax(tf.matmul(x,W) + b)
# Truth
y_ = tf.placeholder("float", [None,6])
def angle(v1, v2):
return np.arccos(np.sum(v1*v2,axis=1))
def normVec(y):
return np.cross(y[:,[0,2,4]],y[:,[1,3,5]])
angle_distance = -tf.reduce_sum(angle(normVec(y_),normVec(y)))
# This is the example code they give for cross entropy
cross_entropy = -tf.reduce_sum(y_*tf.log(y))
I get the following error:
TypeError: Bad slice index [0, 2, 4] of type <type 'list'>
At present, tensorflow can't gather on axes other than the first - it's requested.
But for what you want to do in this specific situation, you can transpose, then gather 0,2,4, and then transpose back. It won't be crazy fast, but it works:
tf.transpose(tf.gather(tf.transpose(y), [0,2,4]))
This is a useful workaround for some of the limitations in the current implementation of gather.
(But it is also correct that you can't use a numpy slice on a tensorflow node - you can run it and slice the output, and also that you need to initialize those variables before you run. :). You're mixing tf and np in a way that doesn't work.
x = tf.Something(...)
is a tensorflow graph object. Numpy has no idea how to cope with such objects.
foo = tf.run(x)
is back to an object python can handle.
You typically want to keep your loss calculation in pure tensorflow, so do the cross and other functions in tf. You'll probably have to do the arccos the long way, as tf doesn't have a function for it.
just realized that the following failed:
cross_entropy = -tf.reduce_sum(y_*np.log(y))
you cant use numpy functions on tf objects, and the indexing my be different too.
I think you can use "Wraps Python function" method in tensorflow. Here's the link to the documentation.
And as for the people who answered "Why don't you just use tensorflow's built in function to construct it?" - sometimes the cost function people are looking for cannot be expressed in tf's functions or extremely difficult.
This is because you have not initialized your variable and because of this it does not have your Tensor there right now (can read more in my answer here)
Just do something like this:
def normVec(y):
print y
return np.cross(y[:,[0,2,4]],y[:,[1,3,5]])
t1 = normVec(y_)
# and comment everything after it.
To see that you do not have a Tensor now and only Tensor("Placeholder_1:0", shape=TensorShape([Dimension(None), Dimension(6)]), dtype=float32).
Try initializing your variables
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)
and evaluate your variable sess.run(y). P.S. you have not fed your placeholders up till now.