So I was trying to multiply two tensors (deltasTimesWeights, self.weightedInputs) both of shapes (24,24,3,2), but for some reason numpy thinks that the latter of them is of shape (24,24,3,10). Specifically it gives the error ValueError: could not broadcast input array from shape (24,24,2) into shape (24,24,10)
print(self.weightedInputs.shape , deltasTimesWeights.shape)
for index in range(c[2]):
self.errors[:, :, index, :] = self.activationFuncs[index].prime(
self.weightedInputs[:, :, index, :] ) * deltasTimesWeights[:, :, index, :]
Note that self.activationFuncs[index].prime just returns the same tensor shape back. This code is part of a bigger loop in which previous iterations, both tensors were in fact both tensors were (24,24,3,10).
Are there any known reasons why this happens ? Does numpy keep like cache about the shape previously ?
Related
I am creating a tensor of one_hot encodings from an audio file loaded in through librosa. The tensor is massive, and I don't want to print all of it.
In fact this is what it shows me and then never prints when I try to print it: (or maybe it will but I don't want to wait) W tensorflow/core/framework/allocator.cc:124] Allocation of 1387692032 exceeds 10% of system memory.
How would I print only certain values? For example I would like to print every 50th one hot encoding in the tensor.
one_hot = _one_hot(load_audio()) # Tensor
sess = tf.InteractiveSession()
one_hot_prnt = tf.Print(one_hot, [one_hot], "One hot encoding:")
evaluator = tf.add(one_hot_prnt, one_hot_prnt)
evaluator.eval()
Tensors in tensorflow support fancy indexing similar to numpy. You can iterate over some dimension of the tensor.
Consider the following tensor(t) with shape(10000, 10). Now you can iterate over the first dimension one index at a time, and get array with shape (10, )
e.g
t = tf.random.uniform(shape=(10000, 10)
print(t[0, :].eval(session=session)) # This prints first row of the tensor. The result is array with shape (10, )
You can also access value individual (cell) position inside the tensor by specify the coordinate([row, col]) value.
t = tf.random.uniform(shape=(10000, 10)
print(t[0, 0].eval(session=session)) # This prints first element of first row. If the tensor has dimensions more than two, is this value would be a matrix or a tensor.
I'm creating a layer which uses its input tensors to create a SparseTensor, i.e., the input tensors are the respective indices and values of the SparseTensor.
Since:
indices: A 2-D int64 tensor of shape
And because tf.cast(x, tf.int64) is not differentiable, I'm not sure if this is achievable.
Another option is to find a turnaround based on tf.round(), but SparseTensor won't accept a different type of tensor as indices.
Is there a way to cast a tensor to integer and not having None for gradient ?
How can I create a SparseTensor using previous layers outputs, which are float tensors ?
I have a Tensor tensor of shape (?, 1082) and I want to slice this Tensor into n subparts in a for-loop but I want to keep the original shape, including the unknown dimension ?.
Example:
lst = []
for n in range(15):
sub_tensor = tensor[n] # this will reduce the first dimension
print(sub_tensor.get_shape())
Print output I'm looking for:
(?, 1082)
(?, 1082)
etc.
How can this be achieved in TensorFlow?
Considering that your problem can have many constraints, I can think of at least 3 solutions.
You can use tf.split. I'll use tf.placeholder, but it's applicable to tensors and variables as well.
p = tf.placeholder(shape=[None,10], dtype=tf.int32)
s1, s2 = tf.split(value=p, num_or_size_splits=2, axis=1)
However, this approach can become unfeasible if number of splits required is large. Note that it can split None axis as well.
for n in range(15):
sub_tensor = tensor[n, :]
s = tf.slice(p, [0,2], [-1, 2])
Slice can be used for multidimensional tensors, but it' pretty tricky to use. And you can use tf.Tensor.getitem method, almost as you described in your question. It acts similar to NumPy. So this should do the job:
for n in range(10):
print(p[n, :])
However, usage of these methods heavily depend on your particular application. Hope this helps.
I have a tensor tf.shape(X) == [M, N, N] and a set of indices tf.shape(IDX) == [N, N]. How can I form a tensor tf.shape(Y) = [N, N], which equals to the slice of X using indices IDX in the first dimension? I.e.
Y[i, j] = X[IDX[i, j], i, j] for all i,j = 1..N.
I have tried to play with tf.gather_nd but with no result :(
Update 10-12-2016:
As of tensorflow version 0.11 and up one can index into tensors in the same way as numpy.
a = tf.Variable([9,10,11])
b = tf.constant([[1,2,3,4],[5,6,7,8]])
a = b[0,1:]
Gradients are also supported on the indexing.
What did you try already?
It seems like there's a bug with tf.gather_nd that I reported.
Here's the response
Support for partial indices in gather_nd (fewer indices than dimensions) was added quite recently. You are you using a version of TensorFlow where each index tensor must have exactly the number of tensor dimensions. The code should work at HEAD.
so by version 0.10 or above gather_nd should work like you want.
However this below works
import tensorflow as tf
x = tf.constant([[1,1,1,1],[1,2,3,4]],shape=(2,4))
indices = [[0,0],[0,1]]
y = tf.gather_nd(x,indices)
so it seems like you need the full index description at the moment, not just slice 0. You also try tf.pack.
You can also track the progress of indexing tensors in tensorflow here:
https://github.com/tensorflow/tensorflow/issues/206
Say I have a tensor of size BxWxHxD. I want to process the tensor such that I have a new BxWxHxD tensor where only the maximum element in each WxH slice is kept, and all other values are zero.
In other words, I think the best way to achieve this is to somehow take a 2D argmax across the WxH slices, resulting in BxD index tensors for the rows and colums that can then be converted to a one-hot BxWxHxD tensor to be used as a mask. How do I make this work?
You can use the following function as a starting point. It calculates the indexes of the maximum element for each batch and for each channel. The resulting array is in the format (batch size, 2, number of channels).
def argmax_2d(tensor):
# input format: BxHxWxD
assert rank(tensor) == 4
# flatten the Tensor along the height and width axes
flat_tensor = tf.reshape(tensor, (tf.shape(tensor)[0], -1, tf.shape(tensor)[3]))
# argmax of the flat tensor
argmax = tf.cast(tf.argmax(flat_tensor, axis=1), tf.int32)
# convert indexes into 2D coordinates
argmax_x = argmax // tf.shape(tensor)[2]
argmax_y = argmax % tf.shape(tensor)[2]
# stack and return 2D coordinates
return tf.stack((argmax_x, argmax_y), axis=1)
def rank(tensor):
# return the rank of a Tensor
return len(tensor.get_shape())