I have two tensors with shapes
voxels :: (4, 64, 64, 64)
indices :: (4096, 3)
Here voxels has dtype float32, while indices are integers between 0 and 63.
I would like to index the voxels, creating a tensor tensor output:
output :: 4, 4096
Such that the following constraint holds
output[i,j] = voxels[i, indices[j,0], indices[j,1]. indices[j,2]]
How can I do this in tensorflow?
There is some talk of more advanced slicing here, but it's not clear that the suggestions there apply, or that the gradients have been implemented.
For reference, if the above tensors were numpy arrays, the following code does what I want:
output = voxels[:, indices[:,0],indices[:,1],indices[:,2]]
gather_nd is indeed a possibility, but gradients are not yet implemented. However, it is also possible to use a trick involving gather, by flattening the tensor and re-computing appropriate indices.
I believe gather_nd does what you want, though I don't think the gradients have been implemented.
Related
I'm struggling to work with different tensor types and operations between them. For example, a basic division tf.divide(a, b) is giving me the following error:
TypeError: Failed to convert elements of SparseTensor(indices=Tensor("inputs_8_copy:0", shape=(None, 2), dtype=int64), values=Tensor("cond/Cast_1:0", shape=(None,), dtype=float64), dense_shape=Tensor("inputs_10_copy:0", shape=(2,), dtype=int64)) to Tensor. Consider casting elements to a supported type. See https://www.tensorflow.org/api_docs/python/tf/dtypes for supported TF dtypes.
I was able to work around this by calling tf.sparse.to_dense on a and b. But the approach doesn't scale when the dataset is large. Nor does it work in general because I don't know the tensor type of all of the features (I'm working within a preprocessing_fn in TFT and the data comes from BigQuery).
This seems like a very common issue that should have a simple answer but I'm not able to find any information on it. Something like basic divisions shouldn't cause this much trouble?
It is a difficult question, in fact.
For element-wise division in particular, let say ai and bi are scalars. if ai = 0 and bi is not zero, then ai/bi = 0, but what if ai = 0 and bi = 0, ai/bi = ? 0?
Even worse, if ai is not zero and bi = 0 then ai/bi is NaN!
So if the divisor is a sparse tensor, it will raise (possibly lots of) NaNs, unless the indices of both sparse matrices are the same. Same problem if you divide a dense matrix by a sparse matrix.
There is a nice a workaround to multiply two sparse tensors element-wise here, based on the relation (a+b)^2 = a^2 + b^2 + 2 ab.
It is also possible to compute the inverse of a sparse tensor C: tf.SparseTensor(indices=C.indices, values=1/C.values, dense_shape=C.dense_shape).
So there is this NaN issue for division, and concerning the mixture of dense tensor and sparse tensor, one option consists in converting the sparse tensor to a dense tensor. But we want to avoid this. In the other direction, from converting the dense tensor to a sparse tensor, this can be very ineffective if the tensor is not really sparse.
All this to say that it does not seem to be a simple problem.
I have a generator that yields tf.sparse.SparseTensors. I want to turn this into a Tensorflow Dataset, but am running into some issues. I am using TF2. First, unlike regular Tensors, you cannot simply pass them in (and providing the correct data types for output_types). For a sparse tensor of [1,0,0,0,5,0], the error looks like
tensorflow.python.framework.errors_impl.InvalidArgumentError: TypeError: `generator` yielded an element that could not be converted to the expected type. The expected type was int64, but the yielded element was SparseTensor(indices=tf.Tensor(
E [[0]
E [4]], shape=(2, 1), dtype=int64), values=tf.Tensor([1 5], shape=(2,), dtype=int64), dense_shape=tf.Tensor([6], shape=(1,), dtype=int64)).
After doing some looking around on the internet, I found this open issue and tried to do something similar https://github.com/tensorflow/tensorflow/issues/16689 - read the indices, values, and shape as separate tensors into a TF Dataset, and then mapping over the dataset to create the sparse tensor. This is not working as shown in some of the examples in the github issue - tf.sparse.SparseTensor(indices, values, shape) does not seem to accept indices and shape in the form of a tf.Tensor - it will happily take in a list or numpy array, but not a Tensor. Since map is not eager, I also cannot call .numpy() on the Tensor either. What is best way to get this to work? I see there is tf.py_function/tf.numpy_function which could help, but constructing the output type can be tricky (though not impossible) for my use case - the incoming data is not fixed and can have a mix of sparse and dense tensors.
Let's say I have a tensor (None, 2, 56, 56, 256). Now I want to have my tensor with shape (None, 2, 55, 55, 256) by dropping last col and last row. How can I acheive this using Keras/Tensorflow?
In tensorflow we can slice tensors using python slice notation. SO, given a tensor X with shape (20,2,56,56,256) say (as you have described but with a batch size of 20), we can easily slice it taking all but the last 'row' in the 2nd and 3rd dimension as follows:
X[:,:,:-1,:-1,:]
Note the use of :-1 to denote "everything before the last 'row'".
Given this know-how about slicing the tensor in tensorflow we just need to adapt it for keras. We could, of course, write a full blown custom layer implementing this (or possibly even find one out there someone else has written - I've not looked but slicing is pretty common so suspect someone has written something somewhere!).
However, for something as simple as this, I'd advocate just using a Lambda layer which we can define as follows:
my_slicing_layer = Lambda(lambda x: x[:,:,:-1,:-1,:], name='slice')
And can use in our keras models as normal:
my_model = Sequential([
Activation('relu', input_shape=(2,56,56,256)),
my_slicing_layer
])
I have a Tensor which needs to be cropped with the indices of a tensor.
ex - Input (None,5x5x10) tensor
BoundingBox (None, 2) -- tensor
I want to have an operation that does the following
Output (None,3x2x10) --tensor
if BoundingBox[0,0] = 3, BoundingBox[0,1] = 2
This is same as tf.image.crop_to_bounding_box but this function does not tensor type bounding box as input. Please help.
Unfortunately this isn't possible with 'standard' tensor operations because the dimensions of the output could vary.
Consider the example where bounding_box[0] == [3,2] and bounding_box[1] == [4,2] then your output shape needs to be (None, 3 or 4, 2, 10) and (of course) having a dimension 3 or 4 is not allowed for standard tensors.
TensorFlow does, however, have the concept of a Ragged Tensor which could conceivably be used to represent crops of different dimensions but this is an unusual case and is unlikely to be suited to most mainstream downstream training operations. Still, it could be worth reading up on this to see if it fits your use case: link
sparse tensor.shape method returns a tensor object which seems to be of no use to extract the actual shape of the sparse tensor without resorting to run function.
To clarify what I mean, first consider a sparse tensor:
a = tf.SparseTensor(indices=[[0, 0, 0], [1, 2, 1]], values=[1.0+2j, 2.0], shape=[3, 4, 2])
a.shape returns:
tf.Tensor 'SparseTensor_1/shape:0' shape=(3,) dtype=int64
This is kind of no use.
Now, consider a dense tensor:
a = tf.constant(np.random.normal(0.0, 1.0, (4, 4)).astype(dtype=np.complex128))
a.get_shape() returns:
TensorShape([Dimension(4), Dimension(4)])
I can use this output and cast it into a list or tuple of integers without ever invoking run(). However, I cannot do the same for sparse tensor, unless I first convert sparse tensor to dense (which is not implemented for complex sparse tensor yet) and then call get_shape() method on it, but this is kind of redundant, defeats the purpose of using a sparse tensor in the first place and also leads to error down the road if the input sparse tensor is complex.
Is there a way to obtain the shape of a sparse tensor without invoking run() or converting it to a dense tensor first?
tf.SparseTensor is implemented as a triple of dense Tensors under the hood. The shape of a SparseTensor is just a Tensor; if you want to know its value, your best bet is to evaluate it using session.run:
print(sess.run(a.shape))
In general, Tensorflow does not promise to compute an exact shape even for dense tensors at graph construction time; shapes are best effort and may not even have a fixed value. So even for a dense Tensor you may have to evaluate the Tensor using run to get a precise shape.