sparse tensor.shape method returns a tensor object which seems to be of no use to extract the actual shape of the sparse tensor without resorting to run function.
To clarify what I mean, first consider a sparse tensor:
a = tf.SparseTensor(indices=[[0, 0, 0], [1, 2, 1]], values=[1.0+2j, 2.0], shape=[3, 4, 2])
a.shape returns:
tf.Tensor 'SparseTensor_1/shape:0' shape=(3,) dtype=int64
This is kind of no use.
Now, consider a dense tensor:
a = tf.constant(np.random.normal(0.0, 1.0, (4, 4)).astype(dtype=np.complex128))
a.get_shape() returns:
TensorShape([Dimension(4), Dimension(4)])
I can use this output and cast it into a list or tuple of integers without ever invoking run(). However, I cannot do the same for sparse tensor, unless I first convert sparse tensor to dense (which is not implemented for complex sparse tensor yet) and then call get_shape() method on it, but this is kind of redundant, defeats the purpose of using a sparse tensor in the first place and also leads to error down the road if the input sparse tensor is complex.
Is there a way to obtain the shape of a sparse tensor without invoking run() or converting it to a dense tensor first?
tf.SparseTensor is implemented as a triple of dense Tensors under the hood. The shape of a SparseTensor is just a Tensor; if you want to know its value, your best bet is to evaluate it using session.run:
print(sess.run(a.shape))
In general, Tensorflow does not promise to compute an exact shape even for dense tensors at graph construction time; shapes are best effort and may not even have a fixed value. So even for a dense Tensor you may have to evaluate the Tensor using run to get a precise shape.
Related
I'm working with RaggedTensors to manipulate a dense tensor. Something like this :
out_left = tf.ragged.boolean_mask(input, index)
index = tf.math.logical_not(index)
out_right = tf.ragged.boolean_mask(input, index)
reconstruced_tensor = tf.concat([out_left, out_right], axis=-1)
reconstruced_tensor = reconstruced_tensor.to_tensor()
As you can see, in my example, I'm just splitting my input tensor using RaggedTensors and reconstructing it (I know its useless, but it's just for the sake of simplicity)
The problem I'm having is that I'm getting the following warning:
IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model_14/channel_roll_13/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model_14/channel_roll_13/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None,), dtype=float32), dense_shape=Tensor("gradient_tape/model_14/channel_roll_13/RaggedToTensor/Shape:0", shape=(1,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory. "shape. This may consume a large amount of memory." % value)
Since I know the shape of the output Tensor, I would have thought Tensorflow would too. Is there any way I can explicitly specify the shape of the output tensor since Tensorflow doesn't seem to deduce it on its own?
I have a tf.data.Dataset object d, where each element is an integer tf.sparse.SparseTensor, and I would like to sum them, returning a sparse tensor. One way I see is the following:
d.reduce(tf.sparse.SparseTensor(tf.zeros([0, 1], tf.int64),
tf.zeros([0], tf.int32),
dense_shape),
tf.sparse.add)
Problem:
How do I construct the zero for the reduce operation if I do not know the dense_shape ahead of time? I know all the sparse tensors in the dataset will have same shape, but it is not statically known. Perhaps it depends on data from which the sparse tensors are constructed, and setting this interactively in eager mode is not a viable option.
I have a generator that yields tf.sparse.SparseTensors. I want to turn this into a Tensorflow Dataset, but am running into some issues. I am using TF2. First, unlike regular Tensors, you cannot simply pass them in (and providing the correct data types for output_types). For a sparse tensor of [1,0,0,0,5,0], the error looks like
tensorflow.python.framework.errors_impl.InvalidArgumentError: TypeError: `generator` yielded an element that could not be converted to the expected type. The expected type was int64, but the yielded element was SparseTensor(indices=tf.Tensor(
E [[0]
E [4]], shape=(2, 1), dtype=int64), values=tf.Tensor([1 5], shape=(2,), dtype=int64), dense_shape=tf.Tensor([6], shape=(1,), dtype=int64)).
After doing some looking around on the internet, I found this open issue and tried to do something similar https://github.com/tensorflow/tensorflow/issues/16689 - read the indices, values, and shape as separate tensors into a TF Dataset, and then mapping over the dataset to create the sparse tensor. This is not working as shown in some of the examples in the github issue - tf.sparse.SparseTensor(indices, values, shape) does not seem to accept indices and shape in the form of a tf.Tensor - it will happily take in a list or numpy array, but not a Tensor. Since map is not eager, I also cannot call .numpy() on the Tensor either. What is best way to get this to work? I see there is tf.py_function/tf.numpy_function which could help, but constructing the output type can be tricky (though not impossible) for my use case - the incoming data is not fixed and can have a mix of sparse and dense tensors.
I'm creating a layer which uses its input tensors to create a SparseTensor, i.e., the input tensors are the respective indices and values of the SparseTensor.
Since:
indices: A 2-D int64 tensor of shape
And because tf.cast(x, tf.int64) is not differentiable, I'm not sure if this is achievable.
Another option is to find a turnaround based on tf.round(), but SparseTensor won't accept a different type of tensor as indices.
Is there a way to cast a tensor to integer and not having None for gradient ?
How can I create a SparseTensor using previous layers outputs, which are float tensors ?
I have two tensors with shapes
voxels :: (4, 64, 64, 64)
indices :: (4096, 3)
Here voxels has dtype float32, while indices are integers between 0 and 63.
I would like to index the voxels, creating a tensor tensor output:
output :: 4, 4096
Such that the following constraint holds
output[i,j] = voxels[i, indices[j,0], indices[j,1]. indices[j,2]]
How can I do this in tensorflow?
There is some talk of more advanced slicing here, but it's not clear that the suggestions there apply, or that the gradients have been implemented.
For reference, if the above tensors were numpy arrays, the following code does what I want:
output = voxels[:, indices[:,0],indices[:,1],indices[:,2]]
gather_nd is indeed a possibility, but gradients are not yet implemented. However, it is also possible to use a trick involving gather, by flattening the tensor and re-computing appropriate indices.
I believe gather_nd does what you want, though I don't think the gradients have been implemented.