I'm trying to update rank3 tensor using tensor_scatter_nd_update fn.
For rank2 tensor I used the following snippet:
tensor = tf.zeros((3, 2))
indices = [[0], [2]]
updates = [[5, 5], [10, 10]]
output = tf.tensor_scatter_nd_update(tensor, indices, updates)
print(output)
tf.Tensor(
[[ 5. 5.]
[ 0. 0.]
[10. 10.]], shape=(3, 2), dtype=float32)
Now I want to perform the same op for batches -> basically on rank3 tensor:
tensor = tf.zeros((4, 3, 2))
indices = [[[0], [2]],
[[1], [0]],
[[0], [1]],
[[0], [2]]]
updates = [[[5, 5], [10, 10]],
[[1, 1], [7, 7]],
[[3, 3], [2, 2]],
[[5, 5], [1, 1]]]
output = tf.tensor_scatter_nd_update(tensor, indices, updates)
However, I'm getting a shape mismatch error:
Inner dimensions of output shape must match inner dimensions of updates shape Output: [4,3,2] updates: [4,2,2]
If you want to update by the first dimension, for example you want to change [0], [2] and [3], then each of the updates has shape (1, 3, 2) but you have three updates so (3, 3, 2):
tensor = tf.zeros((4, 3, 2))
indices = [[0], [2], [3]] # shape (3, 1)
updates = [[[1, 1], [2, 2], [3, 3]], # <- this is [0] and will go to output[0,:,:]
[[10, 10], [10, 10], [10, 10]], # <- this is [2]
[[101,102], [103, 104], [105, 106]]] # shape (3, 3, 2)
output = tf.tensor_scatter_nd_update(tensor, indices, updates)
output:
tf.Tensor(
[[[ 1. 1.]
[ 2. 2.]
[ 3. 3.]]
[[ 0. 0.]
[ 0. 0.]
[ 0. 0.]]
[[ 10. 10.]
[ 10. 10.]
[ 10. 10.]]
[[101. 102.]
[103. 104.]
[105. 106.]]], shape=(4, 3, 2), dtype=float32)
If you want to update by the first two dimensions, for example you want to update the positions [0, 1] and [2, 2], then each of updates has shape (1, 1, 2), but you have two updates, so updates will have shape (2, 1, 2):
tensor = tf.zeros((4, 3, 2))
indices = [[[0,1]], [[2,2]]] # shape (2, 1, 2)
updates = [[[1, 1]], # <- this will go to output[0,1,:]
[[10, 10]]] # shape (2, 1, 2)
output = tf.tensor_scatter_nd_update(tensor, indices, updates)
output:
tf.Tensor(
[[[ 0. 0.]
[ 1. 1.]
[ 0. 0.]]
[[ 0. 0.]
[ 0. 0.]
[ 0. 0.]]
[[ 0. 0.]
[ 0. 0.]
[10. 10.]]
[[ 0. 0.]
[ 0. 0.]
[ 0. 0.]]], shape=(4, 3, 2), dtype=float32)
It is simply as Matrix multiplication that follows the sample it reveals your works, and the index indicates the target.
As in programming tools built carefully it will consider the orders and the number of data contain as a basic program.
The outer and inner is indicates the multiplying matrix dimensions as we learn in mathematics and science programs. Sample: ( 8, 2 ) * ( 2 * 8 )
Specify types of inputs, index can be int, scatters expecting float32 since the calculation of Eigaint and their relationship.
For more than 2 dimensions you had options to convert it back to 2 dimensions or vary each index input, see some example create function and loop though the indexes.
Sample: Simple solution
tensor = tf.zeros((8, 2))
tensor = tf.cast( tensor, dtype=tf.int32 )
indices = [[[0], [2]],
[[1], [0]],
[[0], [1]],
[[0], [2]]]
indices = tf.constant(indices, shape=(8, 1), dtype=tf.int32)
updates = [[[5, 5], [10, 10]],
[[1, 1], [7, 7]],
[[3, 3], [2, 2]],
[[5, 5], [1, 1]]]
updates = tf.constant(updates, shape=(8, 2), dtype=tf.int32)
output = tf.tensor_scatter_nd_update(tensor, indices, updates)
print(output)
Output
tf.Tensor(
[[5 5]
[2 2]
[1 1]
[0 0]
[0 0]
[0 0]
[0 0]
[0 0]], shape=(8, 2), dtype=int32)
Sample 2: Transform input is easiest way
tensor = tf.zeros((4, 3, 2))
tensor = tf.cast( tensor, dtype=tf.int32 )
tensor = tf.constant(tensor, shape=(12, 2), dtype=tf.int32)
indices = [[[0], [2], [1]],
[[1], [0], [1]],
[[0], [1], [1]],
[[0], [2], [1]]]
indices = tf.constant(indices, shape=(12, 1), dtype=tf.int32)
updates = [[[5, 5, 1], [10, 10, 1]],
[[1, 1, 1], [7, 7, 1]],
[[3, 3, 1], [2, 2, 1]],
[[5, 5, 1], [1, 1, 1]]]
updates = tf.constant(updates, shape=(12, 2), dtype=tf.int32)
output = tf.tensor_scatter_nd_update(tensor, indices, updates)
print(output)
Output
tf.Tensor(
[[5 5]
[1 1]
[1 1]
[0 0]
[0 0]
[0 0]
[0 0]
[0 0]
[0 0]
[0 0]
[0 0]
[0 0]], shape=(12, 2), dtype=int32)
Sample 3: Repeating where you can create recursive
global index
index = 5
#tf.function
def reverse_count( index_input ):
global index
index = index - 1
if index > 0 :
return True
else :
return False
tensor = tf.zeros((3, 2))
tensor = tf.cast( tensor, dtype=tf.int32 )
index = 5
indices = [[[0], [2], [1]],
[[1], [0], [1]],
[[0], [1], [1]],
[[0], [2], [1]]]
indices = tf.constant(indices, shape=(4, 3, 1), dtype=tf.int32)
updates = [[5, 5], [10, 10], [1, 1]]
updates = tf.constant(updates, shape=(3, 2), dtype=tf.int32)
print( index )
indice = tf.where([reverse_count( index )], indices[index - 1,:,:1], tf.constant([[-1, -1, -1]], shape=(3, 1)).numpy() )
print( 'tensor' )
print( tensor )
print( 'indice' )
print( indice )
print( 'updates' )
print( updates )
output = tf.tensor_scatter_nd_update(tensor, indice, updates)
print( 'output' )
print( output )
Output
5
tensor
tf.Tensor(
[[0 0]
[0 0]
[0 0]], shape=(3, 2), dtype=int32)
indice
tf.Tensor(
[[0]
[2]
[1]], shape=(3, 1), dtype=int32)
updates
tf.Tensor(
[[ 5 5]
[10 10]
[ 1 1]], shape=(3, 2), dtype=int32)
output
tf.Tensor(
[[ 5 5]
[ 1 1]
[10 10]], shape=(3, 2), dtype=int32)
Related
For example, if I have an input tensor:
x = [[[0, 1], [2, -1]], [[5, 1], [-10, -100]]] batch x 2 x 2 dimensionality
and a indices tensor:
ind = [[[1], [0]], [[0], [0]]]
How to go about indexing x with ind in order to obtain:
out = [[[1], [2]], [[5], [-10]]]
You can use torch.Tensor.gather
x = torch.tensor([[[0, 1], [2, -1]], [[5, 1], [-10, -100]]])
ind = torch.tensor([[[1], [0]], [[0], [0]]])
out = x.gather(2, ind)
which results in
>>> out
tensor([[[ 1],
[ 2]],
[[ 5],
[-10]]])
Is there a way to directly update the elements in tf.Variable X at indices without creating a new tensor having the same shape as X?
tf.tensor_scatter_nd_update create a new tensor hence it appears not updateing the original tf.Variable.
This operation creates a new tensor by applying sparse updates to the input tensor.
tf.Variable assign apparently needs a new tensor value which has the same shape of X to update the tf.Variable X.
assign(
value, use_locking=False, name=None, read_value=True
)
value A Tensor. The new value for this variable.
About the tf.tensor_scatter_nd_update, you're right that it returns a new tf.tensor (and not tf.Variable). But about the assign which is an attribute of tf.Variable, I think you somewhat misread the document; the value is just the new item that you want to assign in particular indices of your old variable.
AFAIK, in tensorflow all tensors are immutable like python numbers and strings; you can never update the contents of a tensor, only create a new one, source. And directly updating or manipulating of tf.tensor or tf.Variable such as numpy like item assignment is still not supported. Check the following Github issues to follow up the discussions: #33131, #14132.
In numpy, we can do an in-place item assignment that you showed in the comment box.
import numpy as np
a = np.array([1,2,3])
print(a) # [1 2 3]
a[1] = 0
print(a) # [1 0 3]
A similar result can be achieved in tf.Variable with assign attribute.
import tensorflow as tf
b = tf.Variable([1,2,3])
b.numpy() # array([1, 2, 3], dtype=int32)
b[1].assign(0)
b.numpy() # array([1, 0, 3], dtype=int32)
Later, we can convert it to tf. tensor as follows.
b_ten = tf.convert_to_tensor(b)
b_ten.numpy() # array([1, 0, 3], dtype=int32)
We can do such item assignment in tf.tensor too but we need to convert it to tf.Variable first, (I know, not very intuitive).
tensor = [[1, 1], [1, 1], [1, 1]] # tf.rank(tensor) == 2
indices = [[0, 1], [2, 0]] # num_updates == 2, index_depth == 2
updates = [5, 10] # num_updates == 2
x = tf.tensor_scatter_nd_update(tensor, indices, updates)
x
<tf.Tensor: shape=(3, 2), dtype=int32, numpy=
array([[ 1, 5],
[ 1, 1],
[10, 1]], dtype=int32)>
x = tf.Variable(x)
x
<tf.Variable 'Variable:0' shape=(3, 2) dtype=int32, numpy=
array([[ 1, 5],
[ 1, 1],
[10, 1]], dtype=int32)>
x[0].assign([5,1])
x
<tf.Variable 'Variable:0' shape=(3, 2) dtype=int32, numpy=
array([[ 5, 1],
[ 1, 1],
[10, 1]], dtype=int32)>
x = tf.convert_to_tensor(x)
x
<tf.Tensor: shape=(3, 2), dtype=int32, numpy=
array([[ 5, 1],
[ 1, 1],
[10, 1]], dtype=int32)>
data = tf.constant( [ [ [0, 2, 4, 1], [1, 0, 0, 2] ], [ [1, 0, 4, 6], [2, 6, 3, 1] ] ] )
indices = tf.argmax(data, axis=2)
How to get the result [ [4 2], [6 6] ] in tensorflow?
Please help me!!!!
Use tf.reduce_max(), in your case
data = tf.constant([[ [0, 2, 4, 1],[1, 0, 0, 2]], [ [1, 0, 4, 6], [2, 6, 3, 1] ]])
maximum_values = tf.reduce_max(data, reduction_indices=[2])
with tf.Session() as sess:
p=sess.run(maximum_values)
print(p)
[[4 2]
[6 6]]
Edit: To access other values you could slice by indexes and then use tf.concat or tf.stack. For example, if you want to get [[0 2] [3 1]] you could try
with tf.Session() as sess:
p=sess.run(data[0][0][0:2])
print(p)
[0 2]
q=sess.run(data[1][1][2:])
print(q)
[3 1]
r=sess.run(tf.stack([p,q],0))
print(r)
[[0 2]
[3 1]]
I have a tensor of shape (2, 3), say input = [[1 2 3] [4 5 6]], I have a index tensor of shape (2, 3) which I hope to be used to retrieve the values from input, index = [[1 0] [2 0]]. My expected result is result = [[2 1] [6 4]]. However, simply using tf.gather(input, index) does not seem to work.
If you want to extract element from the array, you can use gather_nd and the index should be of the form (i,j) for each element. In your example, your index should be:
inputs = tf.Variable([[1, 2, 3], [4, 5, 6]])
index = tf.Variable([[[0,1],[0,0]], [[1,2],[1,0]]])
result = tf.gather_nd(inputs, new_index)
sess = tf.InteractiveSession()
tf.global_variables_initializer().run()
print(result.eval())
# output
# [[2 1]
# [6 4]]
If you want to generate the index from the form you have mentioned, you can do:
index = tf.Variable([[1, 0], [2, 0]])
r = tf.tile(tf.expand_dims(tf.range(tf.shape(index1)[0]), 1), [1, 2])
new_index = tf.concat([tf.expand_dims(r,-1), tf.expand_dims(index, -1)], axis=2)
sess = tf.InteractiveSession()
tf.global_variables_initializer().run()
print(new_index.eval())
#output
#[[[0 1]
#[0 0]]
#[[1 2]
#[1 0]]]
The problem is in index, you have to use 0 or 1 values in index only, because your input array has shape (2,3). If you add additional row to input array all work fine:
import tensorflow as tf
input = tf.Variable([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
index = tf.Variable([[1, 0], [2, 0]])
result = tf.gather(input, index)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print(sess.run(result))
# results [[[4 5 6] [1 2 3]] [[7 8 9] [1 2 3]]]
Anyway, index describe which slice gather from input array, not which element.
For example:
array = [[1, 2, 3], [4, 5, 6]]
slice = [[0, 0, 1], [0, 1, 2]]
output = [[1, 1, 2], [4, 5,6]]
I've tried array[slice], but that didn't work. I also couldn't get tf.gather or tf.gather_nd to work, although these initially seemed like the correct functions to use. Note that these are all tensors in-graph.
How can I select these values in my array according to slice?
You need to add a dimension to your slice tensor which you can do with tf.pack and then we can use tf.gather_nd no problem.
import tensorflow as tf
tensor = tf.constant([[1, 2, 3], [4, 5, 6]])
old_slice = tf.constant([[0, 0, 1], [0, 1, 2]])
# We need to add a dimension - we need a tensor of rank 2, 3, 2 instead of 2, 3
dims = tf.constant([[0, 0, 0], [1, 1, 1]])
new_slice = tf.pack([dims, old_slice], 2)
out = tf.gather_nd(tensor, new_slice)
If we run the follow code:
with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
run_tensor, run_slice, run_out = sess.run([tensor, new_slice, out])
print 'Input tensor:'
print run_tensor
print 'Correct param for gather_nd:'
print run_slice
print 'Output:'
print run_out
This should give the correct output:
Input tensor:
[[1 2 3]
[4 5 6]]
Correct param for gather_nd:
[[[0 0]
[0 0]
[0 1]]
[[1 0]
[1 1]
[1 2]]]
Output:
[[1 1 2]
[4 5 6]]
An even easier way to calculate the results, which is also of more general nature, is to directly leverage the batch_dims argument of tf.gather:
>>> array = tf.constant([[1,2,3], [4,5,6]])
>>> slice = tf.constant([[0,0,1], [0,1,2]])
>>> output = tf.constant([[1,1,2], [4,5,6]])
>>> tf.gather(array, slice, batch_dims=1, axis=1)
<tf.Tensor: shape=(2, 3), dtype=int32, numpy=
array([[1, 1, 2],
[4, 5, 6]], dtype=int32)>