I have a problem with using Tensorflow. I have four images with their corresponding indices. I want to make an image from them. I tried for loops, tf.gather, tf.assign, and so on but all show error. If somebody help me, it would be really appreciated. I explain my question with one small example:
We have 4 tensors and their indices from tensor tf.ktop function: (I write like MATLAB for just for simplicity)
a = [1, 2; 5, 6] a_idx = [0, 1; 2, 3]
b = [3, 4; 7, 8] b_idx = [0, 1; 2, 3]
c = [9, 10; 13, 14] c_idx = [0, 1; 2, 3]
d = [11, 12; 15, 16] d_idx = [0, 1; 2, 3]
I am looking for a big image from a, b, c, and d and their indices like:
image = [a b; c d]
image = [1, 2, 3, 4; 5, 6, 7, 8;9 10, 11, 12;13, 14, 15, 16]
In python I have something like:
a, a_idx, b, b_idx, c, c_idx, d, d_idx
n_x = tf.Variable(tf.zeros([1, 4, 4, 1]))
n_patches = tf.extract_image_patches(
n_x,
[1, 2, 2, 1],
[1, 2, 2, 1],
[1, 1, 1, 1],
"SAME"
)
So, n_patches is 4 tensors and I need to put a to d values to each patch corresponding to a_idx to d_idx. Its really easy for me in MATLAB or Numpy to do that using for loop but in tensorflow I can not
In your comments, I suspect you made a tiny error in your desired output, image.
I interpret that you want is given
values = np.array([[2, 5],\
[4, 6]])
indices = np.array([[0, 3],\
[2, 1]])
your result would be
[[2. 0. 0. 0.]
[0. 0. 0. 5.]
[0. 0. 4. 0.]
[0. 6. 0. 0.]]
So you want to obtain a sort of one hot encoded matrix, but with values corresponding to given indices. This can be obtained like so:
import numpy as np
values = np.array([[2, 5],\
[4, 6]])
indices = np.array([[0, 3],\
[2, 1]])
# Make a matrix with only zeros
n_hots = np.zeros_like((indices))
# Now row 0,1,2 and 3 should have values corresponding to the
# indices. That is we should first "unpack" the values and indices:
indices=indices.ravel()
values=values.ravel()
# values are now: [2,5,4,6]
# indices are now: [0,3,2,1]
# values:
# n_hots[row,indices[row]]=values[indices[row]]
# e.g.
# n_hots[0,0]=2
# n_hots[1,3]=5
# n_hots[2,2]=4
# n_hots[3,1]=6
# Notice how the first slices are a ascending range of values:
# [0,1,2,3], and the second slice are the raveled indices, and the
# right hand side of the equal sign are the ravele values!
# That means we can just do the following:
n_hots[np.arange(4),indices]=values
print(n_hots)
In tensorflow it would be a bit different. First generating a one_hot tensor that have ones at the 2nd axis value: at the indices, and then multiplying that with the corresponding indices:
import numpy as np
import tensorflow as tf
indices=tf.placeholder(shape=(None),dtype=tf.int32)
values=tf.placeholder(shape=(None),dtype=tf.float32)
one_hots=tf.one_hot(indices, tf.shape(indices)[0])
n_hots=one_hots*tf.gather(values, indices)
with tf.Session() as sess:
_values = np.array([[2, 5],\
[4, 6]])
_indices = np.array([[0, 3],\
[2, 1]])
n_h=sess.run(n_hots, {indices: _indices.ravel(), values:_values.ravel()})
print(n_h)
Related
How can I retrieve a column vector from a 2d array given an indicator column vector?
Suppose I have
X = np.array([[1, 4, 6],
[8, 2, 9],
[0, 3, 7],
[6, 5, 1]])
and
S = np.array([0, 2, 1, 2])
Is there an elegant way to get from X and S the result array([1, 9, 3, 1]), which is equivalent to
np.array([x[s] for x, s in zip(X, S)])
You can achieve this using np.take_along_axis:
>>> np.take_along_axis(X, S[..., None], axis=1)
array([[1],
[9],
[3],
[1]])
You need to make sure both array arguments are of the same shape (or broadcasting can be applied), hence the S[..., None] broadcasting.
Of course your can reshape the returned value with a [:, 0] slice.
>>> np.take_along_axis(X, S[..., None], axis=1)[:, 0]
array([1, 9, 3, 1])
Alternatively you can just use indexing with an arangement:
>>> X[np.arange(len(S)), S[np.arange(len(S))]]
array([1, 9, 3, 1])
I believe this is also equivalent to np.diag(X[:, S]) but with unnecessary copying...
For 2d arrays
# Mention row numbers as one list and S which is column number as other
X[[0, 1, 2, 3], S]
# more general
X[np.indices(S.shape), S]
indexing_basics
Suppose that you have a 3-tensor
data = np.reshape(np.arange(12), [2, 2, 3])
x = tf.constant(data)
Thinking of this as 2x2 matrices indexed by the last index, I would like to get the first column from the first matrix, the second column from the second matrix and the second column from the third matrix.
How can I use tf.gather_nd to do this?
You need first generate the indices you want.
import tensorflow as tf
import numpy as np
indices = [[i,min(j,1),j] for j in range(3) for i in range(2)] # According to your description
# [[0, 0, 0], [1, 0, 0], [0, 1, 1], [1, 1, 1], [0, 1, 2], [1, 1, 2]]
a = tf.constant(np.arange(12).reshape(2,2,3))
res = tf.gather_nd(a, indices)
sess = tf.InteractiveSession()
a.eval()
# array([[[ 0, 1, 2],
# [ 3, 4, 5]],
# [[ 6, 7, 8],
# [ 9, 10, 11]]])
res.eval()
# array([ 0, 6, 4, 10, 5, 11])
I found the following tutorial online explaining how to deal with this kind of problems: https://geekyisawesome.blogspot.com/2018/05/fancy-indexing-in-tensorflow-getting.html
Suppose we have a 4x3 matrix
M = tf.constant(np.arange(12).reshape(4,3))
Now let's say that you wanted the third element of the first row, the second element of the second row, the first element of the third row, and the second element of the fourth row. As explained in the tutorial, this could be accomplished like:
idx = tf.constant([2,1,0,1], tf.int32)
x = tf.gather_nd(M, tf.stack([tf.range(M.shape[0]), idx], axis=1))
But what if M has an unknown number of rows? (and idx as a tensor of integers of the appropriate size) Then tf.range(M.shape[0]) will raise an error. How can I go around that?
I'm trying to do some very simple average pooling on a Keras / Tensorflow Tensor (not a layer in a network). I have a list of 18 embeddings (embedding = 2D vector) and want to average pool them with a pool-size of 3 with no overlap. So the first 3 embeddings should be averaged to an embedding, then the next 3 and so on. In the end, I need a list of 18/3 = 6 embeddings.
Here's my code so far:
sequence_len = 3 # The number of embeddings that should be averaged
feature_dim = 2 # the dimensions of embeddings
input = [
[1, 3], # Anchor 1
[2, 2],
[3, 1],
[4, 1], # Anchor 2
[6, 1],
[8, 1],
[12, 3], # Positive 1
[0, 6],
[0, 0],
[1, 1], # Positive 2
[1, 1],
[1, 1],
[5, 0], # Negative 1
[5, 12],
[5, 0],
[1, 1], # Negative 2
[1, 1],
[1, 1],
]
expected_out = [
[2, 2], # Anchor 1
[6, 1], # Anchor 2
[4, 3], # Positive 1
[1, 1], # Positive 2
[5, 4], # Negative 1
[1, 1], # Negative 2
]
input_tensor = K.variable(input)
output_tensor = a_function_i_cant_figure_out(input_tensor) # What's the API to call?
assert numpy.array_equal(K.eval(output_tensor), expected_out)
I've spent over 2 hours on it now and I can't figure it out. I don't understand how I would have to reshape my data to get it to work with the pooling function.
The closes I've come to a working solution is wrapping my input in another 2 lists (making it shape (1,1,18,2)) and calling this
output_tensor = K.pool2d(foo, (3, 2), (3, 2), pool_mode="avg", data_format="channels_first")
But this also averages the embeddings themselves. So instead of [[2,2], [6,1], ...] I'd receive [2, 3.5, ...]
This seems like such a trivial operation - how can I do this?
Ok, figured it out:
output_tensor = K.tf.nn.pool(input_tensor, [1, sequence_len], strides=(1, sequence_len), pooling_type="AVG", padding="VALID")
Think of my data as a 2D matrix.
[1, sequence_len] is the pool/window size. So in X direction, we have a width of 1 and in Y direction a height of sequence_len. This means: The first value of sequence_len rows will be combined to one value. Then the stride of (1, sequence_len) means: We move the pool by 1 in X direction and once we're done with the row we move the pool down by sequence_len.
We go with a width of 1 and a X-step of 1 column by column through the data which means our output has the same width. In Y direction we pool sequence_len rows to 1 row and then take the next sequence_len rows with no overlap. So the height of our output is original_height / sequence_len
I read the doc of tf.stack() on tensorflow stack . There is an example on the page:
>>> x = tf.constant([1, 4])
>>> y = tf.constant([2, 5])
>>> z = tf.constant([3, 6])
>>> sess=tf.Session()
>>> sess.run(tf.stack([x, y, z]))
array([[1, 4],
[2, 5],
[3, 6]], dtype=int32)
>>> sess.run(tf.stack([x, y, z], axis=1))
array([[1, 2, 3],
[4, 5, 6]], dtype=int32)
what I don't understand is the second example where axis=1.
From the result it seems it converts the three inputs rows to columns first
and then put them toghter along the axis=1, but
I think the result should be
array([[1,4, 2, 5, 3, 6 ]] dtype=int32 )
can anyone help explain this?
Thanks!
tf.stack always adds a new dimension, and always concatenates the given tensor along that new dimension. In your case, you have three tensors with shape [2]. Setting axis=0 is the same as adding a new first dimension, so each tensor would now have shape [1, 2], and concatenating across that dimension, so the final shape would be [3, 2]. That is, each tensor would be a "row" of the final tensor. With axis=1 the shapes of each individual tensor would be extended to [2, 1], and the result would have shape [2, 3]. So each given tensor would be a "column" of the resulting tensor.
In other words, tf.stack is functionally equivalent to this:
def tf.stack(tensors, axis=0):
return tf.concatenate([tf.expand_dims(t, axis=axis) for t in tensors], axis=axis)
But the result that you expect would be obtained with something like this:
tf.concatenate([tf.expand_dims(t, axis=0) for t in tensors], axis=1)
Note that the added dimension and the concatenation dimension are different in this case.
Lets say I have a sparse tensor with duplicate indices and where they are duplicate I want to merge values (sum them up)
What is the best way to do this?
example:
indicies = [[1, 1], [1, 2], [1, 2], [1, 3]]
values = [1, 2, 3, 4]
object = tf.SparseTensor(indicies, values, shape=[10, 10])
result = tf.MAGIC(object)
result should be a spare tensor with the following values (or concrete!):
indicies = [[1, 1], [1, 2], [1, 3]]
values = [1, 5, 4]
The only thing I have though of is to string concat the indicies together to create an index hash apply it to a third dimension and then reduce sum on that third dimension.
indicies = [[1, 1, 11], [1, 2, 12], [1, 2, 12], [1, 3, 13]]
sparse_result = tf.sparse_reduce_sum(sparseTensor, reduction_axes=2, keep_dims=true)
But that feels very very ugly
Here is a solution using tf.segment_sum. The idea is to linearize the indices in to a 1-D space, get the unique indices with tf.unique, run tf.segment_sum, and convert the indices back to N-D space.
indices = tf.constant([[1, 1], [1, 2], [1, 2], [1, 3]])
values = tf.constant([1, 2, 3, 4])
# Linearize the indices. If the dimensions of original array are
# [N_{k}, N_{k-1}, ... N_0], then simply matrix multiply the indices
# by [..., N_1 * N_0, N_0, 1]^T. For example, if the sparse tensor
# has dimensions [10, 6, 4, 5], then multiply by [120, 20, 5, 1]^T
# In your case, the dimensions are [10, 10], so multiply by [10, 1]^T
linearized = tf.matmul(indices, [[10], [1]])
# Get the unique indices, and their positions in the array
y, idx = tf.unique(tf.squeeze(linearized))
# Use the positions of the unique values as the segment ids to
# get the unique values
values = tf.segment_sum(values, idx)
# Go back to N-D indices
y = tf.expand_dims(y, 1)
indices = tf.concat([y//10, y%10], axis=1)
tf.InteractiveSession()
print(indices.eval())
print(values.eval())
Maybe you can try:
indicies = [[1, 1], [1, 2], [1, 2], [1, 3]]
values = [1, 2, 3, 4]
object = tf.SparseTensor(indicies, values, shape=[10, 10])
tf.sparse.to_dense(object, validate_indices=False)
Using unsorted_segment_sum could be simpler:
def deduplicate(tensor):
if not isinstance(tensor, tf.IndexedSlices):
return tensor
unique_indices, new_index_positions = tf.unique(tensor.indices)
summed_values = tf.unsorted_segment_sum(tensor.values, new_index_positions, tf.shape(unique_indices)[0])
return tf.IndexedSlices(indices=unique_indices, values=summed_values, dense_shape=tensor.dense_shape)
Another solution is to use tf.scatter_nd which will create a dense tensor and accumulate values on duplicate indices. This behavior is clearly stated in the documentation:
If indices contains duplicates, the duplicate values are accumulated (summed).
Then eventually we can convert it back to a sparse representation.
Here is a code sample for TensorFlow 2.x in eager mode:
import tensorflow as tf
indicies = [[1, 1], [1, 2], [1, 2], [1, 3]]
values = [1, 2, 3, 4]
merged_dense = tf.scatter_nd(indices, values, shape=(10, 10))
merged_sparse = tf.sparse.from_dense(merged_dense)
print(merged_sparse)
Output
SparseTensor(
indices=tf.Tensor(
[[1 1]
[1 2]
[1 3]],
shape=(3, 2),
dtype=int64),
values=tf.Tensor([1 5 4], shape=(3,), dtype=int32),
dense_shape=tf.Tensor([10 10], shape=(2,), dtype=int64))
So. As per the solution mentioned above.
Another example.
For the shape [12, 5]:
Lines to be changed in the code:
linearized = tf.matmul(indices, [[5], [1]])
indices = tf.concat([y//5, y%5], axis=1)