enter image description hereI worked on my problem and still I am overwhelmed with many functions. I am looking to use tf.top_k and return to the first image. Could somebody familiar with tensorflow help me to solve this problem?
Question in detail: image (4 by 4) as a tensor --> tf.top_k --> 4 values(2 by 2) and 4 indices(2 by 2) --> snippet --> image (4, 4) as a tensor
For example imagine that we have one image
image = np.array([[1, 2, 3, 4],
[7, 8, 9, 10],
[19, 20, 21, 22],
[25, 26, 27, 28]])
x = tf.placeholder(tf.float32, [None, img_height, img_width, 1], name='x')
patches = tf.extract_image_patches(x, [1, 2, 2, 1], [1, 2, 2, 1], [1, 1, 1, 1], "SAME")
ktop, indices = tf.nn.top_k(patches, k=4, sorted=True, name=None)
Now I am looking to give ktop and indices to get image again like the first state
image = np.array([[1, 2, 3, 4],
[7, 8, 9, 10],
[19, 20, 21, 22],
[25, 26, 27, 28]])
I tried many things such as tf.one_hot, tf.gatter, and so on but I could not get a (4, 4) image. I should have a tensor in input and output and it seems that I can not use numpy or for loops. I asked similar question before but I ask here again with more clear explanation.
Could you please somebody help to solve this question. It should be easy for others but for me its difficult.
This snippet gave me the first image but not as (1, 4, 4, 1) and instead it give as (2, 2, 2, 2) or other shapes.
z1 = tf.assign(z1, tf.reshape(tf.gather(ktop[0, 0, 0, :], [indices[0, 0, 0, :]]), [2, 2]))
z2 = tf.assign(z2, tf.reshape(tf.gather(ktop[0, 0, 1, :], [indices[0, 0, 1, :]]), [2, 2]))
z3 = tf.assign(z3, tf.reshape(tf.gather(ktop[0, 1, 0, :], [indices[0, 1, 0, :]]), [2, 2]))
z4 = tf.assign(z4, tf.reshape(tf.gather(ktop[0, 1, 1, :], [indices[0, 1, 1, :]]), [2, 2]))
z = tf.concat([[z1, z2], [z3, z4]], 0)
In the absence of a clear explanation of how you want to go from the 2x2x4 tensors to the 4x4x1 I can only suggest this:
img_out = tf.reshape(ktop, [-1, 4, 4, 1])
Finally I found the answer which TENSORFLOW followers did not replay so far. For returning from top_k to image we should use depth_to_space function:
output_image = tf.depth_to_space(
output_image,
2,
name=None,
data_format='NHWC'
)
Related
I have tensor T of shape [batch_size, A] with values and tensor S of shape [batch_size] with shift parameters.
I would like to shift values in T[b] by S[b] positions to the right, the last S[b] elements of T[b] should be dropped and new elements should be set to 0.
So basically want to do something like:
for i in range(batch_size):
T[i] = zeros[:S[i]] + T[i, :A-S[i]]
Example:
For:
T = [[1, 2, 3], [4, 5, 6]]
S = [1, 2]
Return:
T' = [[0, 1, 2], [0, 0, 4]]
Is there some easy way to do it?
You can use tf.concat and tf.stack for that purpose:
T_shift = tf.zeros((batch_size, A), tf.float32)
tmp = []
for i in xrange(batch_size):
tmp.append(tf.concat([T_shift[i, :S[i, 0]],T[i, :17 - S[i,0]]], axis = 0))
T_shift = tf.stack(tmp)
If you are working in Tensorflow 2, you can use the tf.roll for that purpose:
"The elements are shifted positively (towards larger indices) by the
offset of shift along the dimension of axis. Negative shift values
will shift elements in the opposite direction. Elements that roll
passed the last position will wrap around to the first and vice versa.
Multiple shifts along multiple axes may be specified."
tf.roll(
input, shift, axis, name=None
)
# 't' is [0, 1, 2, 3, 4]
roll(t, shift=2, axis=0) ==> [3, 4, 0, 1, 2]
# shifting along multiple dimensions
# 't' is [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]
roll(t, shift=[1, -2], axis=[0, 1]) ==> [[7, 8, 9, 5, 6], [2, 3, 4, 0, 1]]
# shifting along the same axis multiple times
# 't' is [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]
roll(t, shift=[2, -3], axis=[1, 1]) ==> [[1, 2, 3, 4, 0], [6, 7, 8, 9, 5]]
In NumPy, suppose I have a matrix X:
X = array([[3, 1, 4, 5], [5, 1, 2, 1], [4, 4, 0, 1], [0, 3, 0, 3], [1, 2, 3, 4])
How can I construct a new matrix using the first (row 0), last second and last (row 3, 4) of X?
The resulting matrix is:
Y = array([[3, 1, 4, 5], [0, 3, 0, 3], [1, 2, 3, 4])
I cannot list all the rows I want to include for the new matrix because for the data I have, it will be like choosing the (20, 60), (90, 120) row of the original matrix to construct a new matrix.
Use np.r_ to get those concatenated row indices and simply index into the rows of the input array, like so -
X[np.r_[0, 3:5]] # for sample case
X[np.r_[20:60, 90:120]] # for actual case
Sample run -
In [146]: X
Out[146]:
array([[3, 1, 4, 5],
[5, 1, 2, 1],
[4, 4, 0, 1],
[0, 3, 0, 3],
[1, 2, 3, 4]])
In [147]: X[np.r_[0, 3:5]]
Out[147]:
array([[3, 1, 4, 5],
[0, 3, 0, 3],
[1, 2, 3, 4]])
Sample run for shape test on a bigger random array -
In [150]: X = np.random.rand(200,10)
In [151]: X[np.r_[20:60, 90:120]].shape
Out[151]: (70, 10) # 70 rows selected
I was using dynamic_rnn with an LSTMCell, which put out an LSTMStateTuple containing the inner state. Calling reshape on this object (by my mistake) results in a tensor without causing any error at graph creation. I didn't get any error at runtime when feeding input through the graph, either.
Code:
cell = tf.contrib.rnn.LSTMCell(size, state_is_tuple=True, ...)
outputs, states = tf.nn.dynamic_rnn(cell, inputs, ...)
print(states) # state is an LSTMStateTuple
states = tf.reshape(states, [-1, size])
print(states) # state is a tensor of shape [?, size]
Is this a bug (I ask because it's not documented anywhere)? What is the reshaped tensor holding?
I have conducted a similar experiment which may gives you some hints:
>>> s = tf.constant([[0, 0, 0, 1, 1, 1],
[2, 2, 2, 3, 3, 3]])
>>> t = tf.constant([[4, 4, 4, 5, 5, 5],
[6, 6, 6, 7, 7, 7]])
>>> g = tf.reshape((s, t), [-1, 3]) # <tf.Tensor 'Reshape_1:0' shape=(8, 3) dtype=int32>
>>> sess.run(g)
array([[0, 0, 0],
[1, 1, 1],
[2, 2, 2],
[3, 3, 3],
[4, 4, 4],
[5, 5, 5],
[6, 6, 6],
[7, 7, 7]], dtype=int32)
We can see that it just concatenates the two tensors in the first dimension and performs the reshaping. Since the LSTMStateTuple is like a namedtuple then it has the same effect as tuple and I think this is also what happens in your case.
Let's go further,
>>> st = tf.contrib.rnn.LSTMStateTuple(s, t)
>>> gg = tf.reshape(st, [-1, 3])
>>> sess.run(gg)
array([[0, 0, 0],
[1, 1, 1],
[2, 2, 2],
[3, 3, 3],
[4, 4, 4],
[5, 5, 5],
[6, 6, 6],
[7, 7, 7]], dtype=int32)
We can see that if we create a LSTMStateTuple, the result verifies our assumption.
We know that tf.multiply can broadcast like this:
import tensorflow as tf
import numpy as np
a = tf.Variable(np.arange(12).reshape(3, 4))
b = tf.Variable(np.arange(4))
sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())
sess.run(tf.multiply(a, b))
This will give us
[[0, 1, 4, 9],
[0, 5, 12, 21],
[0, 9, 20, 33]]
But my question is, what should I do if both a and b are in batches? That is,
a = tf.Variable(np.arange(24).reshape(2, 3, 4))
b = tf.Variable(np.arange(8).reshape(2, 4))
Then how can I get the result of multiplying (broadcasting) the vector onto the matrix in each batch? Like the following answer:
[[[0, 1, 4, 9],
[0, 5, 12, 21],
[0, 9, 20, 33]],
[[48, 65, 84, 105],
[64, 85, 108, 133],
[80, 105, 132, 161]]]
Thanks!
Broadcasting first adds singleton dimensions to the left until rank is matched. In first case that adds batch dimension. But in second case you already have batch dimension so you need to insert singleton dimension manually in the second position:
a = tf.reshape(tf.range(24), (2, 3, 4))
b = tf.reshape(tf.range(8), (2, 4))
sess.run(tf.mul(a, tf.expand_dims(b, 1)))
I have a 2-d numpy array as follows:
a = np.array([[1,5,9,13],
[2,6,10,14],
[3,7,11,15],
[4,8,12,16]]
I want to extract it into patches of 2 by 2 sizes with out repeating the elements.
The answer should exactly be the same. This can be 3-d array or list with the same order of elements as below:
[[[1,5],
[2,6]],
[[3,7],
[4,8]],
[[9,13],
[10,14]],
[[11,15],
[12,16]]]
How can do it easily?
In my real problem the size of a is (36, 72). I can not do it one by one. I want programmatic way of doing it.
Using scikit-image:
import numpy as np
from skimage.util import view_as_blocks
a = np.array([[1,5,9,13],
[2,6,10,14],
[3,7,11,15],
[4,8,12,16]])
print(view_as_blocks(a, (2, 2)))
You can achieve it with a combination of np.reshape and np.swapaxes like so -
def extract_blocks(a, blocksize, keep_as_view=False):
M,N = a.shape
b0, b1 = blocksize
if keep_as_view==0:
return a.reshape(M//b0,b0,N//b1,b1).swapaxes(1,2).reshape(-1,b0,b1)
else:
return a.reshape(M//b0,b0,N//b1,b1).swapaxes(1,2)
As can be seen there are two ways to use it - With keep_as_view flag turned off (default one) or on. With keep_as_view = False, we are reshaping the swapped-axes to a final output of 3D, while with keep_as_view = True, we will keep it 4D and that will be a view into the input array and hence, virtually free on runtime. We will verify it with a sample case run later on.
Sample cases
Let's use a sample input array, like so -
In [94]: a
Out[94]:
array([[2, 2, 6, 1, 3, 6],
[1, 0, 1, 0, 0, 3],
[4, 0, 0, 4, 1, 7],
[3, 2, 4, 7, 2, 4],
[8, 0, 7, 3, 4, 6],
[1, 5, 6, 2, 1, 8]])
Now, let's use some block-sizes for testing. Let's use a blocksize of (2,3) with the view-flag turned off and on -
In [95]: extract_blocks(a, (2,3)) # Blocksize : (2,3)
Out[95]:
array([[[2, 2, 6],
[1, 0, 1]],
[[1, 3, 6],
[0, 0, 3]],
[[4, 0, 0],
[3, 2, 4]],
[[4, 1, 7],
[7, 2, 4]],
[[8, 0, 7],
[1, 5, 6]],
[[3, 4, 6],
[2, 1, 8]]])
In [48]: extract_blocks(a, (2,3), keep_as_view=True)
Out[48]:
array([[[[2, 2, 6],
[1, 0, 1]],
[[1, 3, 6],
[0, 0, 3]]],
[[[4, 0, 0],
[3, 2, 4]],
[[4, 1, 7],
[7, 2, 4]]],
[[[8, 0, 7],
[1, 5, 6]],
[[3, 4, 6],
[2, 1, 8]]]])
Verify view with keep_as_view=True
In [20]: np.shares_memory(a, extract_blocks(a, (2,3), keep_as_view=True))
Out[20]: True
Let's check out performance on a large array and verify the virtually free runtime claim as discussed earlier -
In [42]: a = np.random.rand(2000,3000)
In [43]: %timeit extract_blocks(a, (2,3), keep_as_view=True)
1000000 loops, best of 3: 801 ns per loop
In [44]: %timeit extract_blocks(a, (2,3), keep_as_view=False)
10 loops, best of 3: 29.1 ms per loop
Here's a rather cryptic numpy one-liner to generate your 3-d array, called result1 here:
In [60]: x
Out[60]:
array([[2, 1, 2, 2, 0, 2, 2, 1, 3, 2],
[3, 1, 2, 1, 0, 1, 2, 3, 1, 0],
[2, 0, 3, 1, 3, 2, 1, 0, 0, 0],
[0, 1, 3, 3, 2, 0, 3, 2, 0, 3],
[0, 1, 0, 3, 1, 3, 0, 0, 0, 2],
[1, 1, 2, 2, 3, 2, 1, 0, 0, 3],
[2, 1, 0, 3, 2, 2, 2, 2, 1, 2],
[0, 3, 3, 3, 1, 0, 2, 0, 2, 1]])
In [61]: result1 = x.reshape(x.shape[0]//2, 2, x.shape[1]//2, 2).swapaxes(1, 2).reshape(-1, 2, 2)
result1 is like a 1-d array of 2-d arrays:
In [68]: result1.shape
Out[68]: (20, 2, 2)
In [69]: result1[0]
Out[69]:
array([[2, 1],
[3, 1]])
In [70]: result1[1]
Out[70]:
array([[2, 2],
[2, 1]])
In [71]: result1[5]
Out[71]:
array([[2, 0],
[0, 1]])
In [72]: result1[-1]
Out[72]:
array([[1, 2],
[2, 1]])
(Sorry, I don't have time at the moment to give a detailed breakdown of how it works. Maybe later...)
Here's a less cryptic version that uses a nested list comprehension. In this case, result2 is a python list of 2-d numpy arrays:
In [73]: result2 = [x[2*j:2*j+2, 2*k:2*k+2] for j in range(x.shape[0]//2) for k in range(x.shape[1]//2)]
In [74]: result2[5]
Out[74]:
array([[2, 0],
[0, 1]])
In [75]: result2[-1]
Out[75]:
array([[1, 2],
[2, 1]])