the image size is [m,32,32,3] (m = no. of training examples)
the filter size is [3,3,3,10]
stride = 1
padding = None
if I convolve this using tensorflow.nn.conv2d then the output shape should be this, according to the formula
out ={ ( 32 - 3 + 2*(0) ) / 1 }+ 1 = 30
so the output size should be [m, 30, 30, 10]
but the output shape i am getting is [m, 32, 32, 10]
why is this happening?
# convolution layer 1
c1 = tf.nn.conv2d(x_train, w1, strides = [1,1,1,1], padding = 'SAME')
print('c1 size: ', c1.shape)
# activation function for c1: relu
r1 = tf.nn.relu(c1)
# maxpooling
p1 = tf.nn.max_pool(r1, ksize = [1,2,2,1], strides = [1,2,2,1], padding = 'SAME')
padding = "SAME" means:
input = [1, 2, 3, 4, 5, 6, 7, 8]
filter size = [1, 3]
stride = [2]
so input to filter will be [[1, 2, 3], [3, 4, 5], [5, 6, 7], [7, 8, 0]]
padding = "VALID" means:
input = [1, 2, 3, 4, 5, 6, 7, 8]
filter size = [1, 3]
stride = [2]
so input to filter will be [[1, 2, 3], [3, 4, 5], [5, 6, 7]]
Last pixel got dropped in this case.
So padding "VALID" will give you the output you expect.
Related
tf.sparse_to_dense() fucntion in tensorflow only support ((data, (row_ind, col_ind)), [shape=(M, N)]) format. How can I convert standard CSR tensor (((data, indices, indptr), [shape=(M, N)])) to dense representation in tensorflow?
For example given, data, indices and indptr the function will return dense tensor.
e.g., inputs:
indices = [1 3 3 0 1 2 2 3]
indptr = [0 2 3 6 8]
data = [2 4 1 3 2 1 1 5]
expected output:
[[0, 2, 0, 4],
[0, 0, 0, 1],
[3, 2, 1, 0],
[0, 0, 1, 5]]
According to Scipy documentation, we can convert it back by the following:
the column indices for row i are stored in indices[indptr[i]:indptr[i+1]] and their
corresponding values are stored in data[indptr[i]:indptr[i+1]].
If the shape parameter is not supplied, the matrix dimensions are
inferred from the index arrays.
It is relatively easily to convert from the CSR format to the COO by expanding the indptr argument to get the row indices. Here is an example using a subtraction, tf.repeat and tf.range. The shape of the final sparse tensor is inferred from the max indices in the rows/columns respectively (but can also be provided explicitly).
def csr_to_sparse(data, indices, indptr, dense_shape=None):
rep = tf.math.subtract(indptr[1:], indptr[:-1])
row_indices = tf.repeat(tf.range(tf.size(rep)), rep)
sparse_indices = tf.cast(tf.stack((row_indices, indices), axis=-1), tf.int64)
if dense_shape is None:
max_row = tf.math.reduce_max(row_indices)
max_col = tf.math.reduce_max(indices)
dense_shape = (max_row + 1, max_col + 1)
return tf.SparseTensor(indices=sparse_indices, values=data, dense_shape=dense_shape)
With your example:
>>> indices = [1, 3, 3, 0, 1, 2, 2, 3]
>>> indptr = [0, 2, 3, 6, 8,]
>>> data = [2, 4, 1, 3, 2, 1, 1, 5]
>>> tf.sparse.to_dense(csr_to_sparse(data, indices, indptr))
<tf.Tensor: shape=(4, 4), dtype=int32, numpy=
array([[0, 2, 0, 4],
[0, 0, 0, 1],
[3, 2, 1, 0],
[0, 0, 1, 5]], dtype=int32)>
Say v1 and v2 has the same shape. Is it possible in tensorflow to concat v1 and the transposed version of v2 using the broadcast semantic?
For example,
v1 = tf.constant([[1,1,1,1],[3,3,3,3],[5,5,5,5]])
v2 = tf.constant([[2,2,2,2],[4,4,4,4]])
I want to produce something like
[
[[[1,1,1,1], [2,2,2,2]],
[[1,1,1,1], [4,4,4,4]]],
[[[3,3,3,3], [2,2,2,2]],
[[3,3,3,3], [4,4,4,4]]],
[[[5,5,5,5], [2,2,2,2]],
[[5,5,5,5], [4,4,4,4]]]]
that is, with v1 as [3, 4] and v2 as [2,4], I want to do
tf.concat([v1, tf.transpose(v2)], axis=0)
and produce a [3,2,2,4] matrix.
Is there any trick for doing that?
If you mean by trick an elegant solution, I don't think so. However, a working solution would be to tile and repeat the incoming v1, v2
import tensorflow as tf
v1 = tf.constant([[1, 1, 1, 1],
[3, 3, 3, 3],
[7, 7, 7, 7],
[5, 5, 5, 5]])
v2 = tf.constant([[2, 2, 2, 2],
[6, 6, 6, 6],
[4, 4, 4, 4]])
def my_concat(v1, v2):
v1_m, v1_n = v1.shape.as_list()
v2_m, v2_n = v2.shape.as_list()
v1 = tf.concat([v1 for i in range(v2_m)], axis=-1)
v1 = tf.reshape(v1, [v2_m * v1_m, -1])
v2 = tf.tile(v2, [v1_m, 1])
v1v2 = tf.concat([v1, v2], axis=-1)
return tf.reshape(v1v2, [v1_m, v2_m, 2, v2_n])
with tf.Session() as sess:
ret = sess.run(my_concat(v1, v2))
print ret.shape
print ret
Here's my attempt to add two more elegant solutions to this Cartesian Product problem as follows (both tested); first one using tf.map_fn():
import tensorflow as tf
v1 = tf.constant([[1, 1, 1, 1],
[3, 3, 3, 3],
[5, 5, 5, 5]])
v2 = tf.constant([[2, 2, 2, 2],
[4, 4, 4, 4]])
cartesian_product = tf.map_fn( lambda x: tf.map_fn( lambda y: tf.stack( [ x, y ] ), v2 ), v1 )
with tf.Session() as sess:
print( sess.run( cartesian_product ) )
or this one taking advantage of the implicit broadcasting of add:
import tensorflow as tf
v1 = tf.constant([[1, 1, 1, 1],
[3, 3, 3, 3],
[5, 5, 5, 5]])
v2 = tf.constant([[2, 2, 2, 2],
[4, 4, 4, 4]])
v1, v2 = v1[ :, None, None, : ], v2[ None, :, None, : ]
cartesian_product = tf.concat( [ v1 + tf.zeros_like( v2 ),
tf.zeros_like( v1 ) + v2 ], axis = 2 )
with tf.Session() as sess:
print( sess.run( cartesian_product ) )
both output:
[[[[1 1 1 1]
[2 2 2 2]]
[[1 1 1 1]
[4 4 4 4]]]
[[[3 3 3 3]
[2 2 2 2]]
[[3 3 3 3]
[4 4 4 4]]]
[[[5 5 5 5]
[2 2 2 2]]
[[5 5 5 5]
[4 4 4 4]]]]
as desired.
I have tensor T of shape [batch_size, A] with values and tensor S of shape [batch_size] with shift parameters.
I would like to shift values in T[b] by S[b] positions to the right, the last S[b] elements of T[b] should be dropped and new elements should be set to 0.
So basically want to do something like:
for i in range(batch_size):
T[i] = zeros[:S[i]] + T[i, :A-S[i]]
Example:
For:
T = [[1, 2, 3], [4, 5, 6]]
S = [1, 2]
Return:
T' = [[0, 1, 2], [0, 0, 4]]
Is there some easy way to do it?
You can use tf.concat and tf.stack for that purpose:
T_shift = tf.zeros((batch_size, A), tf.float32)
tmp = []
for i in xrange(batch_size):
tmp.append(tf.concat([T_shift[i, :S[i, 0]],T[i, :17 - S[i,0]]], axis = 0))
T_shift = tf.stack(tmp)
If you are working in Tensorflow 2, you can use the tf.roll for that purpose:
"The elements are shifted positively (towards larger indices) by the
offset of shift along the dimension of axis. Negative shift values
will shift elements in the opposite direction. Elements that roll
passed the last position will wrap around to the first and vice versa.
Multiple shifts along multiple axes may be specified."
tf.roll(
input, shift, axis, name=None
)
# 't' is [0, 1, 2, 3, 4]
roll(t, shift=2, axis=0) ==> [3, 4, 0, 1, 2]
# shifting along multiple dimensions
# 't' is [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]
roll(t, shift=[1, -2], axis=[0, 1]) ==> [[7, 8, 9, 5, 6], [2, 3, 4, 0, 1]]
# shifting along the same axis multiple times
# 't' is [[0, 1, 2, 3, 4], [5, 6, 7, 8, 9]]
roll(t, shift=[2, -3], axis=[1, 1]) ==> [[1, 2, 3, 4, 0], [6, 7, 8, 9, 5]]
I was using dynamic_rnn with an LSTMCell, which put out an LSTMStateTuple containing the inner state. Calling reshape on this object (by my mistake) results in a tensor without causing any error at graph creation. I didn't get any error at runtime when feeding input through the graph, either.
Code:
cell = tf.contrib.rnn.LSTMCell(size, state_is_tuple=True, ...)
outputs, states = tf.nn.dynamic_rnn(cell, inputs, ...)
print(states) # state is an LSTMStateTuple
states = tf.reshape(states, [-1, size])
print(states) # state is a tensor of shape [?, size]
Is this a bug (I ask because it's not documented anywhere)? What is the reshaped tensor holding?
I have conducted a similar experiment which may gives you some hints:
>>> s = tf.constant([[0, 0, 0, 1, 1, 1],
[2, 2, 2, 3, 3, 3]])
>>> t = tf.constant([[4, 4, 4, 5, 5, 5],
[6, 6, 6, 7, 7, 7]])
>>> g = tf.reshape((s, t), [-1, 3]) # <tf.Tensor 'Reshape_1:0' shape=(8, 3) dtype=int32>
>>> sess.run(g)
array([[0, 0, 0],
[1, 1, 1],
[2, 2, 2],
[3, 3, 3],
[4, 4, 4],
[5, 5, 5],
[6, 6, 6],
[7, 7, 7]], dtype=int32)
We can see that it just concatenates the two tensors in the first dimension and performs the reshaping. Since the LSTMStateTuple is like a namedtuple then it has the same effect as tuple and I think this is also what happens in your case.
Let's go further,
>>> st = tf.contrib.rnn.LSTMStateTuple(s, t)
>>> gg = tf.reshape(st, [-1, 3])
>>> sess.run(gg)
array([[0, 0, 0],
[1, 1, 1],
[2, 2, 2],
[3, 3, 3],
[4, 4, 4],
[5, 5, 5],
[6, 6, 6],
[7, 7, 7]], dtype=int32)
We can see that if we create a LSTMStateTuple, the result verifies our assumption.
I want to get the min value index of a tensor, but the value is not 0.
a = np.array([[0, 3, 9, 0],
[0, 0, 5, 7]])
tensor_a = tf.constant(a, dtype=tf.int32)
max_index = tf.argmax(tensor_a, axis=1)
The above code defined a constant tensor, if I use tf.argmax, I will get the index [2, 3]. How could I get the index of 3 in row one and 5 in row two, the min value but not zero. The true indexes I want to get is [1, 2].
How to implement it in tensorflow, Thanks.
It's hideous, but it works:
with tf.Session() as sess:
a = np.array([[0, 3, 9, 0],
[0, 0, 5, 7]])
tensor_a = tf.constant(a, dtype=tf.int64)
row_max = tf.reshape(tf.reduce_max(a, axis=-1), [-1, 1]) + 1
max_index = tf.argmin(tf.where(tensor_a > 0, tensor_a, row_max * tf.ones_like(tensor_a)), axis=1)
print(max_index.eval()) # -> [1 2]