What I want to do
M = tf.concat([tensor]*N, axix = 0)
But now, N is a tensor that decided in run time.
other_tensor = tf.placeholder(dtype=tf.int32, shape=[None, 2])
N = tf.shape(other_tensor)[0] # N is None, and it is decided in run time.
So, how to do this?
You should use tf.tile, not concat. To get the shape, use tensor.get_shape Here is an example:
import tensorflow as tf
a = tf.constant([[1, 2], [3, 4]])
b = tf.constant([1, 2])
c = tf.tile(a, (1, int(a.get_shape()[0])))
with tf.Session() as sess:
print sess.run(c)
If you need your tensor to have a slightly different shape, read about the second parameter in tile function and also use tf.reshape
Related
I have two tensors of different sizes and want to write a custom merge function
a = tf.constant([[1,2,3]])
b = tf.constant([[1,1,2,2,3,3]])
I want to take the dot product of each point in tensor a with two points in tensor b. So in the example above element 1 in a is multiplied with the first two elements in b and so on. I'm unsure of how to do loops in tensorflow:
def customMergeFunct(x):
# not sure how to write a loop over a tensor
The output should be:
c = Lambda(customMergeFunct)([a,b])
with tf.Session() as sess:
print(c.eval())
=> [[2,8,18]]
I'm not exactly sure why you call this a merge function. You don't really need to define a custom function. You can do this with a simple lambda function. Here's my solution.
import tensorflow as tf
from tensorflow.keras.layers import Lambda
import tensorflow.keras.backend as K
a = tf.constant([[1,2,3]])
b = tf.constant([[1,1,2,2,3,3]])
a_res = tf.reshape(a,[-1,1]) # make a.shape [3,1]
b_res = tf.reshape(b,[-1,2]) # make b.shape [3,2]
layer = Lambda(lambda x: K.sum(x[0]*x[1],axis=1))
res = layer([a_res,b_res])
with tf.Session() as sess:
print(res.eval())
You can do something following:
a = tf.constant([[1,2,3]]) # Shape: (1, 3)
b = tf.constant([[1,1,2,2,3,3]]) # Shape: (1, 6)
def customMergeFunct(x):
# a_ = tf.tile(x[0], [2, 1]) # Duplicating 2 times (Original) # Update: No need of doing this as tf.multiply will use Broadcasting
b_ = tf.transpose(tf.reshape(x[1], [-1, 2])) # reshaping followed by transpose to make a shape of (2, 3) to serve the purpose + multiplication rule
return tf.reduce_sum(tf.multiply(x[0], b_), axis=0) # Element-wise multiplication followed by sum
# Using function
c = Lambda(customMergeFunct)([a,b])
# OR in a reduced form
c = Lambda(lambda x: tf.reduce_sum(tf.multiply(x[0], tf.transpose(tf.reshape(x[1], [-1, 2]))), axis=0))([a,b])
Output:
with tf.Session() as sess:
print(c.eval()) # Output: [2 8 18]
# OR in eager mode
print(c.numpy()) # Output: [2 8 18]
Updated solution is computationally efficient than the original solution as we don't actually need to apply tile on x[0]
What is the difference between SparseTensor and SparseTensorValue? Is there anything I should keep in mind if I want to build the sparse tensor based on fed indices and values? I could only find a few toy examples.
It depends on where you define your Sparse Tensor.
If you would like to define the tensor outside the graph, e.g. define the sparse tensor for later data feed, use SparseTensorValue. In contrast, if the sparse tensor is defined in graph, use SparseTensor
Sample code for tf.SparseTensorValue:
x_sp = tf.sparse_placeholder(dtype=tf.float32)
W = tf.Variable(tf.random_normal([6, 6]))
y = tf.sparse_tensor_dense_matmul(sp_a=x_sp, b=W)
init = tf.global_variables_initializer()
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.2)
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))
sess.run(init)
stv = tf.SparseTensorValue(indices=[[0, 0], [1, 2]], values=[1.1, 1.2],
dense_shape=[2,6])
result = sess.run(y,feed_dict={x_sp:stv})
print(result)
Sample code for tf.SparseTensor:
indices_i = tf.placeholder(dtype=tf.int64, shape=[2, 2])
values_i = tf.placeholder(dtype=tf.float32, shape=[2])
dense_shape_i = tf.placeholder(dtype=tf.int64, shape=[2])
st = tf.SparseTensor(indices=indices_i, values=values_i, dense_shape=dense_shape_i)
W = tf.Variable(tf.random_normal([6, 6]))
y = tf.sparse_tensor_dense_matmul(sp_a=st, b=W)
init = tf.global_variables_initializer()
gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.2)
sess = tf.Session(config=tf.ConfigProto(gpu_options=gpu_options))
sess.run(init)
result = sess.run(y,feed_dict={indices_i:[[0, 0], [1, 2]], values_i:[1.1, 1.2], dense_shape_i:[2,6]})
print(result)
Hope this help~
Now i have a tensor with shape (3*2, 2) look like
and i want to reshape it with the shape (3, 2*2) with the specific axis like following:
what should i do? The default tf.reshape() will reshape it to
SOLUTION： I found that use slice in tensorflow and tf.concat() can solve the problem.You can slice sub-tensors and concat them which solve my problem exactly
I tried the following code and got the result that you need. But not sure whether the number of steps can be reduced.
import tensorflow as tf
x = [[1, 2],
[3,4],
[5,6],
[7,8],
[9,10],
[11,12]]
a = tf.reshape(x,[-1,6])
b = tf.split(a,3, 1)
c = tf.reshape(b,[-1,4])
X=tf.placeholder(tf.float32, shape=[6, 2], name='input')
with tf.Session() as sess:
c = sess.run(c, feed_dict={X: x})
print(c)
Hope this helps.
As generalized slicing is being worked on in this issue, what would be the best way to achieve an op gathering columns of a 2D tensor (matrix)? For example, for tensor t:
1 2 3 4
5 6 7 8
and indices [1,3], I would like to get:
2 4
6 8
which is equivalent to numpy t[:, [1,3]].
Meanwhile the gather method has an axis parameter.
import tensorflow as tf
params = tf.constant([[1,2,3],[4,5,6]])
indices = [0,2]
op = tf.gather(params, indices, axis=1)
produces the output
[[1 3]
[4 6]]
There is a function named tf.nn.embedding_lookup(params, ind) which retrieves the rows of the params tensor.
To achieve what you want, we can first transpose the tensor t from which you want to select certain columns from. Then look up the rows of tf.transpose(t) (columns of t). After the selection, we transpose the result back.
import tensorflow as tf
t = tf.constant([[1, 2, 3],
[4, 5, 6]])
ind = tf.constant([0, 2])
result = tf.transpose(tf.nn.embedding_lookup(tf.transpose(t), ind))
with tf.Session() as sess:
print(sess.run(result))
So far, I created a workaround by flattening the input and using gather:
def gather_cols(params, indices, name=None):
"""Gather columns of a 2D tensor.
Args:
params: A 2D tensor.
indices: A 1D tensor. Must be one of the following types: ``int32``, ``int64``.
name: A name for the operation (optional).
Returns:
A 2D Tensor. Has the same type as ``params``.
"""
with tf.op_scope([params, indices], name, "gather_cols") as scope:
# Check input
params = tf.convert_to_tensor(params, name="params")
indices = tf.convert_to_tensor(indices, name="indices")
try:
params.get_shape().assert_has_rank(2)
except ValueError:
raise ValueError('\'params\' must be 2D.')
try:
indices.get_shape().assert_has_rank(1)
except ValueError:
raise ValueError('\'indices\' must be 1D.')
# Define op
p_shape = tf.shape(params)
p_flat = tf.reshape(params, [-1])
i_flat = tf.reshape(tf.reshape(tf.range(0, p_shape[0]) * p_shape[1],
[-1, 1]) + indices, [-1])
return tf.reshape(tf.gather(p_flat, i_flat),
[p_shape[0], -1])
Which for:
params = tf.constant([[1, 2, 3],
[4, 5, 6]])
indices = [0, 2]
op = gather_cols(params, indices)
produces the expected output:
[[1 3]
[4 6]]
I have an input to tensorflow of shape [None, 9, 2] (where the None is batch).
To perform further actions (e.g. matmul) on it I need to transform it to [None, 18] shape. How to do it?
You can do it easily with tf.reshape() without knowing the batch size.
x = tf.placeholder(tf.float32, shape=[None, 9,2])
shape = x.get_shape().as_list() # a list: [None, 9, 2]
dim = numpy.prod(shape[1:]) # dim = prod(9,2) = 18
x2 = tf.reshape(x, [-1, dim]) # -1 means "all"
The -1 in the last line means the whole column no matter what the batchsize is in the runtime. You can see it in tf.reshape().
Update: shape = [None, 3, None]
Thanks #kbrose. For the cases where more than 1 dimension are undefined, we can use tf.shape() with tf.reduce_prod() alternatively.
x = tf.placeholder(tf.float32, shape=[None, 3, None])
dim = tf.reduce_prod(tf.shape(x)[1:])
x2 = tf.reshape(x, [-1, dim])
tf.shape() returns a shape Tensor which can be evaluated in runtime. The difference between tf.get_shape() and tf.shape() can be seen in the doc.
I also tried tf.contrib.layers.flatten() in another . It is simplest for the first case, but it can't handle the second.
flat_inputs = tf.layers.flatten(inputs)
You can use dynamic reshaping to get value of batch dimension through tf.batch during runtime, calculate the whole set of new dimensions into tf.reshape. Here's an example of reshaping flat list into square matrix without knowing list length.
tf.reset_default_graph()
sess = tf.InteractiveSession("")
a = tf.placeholder(dtype=tf.int32)
# get [9]
ashape = tf.shape(a)
# slice the list from 0th to 1st position
ashape0 = tf.slice(ashape, [0], [1])
# reshape list to scalar, ie from [9] to 9
ashape0_flat = tf.reshape(ashape0, ())
# tf.sqrt doesn't support int, so cast to float
ashape0_flat_float = tf.to_float(ashape0_flat)
newshape0 = tf.sqrt(ashape0_flat_float)
# convert [3, 3] Python list into [3, 3] Tensor
newshape = tf.pack([newshape0, newshape0])
# tf.reshape doesn't accept float, so convert back to int
newshape_int = tf.to_int32(newshape)
a_reshaped = tf.reshape(a, newshape_int)
sess.run(a_reshaped, feed_dict={a: np.ones((9))})
You should see
array([[1, 1, 1],
[1, 1, 1],
[1, 1, 1]], dtype=int32)