Suppose that I have two tensors
x = tf.constant([["a", "b"], ["c", "d"]]),
y = tf.constant([["b", "c"], ["d", "c"]])
Then, I want to get the following tensors:
x_ = [[0, 1], [1, 1]]
y_ = [[1, 0], [1, 1]]
How is x_ constructed?
The (0,1) entry in the first row of x is in the first row of y, so we set the (0,1) entry of x_ equal to 1. In addition, both the (1,0) and (1,1) entries of x are in the second row of y, so we set the (1,0) and (1,1) entries of x_ equal to 1.
How is y_ constructed?
The (0,0) entry of the first row of y is in the first row of x, so we set the (0,0) entry of y_ equal to 1. In addition, both the (1,0) and (1,1) entries of y are in the second row of x, so we set the (1,0) and (1,1) entries of y_ equal to 1.
Ugliest solution possible:
import tensorflow as tf
x = tf.constant([["a", "b"], ["c", "d"]])
y = tf.constant([["b", "c"], ["d", "c"]])
def func(x, y):
def compare(v, w):
return tf.cast(tf.equal(v, w), tf.int32)
y_rows = tf.range(tf.shape(y)[0])
x_cols = tf.range(tf.shape(x)[1])
res = tf.map_fn(lambda t: tf.map_fn(lambda z: compare(x[t,z],y[t]),
x_cols, dtype=tf.int32), y_rows, dtype=tf.int32)
res = tf.reduce_max(res, axis=-1)
return res
res1 = func(x,y)
res2 = func(y,x)
sess = tf.Session()
print(sess.run(res1))
print(sess.run(res2))
[[0 1]
[1 1]]
[[1 0]
[1 1]]
Don't know whether it has gradient. Perhaps someone will propose something better
Related
I have got a X = mx3 array of input data and y = mx1 array of desired outcome data. Now I want to normalize the features accordingly: X[:, 1] holds size of a house, X[:, 2] holds number of rooms, X[:, 0] = 1 is just a constant term. I proceeded as follows:
sigma = np.std(X[:, 0])
mean = np.mean(X[:, 0])
X[:, 0] = (X[:, 0] - mean)/sigma
sigma = np.std(X[:, 1])
mean = np.mean(X[:, 1])
X[:, 1] = (X[:, 1] - mean)/sigma
X = np.c_[np.ones((p,1)), X]
I determined the mean value and the standard deviation of each column separately, does this procedure seem valid?
Suppose I have a tensor A of shape (m, n), I would like to randomly sample k elements (without replacement) from each row, resulting in a tensor B of shape (m, k). How to do that in tensorflow?
An example would be:
A: [[1,2,3], [4,5,6], [7,8,9], [10,11,12]]
k: 2
B: [[1,3],[5,6],[9,8],[12,10]]
This is a way to do that:
import tensorflow as tf
with tf.Graph().as_default(), tf.Session() as sess:
tf.random.set_random_seed(0)
a = tf.constant([[1,2,3], [4,5,6], [7,8,9], [10,11,12]], tf.int32)
k = tf.constant(2, tf.int32)
# Tranpose, shuffle, slice, undo transpose
aT = tf.transpose(a)
aT_shuff = tf.random.shuffle(aT)
at_shuff_k = aT_shuff[:k]
result = tf.transpose(at_shuff_k)
print(sess.run(result))
# [[ 3 1]
# [ 6 4]
# [ 9 7]
# [12 10]]
My Question is for the below equation
The equation above of single vector. But if I have a batches of vectors, like my X and Y having the dimension of (None, 32), then there will some issue.
Also remember in coding environment, one example inside the batch is already in transpose shape. My problem is when we need to do transpose on [None, 32] the code will not accept and transpose for None dimenation.So I solve it in the following way:
def Cosine_similarity(X, Y, feature_dim):
L = tf.compat.v1.initializers.glorot_normal()(shape=[feature_dim, feature_dim])
out1 = tf.matmul(X, L)
out2 = tf.matmul(Y, L)
out_numerator = tf.reduce_sum(tf.multiply(out1, out2), axis = 1)
out3 = tf.reduce_sum(tf.multiply(out1, out1), axis = 1)
out3 = tf.sqrt(out3)
out4 = tf.reduce_sum(tf.multiply(out2, out2), axis = 1)
out4 = tf.sqrt(out4)
out_denominator = tf.multiply(out3, out4)
final_out = tf.divide(out_numerator, out_denominator)
return final_out
And this is coming from the following:
<XA.YA> = (XA)^T (YA)
= tf.reduce_sum(tf.multiply((X A) , (Y A)), axis = 1)
So I just to know if this implementation is right? Or you can correct me if I am missing something
Not sure I understand your concern for the (none) dimension.
If I understand correctly the cosine similarity between two identically shaped matrix X and Y ([batch, target_dim]) is just a matrix multiplication of X * Y^T with some L2 normalization. Note X would be your out1 and Y would be your out2.
def Cosine_similarity(x, y, A):
"""Pair-wise Cosine similarity.
First `x` and `y` are transformed by A.
`X = xA^T` with shape [batch, target_dim],
`Y = yA^T` with shape [batch, target_dim].
Args:
x: shaped [batch, feature_dim].
y: shaped [batch, feature_dim].
A: shaped [targte_dim, feature_dim]. Transformation matrix to project
from `feature_dim` to `target_dim`.
Returns:
A cosine similarity matrix shaped [batch, batch]. The entry
at (i, j) is the cosine similarity value between vector `X[i, :]` and
`Y[j, :]` where `X`, `Y` are the transformed `x` and y` by `A`
respectively. In the other word, entry at (i, j) is the pair-wise
cosine similarity value between the i-th example of `x` and the j-th
example of `y`.
"""
x = tf.matmul(x, A, transpose_b=True)
y = tf.matmul(y, A, transpose_b=True)
x_norm = tf.nn.l2_normalize(x, axis=-1)
y_norm = tf.nn.l2_normalize(y, axis=-1)
y_norm_trans = tf.transpose(y_norm, [1, 0])
sim = tf.matmul(x_norm, y_norm_trans)
return sim
import numpy as np
feature_dim = 8
target_dim = 4
batch_size = 2
x = tf.placeholder(tf.float32, shape=(None, dim))
y = tf.placeholder(tf.float32, shape=(None, dim))
A = tf.placeholder(tf.float32, shape=(target_dim, feature_dim))
sim = Cosine_similarity(x, y, A)
with tf.Session() as sess:
x, y, sim = sess.run([x, y, sim], feed_dict={
x: np.ones((batch_size, feature_dim)),
y: np.random.rand(batch_size, feature_dim),
A: np.random.rand(target_dim, feature_dim)})
print 'x=\n', x
print 'y=\n', y
print 'sim=\n', sim
Result:
x=
[[ 1. 1. 1. 1. 1. 1. 1. 1.]
[ 1. 1. 1. 1. 1. 1. 1. 1.]]
y=
[[ 0.01471654 0.76577073 0.97747731 0.06429122 0.91344446 0.47987637
0.09899797 0.773938 ]
[ 0.8555786 0.43403915 0.92445409 0.03393625 0.30154493 0.60895061
0.1233703 0.58597666]]
sim=
[[ 0.95917791 0.98181278]
[ 0.95917791 0.98181278]]
I've two tensor, for example x = [1,2] and y=[3], and I want replicate the last along an axis of the other, obtaining z = [[1,3],[2,3]]. Ideally in tensorflow:
x = tf.placeholder(shape=[None, 2], dtype = tf.float32)
y = tf.placeholder(shape=[1], dtype = tf.float32)
z = tf.concat(x, tf.tile(y, [ x.shape[0] ]) , 1)
The problem is that x placeholder first dimension is not determined, how can I fix this?
import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
x = tf.placeholder(shape=[None, 2], dtype = tf.float32)
y = tf.placeholder(shape=[1], dtype = tf.float32)
dim = tf.shape(x)[0]
y1 = tf.expand_dims(y, axis = 1)
y1 = tf.tile(y1, [dim, 1])
z = tf.concat((x, y1), axis = 1)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
z_val = sess.run(z, feed_dict = {x:[[2,5],[5,7],[8,9]], y:[3]})
print(z_val)
Output:
[[ 2. 5. 3.]
[ 5. 7. 3.]
[ 8. 9. 3.]]
I have two embeddings tensor A and B, which looks like
[
[1,1,1],
[1,1,1]
]
and
[
[0,0,0],
[1,1,1]
]
what I want to do is calculate the L2 distance d(A,B) element-wise.
First I did a tf.square(tf.sub(lhs, rhs)) to get
[
[1,1,1],
[0,0,0]
]
and then I want to do an element-wise reduce which returns
[
3,
0
]
but tf.reduce_sum does not allow my to reduce by row. Any inputs would be appreciated. Thanks.
Add the reduction_indices argument with a value of 1, eg.:
tf.reduce_sum( tf.square( tf.sub( lhs, rhs) ), 1 )
That should produce the result you're looking for. Here is the documentation on reduce_sum().
According to TensorFlow documentation, reduce_sum function which takes four arguments.
tf.reduce_sum(input_tensor, axis=None, keep_dims=False, name=None, reduction_indices=None).
But reduction_indices has been deprecated. Better to use axis instead of. If the axis is not set, reduces all its dimensions.
As an example,this is taken from the documentation,
# 'x' is [[1, 1, 1]
# [1, 1, 1]]
tf.reduce_sum(x) ==> 6
tf.reduce_sum(x, 0) ==> [2, 2, 2]
tf.reduce_sum(x, 1) ==> [3, 3]
tf.reduce_sum(x, 1, keep_dims=True) ==> [[3], [3]]
tf.reduce_sum(x, [0, 1]) ==> 6
Above requirement can be written in this manner,
import numpy as np
import tensorflow as tf
a = np.array([[1,7,1],[1,1,1]])
b = np.array([[0,0,0],[1,1,1]])
xtr = tf.placeholder("float", [None, 3])
xte = tf.placeholder("float", [None, 3])
pred = tf.reduce_sum(tf.square(tf.subtract(xtr, xte)),1)
# Initializing the variables
init = tf.global_variables_initializer()
# Launch the graph
with tf.Session() as sess:
sess.run(init)
nn_index = sess.run(pred, feed_dict={xtr: a, xte: b})
print nn_index