I have a rank 3 tensor of shape (100, 257, 121) that we'll call y_pred.
I have extracted one rank 2 tensor from this of shape (257, 121) that we'll call y_element.
Is there a method similar to tensorflow.not_equal() that will compare y_element to every other rank 2 tensor element along axis 0 of y_pred, and return a tensor of bools of shape (100)?
Calling tensorflow.not_equal(y_pred, y_element) does return a tensor of bools, but of the same shape as y_pred, suggesting it's doing something like comparing the y_element tensor to all 3109700 values in y_pred instead.
y_pred = tf.Variable(tf.ones((100, 257, 121)))
y_element = tf.Variable(tf.ones((257, 121)))
y_element[-1,:].assign(tf.zeros(121))
tf.reduce_all(tf.equal(y_pred, tf.expand_dims(y_element,0)), axis=[1,2])
this piece of code is based on the method you introduced and it compares element-wise the 2 tensors on the axis zeros. it returns an array of shape equal to the first axis (100 in our case). every element of the tensor of rank 3 is compared with the tensor of rank 2. True is return if ALL the tensor are equal otherwise false
Related
My codes are as follow:
v = tf.Variable(initial_value=v, trainable=True)
v.shape is (1, 768)
In the model:
inputs_sents = keras.Input(shape=(50,3))
inputs_events = keras.Input(shape=(50,768))
x_1 = tf.matmul(v,tf.transpose(inputs_events))
x_2 = tf.matmul(x_1,inputs_sents)
But I got an error,
ValueError: Dimensions must be equal, but are 768 and 50 for
'{{node BatchMatMulV2_3}} =
BatchMatMulV2[T=DT_FLOAT,
adj_x=false,
adj_y=false](BatchMatMulV2_3/ReadVariableOp,
Transpose_3)' with input shapes: [1,768], [768,50,?]
I think it takes consideration of the batch? But how shall I deal with this?
v is a trainable vector (or 2d array with first dimension being 1), I want it to be trained in the training process.
PS: This is the result I got using the codes provided by the first answer, I think it is incorrect cause keras already takes consideration of the first batch dimension.
Plus, from the keras documentation,
shape: A shape tuple (integers), not including the batch size. For instance, shape=(32,) indicates that the expected input will be batches of 32-dimensional vectors. Elements of this tuple can be None; 'None' elements represent dimensions where the shape is not known.
https://keras.io/api/layers/core_layers/input/
Should I rewrite my codes without keras?
The shape of a batch is denoted by None:
import numpy as np
inputs_sents = keras.Input(shape=(None,1,3))
inputs_events = keras.Input(shape=(None,1,768))
v = np.ones(shape=(1,768), dtype=np.float32)
v = tf.Variable(initial_value=v, trainable=True)
x_1 = tf.matmul(v,tf.transpose(inputs_events))
x_2 = tf.matmul(x_1,inputs_sents)
I would like to multiply a sparse tensor by a dense tensor but do so within a batch.
For example I have a sparse tensor with the corresponding dense shape of (20,65536,65536) where 20 is the batch size. I would like to multiply each (65536,65536) in the batch with the corresponding (65536x1) from a tensor shape (20,65536) which has a dense representation. tf.sparse_tensor_dense_matmul only accepts a rank 2 sparse tensor. Is there a way to perform this over a batch?
I would like to avoid converting the sparse matrix to a dense matrix if possible due to memory constraints.
Assuming that a is a sparse tensor with shape (20, 65536, 65536) and b a dense tensor with shape (20, 65536), you could perform the batch sparse-dense matrix multiplication as follows:
y_sparse = tf.sparse.reduce_sum_sparse(a * b[:, None, :], axis=-1)
This solution expands the second dimension of tensor b to enable implicit broadcasting. Then, the batch matrix multiplication takes place by performing a sparse-dense multiplication and a sparse sum along the last axis.
If b has got a third dimension so it is a batch of matrices, you can multiply their columns individually and concatenate them later:
multiplied_dims = []
for i in range (b.shape[-1]):
multiplied_dims.append(tf.expand_dims(tf.sparse.reduce_sum(a * b[:, :, i][:, None, :], axis=-1), -1))
result = tf.concat(multiplied_dims, -1)
The answer is simple - you reshape the sparse tensor first and then multiply it by the dense matrix. Something like this would work:
sparse_tensor_rank2 = tf.sparse_reshape(sparse_tensor, [-1, 65536])
I would like to choose only particular columns in each row of a tensor, using it for an RNN
seq_len=[11,12,20,30] #This is the sequence length, assume 4 sequences
array=tf.ones([4,30]) #Assuming this is the array I want to index from
function(array,seq_len) #apply required function
Output=(first 11 elements from row 0, first 12 from row 2, first 20 from row 3 etc), perhaps obtained as a flat tensor
You can use tf.sequence_mask and tf.boolean_mask to get them flattened:
mask = tf.sequence_mask(seq_len, MAX_LENGTH) # Replace MAX_LENGTH with the size of array on the right dimension, 30 in your case
output= tf.boolean_mask(array, mask=mask)
A tensor in tensorflow can be sliced just like a numpy array and then concatenated into one tensor. Assuming you measure the sequence length from the first element.
Use [row_idx,column_idx] to slice the tensor. slice = array[0,:] would assign the first row to slice.
flat_slices = tf.concat([slice,slice]) will flatten them into one tensor.
import tensorflow as tf
seq_len = [11,12,20,30]
array = tf.ones([4,30])
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
flatten = array[0,:seq_len[0]]
for i in range(1,len(seq_len)):
row = array[i,:seq_len[i]]
flatten = tf.concat([flatten, row])
print(sess.run(flatten))
What does it mean if this is the return value for tf.losses.softmax_cross_entropy_loss?
<tf.Tensor 'softmax_cross_entropy_loss/value:0' shape=() dtype=float32>
Does the fact that is states value:0 mean and shape=() mean that nothing was computed?
Nothing has been computed because you are displaying the tensors in the graph before any data has been passed through them. Let's say
sce = tf.losses.softmax_cross_entropy_loss(input)
Then to actually get the loss value you have to feed data into it using
sess = tf.Session()
...
loss = sess.run(sce, feed_dict)
where feed_dict is the dictionary for your data. Loss will now return the actual numerical loss value.
value is just an indicator for the group of computations that the value belongs to. For example: tf.reduce_mean returns tf.Tensor 'Mean_1:0' shape=() dtype=float32 because it is a mean calculation. The 0 does not mean its current value is 0, it is just used for indexing.
Additionally, your tensor shape is () because the single loss value doesn't have a batch size, x or y directions, or channels (assuming you are working with 4D tensors) so that is also ok.
I ma trying to understand tf.rank function in tensorflow. From the documentation here, I understood that rank should return the number of distinct elements in the tensor.
Here x and weights are 2 distinct 2*2 tensors with 4 distinct elemnts in each of them. However, rank() function outputs are:
Tensor("Rank:0", shape=(), dtype=int32) Tensor("Rank_1:0", shape=(),
dtype=int32)
Also, for the tensor x, I used tf.constant() with dtype = float to convert ndarray into float32 tensor but the rank() still outputs as int32.
g = tf.Graph()
with g.as_default():
weights = tf.Variable(tf.truncated_normal([2,2]))
x = np.asarray([[1 , 2], [3 , 4]])
x = tf.constant(x, dtype = tf.float32)
y = tf.matmul(weights, x)
print (tf.rank(x), tf.rank(weights))
with tf.Session(graph = g) as s:
tf.initialize_all_variables().run()
print (s.run(weights), s.run(x))
print (s.run(y))
How should I interpret the output.
Firstly, tf.rank returns the dimension of a tensor, not the number of elements. For instance, the output from tf.rank called for the 2x2 matrix would be 2.
To print the rank of a tensor, create an appropriate node, e.g. rank = tf.rank(x) and then evaluate this node using a Session.run(), as you've done for weights and x. Execution of print (tf.rank(x), tf.rank(weights)) expectedly prints out description of tensors, as tf.rank(x), tf.rank(weights) are nodes of the graph, not the variables with defined values.