I have a dictionary with two keys and the shape for each of the two keys is ([612, 41, 41, 1]).
I would like to reshape it in order to have shape ([612, 41, 41, 2]) in order to feed it in the cnn with input channels = 2. If I use the command tf.expand_dims(my_dict[1], -2)} does not work.
Any ideas?
In my opinion, because of 612x41x41x1 is different from 612x41x41x2 so you can't reshape to this shape. But if you mean that you want to duplicate the last dimension so you can try this code:
my_tensor = tf.tile(my_tensor, (1, 1, 1, 2))
Related
Say I have 2 tensors, one with shape (10,1) and another one with shape (10, 11, 1)... what I want is to multiply those broadcasting along the first axis, and not the last one, as used to
tf.zeros([10,1]) * tf.ones([10,12,1])
however this is not working... is there a way to do it without transposing it using perm?
You cannot change the broadcasting rules, but you can prevent broadcasting by doing it yourself. Broadcasting takes effect if the ranks are different.
So instead of permuting the axes, you can also repeat along a new axis:
import tensorflow as tf
import einops as ops
a = tf.zeros([10, 1])
b = tf.ones([10, 12, 1])
c = ops.repeat(a, 'x z -> x y z', y=b.shape[1]) * b
c.shape
>>> TensorShape([10, 12, 1])
For the above example, you need to do tf.zeros([10,1])[...,None] * tf.ones([10,12,1]) to satisfy broadcasting rules: https://numpy.org/doc/stable/user/basics.broadcasting.html#general-broadcasting-rules
If you want to do this for any random shapes, you can do the multiplication with the transposed shape, so that the last dimensions of both the matrices match, obeying broadcasting rule and then do the transpose again, to get back to the required output,
tf.transpose(a*tf.transpose(b))
Example,
a = tf.ones([10,])
b = tf.ones([10,11,12,13,1])
tf.transpose(b)
#[1, 13, 12, 11, 10]
(a*tf.transpose(b))
#[1, 13, 12, 11, 10]
tf.transpose(a*tf.transpose(b)) #Note a is [10,] not [10,1], otherwise you need to add transpose to a as well.
#[10, 11, 12, 13, 1]
Another approach is to expanding the axis:
a = tf.ones([10])[(...,) + (tf.rank(b)-1) * (tf.newaxis,)]
I have a 2d tensor of shape [32,768] also a 3d tensor of [32,512,768]. I want the stack them and get output to have shape of [32,512,1536].
If I expand dimensions at axis=1 for 2d and concat. I am getting [32,513,768]. So how to get [32,512,1536] as my output shape of tensor?
Short answer: you will have to repeat the 2D tensor along axis 1 512 times to get a 3D tensor of shape [32, 512, 768]. This 3D tensor when concatenated with the other 3D tensor along the last dimension will give a tensor of shape [32, 512, 1536]. You need to make sure this repetition in desired.
Longer extension:
Let's take a much simpler case:
Take a 1D tensor (1, 2, 3, 4, 5). Say you need to concatenate this to a 2D tensor of shape [2, 5], say ((6, 7, 8, 9, 10), (11, 12, 13, 14, 15)). Note that this is a simplified version of your problem, with smaller tensors and no batch dimension.
One way to combine these tensors is to get a tensor of shape [3, 5]. Here, you would expand the 1D tensor to a 2D tensor having shape [1, 5], and concatenate along axis 0. This will give the result ((1, 2, 3, 4, 5), (6, 7, 8, 9, 10), (11, 12, 13, 14, 15)). When applied to your problem, this gives the resulting [32, 513, 768] tensor you have.
The second way would give the tensor ((1, 2, 3, 4, 5, 6, 7, 8, 9, 10), (1, 2, 3, 4, 5, 11, 12, 13, 14, 15)), having shape [2, 10]. As you can see, this requires (1, 2, 3, 4, 5) to be repeated twice. So, you'll have to expand the 1D tensor to get the shape [1, 5], and repeat it to get a tensor of shape [2, 5]. This tensor can then be concatenated with the other 2D tensor. In your case, you will expand the 2D tensor to shape [32, 1, 768], then repeat it 512 times along axis 1 to get a tensor of shape [32, 512, 768], which will be concatenated with the other 3D tensor.
When going for the second method, ensure that you really want to repeat the smaller tensor across all entries of the second tensor.
You can try this:
import torch
a = torch.ones([32,768])
b = torch.ones([32,512,768])
result = torch.cat([a[:, None, :].repeat(1, 512,1), b], dim=2)
I am trying to embedding the positional information 'index' to some vector and use in Keras, for instance
inputs = Input(shape=(23,))
Which usually 23 represents as the number of features. I want to embed the position of the features to be one dimentional vector, from position 0 to position 22.
But I don't know how to get the position index of the features (I want something like 'enumerate' function for keras layer), I added
pos = K.constant([[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22]])
embedding_pos = Embedding(23, 1)(pos)
And I actually this embedding_pos to multiply with inputs and do the rest of the algorithms go on.
But I get this message
AttributeError: 'NoneType' object has no attribute '_inbound_nodes'
If I get rid of that embedding layer and multiply layer, the algorithm works fine. How am I supposed to get the embedding vectors using the position index of the features of inputs?
======
Adding more information, I moved around the layers to see the model.summary(), it seems like it has the embedding_pos = [None, 1] shape, which is missing batch size.
I don't think it is good to use 'Constant'. I'd like to know if there is some kind of 'enumerate' function for keras layer
=====
By the request, example inputs is like this
batch_size x number_of_features = 1 x 10
[[1.0, 4719.0, 0.0001, 472818.44, 958, 6402818., 1.828, 24.321, 55.0, 127.44]]
and so on...
I want to get the index of the features
[[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]
to use this value as the input for Embedding.
But if I make it with constant, it doesn't know the batch size.
I have a tensor with probabilities. This is a dynamic tensor with shape (?, 30) and I am selecting index with the best probability of these 30 values as :
best_probability = tf.argmax(probability, axis = 1)
Now the dimensions of tensor best_probability is (?,). Now I want to select the values with these indices from another tensor called data with dimensions (?, 30, 1024, 3). Essentially from each of the 30 values select one with best probability using best_probability tensor.
The final output should have dimensions of (?, 1024, 3).
PS:- I tried gather_nd but it need indexing of best_probability tensor something like [[0, 9], [1, 10], [2, 15], [3, 25]]. To do so I wrote following snippet.
selected_data = tf.stack(tf.range(probability.shape[0]),
tf.argmax(probability, axis = 1))
This doesn't work as I am dealing with a dynamic tensor. Is there any alternative to solve this problem.
I was able to solve this issue using tf.batch_gather and tf.reshape
selected_data = tf.reshape(tf.batch_gather(data, best_probability),
(-1, data.shape[2],data.shape[3]))
Why does numpy.random.choice not work the same as random.choice? When I do this :
>>> random.choice([(1,2),(4,3)])
(1, 2)
It works.
But when I do this:
>>> np.random.choice([(1,2), (3,4)])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "mtrand.pyx", line 1393, in mtrand.RandomState.choice
(numpy/random/mtrand/mtrand.c:15450)
ValueError: a must be 1-dimensional
How do I achieve the same behavior as random.choice() in numpy.random.choice()?
Well np.random.choice as noted in the docs, expects a 1D array and your input when expressed as an array would be 2D. So, it won't work simply like that.
To make it work, we can feed in the length of the input and let it select one index, which when indexed into the input would be the equivalent one from random.choice, as shown below -
out = a[np.random.choice(len(a))] # a is input
Sample run -
In [74]: a = [(1,2),(4,3),(6,9)]
In [75]: a[np.random.choice(len(a))]
Out[75]: (6, 9)
In [76]: a[np.random.choice(len(a))]
Out[76]: (1, 2)
Alternatively, we can convert the input to a 1D array of object dtype and that would allow us to directly use np.random.choice, as shown below -
In [131]: a0 = np.empty(len(a),dtype=object)
In [132]: a0[:] = a
In [133]: a0.shape
Out[133]: (3,) # 1D array
In [134]: np.random.choice(a0)
Out[134]: (6, 9)
In [135]: np.random.choice(a0)
Out[135]: (4, 3)
Relatedly, if you want to randomly sample rows of a 2D matrix like this
x = np.array([[1, 100], [2, 200], [3, 300], [4, 400]])
then you can do something like this:
n_rows = x.shape[0]
x[np.random.choice(n_rows, size=n_rows, replace=True), :]
Should work for a 2D matrix with any number of columns, and you can of course sample however many times you want with the size kwarg, etc.