I am tackling a machine learning problem in which I feed my network with data of shape (batch_size, n_objects, n_features). So, each training instance comes with a given number of objects, each of them having a given number of features. Among these features I have electric charge, and while writing a custom loss function I would like to use only the neutral objects to compute it. Thus, starting from a tensor of shape (batch_size, n_objects, n_features) I would like to get a tensor of shape (batch_size, n_neutral_objects, n_features). In doing this, I'm facing a couple of problems.
First of all, I made a try by creating a tensor by hand. I have 3 training instances, each one having 2 objects, each one having 3 features. I try to get the neutral objects using tf.where() and tf.gather() methods in the following way (suppose that electric charge is the 2nd feature):
a = tf.constant([[[3.5, 0, 6], [2.1, 1, 2.9]], [[1.5, 1, 4.5], [2.0, 0, 4.2]], [[6.2, 0, 6.1], [4.8, 1, 3.4]]]) #toy input tensor
b = tf.where(a[:,:,1] == 0) #find neutral objects (charge is 2nd feature)
c = tf.gather_nd(a,b) #gather them
print(c)
This kind of works, as I get
[[3.5 0. 6. ]
[2. 0. 4.2]
[6.2 0. 6.1]], shape=(3, 3), dtype=float32)
as an output, which are the desired objects. But I've somehow lost the first dimension, as I don't want a tensor of shape (3, 3), but rather one of shape (3, 1, 3), namely still 3 input instances, each one having only one neutral object, each of them having 3 features.
Things get worse if I plug my approach into my TF model. In this real-life case, my batch size is None and I am thus dealing with tensors of shape (None, 4000, 14) (so 4000 objects for each training instance, 14 features each). This is the code I tried
def get_neutrals(tensor):
print("tensor.get_shape()", tensor.get_shape())
charges = tensor[:,:,4] #charge is the 5th feature in this case
print("charges.get_shape()", charges.get_shape())
where_neutrals = tf.where(charges == 0) # get the neutrals only
print("where_neutrals.get_shape()", where_neutrals.get_shape())
print("tf.gather_nd(tensor, where_neutrals).get_shape()", tf.gather_nd(tensor, where_neutrals).get_shape())
return tf.gather_nd(tensor, where_neutrals)
and this is what I get printed if I call my method:
tensor.get_shape() (None, 4000, 14)
charges.get_shape() (None, 4000)
where_neutrals.get_shape() (None, 2)
tf.gather_nd(tensor, where_neutrals).get_shape() (None, 14)
The last two shapes are completely unexpected and I don't know why they look like this. Can anyone here help with this?
Thanks a lot, cheers,
F.
Related
Thanks for your reading.
I train a LSTM predictor with fixed dimension (None, 5, 2), and I test the predictor with smaller dimension (None, 1, 2), and I got the warning:
WARNING:tensorflow:Model was constructed with shape (None, 5, 2) for input Tensor("input_1_1:0", shape=(None, 5, 2), dtype=float32), but it was called on an input with incompatible shape (None, 1, 2).
However, the results are fine.
I just wonder what tensorflow actually do when the case happens? Say it will automatically pad zero, such that to match the dimensions?
Again, thanks for your reading and looking forward to an answer.
Tensor computations are executed as a TensorFlow graph - see https://www.tensorflow.org/guide/intro_to_graphs. Normally graph execution is faster.
The second dimension of LSTM is dynamic. In such cases keras have to rebuild the graph every time when the input shape changing. It is slow. If your input shape is changing frequently - graph execution could be slower than eager execution. Because of that - keras issuing a warning.
Keras don't pad your data.
This is definitely a basic question, but I'm having trouble understanding exactly what is going on with Keras's layers.Conv2DTranspose function. I have the following three lines:
Setup
model = tf.keras.Sequential()
...
model.add(layers.Reshape((10, 10, 256)))
model.add(layers.Conv2DTranspose(128, (5, 5), strides=(1, 1), padding='same', use_bias=False))
assert model.output_shape == (None, 10, 10, 128)
The first occurrence of Reshape gets me a tensor of shape [10x10x256].
In the Conv2DTranspose layer, somehow I'm sliding a filter of shape [5x5] along this tensor and ending up with a new tensor of shape [10x10x128].
Question
What mathematically is happening to get me from the first tensor [10x10x256] to the second [10x10x128]?
It's almost the same as a convolution, but with fancy paddings to get the feeling of doing a backward convolution.
The sliding window in your picture is correctly positioned.
But it's not a "window", it is actually a "sliding block". The size of the block is 256 in depth.
So, it goes multiplying and summing all the channels for each stride.
But then there are 128 different sliding blocks (as you defined in your layer with filters=128). Each of these 128 sliding blocks produce separate output channel.
Great explanations about transposed convolutions: https://datascience.stackexchange.com/questions/6107/what-are-deconvolutional-layers
Let's say I have a tensor (None, 2, 56, 56, 256). Now I want to have my tensor with shape (None, 2, 55, 55, 256) by dropping last col and last row. How can I acheive this using Keras/Tensorflow?
In tensorflow we can slice tensors using python slice notation. SO, given a tensor X with shape (20,2,56,56,256) say (as you have described but with a batch size of 20), we can easily slice it taking all but the last 'row' in the 2nd and 3rd dimension as follows:
X[:,:,:-1,:-1,:]
Note the use of :-1 to denote "everything before the last 'row'".
Given this know-how about slicing the tensor in tensorflow we just need to adapt it for keras. We could, of course, write a full blown custom layer implementing this (or possibly even find one out there someone else has written - I've not looked but slicing is pretty common so suspect someone has written something somewhere!).
However, for something as simple as this, I'd advocate just using a Lambda layer which we can define as follows:
my_slicing_layer = Lambda(lambda x: x[:,:,:-1,:-1,:], name='slice')
And can use in our keras models as normal:
my_model = Sequential([
Activation('relu', input_shape=(2,56,56,256)),
my_slicing_layer
])
I have a Tensor which needs to be cropped with the indices of a tensor.
ex - Input (None,5x5x10) tensor
BoundingBox (None, 2) -- tensor
I want to have an operation that does the following
Output (None,3x2x10) --tensor
if BoundingBox[0,0] = 3, BoundingBox[0,1] = 2
This is same as tf.image.crop_to_bounding_box but this function does not tensor type bounding box as input. Please help.
Unfortunately this isn't possible with 'standard' tensor operations because the dimensions of the output could vary.
Consider the example where bounding_box[0] == [3,2] and bounding_box[1] == [4,2] then your output shape needs to be (None, 3 or 4, 2, 10) and (of course) having a dimension 3 or 4 is not allowed for standard tensors.
TensorFlow does, however, have the concept of a Ragged Tensor which could conceivably be used to represent crops of different dimensions but this is an unusual case and is unlikely to be suited to most mainstream downstream training operations. Still, it could be worth reading up on this to see if it fits your use case: link
I have a variable a of dimension (1, 5) which I want to 'tile' as many times as the size of my mini-batch. For example, if the mini-batch size is 32 then I want to construct a tensor c of dimension (32, 5) where each row has values the same as the original (1, 5) variable a.
But I only know the mini-batch size at run time: it's the size of dimension 0 of a placeholder b: tf.shape(b)[0]
Here's my code to construct c:
a = tf.Variable(np.random.uniform(size=(1,5)))
b = tf.placeholder(shape=[None, 12], dtype=tf.float32)
batch_size = tf.shape(b)[0]
c = tf.tile(a, tf.pack([batch_size, 1]))
This runs fine. Howeverc.get_shape() returns (?, ?). I don't understand why this doesn't return (?, 5) instead.
This is causing an issue later in my code when I construct a matrix variable W with number of columns c.get_shape()[1] which I expect to return 5 rather than ?.
Any help would be appreciated. Thanks.
[EDIT: This was fixed in a commit to TensorFlow on August 10, 2016.]
This is a known limitation of TensorFlow's shape inference: when the multiples argument to tf.tile() is a computed value (such as the result of tf.pack() here), and its value is not trivially computable at graph construction time (in this case, because it depends on a tf.placeholder(), which has no value until it is fed), the current shape inference will throw its hands up and declare that the shape is unknown (but with the same rank as the input, a).
The current workaround is to use Tensor.set_shape(), which allows you as the programmer to provide additional shape information when you know more than the shape inference does. For example, you could do:
a = tf.Variable(np.random.uniform(size=(1, 5)))
b = tf.placeholder(shape=[None, 12], dtype=tf.float32)
batch_size = tf.shape(b)[0]
c = tf.tile(a, tf.pack([batch_size, 1]))
c.set_shape([None, a.get_shape()[1]]) # or `c.set_shape([None, 5])`
However, we recently added some features that make it possible to propagate partially computed values that may be used as shapes, and this can be adapted to aid the shape function for tf.tile(). I have created a GitHub issue to track this, and I have a fix being tested right now.