I'm working with video sequences and optical flows. I'd like to know, whether Tensorflow has operation for warping images. Analog to image.warp in Torch https://github.com/torch/image/blob/master/doc/paramtransform.md
If there is no such operation build in, maybe there is open source code for that. Or you could provide pointers for me to implement this operation in TF.
Thanks!
I haven't found a built in function yet. But the answer to this question might help you.
Its built with standard Tensorflow Ops and doing a bilinear interpolation, but I guess it won't be very fast in comparison to a truly CUDA optimized Op. Also you need to expand it for batches, color images and the kind of padding you want.
Yes. In tf2 it is moved to TensorFlow Addons (here):
#tf.function
tfa.image.dense_image_warp(
image: tfa.types.TensorLike,
flow: tfa.types.TensorLike,
name: Optional[str] = None
) -> tf.Tensor
Opencv have these functions: warpPerspective(), perspectiveTransform()
Related
I have been looking to learn TensorFlow and I have noticed that different functions are used for the same goal. To square a variable for instance, I have seen tf.square(), tf.math.square() and tf.keras.backend.square(). This is the same for most math operations. Are all these the same or is there any difference?
Mathematically, they should produce the same result. However Tensorflow functions in tensorflow.math.somefunction are used for operating Tensorflow tensors.
For example, when you write a custom loss or metric, the inputs and outputs should be Tensorflow tensors. So that Tensorflow knows how to take gradients of the functions. You can also use tf.keras.backend.* functions for custom loss etc.
Try to use tensorflow.math.somefunctions whenever you can, native operations are preferred. Because they are officially documented and guarateed to have backward compatibility between TF versions like TF 1.x and TF 2.x.
As tf.data augmentations are executed only on CPUs. I need a way to run certain augmentations on the TPU for an audio project.
For example,
CPU: tf.recs read -> audio crop -> noise addition.
TPU: spectogram -> Mixup Augmentation.
Most augmentations can be done as a Keras Layer on top of the model, but MixUp requires both changes in input as well as label.
Is there a way to do it using tf keras APIs.
And if there is any way we can transfer part of tf.data to run on TPU that will also be helpful.
As you have rightly mentioned and as per the Tensorflow documentation also the preprocessing of tf.data is done on CPU only.
However, you can do some workaround to preprocess your tf.data using TPU/GPU by directly using transformation function in your model with something like below code.
input = tf.keras.layers.Input((512,512,3))
x = tf.keras.layers.Lambda(transform)(input)
You can follow this Kaggle post for detailed discussion on this topic.
See the Tensorflow guide that discusses preprocessing data before the model or inside the model. By including preprocessing inside the model, the GPU is leveraged instead of the CPU, it makes the model portable, and it helps reduce the training/serving skew. The guide also has multiple recipes to get you started too. It doesn't explicitly state this works for a TPU but it can be tried.
I notice in the code for the Tensorflow Object Detection API there are several references to Mask R-CNN however no mention of it in the documentation. Is it possible to train/run Mask R-CNN through this API, and if so how?
You may not like it, but the answer is (for the moment), is no. The API cannot be used to predict or recover masks
They only use a little part of the Mask R-CNN paper to predict boxes in a certain way, but predicting the instance masks is not yet implemented.
Now we can implement Mask with faster_rcnn_inception_v2 there is samples with 1.8.0 tensorflow version
I have a pytorch model and a tensorflow model, I want to train them together on one GPU, following the process bellow: input --> pytorch model--> output_pytorch --> tensorflow model --> output_tensorflow --> pytorch model.
Is is possible to do this? If answer is yes, is there any problem which I will encounter?
Thanks in advance.
I haven't done this but it is possible but implementing is can be a little bit.
You can consider each network as a function, you want to - in some sense - compose these function to form your network, to do this you can compute the final function by just giving result of one network to the other and then use chain-rule to compute the derivatives(using symbolic differentiation from both packages).
I think a good way for implementing this you might be to wrap TF models as a PyTorch Function and use tf.gradients for computing the backward pass.
Doing gradient updates can really get hard (because some variables exist in TF's computation graph) you can turn TF variables to PyTorch Variable turn them into placeholdes in TF computation graph, feed them in feed_dict and update them using PyTorch mechanisms, but I think it would be really hard to do, instead if you do your updates inside backward method of the function you might be able to do the job(it is really ugly but might do the job).
I want to implement a convolution-deconvolution network for a image segmentation project. In the deconvolution part, I am planning to upsample the feature map by 2. e.g. The original feature map is of dimension 64*64*4 and I want to upsample it into 128*128*4. Does anyone know a tensor operation that does this? Thanks!
You could use tf.image.resize_images(). It takes batches of images or single images and supports the most common methods such as bilinear and nearest_neighbor.
Here's the link to the TensorFlow API reference: resizing
You can also take a look at how the upsampling operation is implemented in a higher-level API such as tflearn. You can find upsample_2d and upscore_layer in their Github repo: conv.py
Note: the output might be cast to tf.float32 in older TF versions