For legacy_seq2seq, it just supports the input with a list of 2D Tensors.
If a model has more parameters for each inputs. says,
[[1,2], [2,3], [3,4]...] instead of [1, 2, 3, 4, ....], it cant use legacy_seq2seq to have batch methods.
So, how to implement a multidimensional input rnn in batch method with tensorflow?
Or it doesnt exist?
Multi-dimensional RNNs are well supported in Tensorflow, but not added to the legacy seq2seq interface. Please use the current (dynamic_rnn) API and you'll see that multi-dimensional tensors work fine (there are even pre-made multi-dimensional RNN cells in contrib).
Related
When I decode a tensor using tf.io.parse_tensor, the shape comes out as shape=<unknown>, which makse sense because tensorflow has no way to know the shape of the data that I will pass into this operation. However, if I know that the data I will provide has a certain shape, such as having exactly 2 dimensions, or having 3 rows and an unknown number of columns, how can I "tell" tensorflow this?
I need to do this because I am using functions like padded_batch later on that behave differently for different shapes (producing a ragged tensor versus a dense tensor).
What are symbolic tensors in TensorFlow and Keras? How are they different than other tensors? Why do they even exist? Where do they come up in TensorFlow and Keras? How should we deal with them or what problems can we face when dealing with them?
In the past, I had faced certain issues related to symbolic tensors, such as the _SymbolicException, but the documentation does not describe this concept. There's also another post where this question is also asked, but, in this post, I am focusing on this specific question, so that answers can be later used as a reference.
According to blog.tensorflow.org, a symbolic tensor differs from other tensors in that they do not specifically hold values.
Let's consider a simple example.
>>> a = tf.Variable(5, name="a")
>>> b = tf.Variable(7, name="b")
>>> c = (b**2 - a**3)**5
>>> print(c)
The output is as follows:
tf.Tensor(1759441920, shape=(), dtype=int32)
For the above, the values are specifically defined in tf.Variable format, and the output is in Tensor format. However, the tensor must contain a value in order to be considered as such.
Symbolic tensors are different in that no explicit values are required to define the tensor, and this has implications in terms of building neural networks with TensorFlow 2.0, which now uses Keras as the default API.
Here is an example of a Sequential neural network that is used to build a classification model for predicting hotel cancellation incidences (full Jupyter Notebook here if interested):
from tensorflow.keras import models
from tensorflow.keras import layers
model = models.Sequential()
model.add(layers.Dense(8, activation='relu', input_shape=(4,)))
model.add(layers.Dense(1, activation='sigmoid'))
This is a symbolically defined model, as no values are explicitly being defined in the network. Rather, a framework is created for the input variables to be read by the network, and then generate predictions.
In this regard, Keras has become quite popular given that it allows for building of graphs using symbolic tensors, while at the same time maintaining an imperative layout.
I don't know how to convert the PyTorch method adaptive_avg_pool2d to Keras or TensorFlow. Anyone can help?
PyTorch mehod is
adaptive_avg_pool2d(14,[14])
I tried to use the average pooling, the reshape the tensor in Keras, but got the error:
ValueError: total size of new array must be unchanged
I'm not sure if I understood your question, but in PyTorch, you pass the spatial dimensions to AdaptiveAvgPool2d. For instance, if you want to have an output sized 5x7, you can use nn.AdaptiveAvgPool2d((5,7)).
If you want a global average pooling layer, you can use nn.AdaptiveAvgPool2d(1). In Keras you can just use GlobalAveragePooling2D.
For other output sizes in Keras, you need to use AveragePooling2D, but you can't specify the output shape directly. You need to calculate/define the pool_size, stride, and padding parameters depending on how you want the output shape. If you need help with the calculations, check this page of CS231n course.
I am interested in studying graphs in tensorflow. The number of nodes varies from graph to graph in my dataset. This is problematic because, in my observation, tensorflow does not easily allow me to input data with variable shapes.
More specifically, could anyone suggest an easy way to feed batches of 2d arrays to a tf.Estimator.estimator , when the shapes of arrays are different? I have been using tf.estimator.inputs.numpy_input_fn for a dataset with fixed sizes, but I now need a different approach.
Ragged tensor has been introduced for this purpose.
https://www.tensorflow.org/guide/ragged_tensor
In reading about deconvolution, it is often mentioned to use the transpose of the weights when upsampling, but in the few examples in Tensorflow that I can find, this is not the case. Does the transpose happen internally? Which of the following is correct?
tf.nn.conv2d_transpose(matrix, tf.transpose(W1, [1, 0, 2, 3]), ...)
tf.nn.conv2d_transpose(matrix, W1, ...)
You don't need to transpose the weights. It's just a naming convention.
You can see why they named it the way they did here. The short summary is that it isn't performing deconvolution and is instead performing a fractionally strided convolution.
Also to answer your question directly the second one is correct.