guys! I have a question to ask.If I want to use maxout as activation function , how should I write the codes in Tensorflow? An input parameter is required in the slim.maxout() function, so it cannot be used for
slim.arg_scope([slim.conv], activation_fn=slim.maxout)?What should I do?
You may have to define maxout in a separate function. For example:
def maxout(inputs,num_inputs):
return slim.maxout(inputs,num_inputs)
slim.arg_scope([slim.conv],activation_fn=maxout)
(I may have the arguments defined incorrectly in the maxout function.)
In any case, I recommend that you switch to tf.layers (tf core API) because tf.slim seems to be in the works of being phased out.
https://github.com/tensorflow/tensorflow/issues/16182#issuecomment-372397483
Related
In tensorflow tf.keras.Model.compile, you can pass a lambda y_true, y_pred: val function as a metric (though, it seems not documented), but I asked my self : "How does it aggregate it over the batches" ?
I searched the documentation, but I've found nowhere how it is done ?
By the way, I don't even know if it is an undefined behavior to do so and one should instead subclass the Metric class ? ( or at least provide the required methods).
Also, is it pertinent to pass a loss as a metric (and in this case, same question : how is it aggregated over the batches ? )
To understand "How does it aggregate (I'm assuming for display in the progress bar)", I suggest you check tf.keras.utils.Progbar. Aggregation over batches is done when you use model.fit, not model.compile.
Is using a lambda as a loss or metric undefined behaviour? No, if defined properely. If you do not write the lambda expression properly, TensorFlow will throw an Exception.
Is using a lambda as a loss or metric recommended? Nope. There is a reason TensorFlow provides separate classes for these. Extending inbuilt classes simplifies other parts of the pipeline, such as saving or loading models. It also makes the code much more readable.
It should just take the average over batches. I don't think it's undefined behavior.
Check out the "Creating Custom Metrics" section here. The metric you use (the lambda) is a stateless, and therefore, during training, it's
the average of the per-batch metric values for all batches seen during a given epoch.
I'm trying to implement a custom layer in Keras where I need to convert a tensor of floats [a, 1+a) to a binary tensor for masking. I can see that Tensorflow has a floor function that can do that, but Keras doesn't seem to have it in keras.backend. Any idea how I can do this?
As requested by OP, I will mention the answer I gave in my comment and elaborate more:
Short answer: you won't encounter any major problems if you use tf.floor().
Long answer: Using Keras backend functions (i.e. keras.backend.*) is necessary in those cases when 1) there is a need to pre-process or augment the argument(s) passed to actual function of Tensorflow or Theano backend or post-process the returned results. For example, the mean method in backend can also work with boolean tensors as input, however the reduce_mean method in TF expects numerical types as input; or 2) you want to write a model that works across all the Keras supported backends.
Otherwise, it is fine to use most of real backend functions directly; however, if the function has been defined in keras.backend module, then it is recommended to use that instead.
When using the GANEstimator in the tensorflow library, I can only pass the mode to the generator function but not the discriminator function. This means I can not perform batch normalisation or dropout in the discriminator. Is there a reason why this is not allowed? Will this be changed in the future?
Link to GANEstimator:
https://www.tensorflow.org/versions/r1.6/api_docs/python/tf/contrib/gan/estimator/GANEstimator
This feature has been added already:
https://github.com/tensorflow/tensorflow/commit/9560054b18ad3fc9faa296d344e062876858817a
I need to write a simple initializer for my convolutional layer biases. I am using tf.slim so I can specify the initializer when calling the convolutional layer, like so.
I want to replace the biases_initializer=init_ops.zeros_initializer() with my own custom function that just initializes the bias to a given constant, for example :
`biases_initializer=custom_initializer(value)`
where I can specify the value, for example value = -5.
Can anyone show me how this is done? I've spent about an hour reading through the existing initializers, but still don't know how to implement this simple function.
I finally found that it is not necessary to define that function since there already is a tf.constant_initializer. The above would just be achieved with:
biases_initializer = tf.constant_initializer(value)
I would like to speed up my LSTM network, but as I am using it for a OCR (where sequences have variable lenght), I can not use plain LSTM implementation. That is why I use "tf.nn.dynamic_rnn".
Based on benchmark of RNN in tensorflow (https://github.com/tensorflow/tensorflow/blob/754048a0453a04a761e112ae5d99c149eb9910dd/tensorflow/contrib/cudnn_rnn/python/kernel_tests/cudnn_rnn_ops_benchmark.py#L77), the CUDNN implementation is used for creating all model at once (it does not use "tf.nn.rnn" structure like others). I assume that it maybe impossible to use CUDNN with variable length, but maybe anybody success it?
Second this is using "tf.nn.bidirectional_dynamic_rnn", as I would like to use Bi-LSTM for OCR. But this should be resolved after implementing the first part.
Edit: It looks like "tf.contrib.cudnn_rnn.CudnnLSTM" have "bidirectional" implementation inside. So the only unknown this is that CUDNN can be used with variable input sequence.
Or maybe any working example which use 'CudnnLSTM' would be helpfull.
Just found this:
tf.contrib.cudnn_rnn.CudnnLSTM currently does not support batches with sequences of different length, thus this is normally not an option to use.
Source: http://returnn.readthedocs.io/en/latest/tf_lstm_benchmark.html
TensorFlow will soon finally have support for variable sequence lengths: https://github.com/tensorflow/tensorflow/blob/2f672ee9562a452f8dbfa259a8ccec56367e9b17/tensorflow/contrib/cudnn_rnn/python/layers/cudnn_rnn.py#L389
It looks like it landed too late for 1.13, so it'll probably only be available on TensorFlow 1.14.
You can try it out today by installing the tf-nightly-gpu package and passing sequence_lengths=lengths where lenghts is a tf.int32 Tensor with shape [batch_size], containing the lengths of each sequence in your batch.