I would like to know how to use
SummarySaver
monitor with contrib.learn.estimator
I understand and used ValidationMonitor but I am struggling on
Understanding the use case for SummarySaver
Using it in conjunction with contrib.learn.estimator
As an aside, what is the best practice to add summaries that are not metrics defined in ValidationMonitor ?
SummarySaver is for writing summary out every n steps
manual summary can be most easily done by using tf.scalar_summary
Related
I'm using TF-Agents library for reinforcement learning,
and I would like to take into account that, for a given state,
some actions are invalid.
How can this be implemented?
Should I define a "observation_and_action_constraint_splitter" function when
creating the DqnAgent?
If yes: do you know any tutorial on this?
Yes you need to define the function, pass it to the agent and also appropriately change the environment output so that the function can work with it. I am not aware on any tutorials on this, however you can look at this repo I have been working on.
Note that it is very messy and a lot of the files in there actually are not being used and the docstrings are terrible and often wrong (I forked this and didn't bother to sort everything out). However it is definetly working correctly. The parts that are relevant to your question are:
rl_env.py in the HanabiEnv.__init__ where the _observation_spec is defined as a dictionary of ArraySpecs (here). You can ignore game_obs, hand_obs and knowledge_obs which are used to run the environment verbosely, they are not fed to the agent.
rl_env.py in the HanabiEnv._reset at line 110 gives an idea of how the timestep observations are constructed and returned from the environment. legal_moves are passed through a np.logical_not since my specific environment marks legal_moves with 0 and illegal ones with -inf; whilst TF-Agents expects a 1/True for a legal move. My vector when cast to bool would therefore result in the exact opposite of what it should be for TF-agents.
These observations will then be fed to the observation_and_action_constraint_splitter in utility.py (here) where a tuple containing the observations and the action constraints is returned. Note that game_obs, hand_obs and knowledge_obs are implicitly thrown away (and not fed to the agent as previosuly mentioned.
Finally this observation_and_action_constraint_splitter is fed to the agent in utility.py in the create_agent function at line 198 for example.
I'm using magenta and tensorflow to generate some music via the pre-trained models from melody_rnn.
As I understand, at the moment the output generated sequence can have notes between a range of MIDI pitches.
Now, let's say I only want to output sequences that only uses MIDI notes between 50 and 60, for example, or only MIDI notes that belongs to a list that I would define.
Is there a way to do this, and if yes, how ?
Thanks !
Probably the easiest way to try this out is to copy one of the default configs referenced in melody_rnn_model.py and make your own config. Just modify the min_note and max_note values.
Note that you'll need to redo the create dataset and training steps before you can try out your new model.
After searching a while, I could not find a function to compute the l2 norm of a tensor. It seems really strange for me that it's not included so I'm probably missing something.
I looked at the l2_normalize and tf.clip_by_norm implementations and all use rsqrt(reduce_sum(x**2)) to do the trick (in that case inverse norm).
I'm probably missing something or is there a reason for not including such common function as a standard operator ?
Edit: a relevant issue from one years ago: https://github.com/tensorflow/tensorflow/issues/424
This has been added as tf.norm(matrix, order="fro") in commit 709fa61b
I am getting a Duplicate tag error when I try to write out histogram summaries for a multi-layer network that I generate procedurally. I think that the problem might be related to naming. Imagine code like the following:
with tf.name_scope(some_unique_name):
...
_ = tf.histogram_summary('weights', kernel_weights)
I'd naively assumed that 'weights' would be scoped to some_unique_name but I'm suspecting that it is not. Are summary names independent of name_scope?
As Dave points out, the tag argument to tf.histogram_summary(tag, ...) is indeed independent of the current name scope. Part of the reason for this is that the tag may be a string Tensor (i.e. computed by part of your graph), whereas name scopes are a purely client-side construct (i.e. Python-only), so there's no good way to make the scoping work consistently across the two modes of use.
However, if you're using TensorFlow build from source (and should be available in the next release, 0.8.0), you can use the following recipe to scope your tags (using Graph.unique_name(..., mark_as_used=False)):
with tf.name_scope(some_unique_name):
# ...
tf.histogram_summary(
tf.get_default_graph().unique_name('weights', mark_as_used=False),
kernel_weights)
Alternatively, you can do the following in the current version:
with tf.name_scope(some_unique_name) as scope:
# ...
tf.histogram_summary(scope + 'weights', kernel_weights)
They are.
I'm with you in thinking this is a bug, but I haven't run it past the designers of the op yet. Go ahead and open an issue for it on GitHub!
(I've run into this also and found it terribly annoying -- it prevents reuse of the model without deliberately parameterizing the summary op invocations.)
I was wondering if anyone who has used the Optimization Library in Dymola has been able to utilize the RealtimeOptimization function without having explicit plant constitutive equations to input into the criteriaFunction? Specifically, I am trying to use some of my model's states as the criteria function, but when I input y[1] := mymodel.state; with or without quotes around the state, Dymola rejects the function. Any thoughts would be very helpful. Thank you.
It is not possible to have access to model variables inside a function. You have to provide the values of the variables through the inputs to the function. The task RealtimeOptimization is aimed to be called during the simulation of a Modelica model. The criteria function has the defined inputs nTuners, nCriteria, tuners[nTuners]. You can add additional input variables, see the Example Optimization.Tasks.RealtimeOptimization.Examples.Example02.RunSampled and the record UserData in the function Optimization.Tasks.RealtimeOptimization.Examples.Example02.criteriaFunc02. By these additional variables you could transfer some state variables into the critiera function.
Maybe we can assist you, if you provide more information about your setting. I am the main developer of the Optimization library in Dymola.
You could try adding outputs (Modelica.Blocks.Interfaces.RealOutputs) to the top level of the model and then linking these to the states by using Modelica.Blocks.Sources.RealExpression blocks.
Then when using the optimisation function goto the criteria page and use the Select button and select these outputs to add them as criteria.