The tensorflow.Estimator seems to gear toward supervised learning but seems to difficult to adopt for other task even if only small changes are needed for the model or training. For example, in reinforcement learning, I would need to feed a reward value which is not part of the features.
Estimators were meant for good-old supervised learning. So, adopting them to reinforcement learning will likely feel awkward. Here is an RL library that might be useful: https://github.com/tensorflow/agents.
Related
Is it really necessary to optimize the initial learning rate when using ADAM as optimizer in tensorflow/keras? How can this be done (in tensorflow 2.x)?
It is. Like with any hyperparameter, an optimal learning rate should be search for. It might be the case that your model will not learn if the learning rate is too big or too small even with an optimizer like ADAM which has a nice properties regarding decay etc.
Example of behavior of a model under ADAM optimizer with respect to a learning rate can be seen in this article How to pick the best learning rate for your machine learning project
Looking for right hyperparameters is called hyperparameter tuning. I am not using TF 2.* in my projects so I will give a reference to what TensorFlow itself offers Hyperparameter Tuning with the HParams Dashboard
I am trying to explore computer vision using deep learning techniques. I have gone through basic literature, made a NN of my own to classify digits using MNIST data(without using any library like TF,Keras etc, and in the process understood concepts like loss function, optimization, backward propagation etc), and then also explored Fashion MNIST using TF Keras.
I applied my knowledge gained so far to solve a Kaggle problem(identifying a plant type), but results are not very encouraging.
So, what should be my next step in progress? What should I do to improve my knowledge and models to solve more complex problems? What more books, literature etc should I read to move ahead of beginner stage?
You should try hyperparameter tuning, it will help improve your model performance. Feel free to surf around various articles, fine tuning your model will be the next step as you have fundamental knowledge regarding how model works.
I'm trying to control an actual robot manipulator using reinforcement learning. For reinforcement learning, I'm using Google tensorflow.
To control a robot manipulator, I need my controller to have real-time capability. However, as far as I know, python and thus tensorflow is not real-time friendly. I want to control the robot at about 100 ~ 1000 Hz.
I've considered implementing my own reinforcement learning algorithm in C++, but it would be too much work, and take too much time.
Is there anyway of using Tensorflow reinforcement learning algorithms in C++? Or, is there any other way of implementing reinforcement learning algorithm to a C++ real-time controller?
Any help would be appreciated.
Sincerely,
Steve
I don't see a reason why Tensorflow is not good for real-time control since a TF model is not subject to the limitations of the Python interpreter.
In case you find that standard TF is not fast enough you can also have a look at TF-lite: https://www.tensorflow.org/lite.
**TLDR at bottom
I have been searching for days and the chances are I just don't know what to google since I'm very new to machine learning. After doing research I've decided that tensorflow is a good starting point (I'm open to other suggestions). When I looked for examples of tensorflow they all required a large repository of data to feed into the program, but I'm more interested in creating AI that learns while it plays a game, such as Tic Tac Toe. I'm having difficulty figuring out how to do this. Any advice helps, thanks!
TLDR: Are there any good, simple examples of a machine learning program (preferably tensorflow) that can help me make a Tic-Tac-Toe AI.
I have also just started to learn more about machine learning, and it looks like the tutorial and library depend on the kind of machine learning you wish to pursue.
As you probably know there is supervised and unsupervised learning and reinforcement learning.
If you are curious about supervised and unsupervised learning, Tensorflow and SciKit Learn are the way to go.
If it's reinforcement learning, then openAI Gym would work best.
Here are some links to some tic-tac-toe repositories for each three.
Good Luck!
Supervised and Unsupervised Learning
https://github.com/3cky/tensorflow-rl-tictactoe (TensorFlow)
https://github.com/akapoorx00/tic-tac-toe-ml-project (SciKit Learn)
Reinforcement Learning
https://github.com/haje01/gym-tictactoe
https://gym.openai.com/docs/
A quick search brought these up:
https://github.com/jamesq9/Tic-Tac-Toe-Machine-Learning-Using-Reinforcement-Learning
https://github.com/3cky/tensorflow-rl-tictactoe
I'm really confused about the purpose of DeepMind's Sonnet library for TensorFlow. As far as I can tell from the documentation, it seems to do essentially what Keras does (flexible functional abstractions). Can someone tell me what the advantage of Sonnet is?
There isn't much difference between them. They are both:
High-level object oriented libraries that bring about abstraction when developing neural networks (NN) or other machine learning (ML) algorithms.
Built on top of TensorFlow (with the addition of Theano for Keras).
So why did they make Sonnet? It appears that Keras doesn't seem to suit the needs of DeepMind. So DeepMind came up with Sonnet, a high-level object oriented programming library built on top of TensorFlow to address its research needs.
Keras and Sonnet are both trying to simplify deep reinforcement learning, with the major difference being Sonnet is specifically adapted to the problems that DeepMind explores.
The main advantage of Sonnet, from my perspective, is you can use it to reproduce the research demonstrated in DeepMind's papers with greater ease than keras, since DeepMind will be using Sonnet themselves. Aside from that advantage, it's just yet another framework with which to explore deep RL problems.