Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I am trying to create a bot that mimics my playing behavior. The game is real time. I have access to the game's internal state. The player has a set of actions that they can perform.
Implementing an if/then decision tree bot based on the games state is easy to do, but it does not result in a realistic human player.
I though using machine learning and neural networks could solve this problem. My initial approach was to log the game's state and my actions every 100 ms. I fed a sequence of game states and my actions into a LSTM and attempted to predict what action should be performed in the next 100ms. The problem with this is that 95+% of the time, me (the player) is idle and not sending any input into the game. So the result of training is that the network predicts that the next action after a sequence of game states should be nothing/idle.
I thought about using a different approach where game state is only logged when the player sends an input. That way the network will not predict that the player should be idle. This misses out on potentially vital information in the game state when the player was not sending input.
Any ideas on how to approach this?
Another approach you can take is to use cost function weights to bring each action (i.e. idle and others) on equal footing. The problem here is presumably that you have an unbalanced dataset (in terms of labels) e.g. you might have 10 "idle" for each "jump" label/action that you have; this in return pushes the neural network to go with the dominant label.
In that case, you can set the class weights of your cross entropy as [1 10] i.e. inversely proportional to frequency of each label that you have. See https://www.tensorflow.org/api_docs/python/tf/losses/softmax_cross_entropy
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
So this might be a bit of a strange one, but I'm trying to find a tool that would help me visualize real time data in a form of a table rather than a graph/chart. There are a lot of tools out there like Grafana, Kibana, Tableau that kind of fit a similar purpose, but for a very different application and they're primarily made for aggregated data visualization.
I am essentially looking to build something like what a departure board is at an airport. You got flight flight AAA that landed 20 minutes ago, XXX departing in 50 minutes, once flight AAA is clear it disappears from the departure board etc. Only I want to have that real-time, as the input will be driven by actions users are performing on the shop floor on their RF guns.
I'd be connecting to a HANA database for this. I know it's definitely possible to build it using HTML5, Ajax and Websocket but before I get on the journey of building it myself I want to see if there's anything out there that somebody else has already done better.
Surely there's something there already - especially in the manufacturing/warehousing space where having real-time information on big screens is of big benefit?
Thanks,
Linas M.
Based on your description I think you might be looking for a dashboard solution.
Dashboards are used in many scenarios, especially where an overview of the past/current/expected state of a process is required.
This can be aggregated data (e.g. how long a queue is, how many tellers are occupied/available, what the throughput of your process is, etc.) or individual data (e.g. which cashier desk is open, which team player is online, etc.).
The real-time part of your question really boils down to what you define to be real-time.
Commonly, it’s something like “information delivery that is quick enough to be able to make a difference”.
So, if, for example, I have a dashboard that tells me that I will likely be short of, say, service staff tomorrow evening (based on my reservations) then to make a difference I need to know this as soon as possible (so I can call more staff for tomorrows shift). It won’t matter much if the data takes 5 or 10 minutes from the system entry to the dashboard, but when I only learn about it tomorrow afternoon, that’s too late.
Thus, if you’re after a dashboard solution, then there are in fact many tools available and you already mentioned some of them.
Others would be e.g. SAP tools like Business Objects Platform or SAP Cloud Analytics. To turn those into “real-time” all you need to do is to define how soon the data needs to be present in the dashboard and set the auto-refresh period accordingly.
We know q-learning need tons of calculations:
The huge amount of states in q-learning calculation
For a gaming AI, it needs much more q-values than OX game, GO game.
How this is to be done to calculate these large amounts of q-values?
Thanks.
MCTS didn't actually reduce any calculation for q-values.
For a very simple Atari gaming AI, it needs much more than 3^(19x19) q values.
Check the deep q network, that solved your problem.
We could represent our Q-function with a neural network, that takes
the state (four game screens) and action as input and outputs the
corresponding Q-value. Alternatively we could take only game screens
as input and output the Q-value for each possible action. This
approach has the advantage, that if we want to perform a Q-value
update or pick the action with highest Q-value, we only have to do one
forward pass through the network and have all Q-values for all actions
immediately available.
https://neuro.cs.ut.ee/demystifying-deep-reinforcement-learning/
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I would like to deploy tensorflow on a production server. I expect about 200 concurrent users. I will be using the parser and a few of my own neural network deep learning models. I would like to know the peak memory and cpu usage for the same.
Appreciate your help.
Trying a simple (But very variable) guess:
If you talk about deep learning, I infer you are talking at least of 3 or more layers, including some CNN and probably RNNs.
If you are using simple 2D or 3D inputs, but a complex architecture it can be safely said that your bottleneck will be on CPU, and thus implementing the algorithms on GPU will be needed.
You also need to prepare to scale for any number of clients, so a scaling mechanism will be useful from the start.
Also you need to know how the workload will be handled, will you have to serve real time, or a batch queue is needed? This change the requirements enormously.
Once you can figure out this and maybe other details, you can refine your estimation.
Best Regards.
For the memory it depends on 3 factors:
graph size
batch size for the graph
the queue size with the incoming request
Among the 3 factors probably the batch size has the most impact as memory of the graph is: graph size x batch size
About the CPU I suggest you to use the GPU for the graph. You can make some tests and count the number of inferences per second you can do with your graph and the selected batch size. Tensorflow serving well implemented handles concurrency very nicely and you bound is going to be the graph speed
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 months ago.
Improve this question
The TensorFlow home page describes its purpose as 'a software library for numerical computation'. Looking through the sample problems it looks like a problem is always formulated as follows:
Input
Model parameters
Desired output
Given some training data for 1) and 3), 2) can be computed.
I can see how this can be used to create bots, self-driving cars, image classifiers etc.
Given the broad definition of 'numerical computation', am I missing a class of other problems this can be used for? Can this be used for, say, more classical numerical computations such as the airflow around an aircraft or deformation of a structure under stress? Do you have any examples of how these classical problems would have to be formulated to fit the form above?
A nice discussion on what artificial neural networks could do, the fact that our brain is a neural network might imply that eventually an artificial neural network will be able to to the same tasks.
Some more examples of artificial neural networks used today: music creation, image based location, page rank, google voice, stock trade predictions, nasa star classifiaction, traffic management
Some fields i know of but do not have a good reference for:
optical quantum mechanics test set-up generator
medical diagnosis, reference only about safety
The Sharp LogiCook microwave oven, wiki, nasa mention
I think there are many millions of "problems" that can be solved with an ANN, deciding on the data representation (input,output) will be a challenge for some of these. some useful and useless examples i have been thinking about:
home thermostat that learns your wishes with certain weather types.
bakery production prediction
recognize go-stones on a board and map their locations
personal activity guesser and turn on appropriate device.
recognize person based on mouse movement
Given the right data and network these examples will work.
Dad has a pc controlling the heating system back home, i trained a network based on his 10years of heating data (outside temp, inside temp, humidity etc.) unfortunately i am not allowed to hook it up.
My aunt and uncle have a bakery, based on 6years of sales data i trained a network predicting how many breads and buns they should make. It showed me how important the correct inputs are. first i used the day of the year but when i switched to day of the week i saw a 15% increase in accuracy.
Currently i am working on a network that will detect a go board in a given image and map all 361 locations telling me if there is a black, white or no stone present.
Two examples that showed me how much information can be stored in a single neuron and of different ways to represent data:
Image example, neuron example (unfortunately you have to train both examples yourself so give them a little time.)
On to your example airflow around an aircraft.
I know none to nothing about airflow calculations and my try would be a really huge 3D input layer where you can "draw" an airplane and the direction and speed of the airflow.
It might work but it will require a tremendous amount of computation power, somebody knowing more about this specific topic probably knows a more abstract way of representing the data resulting in a more manageable network.
This nasa paper talks about a neural network for calculating airflow around a wing. Unfortunately i do not understand what kind of input they use, maybe it is more clear to you.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 8 years ago.
Improve this question
I am not sure about an implementation. I've a multiplexer 8 input, 1 output and 3 select signal. One of these selects signal sequentialy acquires all value of a bit vector. Now I can choose 2 way.
The first way is to use another multiplexer where the input is the bit vector and the select signal is a counter log2 of the input.
The second way is to use a shift register and take the LSB bit of the vector.
What is the best solution in terms of area in a FPGA ?
Lets (for a incorrect start) assume that the FPGA only provides plain logical elements and flip-flops, and that the bit vector is N bits and holds the value while the used bit is selected, then the resources used by the two solutions are:
Shift solution:
N flip-flops for the shifting
N 2-1 muxes for flip-flop loading
Mux solution:
(N - 1) 2-1 muxes for the larger N-1 mux
log2(N) flip-flops for the counter
So this weights towards the mux solution, since the both needs almost the same number of 2-1 muxes due to the required initial loading of the N bit shift register, but the mux solution requires less flip-flops.
However, FPGAs depends heavily on LUTs which are usually based on small memory, and an additional feature is that these small memories can often be used as shift registers also. FPGAs with this feature allows LUT memory to be written directly, and then shifted, and this can be done without any 2-1 muxes for loading. So in this case the resources used for the shift solution are:
Shift solution with LUT for shifting:
? LUT for the loading and shifting
So the conclusion is that the best way to determine and achieve an optimal solution for an FPGA, is to know and utilize the feature of the specific target FPGA, and then try the implementing to be sure the synthesis tool maps the design correctly.