I'm recently interested in functional programming (FP).
However, I wonder whether simulation with FP is difficult or not.
For example, with object-oriented programming (OOP), I would assign a state to each agent (class, probably) and update their states causally. The history of states is recorded to a certain array. The history will be saved and flushed if necessary at each time period.
As I know, FP avoids mutable data so time-varying state of each agent cannot be contained in the agent.
Then, for simulations in FP, do I have to record states at each time instant to a certain array and not assign states to each agent? Then, what if I would like to see a state of a specific agent for debugging? Should I find the state in the recorded array?
Like the above, it's hard for me to think in an FP way for numerical simulation.
Honestly, OOP style looks natural for this task.
Can anybody explain some routines for writing numerical simulation codes in an FP way?
Related
Having an objective function E(s) in Simulated Annealing (SA) defines the transition probability of moving from one state s to another s'. Ideally, the objective function minimum corresponds to the optimal solution.
In Reinforcement learning (RL), we have a value function v(s) that gives a value of how good it is to be in the current state s.
There is also in function which gives a value to a combination of the current state and an action, but I don't want to compare this to SA.
So my question is now, what is the difference between E(s) and v(s)?
Simulated Annealing (SA) and Reinforcement Learning (RL) algorithms are meant to solve different classes of problems. The former is meant to find a global optimum while the later is meant to find a policy that maximize a reward (not directly a reward nor a state). More precisely, in RL, agents do actions regarding a reward and their current state (feedback). The policy of an agent can be seen as a map defining the probability of doing an action given a state and the value function defined how good is it to be in a state considering all future actions.
RL algorithms can be applied to optimize the policy of an agent in game as long as you can attribute a score to the players. The reward can typically be the score difference between two time-step (ie. rounds). For many games, like chess for example, an opponent can impact the state of the agent and the agent can just react to it based on a feedback loop. The goal in such case is to find the sequence of operation that maximize the chance to win. Using naively SA for such a problem does not make much sense: there is no need to find the best global state. In fact, if we try to apply SA in this case, a good opponent will quickly prevent SA to converge to a good global optimal. In fact, SA does not consider the opponent and do not care about the sequence of operation, only the result matters in SA.
Alternatively, if you want to find the minimum value of a derivable mathematical function (eg. high-order polynomials), then RL algorithm are quite useless (and inefficient) because they focus on optimizing the optimal policy while you do not need that (though an optimal policy can help to find a global optimal, SA is already good for that), you only want the optimal state (and possibly its associated objective value).
Another key difference is that AFAIK E(s) is predefined in SA, while V(s) is generally unknown and must be found by RL algorithms. This is a huge difference since in practice V(s) is is dependent of the policy which the RL algorithm need to also find. If V(s) is known, then the policy can be trivially deduced (the agent needs to perform the action that maximize V(s)) and if an optimal policy is known, then V(s) can be approximated computed from the Markov chain.
I would like to use AI/TensorFlow/Machine Learning of some description in order to recognise patterns in a set of data.
Most of the samples of machine learning seem to be based on decision making, whether a value falls above or a below a line, then the decision is good or bad. But what I seem to have is a set of data, that may or may not have any relationship, may or may not have a pattern, and a single entity by itself is neither good nor bad, but the whole set of data together needs to be used to work out the type of data.
The data in question is a set of readings over a period of time from an automotive CANBUS reader - so hexadecimal values containing a Command ID, and 1 or more values, usually in the format: FFF#FF:FF:FF:FF:FF:FF:FF:FF
eg:
35C#F4:C6:4E:7C:31:2B:60:28
One canbus command, may contain one or more sensor readers - so for example F4 above might represent the steering position, C6 might indicate the pitch of the vehicle, 4E might indicate the roll.
Each sensor may take up one or more octets, 7C:31 might indicate the speed of the vehicle, and "B" of "2B" might indicate whether the engine is running or not.
I can detect the data with a human eye, and can see that the relevent item might be linear, random, static (ie a limited set of values) or it might be a bell curve.
Im new to statistical analysis and machine learning so do not know the terminology. In the first instance Im looking for the terminology and references to appropriate material that will help me achieve my goal.
My goal is given a sample of data from a CANBUS reader, to scan all the values, using each and every possible combination of numbers (eg octets 1, 1+2, 1+2+3... 2, 2+3, 2+3+4... 3, 3+4 etc) to detect patterns within the data and work out whether those patterns are linear, curve, static, or random.
I want to basically read as many CAN-BUS readings from as many cars as I can, throw it at a program to do some analysis and learning and hopefully provide me with possibilities to investigate further so I can monitor various systems on different cars.
It seems like a relatively simple premise, but extremely hard for me to define.
I'm working on the project where university course is represented as a to-do list, where:
course owner (teacher of the course) can add tasks (containing the URL to the resource needs to be learned and two datetime fields - when to start and when to complete the task)
course subscriber (student) can mark tasks as complete or not complete and their marks are saved individually for each account.
If student marks task as complete - his account + element he marked are shown in the course activity tab for teacher where he can:
initiate a conversation in JavaScript-based chat with him
evaluate the result of the conversation
What optimization algorithm you could recommend me to use for timetable rescheduling (changing datetime fields for to-do element if student procrastinates) here?
Actually, we can use the student activity on the resource + fact that he marked the task as complete + if he clicked or not on the URL placed on the to-do element leading to the external learning material (for example Google Book).
For example, are genetic algorithms suitable for this model and what pitfalls do they have: https://medium.com/#vijinimallawaarachchi/time-table-scheduling-2207ca593b4d ?
I'm not sure I completely understand your problem but it sounds like you have a feasible timetable to begin with and you just need to improve it.
If so genetic algorithms will work very well, but I think representing everything as binary 'chromosomes' like in the link might not be practical.
There are many other ways you can represent a timetable, such as in a 2D array, or giving an event a slot number.
You could look into algorithms such as Tabu search, Simulated Annealing and Great Deluge and Hill Climbing. They are all based on similar ideas but some work better with some problems than others. For example if you have a very rough search space simulated annealing won't be the best and Hill Climbing usually only finds a local optimum.
The general architecture of the algorithms mentioned above and many other genetic algorithms and Metaheuristics is: select a neighbouring solution using a move operator (e.g. swapping the time of one or two or three events or swapping the rooms of two events etc...), check the move doesn't violate any hard constraints, use an acceptance strategy such as, simulated annealing or Great Deluge, to determine if the move is accepted. If it is keep the solution and repeat the steps until the termination criterion is met. This can be max time, number of iterations reached or improving move hasn't been found in x number of iterations.
Whilst this is running keep a log of the 'best' solution so when the algorithm is terminated you have the best solution found. You can determine what is considered 'best' based on how many soft constraints the timetable violates
Hope this helps!
I work with computational geometry in C++. In particular I calculate properties of a triangulation with moving vertices. For optimization purposes I end up considering an interface design for the triangles where a property is re-calculated only when needed, i.e. when a vertex of the triangle has been moved, but otherwise is retrieved from stored memory.
I recently watched videos on the Julia programming language and was introduced to JIT thinking. I am now wondering if this above mentioned type of design pattern is built-in in any popular (preferably scientific) languages. As I envision it, one would register variables to listen to changes in some fields and depending on the activity of these fields, as the variable is used, the value is retrieved or calculated on the fly. All under the hood. Has something similar to this been done?
Our group is building a process modelling application that simulates an industrial process. The final output of this process is a set of number representing chemistry and flow rates.
This application is based on some very old software that uses the exact same underlying mathematical model to create the simulation. Thousands of variables are involved in the simulation.
Although each component has been unit tested, we now need to be able to make sure that the data output produced by our software matches that of the old simulation software. I am wondering how best to approach this issue in a formalised and rigorous manner.
The old program works by specifying the input via a text file, so I was thinking we could programatically take each variable, adjust its value in the file (and correspondingly in our new application), then compare the outputs between the new and old application. We do this for every variable in the model.
We know the allowable range for each variable so I suppose a random sample across each variable of a few values is enough to show correctness for that particular variable.
Any thoughts on this approach? Any other ideas?
The comparison of output of the old and new applications id definitely good idea. This is sometime called back-to-back testing.
Regarding test input samples - get familiarized with following concepts:
Equivalence partitioning
Boundary-value analysis