OptaPlanner - is there a way to trigger code at each Move in CH phase? - optaplanner

I'm currently using OptaPlanner 6.2.0 but can update if it can help.
I need to keep some data of chaining between variables and entities outside the planner, as well as some precalculated data and I need a way to update it after each move but before the rules are fired. I use that data when check the rules. I know it's a hacky way but there are complex rules which need separate data storing, can't think of the way to avoid it. Also this way the Planner has shown good performance.
So the thing is I need a way to trigger some code on each move which would update that separately stored data with the changes from current move, before the rules are applied for this move and decision about how good is the move is taken (because the rules address that data), so in fact before the score for this move gets recalculated. On Local Search phase it's pretty simple because all of my moves are custom, so I can do it inside of doMove. But I really need to do the same thing in CH phase.
I'm aware that we can't change move type to custom in CH phase. I also find it very difficult to implement a full analogue of CH phase in CustomPhaseCommand for this little change. Removing CH phase and replacing it with simple custom phase reduces the quality of allocation for this specific task significantly even thought I had an example where it worked better without CH at all. I've also tried adding PhaseLifecycleListener to my Solver but it only listens to steps and phases, not to separate moves. The closest effort so far was implementing a VariableListener which indeed happens to fire beforeVariableChanged, afterVariableChanged at each move, even on CH phase. But in beforeVariableChanged I can only see the old variable, not the new one (I need both to update my data), and afterVariableChanged seems to fire AFTER the rules where fired and checked so it's useless to update the data in this method (by the way how do shadow variables get updated then on CH phase usually? I thought it should fire after, update the shadow variables there and then recheck the rules with updated shadow variables?)
I also tried to look at the chaining but I'm not sure it can fully cover my complex case with storing data separately and using it in rules...
So is there a way to correctly update my structure at each CH move? Or is there a simple way to handle the complex data dependencies other than moving it to separate structure outside of Planner?
Thanks in advance!

Related

Optaplanner: How to calculate score delta for move

I'm using Optaplanner to automatically solve school timetables. After a timetable has been solved the user will manually change some lessons and will get feedback on how this affects the score via the following call:
scoreManager.updateScore(timetable);
This call takes some 200ms and will, I assume, do a complete evaluation. Im trying to optimize this and want to only pass in a Move object so that Optaplanner only has to recalculate the delta, like:
scoreManager.updateScore(previousTimetable,changeMove);
Is there a way to do this?

There really is no way how to do just a single move. You don't do moves - the solver does moves. You can only make external problem changes to the solution. You should look into the ProblemChange interface and its use in the SolverManager.
However, the problem change will likely reset the entire working solution anyway. And after the external change is done, you're not guaranteed that the solution will still make sense. (What if it breaks hard constraints now?) You simply need to expect and account for the fact that, after users submit their changes, the solver will need to run; possibly even for a prolonged period of time.

ECS / CES shared and dependent components and cache locality

I have been trying to wrap my head around how ECS works when there are components which are shared or dependent. I've read numerous articles on ECS and can't seem to find a definitive answer to this.
Assume the following scenario:
I have an entity which has a ModelComponent (or MeshComponent), a PositionComponent and a ParticlesComponent (or EmitterComponent).
The ModelRenderSystem needs both the ModelComponent and the PositionComponent.
The ParticleRenderSystem needs ParticlesComponent and the PositionComponent.
In the ModelRenderSystem, for cache efficiency / locality, I would like run through all the ModelComponents which are in a compact array and render them, however for each model I need to pull the PositionComponent. I haven't even started thinking about how to deal with the textures, shaders etc for each model (which will definitely blow the cache).
A similar issue with the ParticleRenderSystem.. I need both the ParticlesComponent as well as the PositionComponent, and I want to be able to run through all ParticlesComponents in a cache efficient / friendly manner.
I considered having ModelComponent and ParticlesComponent each having their own position, but they will need to be synched every time the models position changes (imagine a particle effect on a character). This adds another entity or component which needs to track and synch components or values (and potentially negates any cache efficiency).
How does everyone else handle these kinds of dependency issues?

One way to reduce the complexity could be to invert flow of data.
Consider that your ModelRenderSystem has a listener callback that allows the entity framework to inform it that an entity has been added to the simulation that contains both a position and model component. During this callback, the system could register a callback on the position component or the system that owns that component allowing the ModelRenderSystem to be informed when that position object changes.
As the change events from the position changes come in, the ModelRenderSystem can queue up a list of modifications it must replicate during its update phase and then during update, its really a simple lookup each modifications model and set the position to the value in the event.
The benefit is that per frame, you're only ever replicating position changes that actually changed during the frame and you minimize lookups needed to replicate the data. While the update of the position propagates to various systems of interest may not be as cache friendly, the gains you observe otherwise out weigh that.
Lastly, don't forget that systems do not necessarily need to iterate over the components proper. The components in your entity system exist to allow you to toggle plug-able behavior easily. The systems can always manage a more cache friendly data structure and using the above callback approach, allows you to do that and manage data replication super easily with minimal coupling.

Mobx Autorun running too often / need it to skip in some cases / run only after finished

I know its not best practice but its the most performant solution I have found so far. It monitors an object, and transforms that object in the autorun.
This setup is working excellently for small data sets, but when there are a lot of changes, it gets stuck in the "100 cycles and didn't settle" error.
I was wondering if there was some way to get it to iterate just once / collate all changes and apply them in one hit.
I have tried a number of solutions / workarounds that involve timers, deep object comparison (ie don't run if it hasn't changed) etc, but they either disable the autorun or lead to extremely bad performance.
I guess in summary:
Is there some way to make autorun not execute changes on the same frame?
Is there a better solution for autorun to watch every observable in an object?
Is there a way to limit autorun?
Is there a better solution?

It is hard to say anything useful without some minimal code example of what you are doing. But it sounds like you are reading and modifying the same data in the autorun, which introduces cycles. Consider using reaction which allows you to better separate what data you want to listen react, versus the action you want to take once a change occurs

Optaplanner select only entities in conflict

In the change and swap move selector, I would like to only consider moves that involve entities in conflict as they are more likely to improve the heuristic score.
How should this be done? What classes and interfaces do I have to reuse/extend? I looked at ScoreDirector and PhaseLifecycleListener.

A MoveFilter might do that (if it's not in phase or solver cached as it changes ever step). See the course scheduling example and docs for how to use a filter.
I wouldn't recommend it though, as you still want to move non-conflicting entities at times. You might just want to focus more on those conflicting lectures. So I would keep a vanilla move selector in the mix.
The move filter isn't perfect either - the Guided Local Search feature (not yet available) is a better way to deal with this.
However, given the other question about the model and similar cases I 've seen, I 'd say moves are not your problem. A better model will make all these kinds of move tweaking obsolete.

How to quickly analyse the impact of a program change?

Lately I need to do an impact analysis on changing a DB column definition of a widely used table (like PRODUCT, USER, etc). I find it is a very time consuming, boring and difficult task. I would like to ask if there is any known methodology to do so?
The question also apply to changes on application, file system, search engine, etc. At first, I thought this kind of functional relationship should be pre-documented or some how keep tracked, but then I realize that everything can have changes, it would be impossible to do so.
I don't even know what should be tagged to this question, please help.
Sorry for my poor English.

Sure. One can technically at least know what code touches the DB column (reads or writes it), by determining program slices.
Methodology: Find all SQL code elements in your sources. Determine which ones touch the column in question. (Careful: SELECT ALL may touch your column, so you need to know the schema). Determine which variables read or write that column. Follow those variables wherever they go, and determine the code and variables they affect; follow all those variables too. (This amounts to computing a forward slice). Likewise, find the sources of the variables used to fill the column; follow them back to their code and sources, and follow those variables too. (This amounts to computing a backward slice).
All the elements of the slice are potentially affecting/affected by a change. There may be conditions in the slice-selected code that are clearly outside the conditions expected by your new use case, and you can eliminate that code from consideration. Everything else in the slices you may have inspect/modify to make your change.
Now, your change may affect some other code (e.g., a new place to use the DB column, or combine the value from the DB column with some other value). You'll want to inspect up and downstream slices on the code you change too.
You can apply this process for any change you might make to the code base, not just DB columns.
Manually this is not easy to do in a big code base, and it certainly isn't quick. There is some automation to do for C and C++ code, but not much for other languages.
You can get a bad approximation by running test cases that involve you desired variable or action, and inspecting the test coverage. (Your approximation gets better if you run test cases you are sure does NOT cover your desired variable or action, and eliminating all the code it covers).

Eventually this task cannot be automated or reduced to an algorithm, otherwise there would be a tool to preview refactored changes. The better you wrote code in the beginning, the easier the task.
Let me explain how to reach the answer: isolation is the key. Mapping everything to object properties can help you automate your review.
I can give you an example. If you can manage to map your specific case to the below, it will save your life.
The OR/M change pattern
Like Hibernate or Entity Framework...
A change to a database column may be simply previewed by analysing what code uses a certain object's property. Since all DB columns are mapped to object properties, and assuming no code uses pure SQL, you are good to go for your estimations
This is a very simple pattern for change management.
In order to reduce a file system/network or data file issue to the above pattern you need other software patterns implemented. I mean, if you can reduce a complex scenario to a change in your objects' properties, you can leverage your IDE to detect the changes for you, including code that needs a slight modification to compile or needs to be rewritten at all.
If you want to manage a change in a remote service when you initially write your software, wrap that service in an interface. So you will only have to modify its implementation
If you want to manage a possible change in a data file format (e.g. length of field change in positional format, column reordering), write a service that maps that file to object (like using BeanIO parser)
If you want to manage a possible change in file system paths, design your application to use more runtime variables
If you want to manage a possible change in cryptography algorithms, wrap them in services (e.g. HashService, CryptoService, SignService)
If you do the above, your manual requirements review will be easier. Because the overall task is manual, but can be aided with automated tools. You can try to change the name of a class's property and see its side effects in the compiler
Worst case
Obviously if you need to change the name, type and length of a specific column in a database in a software with plain SQL hardcoded and shattered in multiple places around the code, and worse many tables present similar column namings, plus without project documentation (did I write worst case, right?) of a total of 10000+ classes, you have no other way than manually exploring your project, using find tools but not relying on them.
And if you don't have a test plan, which is the document from which you can hope to originate a software test suite, it will be time to make one.

Just adding my 2 cents. I'm assuming you're working in a production environment so there's got to be some form of unit tests, integration tests and system tests already written.
If yes, then a good way to validate your changes is to run all these tests again and create any new tests which might be necessary.
And to state the obvious, do not integrate your code changes into the main production code base without running these tests.
Yet again changes which worked fine in a test environment may not work in a production environment.
Have some form of source code configuration management system like Subversion, GitHub, CVS etc.
This enables you to roll back your changes