OptaPlanner - Explain score of non-optimal solutions - optaplanner

We have a use-case where we want to present the user with some human-readable message with why an "assignment" was rejected based on the score of the constraints.
For e.g. in the CloudBalancing problem with 3 computers (Computer-1,2,3) and 1 process (Process-1) we ended up with the below result:
Computer-1 broke a hard constraint (requiredCpu)
Computer-2 lost due to a soft constraint (min cost)
Computer-3 assigned to Process-1 --> (Optimal solution)
We had implemented the BestSolutionChanged listener where we used solution.explainScore() to get some info and enabled DEBUG logging which provided us the OptaPlanner internal logs for intermediate moves and their scores. But the requirement is to provide some custom human readable information on why all the non-optimal solutions (Computer-1, Computer-2) were rejected even if they were infeasible (basically explanation of scores of these two solutions).
So wanted to know how can we achieve the above ?
We did not want to rely on listening to BestSolutionChanged event as
it might not get triggered for other solutions if the LS/CH
phase starts with a solution which is already a "best solution"
(Computer-3). Is this a valid assumption ?
DEBUG logs do provide us with the
information but building a custom message from this log does not seem
like a good idea so was wondering if there is another
listener/OptaPlanner concept which can be used to achieve this.

By "all the non-optimal solutions", do you mean instead a particular non-optimal solution? Search space can get very large very quickly, and OptaPlanner itself probably won't evaluate the majority of those solutions (simply because the search space is so large).
You are correct that BestSolutionChanged event will not fire again if the problem/solution given to the Solver is already the optimal solution (since by definition, there are no solutions better than it).
Of particular interest is ScoreManager, which allows you to calculate and explain the score of any problem/solution:
(Examples taken from https://www.optaplanner.org/docs/optaplanner/latest/score-calculation/score-calculation.html#usingScoreCalculationOutsideTheSolver)
To create it and get a ScoreExplanation do:
ScoreManager<CloudBalance, HardSoftScore> scoreManager = ScoreManager.create(solverFactory);
ScoreExplanation<CloudBalance, HardSoftScore> scoreExplanation = scoreManager.explainScore(cloudBalance);
Where cloudBalance is the problem/solution you want to explain. With the
score explanation you can:
Get the score
HardSoftScore score = scoreExplanation.getScore();
Break down the score by constraint
Collection<ConstraintMatchTotal<HardSoftScore>> constraintMatchTotals = scoreExplanation.getConstraintMatchTotalMap().values();
for (ConstraintMatchTotal<HardSoftScore> constraintMatchTotal : constraintMatchTotals) {
String constraintName = constraintMatchTotal.getConstraintName();
// The score impact of that constraint
HardSoftScore totalScore = constraintMatchTotal.getScore();
for (ConstraintMatch<HardSoftScore> constraintMatch : constraintMatchTotal.getConstraintMatchSet()) {
List<Object> justificationList = constraintMatch.getJustificationList();
HardSoftScore score = constraintMatch.getScore();
...
}
}
and get the impact of individual entities and problem facts:
Map<Object, Indictment<HardSoftScore>> indictmentMap = scoreExplanation.getIndictmentMap();
for (CloudProcess process : cloudBalance.getProcessList()) {
Indictment<HardSoftScore> indictment = indictmentMap.get(process);
if (indictment == null) {
continue;
}
// The score impact of that planning entity
HardSoftScore totalScore = indictment.getScore();
for (ConstraintMatch<HardSoftScore> constraintMatch : indictment.getConstraintMatchSet()) {
String constraintName = constraintMatch.getConstraintName();
HardSoftScore score = constraintMatch.getScore();
...
}
}

Related

How to speed up construction phase whilst having an trivial overlapping constraint

We're are trying to put together a proof of concept planning constraints solver using OptaPlanner. However the construction phase seems slow for even a trivial set of constraints i.e. assign to one User with no overlapping Tasks for that User.
Problem overview:
We are assigning Tasks to Users
Only one Task can be assigned to User
The Tasks can be variable length: 1-16 hours
Users can only do one Task at a time
Users have 8 hours per day
We are using the Time Grain pattern - 1 grain = 1 hour.
See constraints configuration below.
This works fine (returns in a 20 seconds) for a small number of Users and Tasks e.g. 30 Users / 1000 Tasks but when we start scaling up the performance rapidly drops off. Simply increasing the number of Users without increasing the number of Tasks (300 Users / 1000 Tasks) increases the solve time to 120 seconds.
But we hope to scale up to 300 Users / 10000 Tasks and incorporate much more elaborate constraints.
Is there a way to optimise the constraints/configuration?
Constraint constraint1 = constraintFactory.forEach(Task.class)
.filter(st -> st.getUser() == null)
.penalize("Assign Task", HardSoftLongScore.ONE_HARD);
Constraint constraint2 = constraintFactory.forEach(Task.class)
.filter(st -> st.getStartDate() == null)
.penalize("Assign Start Date", HardSoftLongScore.ONE_HARD);
Constraint constraint3 = constraintFactory
.forEachUniquePair(Task.class,
equal(Task::getUser),
overlapping(st -> st.getStartDate().getId(),
st -> st.getStartDate().getId() + st.getDurationInHours()))
.penalizeLong("Crew conflict", HardSoftLongScore.ONE_HARD,
(st1, st2) -> {
int x1 = st1.getStartDate().getId() > st2.getStartDate().getId() ? st1.getStartDate().getId(): st2.getStartDate().getId();
int x2 = st1.getStartDate().getId() + st1.getDurationInHours() < st2.getStartDate().getId() + st2.getDurationInHours() ?
st1.getStartDate().getId() + st1.getDurationInHours(): st2.getStartDate().getId() + + st2.getDurationInHours();
return Math.abs(x2-x1);
});
constraint1 and constraint2 seem redundant to me. The Construction Heuristic phase will initialize all planning variables (automatically, without being penalized for not doing so) and Local Search will never set a planning variable to null (unless you're optimizing an over-constrained problem).
You should be able to remove constraint1 and constraint2 without impact on the solution quality.
Other than that, it seems you have two planning variables (Task.user and Task.startDate). By default, in each CH step, both variables of a selected entity are initialized "together". That means OptaPlanner looks for the best initial pair of values for that entity in the Cartesian product of all users and all time grains. This scales poorly.
See the Scaling construction heuristics chapter to learn how to change that default behavior and for other ways how to make Construction Heuristic algorithms scale better.

How to define constraint about consecutive periods in optaplanner

In my optaplanner project i have periods with fixed duration.
For some of them there is a medium constraint that they should be scheduled in a row, occupying for example 5 directly adjacent timeslots.
I want to use Java constraints streams but dont manage to define this constraint using the timeslot-pattern.
I know that this constraint can be defined using the time-grain-pattern as suggested by https://stackoverflow.com/a/30702865. I have done this and it works. But I want to compare timeslot-pattern to time-grain-pattern because the have different behaviour when it comes to escape local maxima. The problem with time-grain-pattern is that those 5 periods could also be sheduled in every possible partition of 5 (eg. as 2 + 2 + 1).
Has anyone a hint on how to define the constraint using timeslot-pattern?
You may want to try using the newly added ifExists() building block. Without knowing your actual domain model, I imagine the constraint to look like this:
private Constraint twoConsecutivePeriods(ConstraintFactory constraintFactory) {
return constraintFactory.from(Period.class)
.ifExists(Period.class, equal(Period::getDay, period -> period.getDay() + 1))
.penalize("2 consecutive periods", period -> ...);
}
Consequently, ifNotExists() may be used to achieve the opposite. We have examples of both in Traveling Tournament OptaPlanner example.
Please note that this API is only available since OptaPlanner 7.33.0.Final onward.
I solved the problem using ConstraintCollectors.toList() as follows:
factory.from(Period.class).groupBy(Period::getCourse, ConstraintCollectors.toList())
.penalize(id, score, (course,list)->dayDistributionPenalize(course,list));
and
public int dayDistributionPenalize(Course course, List<Period> list) {
var penalize = 0;
var dayCodes = dayCodesFromPeriodList(list);
for (var dayCode : dayCodes)
if (!getAllowedDayCodes(course).contains(dayCode))
penalize++;
return penalize;
}
Where getAllowedDayCodes() returns for example 0111110000, 0230000000 or 0005000000 etc. from a hash-map if a course has to have 5 consecutive periods.

Non-greedy configurations across steps

I'm using late acceptance as local search algorithm and here is how it actually picks moves:
If my forager is 5, it'll pick 5 moves and then get 1 random move to be applied for every step.
At every step it only picks moves that are increasing scores ie greedy picking across steps.
Forager.pickMove()
public LocalSearchMoveScope pickMove(LocalSearchStepScope stepScope) {
stepScope.setSelectedMoveCount(selectedMoveCount);
stepScope.setAcceptedMoveCount(acceptedMoveCount);
if (earlyPickedMoveScope != null) {
return earlyPickedMoveScope;
}
List<LocalSearchMoveScope> finalistList = finalistPodium.getFinalistList();
if (finalistList.isEmpty()) {
return null;
}
if (finalistList.size() == 1 || !breakTieRandomly) {
return finalistList.get(0);
}
int randomIndex = stepScope.getWorkingRandom().nextInt(finalistList.size());// should have checked for best here
return finalistList.get(randomIndex);
}
I have two questions:
In first, can we make forager to pick the best of 5 instead of pick 1 randomly.
Can we allow move to pick that degrades score but can increase score later(no way to know it)?
Look for acceptedCountLimit and selectedCountLimit in the docs. Those do exactly that.
That's already the case (especially with Late Acceptance and Simulated Annealing). In the DEBUG log, just look at the step score vs the best score. Or ask for the step score statistic in optaplanner-benchmark.

Opta planner incorrect best score

I am trying out the optaplanner for a shift assignment problem.
It is a many to many relationship since one shift can have many employees.
In the trial run , I have two employees and three shifts .
One of the shift needs two employees.
So I have created a new ShiftAssignment class to handle the many to many relationship . ShiftAssignment is the planning entity and employee is the planning variable.
I pass the two employees and four shift assignment class ( because one shift needs two employees )
to the planning solution
I have only one hard rule in the score calculator which is basically the employee should
have the necessary skill needed for the shift
When I run the solver , I print the score in my code below ( I dont have any soft constraints so I have hard coded it to zero )
public HardSoftScore calculateScore(AuditAllocationSolution auditAllocationSolution) {
int hardScore = 0;
for (Auditor auditor : auditAllocationSolution.getAuditors()) {
for (AuditAssignment auditAssignment : auditAllocationSolution.getAuditAssignments()) {
if (auditor.equals(auditAssignment.getAuditor())) {
List<String> auditorSkils = auditor.getQualifications().stream().map(skill -> skill.getSkillName())
.collect(Collectors.toList());
String requiredSkillForThisAuditInstance = auditAssignment.getRequiredSkill().getSkillName();
if ( !auditorSkils.contains(requiredSkillForThisAuditInstance))
{
// increement hard score since skill match contraint is violated
hardScore = hardScore + 1;
}
}
}
}
System.out.println(" hardScore " + hardScore);
return HardSoftScore.valueOf(hardScore, 0);
}
When I print the values of the solution class in the score calculator , I can see that there are few solutions where hard score is zero. The solution satisfies the rules and matches the expected results . But it is not accepted as per the logs
08:16:35.549 [main] TRACE o.o.c.i.l.decider.LocalSearchDecider - Move index (0), score (0hard/0soft), accepted (false), move (AuditAssignment-2 {Auditor-1} <-> AuditAssignment-3 {Auditor-0}).
08:16:35.549 [main] TRACE o.o.c.i.l.decider.LocalSearchDecider - Move index (0), score (0hard/0soft), accepted (false), move (AuditAssignment-2 {Auditor-1} <-> AuditAssignment-3 {Auditor-0}).
One another observation which I want to clarify in the logs.
I understand that every new solution , which is the outcome of each step , is passed to score calculator . But sometimes I see that for a single step , score calculator is invoked more than once with different solution. This is my observation from the logs. Assuming this is single threaded and log sequencing is correct , why does that happen ?
The final output is incorrect . The best score that is selected is something with high hard score. And the solutions with the best score are not accepted
I also see the below line in the logs which I am not able to comprehend. Is there anything wrong in my configuration ?
23:53:01.242 [main] DEBUG o.o.c.i.l.DefaultLocalSearchPhase - LS step (26), time spent (121), score (2hard/0soft), best score (4hard/0soft), accepted/selected move count (1/1), picked move (AuditAssignment-2 {Auditor-1} <-> AuditAssignment-0 {Auditor-0}).
23:53:01.242 [main] DEBUG o.o.c.i.l.DefaultLocalSearchPhase - LS step (26), time spent (121), score (2hard/0soft), best score (4hard/0soft), accepted/selected move count (1/1), picked move (AuditAssignment-2 {Auditor-1} <-> AuditAssignment-0 {Auditor-0}).
This is a small problem size and I feel I have not set it up right . Kindly suggest.
Hard Score has to be decremented when a constraint is violated. In the above code , I had incremented the hard score which probably had led to the erroneous result.
It worked as expected once I fixed the above.

options for questions in Watson conversation api

I need to get the available options for a certain question in Watson conversation api?
For example I have a conversation app and in some cases Y need to give the users a list to select an option from it.
So I am searching for a way to get the available reply options for a certain question.
I can't answer to the NPM part, but you can get a list of the top 10 possible answers by setting alternate_intents to true. For example.
{
"context":{
"conversation_id":"cbbea7b5-6971-4437-99e0-a82927607079",
"system":{
"dialog_stack":["root"
],
"dialog_turn_counter":1,
"dialog_request_counter":1
}
},
"alternate_intents":true,
"input":{
"text":"Is it hot outside?"
}
}
This will return at most the top ten answers. If there is a limited number of intents it will only show them.
Part of your JSON response will have something like this:
"intents":[{
"intent":"temperature",
"confidence":0.9822100598134365
},
{
"intent":"conditions",
"confidence":0.017789940186563623
}
This won't get you the output text though from the node. So you will need to have your answer store elsewhere to cross reference.
Also be aware that just because it is in the list, doesn't mean it's a valid answer to give the end user. The confidence level needs to be taken into account.
The confidence level also does not work like a normal confidence. You need to determine your upper and lower bounds. I detail this briefly here.
Unlike earlier versions of WEA, the confidence is relative to the
number of intents you have. So the quickest way to find the lowest
confidence is to send a really ambiguous word.
These are the results I get for determining temperature or conditions.
treehouse = conditions / 0.5940327076534431
goldfish = conditions / 0.5940327076534431
music = conditions / 0.5940327076534431
See a pattern?🙂 So the low confidence level I will set at 0.6. Next
is to determine the higher confidence range. You can do this by mixing
intents within the same question text. It may take a few goes to get a
reasonable result.
These are results from trying this (C = Conditions, T = Temperature).
hot rain = T/0.7710267712183176, C/0.22897322878168241
windy desert = C/0.8597747113239446, T/0.14022528867605547
ice wind = C/0.5940327076534431, T/0.405967292346557
I purposely left out high confidence ones. In this I am going to go
with 0.8 as the high confidence level.