OptaPlanner: Stuck in local optima. Issue with domain model? Need better moves? Score traps? etc - optaplanner

Question

Forewarning: This is a very detailed and long question. I wouldn't normally make a post like this unless I was really stuck. I've been working on this problem for a few weeks now and I've hit a breaking point. I cannot find a way for the solver to even solve for, much less optimize around, my hard constraints. I'm a fresh CS grad and I'm doing this freelance project for free. I'm doing it for the hospital my mom works for. Optimizing their provider schedule will make her life a lot easier. She's the RN in charge of staffing (which is hell) and works 11-12+ hour days at 58 because she also has regular RN responsibilities, I really want to help her out! Plus, the experience doesn't hurt and I want something like this on my resume.
I'll try to give a rundown of the problem and what I've done so far as succinctly as possible but that is going to be difficult.
The Problem
I'm writing an auto-scheduler for healthcare providers. This hospital dept. has a static scheduling system and they have around 23 providers. They want to load balance the staff required for the providers working in their dept. for each shift. Providers also have obligations to other departments which have to be met.
Each provider must fulfill, on average, a certain number of shifts in a particular location in a month.
How they lay out their schedule, and what I am kind of beholden to, is this. The shifts can be scheduled in a Slot, a Day, and a Time, each of which are discrete.
Slot: 1st, 2nd, 3rd, 4th
Day: M, T, W, Th, F
Time: AM, PM
So (1st, M, AM) refers to the 1st Monday of the month in the morning. Lets call this a TimeSlot. Thus, all of the possible TimeSlots available then is just the Cartesian product of these three sets. Note there could be a 5th slot since sometimes days appear 5 times in a month but the 5th's schedule can just simply mimic the 3rd for simplicity.
Some shifts must be full days. That is they need to span a slot, a day, and both times. Some shifts must be full days and must occur every week on the same day. That means they span a day, every slot and every time. The last shift type is a half day that occurs on the same day every week. These span a day, a time and every slot.
There are two different location types: In the department and outside the department. Dept. Locations have a limit on the number of rooms in a TimeSlot. Outside Locations have a limit on the number of providers that can be scheduled in a TimeSlot.
Hard constraints:
A provider cannot work two different shifts at the same time.
All the shifts scheduled in a TimeSlot at a dept. location cannot exceed the room availability.
All the shifts scheduled in a TimeSlot at an outside location cannot exceed the provider limit.
Medium Constraints:
Even out the number of RNs in every dept. Location in every TimeSlot.
Even out the number of STs in every dept. Location in every TimeSlot.
Even out the number of MAs in every dept. Location in every TimeSlot.
Soft Constraints:
Left out for now for simplicity
What I've Done
I have 4 planning entities for each of the shift types. I'll outline their planning variables and the types below:
HalfDayShift: Slot, Day, Time
HalfDayNoSplitShift: Day, Time
FullDayShift: Slot, Day
FullDayNoSplitShift: Day
Each of these extend an abstract Shift class so I can interact with them all easily. I could've done this with only one planning entity being the "smallest" shift (HalfDayShift) and instead defined the "larger" shifts as groups of these smaller ones. However I found this way easier for defining custom moves, as the moves are the weirdest part of this problem. I'll try to explain the move mechanics as best I can.
In thinking of the moves for this schedule, the first type applies only to the construction. For each shift type, the only move is the Cartesian product of the change moves for each of that types planning variables. Simple enough.
For Local Search, I figured the simplest move would be to pick a shift, assign it a new value based on the possible values of that type, i.e. a FullDayShift will pick a new Slot/Day and a FullDayNoSplitShift will just pick a new Day, and any shifts occurring on that new value with the same provider will be swapped. However, this only works if a shift is moving to a value where all the shifts there are smaller or equal in size. I.e. a HalfDayShift moving onto a day which a FullDayNosplitShift fills doesn't make sense because all the shifts on the original Day should be swapped too. So once we find a valid set of shifts, to make a move, shifts just swap any planning variables in common. I.e. A FullDayNoSplitShift just swaps a Day with any shifts with the same provider on the other day.
This shows how the different shifts can move on top of eachother
I've only implemented the hard constraints and I cannot generate a feasible solution. A couple notes:
Some shifts are concurrent shifts. This means they occur in two Locations simultaneously (usually in nearby locations). All this really means, is the shift has, and counts towards resources in, a second location.
I believe there are not any score traps here and my move is coarse grained enough that it swaps things together as needed. However, please let me know if I'm missing something.
rule "Provider cannot work two shifts in same timeslot"
when
$leftShift : Shift (
isInitialized(),
$leftProv : provider
)
Shift (
this != $leftShift,
provider == $leftProv,
isInitialized(),
conflictsWith($leftShift)
)
then
scoreHolder.addHardConstraintMatch(kcontext, -100);
end
rule "Cannot exceed available slots resource constraint"
when
$loc : Location(rooms == false, $avail: availMap)
$s : Slot()
$d : Day()
$t : Time()
accumulate (
Shift (
isInitialized(),
occursOn($s,$d,$t),
isConsumingResc(),
primaryAt($loc) || concAt($loc)
);
$c : count();
((Integer)$avail.get($s,$d,$t)) != null &&
((Integer)$avail.get($s,$d,$t)) < $c
)
then
scoreHolder.addHardConstraintMatch(kcontext, (int)(((Integer)$avail.get($s,$d,$t)) - ($c)));
end
rule "Cannot exceed room resource constraint"
when
$loc : Location(rooms == true, $avail: availMap)
$s : Slot()
$d : Day()
$t : Time()
accumulate (
Shift (
isInitialized(),
occursOn($s,$d,$t),
isConsumingResc(),
primaryAt($loc),
$primRI : primaryResc
);
$s1 : sum($primRI.getNumRMs())
)
accumulate (
Shift (
isInitialized(),
occursOn($s,$d,$t),
isConsumingResc(),
concAt($loc),
$concRI : concurrentResc
);
$s2 : sum($concRI.getNumRMs())
)
eval(
((Integer)$avail.get($s,$d,$t)) != null &&
((Integer)$avail.get($s,$d,$t)) < ($s1 + $s2)
)
then
scoreHolder.addHardConstraintMatch(kcontext,(int)(((Integer)$avail.get($s,$d,$t)) - ($s1 + $s2)) );
end
Here's my configuration as well.
<?xml version="1.0" encoding="UTF-8"?>
<solver>
<!--<environmentMode>FAST_ASSERT</environmentMode>-->
<!--<environmentMode>FULL_ASSERT</environmentMode>-->
<scanAnnotatedClasses>
<packageInclude>com.labrador.providerplanner.domain</packageInclude>
</scanAnnotatedClasses>
<scoreDirectorFactory>
<scoreDrl>constraints.drl</scoreDrl>
</scoreDirectorFactory>
<!--<constructionHeuristic>
<constructionHeuristicType>CHEAPEST_INSERTION</constructionHeuristicType>
</constructionHeuristic>-->
<constructionHeuristic>
<queuedEntityPlacer>
<entitySelector id="placerEntitySelector">
<cacheType>PHASE</cacheType>
<entityClass>com.labrador.providerplanner.domain.FullDayNoSplitShift</entityClass>
</entitySelector>
<changeMoveSelector>
<entitySelector mimicSelectorRef="placerEntitySelector"/>
</changeMoveSelector>
</queuedEntityPlacer>
</constructionHeuristic>
<constructionHeuristic>
<queuedEntityPlacer>
<entitySelector id="placerEntitySelector">
<cacheType>PHASE</cacheType>
<entityClass>com.labrador.providerplanner.domain.HalfDayNoSplitShift</entityClass>
</entitySelector>
<cartesianProductMoveSelector>
<changeMoveSelector>
<entitySelector mimicSelectorRef="placerEntitySelector"/>
<valueSelector variableName="time"/>
</changeMoveSelector>
<changeMoveSelector>
<entitySelector mimicSelectorRef="placerEntitySelector"/>
<valueSelector variableName="day"/>
</changeMoveSelector>
</cartesianProductMoveSelector>
</queuedEntityPlacer>
</constructionHeuristic>
<constructionHeuristic>
<queuedEntityPlacer>
<entitySelector id="placerEntitySelector">
<cacheType>PHASE</cacheType>
<entityClass>com.labrador.providerplanner.domain.FullDayShift</entityClass>
</entitySelector>
<cartesianProductMoveSelector>
<changeMoveSelector>
<entitySelector mimicSelectorRef="placerEntitySelector"/>
<valueSelector variableName="slot"/>
</changeMoveSelector>
<changeMoveSelector>
<entitySelector mimicSelectorRef="placerEntitySelector"/>
<valueSelector variableName="day"/>
</changeMoveSelector>
</cartesianProductMoveSelector>
</queuedEntityPlacer>
</constructionHeuristic>
<constructionHeuristic>
<queuedEntityPlacer>
<entitySelector id="placerEntitySelector">
<cacheType>PHASE</cacheType>
<entityClass>com.labrador.providerplanner.domain.HalfDayShift</entityClass>
</entitySelector>
<cartesianProductMoveSelector>
<changeMoveSelector>
<entitySelector mimicSelectorRef="placerEntitySelector"/>
<valueSelector variableName="slot"/>
</changeMoveSelector>
<changeMoveSelector>
<entitySelector mimicSelectorRef="placerEntitySelector"/>
<valueSelector variableName="day"/>
</changeMoveSelector>
<changeMoveSelector>
<entitySelector mimicSelectorRef="placerEntitySelector"/>
<valueSelector variableName="time"/>
</changeMoveSelector>
</cartesianProductMoveSelector>
</queuedEntityPlacer>
</constructionHeuristic>
<localSearch>
<localSearchType>LATE_ACCEPTANCE</localSearchType>
<moveIteratorFactory>
<moveIteratorFactoryClass>com.labrador.providerplanner.solver.SingleProviderShiftSwapMoveIteratorFactory</moveIteratorFactoryClass>
</moveIteratorFactory>
<termination>
<secondsSpentLimit>60</secondsSpentLimit>
</termination>
</localSearch>
</solver>
The Problem With The Problem
Here's the main issue. The following is what the solver spits out. It gets stuck in this local optima. Lets look at, for instance, the Main OR Robot Location which is an outside Location that only three Providers work shifts in. P2 and P3 have 4 FullDayShift's each in the Main OR Robot. P1 has 4 FullDayShifts and 4 HalfDay Shifts. Main OR Robot can only accommodate 1 Provider in every Monday TimeSlot, 1 in Every Tuesday, 1 in Every Thursday and 1 in every Friday AM.
The Main OR Robot shifts are circled. P1 violates the resource constraint on Tuesday. P2 violates another resource constraint on Friday. The arrows show a set of moves which would successfully not break those constraints for Main OR Robot anymore. However, the issue is those spots contain Shifts at FH Cancer. Which is another outside location that is filled to the brim. Thus, moving those causes other constraints to break, probably more. So those have to be shuffled around which could cause problems with other Locations and on... and on.. and on... This really is just the crux of an optimization problem and it seems like the meta-heuristic is simply falling short of breaking out of a this local optima.
So this is where I am. I don't really know how to proceed. Are there coarser grained moves I can make to help the solver bust out of optima like this? I.e. trade location times between two providers. I've tried something like this and it didn't help. Do I need to rethink my domain model? I.e. should Locations have rooms or open slots on certain times based on their availability and should Providers be assigned to these? That might work but how do I encapsulate the larger shifts types in this domain? How do I ensure each provider works the number of shifts they need to in each location?
This is quite a complex problem and a wall of information. So thank you if you actually read all of this! Any help or direction at all is hugely appreciated as I am very stuck right now. If anything is confusing or more info is needed, please let me know and I'll do my best to clear it up.