Hello OptaPlanner community.
I am developing a Rest API to plan the routes of a fleet of vehicles. Looking for a tool that would help me I found Optaplanner, and I have seen that it has several success stories. In a first stage I made a planning taking into account the fastest distance and the capacity of the vehicle to cover all its visits. And I got the results I expected. Now I'm planning for time windows of visits and deposits, but I'm not successful yet.
Requirements
R1- I have a fleet of vehicles. Each vehicle has a capacity and its deposit and this deposit has a window of time. From the example of OptaPlanner for VRP I have only made a variation on the capacity that I handle as a float. As I understand it, all the vehicles in the OptaPlanner example are moved for a single depot. In my case, each vehicle has its own depot, each vehicle has its own fixed depot, and it is possible that several vehicles have the same depot.
R2- I have the visits (delivery services). Each visit has a demand and a window of time. From the example of OptaPlanner for VRP I have only made one modification regarding the demand that I handle it as a type "float".
In this process of adding this variant with TW to my routing problem I have some doubts and problems since I have not obtained a viable solution to my problem by applying TW:
1- I understand that I do not need to make modifications to the OptaPlanner example so that each vehicle cannot transport more items than its capacity. I only need to adjust the constrint provider so that the calculation is on float. I would like to know if I am right ? and on the other hand How I can manage the capacities and demands with dimensions?, in OptaPlanner it is a number but I need to manage it as volume and weight.
In the OptaPlanner domain I modified the variables "capacity" from vehicle and "demand" from visit, both to type "float".
Constraint vehicleCapacity(ConstraintFactory constraintFactory) {
return constraintFactory.from(PlanningVisit.class)
.groupBy(PlanningVisit::getVehicle, sumBigDecimal(PlanningVisit::getDemand))
.filter((vehicle, demand) -> demand.compareTo(vehicle.getCapacity()) > 0)
.penalizeLong(
"vehicle capacity",
HardSoftLongScore.ONE_HARD,
(vehicle, demand) -> demand.subtract(vehicle.getCapacity()).longValue());
}
2- In the OptaPlanner example I understand that the TW is a long that multiplies by a thousand, but I do not know if this long expresses an hour or date, or if it is just the hour converted into minutes and multiplied by a thousand.
What I am doing is converting the TW to minutes and multiplied by a thousand, for example if it is 8am, the ready time is a log equal to '480000'.
In the case of the service duration, I do not multiply it by 1000, I always treat it as 10 minutes. Am I doing the conversion correctly? , is this the long that OptaPlanner expects?
3- I understand that using the example of OptaPlanner for time windows I can solve this requirement (R2), without making variations, however for some reason that I can not find is not giving me back a feasible solution. It returns me for example: time spent (5000), best score (-3313987hard/-4156887soft).
I have thought that the error could be in the conversion of the dates of the time window or maybe some hard constraint that I lack, because the arrival times of the visits do not adapt to the time windows defined for visits nor for deposits.
For example:
I have 4 visits with time windows, 2 in the morning (visit 2 and visit 4) and 2 in the afternoon (visit 1 and visit 3).
I have two vehicles, vehicle 1 leaves a depot 1 that has a time window in a morning schedule and the other vehicle leaves a depot 2 that has a time window in an afternoon schedule.
So I expect vehicle 1 to conduct the visits that have a time window in the morning and vehicle 2 to conduct the visits that have a time window in the afternoon: [vehicle 1: {visit 2, visit 4}, vehicle 2: {visit 1, visit 3}]
I must be doing something very wrong, but I can't find where, not only does it not meet the TW of the deposit, but the arrival times of each visit exceed the defined TW. I don't understand why I get such big arrival times, that even exceed the defined limit for 1 day (all arrival times are over 1440000 = 1400min = 24 = 12am), that is, they arrived after this time.
This is the result I have obtained: score (-3313987hard/-4156887soft)
Route 1 referring to the route followed by vehicle 1
Vehicle 1
Depot 1 with TW (8:00 a 13:00)
ready_time: 480000
due_time: 780000
Visit 2 (8:30 a 12:30)
ready_time: 510000
due_time: 750000
service_duraration 10 = 10
arrival_time: 1823943
departure_time: 1833943
Visit 4 (9:30 a 12:30)
ready_time: 570000
due_time: 750000
service_duraration 10
arrival_time: 1739284
departure_time: 1739294
Visit 3 (14:40 a 15:30)
ready_time: 880000
due_time: 930000
service_duraration 10
arrival_time: 1150398
departure_time: 1150408
Route 2 referring to the route followed by vehicle 2
Vehicle 2
Depot 2 with TW (12:00 a 17:00)
ready_time: 720000
due_time: 1020000
Visit 1 (14:00 a 16:30)
ready_time: 840000
due_time: 990000
service_duraration 10 = 10
arrival_time: 2523243
departure_time: 2523253
This is my code, it can give you a better context.
This is my VariableListerner for updating the shadow variable 'arrival time'. I have not made any modifications, however the arrival times returned to me for each visit do not comply with the TW.
public class ArrivalTimeUpdatingVariableListener implements VariableListener<PlanningVisit> {
...
protected void updateArrivalTime(ScoreDirector scoreDirector, TimeWindowedVisit sourceCustomer) {
Standstill previousStandstill = sourceCustomer.getPreviousStandstill();
Long departureTime = previousStandstill == null ? null
: (previousStandstill instanceof TimeWindowedVisit)
? ((TimeWindowedVisit) previousStandstill).getDepartureTime()
: ((TimeWindowedDepot) ((PlanningVehicle)
previousStandstill).getDepot()).getReadyTime();
TimeWindowedVisit shadowCustomer = sourceCustomer;
Long arrivalTime = calculateArrivalTime(shadowCustomer, departureTime);
while (shadowCustomer != null && !Objects.equals(shadowCustomer.getArrivalTime(),
arrivalTime)) {
scoreDirector.beforeVariableChanged(shadowCustomer, "arrivalTime");
shadowCustomer.setArrivalTime(arrivalTime);
scoreDirector.afterVariableChanged(shadowCustomer, "arrivalTime");
departureTime = shadowCustomer.getDepartureTime();
shadowCustomer = shadowCustomer.getNextVisit();
arrivalTime = calculateArrivalTime(shadowCustomer, departureTime);
}
}
private Long calculateArrivalTime(TimeWindowedVisit customer, Long previousDepartureTime) {
if (customer == null || customer.getPreviousStandstill() == null) {
return null;
}
if (customer.getPreviousStandstill() instanceof PlanningVehicle) {
// PreviousStandstill is the Vehicle, so we leave from the Depot at the best suitable time
return Math.max(customer.getReadyTime(),
previousDepartureTime + customer.distanceFromPreviousStandstill());
}
return previousDepartureTime + customer.distanceFromPreviousStandstill();
}
}
And this service is where I build the domain entities from the data stored in the database (find). This TimeWindowedVehicleRoutingSolution is the one I use in the solver.
public TimeWindowedVehicleRoutingSolution find(UUID jobId) {
//load VRP from DB
RoutingProblem byJobId = routingProblemRepository.findVRP(jobId);
Set<Vehicle> vehicles = byJobId.getVehicles();
Set<Visit> visits = byJobId.getVisits();
//building solution
List<PlanningDepot> planningDepots = new ArrayList<>();
List<PlanningVehicle> planningVehicles = new ArrayList<>();
List<PlanningVisit> planningVisits = new ArrayList<>();
vehicles.forEach(vehicle -> {
//submit to planner location of the deposit, add to matrix for calculating distance
PlanningLocation planningLocation =
optimizer.submitToPlanner(vehicle.getDepot().getLocation());
//Depot, Vehicle and Visit are my persistence JPA entities, they are not the OptaPlanner
domain entities.
//The OptaPlanner domain entities are: PlanningVehicle, PlanningDepot and PlanningVisit
//I build the entities of the optaplanner domain from my persistence entities
Depot depot = vehicle.getDepot();
TimeWindowedDepot timeWindowedDepot = new TimeWindowedDepot();
TimeWindowedDepot timeWindowedDepot = new TimeWindowedDepot(depot.getId(),
planningLocation, depot.getStart(), depot.getEnd());
PlanningVehicle planningVehicle = new PlanningVehicle();
planningVehicle.setId(vehicle.getId());
planningVehicle.setCapacity(vehicle.getCapacity());
// each vehicle has its deposit
planningVehicle.setDepot(timeWindowedDepot);
planningVehicles.add(planningVehicle);
});
visits.forEach(visit -> {
//submit to planner location of the visit, add to matrix for calculating distance
PlanningLocation planningLocation = optimizer.submitToPlanner(visit.getLocation());
TimeWindowedVisit timeWindowedVisit = new TimeWindowedVisit();
TimeWindowedVisit timeWindowedVisit = new TimeWindowedVisit(visit.getId(),
planningLocation, visit.getLoad(),visit.getStart(), visit.getEnd(),
visit.getServiceDuration());
planningVisits.add(timeWindowedVisit);
});
//create TWVRP
TimeWindowedVehicleRoutingSolution solution = new TimeWindowedVehicleRoutingSolution();
solution.setId(jobId);
solution.setDepotList(planningDepots);
solution.setVisitList(planningVisits);
solution.setVehicleList(planningVehicles);
return solution;
}
Then I create the solver, start the optimization and finally save the best:
public void solve(UUID jobId) {
if (!planRepository.isResolved(jobId)) {
logger.info("Starting solver");
TimeWindowedVehicleRoutingSolution solution = null;
TimeWindowedVehicleRoutingSolution timeWindowedVehicleRoutingSolution = find(jobId);
try {
SolverJob<TimeWindowedVehicleRoutingSolution, UUID> solverJob =
solverManager.solve(jobId, timeWindowedVehicleRoutingSolution);
solution = solverJob.getFinalBestSolution();
save(jobId, solution);
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
} else
logger.info("This job already has a solution");
}
Any help on where the error is will be welcome. I am starting with Optaplanner, please any comments will be very helpful. Thank you very much!
Sorry about the calligraphy, English is not my language.
Thank you very much Geoffrey, I applied your suggestions and found the source of my problem. Grateful for your help!
I will comment on what happened, in case it is useful to someone:
It happens that I was using for the calculation of the distance the example of OptaWeb, that uses GrahHopper for this end and by default it returns the minimum distance, reason why calculation is as far as time. And by introducing time windows, I was breaking the score in:
Math.max(customer.getReadyTime(),
previousDepartureTime + customer.distanceFromPreviousStandstill())
My score was broken because I did not use the same conversion for all variables, the TW: ready time and departure time was expressed in minutes and multiplied by a thousand, while the distance was in milliseconds.
Example:
ready_time: 480000 (8:00 * 60 * 1000)
due_time: 780000 (13:00 * 60 * 1000)
As the distance returned to me:
distance: 641263
And therefore my score was broken.
What I did was to convert all my time variables to milliseconds:
"HH:MM", HH * 3 600 000 and MM * 60 000
Example:
ready_time: 28 800 000
due_time: 46 800 000
service_duration: 60 000 (10min per visit)
Now ready! The arrival time of each vehicle to your visits is adjusted to the defined time windows.
Related
I have a constraint with some joiners but the performance are very poor. Is it a way to improve it ?
I need to have the count of WorkingDay ( with ::hasPermission ) within the previous four days of the current day analyzed.
Here is my current constraint :
private Constraint fiveConsecutiveWorkingDaysMax(ConstraintFactory constraintFactory) {
return constraintFactory
.from(WorkingDay.class)
.filter(WorkingDay::hasPermission)
.join(WorkingDay.class,
Joiners.equal(WorkingDay::hasPermission),
Joiners.equal(WorkingDay::getAgent),
Joiners.filtering((wd1, wd2) -> {
LocalDate fourDaysBefore = wd1.getDayJava().minusDays(4);
Boolean wd2IsBeforeWd1 = wd2.getDayJava().isBefore(wd1.getDayJava());
Boolean wd2IsAfterFourDaysBeforeWd1 = wd2.getDayJava().compareTo(fourDaysBefore) >= 0;
return (wd2IsBeforeWd1 && wd2IsAfterFourDaysBeforeWd1);
}))
.groupBy((wd1, wd2) -> wd2, ConstraintCollectors.countBi())
.filter((wd2, count) -> count >= 4)
.penalizeConfigurable(FIVE_CONSECUTIVE_WORKING_DAYS_MAX);
}
Thanx for your help
There is potential for improvement here. First, we pre-filter the right hand side of the join to reduce the size of the cartesian product:
return constraintFactory
.forEach(WorkingDay.class)
.filter(WorkingDay::hasPermission)
.join(constraintFactory.forEach(WorkingDay.class)
.filter(WorkingDay::hasPermission),
Joiners.equal(WorkingDay::getAgent),
Joiners.filtering((wd1, wd2) -> {
LocalDate fourDaysBefore = wd1.getDayJava().minusDays(4);
Boolean wd2IsBeforeWd1 = wd2.getDayJava().isBefore(wd1.getDayJava());
Boolean wd2IsAfterFourDaysBeforeWd1 = wd2.getDayJava().compareTo(fourDaysBefore) >= 0;
return (wd2IsBeforeWd1 && wd2IsAfterFourDaysBeforeWd1);
}))
...
This has the added benefit of simplifying the index as it removes one equals joiner. Next, part of the filter can be replaced by a joiner as well:
return constraintFactory
.forEach(WorkingDay.class)
.filter(WorkingDay::hasPermission)
.join(constraintFactory.forEach(WorkingDay.class)
.filter(WorkingDay::hasPermission),
Joiners.equal(WorkingDay::getAgent),
Joiners.greaterThan(wd -> wd.getDayJava()),
Joiners.filtering((wd1, wd2) -> {
LocalDate fourDaysBefore = wd1.getDayJava().minusDays(4);
Boolean wd2IsAfterFourDaysBeforeWd1 = wd2.getDayJava().compareTo(fourDaysBefore) >= 0;
return wd2IsAfterFourDaysBeforeWd1;
}))
...
Finally, the method does needless boxing of boolean into Boolean, wasting CPU cycles and memory. This is a micro-optimization, but if the filter happens often enough, the benefit will be measurable.
A constraint refactored like this should perform better. That said, large joins are still going to take considerable time and the only way to work around that is to figure out a way to make them smaller.
Also, as Geoffrey said, I'd consider penalizing by the actual count, as what you have here is a textbook example of a score trap.
I don't see why this should be slow. Except maybe because the Cartesian Product explodes for a long time window. How many days is your time window?
Do note that the nurse rostering example has a totally different approach to detecting consecutive working days, using a custom collector. You might want to look at that in optaplanner-examples.
I'm trying to add constraints to School timetabling example. For example: "all groups should have the first lesson".
I tried EasyScore and Streaming - no success. EasyScore cant finds a proper solution, shuffles lessons a lot. Streaming gave me an error: Undo for (Lesson(subj...)) does not exist
Code for Streaming:
from(Lesson::class.java)
.filter { it.timeslot != null }
.groupBy({ it.studentGroup }, { it.timeslot!!.day }, ConstraintCollectors.toList())
.filter { group, day, list ->
list.any { it.timeslot!!.number != 1 }
}
.penalize(
"Student must have first lesson",
HardSoftScore.ONE_HARD
) { group, day, list -> list.count { it.timeslot!!.number != 1 } },
Looks like I'm thinking the wrong direction.
https://github.com/Lewik/timetable
Any help will be greatly appreciated.
update: fixed == -> =!
As far as I understand it, I don't think you're enforcing what you intend to enforce. From what I make from your source code, you penalize every studentgroup's first lesson of the day.
What you should do to enforce the intended goal, is to penalize every studentgroup that does NOT have a timeslot with number == 1 but DOES have one (of the same day) where timeslot number != 1.
So something like :
join all Lesson.class instances with all Lesson.class instances where the first lesson's studentGroup equals the second lesson's studentGroup AND the first lesson's timeSlot's day equals the second lesson's timeSlot's day. You obtain a BiConstraintStream<Lesson, Lesson> this way...
from this, filter all Lesson.class instances where the first lesson's timeSlot's number is less than the second lesson's timeSlot number
then penalise the remaining where the first lesson's timeSlot number differs from 1. That equals penalising all of a studentGroup's days where they have some lesson that day without having any lesson that day during the first timeslot.
If I understood you correctly, that's what you wanted ?
I don't know the real source of the problem, but it's about hashCode. The exception was thrown because HashMap with Object key can't find by that Object.
Lesson class:
#Serializable
#NoArg
#PlanningEntity
data class Lesson(
val subject: String,
val teacher: String,
val studentGroup: String,
#PlanningVariable(valueRangeProviderRefs = ["timeslotRange"])
var timeslot: TimeSlot? = null,
#PlanningId
val id: String = UUID.randomUUID().toString(),
)
The implementation above will not work. It could be fixed if I remove data or add override fun hashCode() = Objects.hash(id). #PlanningId does not help here. Kotlin generates hashCode for data classes and seems it not working with optaplanner (or vise versa)
How about using .ifNotExists()?
First, convert student group from a String into a class and add #ProblemFactCollectionProperty List<StudentGroup> on your solution, then do
from(StudentGroup.class)
.ifNotExists(from(Lesson.class).filter(Lesson::isFirstTimeslot),
equals(this -> this, Lesson::getStudentGroup)
.penalize(...);
I have 3 tables, Users, Tasks, TasksSession.
what I am trying to find is total session conducted and total days the user was involved in that task based on created_at group by statement.
User::with(['Task','TotalSessionsGroupByDate'])
->withCount(['TotalSessions','TotalSessionsGroupByDate'])
->whereId(2)
->first(),
With the above total_sessions_group_by_date_count = 12 and total_sessions_group_by_date_count = 9 records in an array.
function code for the TotalSessionsGroupByDate is as follows
public function TotalSessionsGroupByDate()
{
return $this->hasMany(TaskSession::class)->groupBy('created_at');
}
I have the total number of the session conducted / attendance but how to get the total number of days irrelevant to sessions. as per the example, total_sessions_group_by_date_count should be 9 and not 12 total_sessions_group_by_date_count.
the only solution that I was able to find as of now, able to come up is as below:
$user = User::with(['TaskIssues','TotalSessionsGroupByDate'])
->withCount(['TotalSessions'])
->whereId(16)
->first();
$user->TotalSessionsGroupByDate = $user->TotalSessionsGroupByDate->count();
any better way to do the same in model?
I want to get the User ID and it's average score from every minimum timestamp for each category. Here's the table structure
Skill Table
id | user_id | category | score | timestamp**
0....10............a................11........12
1....10............a................10........9
2....10............b................12........10
3....10............c................11........8
4....11............a................8........9
5....11............b................9........10
6....11............c................10........8
7....11............c................15........14
I want to get the result like this:
user_id | AVG(score)
10........11 (average id: 1,2,3)
11........9 (average id: 4,5,6)
For now I use the looping query for every user
foreach ($userIds as $id){
// in some case I need to get from only specified Ids not all of them
foreach ($category as $cat) {
// get the minimum timestamp's score for each category
$rawCategory = Skill::where("user_id", $id)->where("timestamp", "<=", $start)->where("category",$cat->id)->orderBy("timestamp", "desc")->first();
if($rawCategory){
$skillCategory[$cat->cat_name] = $rawCategory->score;
}
}
//get the average score
$average = array_sum($skillCategory) / count($skillCategory);
}
I want to create better Eloquent query to get the data like this with good performance (< 60 sec). Have anyone faced a similar problem and solved it? If so, can you please give me the link. Thanks
i had an interview with microsoft and they asked me this following question! i didn't knew how to solve it and i'm very interesting to know what's the solution
p.s: it's only for me to improve myself because i was denied..
anyways: please assume that EmployeeRepository and ServiceTicketsRepository are implementing EntityFramework ORM repositories. The actual storage is a SQL database in the cloud.
Bonus: what is the name of the anti-pattern?
//
// Return overall number of pending work tickets for all employees in the repository
//
public int GetTicketsForEmployees()
{
EmployeeRepository employeeRepository = new EmployeeRepository();
ServiceTicketsRepository serviceTicketRepository = new ServiceTicketRepository();
int ticketscount = 0;
var employees = employeeRepository.All.Select(e => new EmployeeSummary { Employee = e }).ToList();
foreach (var employee in employees)
{
var tickets = serviceTicketRepository.AllIncluding(t => t.Customer).Where(t => t.AssignedToID ==employee.Employee.ID).ToList();
ticketscount += tickets.Count();
}
return ticketscount;
{
This is called the 1 + N anti-pattern. It means that you will do 1 + N round trips to the database where N is the number of records in the Employee table.
It will do 1 query to find all employees, then for each employee do another query to find their tickets, in order to count them.
The performance issue is that when N grows, your application will do more and more round trips, each taking a few milliseconds. Even at only 1000 employees this will be slow.
In addition to the round trips, this code is fetching all the columns for all the rows in the Employee table and also from the Ticket table. This will add up to a lot of bytes and in the end might cause an out of memory exception when the number of Employees and Tickets have grown to a big amount.
The fix is to perform one query which counts all the tickets which belongs to employees and then only returning the count. This will become one round trip sending only a few bytes over the network.
I'am not a C# but what I can see from my side is you are not using any join procedure.
If you have 1 million of employees and you have about 1000 tickets per employee.
You will do a 1 billion of query (loop including) :/ and you just want to return a count of ticket reported by your employee
Edit : I supposed you are in a eager loading and during your loop your EntityFramework instance will be open for the all duration of your loop.
Edit 2 : With a inner join you wont have to repeat t => t.AssignedToID ==employee.Employee.ID The join will do that for you.