I am working on Business Problem for factory and developing Linear programming Solution. problem has thousands of Constraints and variables. I want to give priority sequence to constraints so that constraints which are lower in priority can be breached if no optimum solution.
My Question is how to Set the constraint priority sequnece for CPLEX/Gurobi Solver.I am using java as language ,Do we have any specific format/function etc?
This is usually done at the modeling level. Add slacks to the equations, and add a term to the objective that minimizes the slack using a penalty or cost coefficient. Sometimes you can even use some dollar figures for the cost (e.g. storage capacity constraint: then cost is something like the price of renting extra storage space). This process is sometimes called making the model elastic, or introducing hard and soft constraints and is quite often used in practical models.
Related
Apologies as this may be a general question for optimization:
For truly large scale optimization models it feels as if the model becomes quite complex and cumbersome before it is even testable. For small scale optimization problems, up to even 10-20 constraints its feasible to code the entire program and just stare at it and debug.
However, for large scale models with potentially 10s-100 of constraint equations it feels as if there should be a way to test subsections of the optimization model before putting the entire thing together.
Imagine you are writing a optimization for a rocket that needs to land on the moon the model tells the rocket how to fly itself and land on the moon safely. There might be one piece of the model that would dictate the gravitational effects of the earth and moon and their orbits together that would influence how the rocket should position itself, another module that dictates how the thrusters should fire in order to maneuver the rocket into the correct positions, and perhaps a final module that dictates how to optimally use the various fuel sources.
Is there a good practice to ensure that one small section (e.g the gravitational module) works well independently of the whole model. Then iteratively testing the rocket thruster piece, then the optimal fuel use etc. Since, once you put all the pieces together and the model doesn't resolve (perhaps due to missing constraints or variables) it quickly becomes a nightmare to debug.
What are the best practices if any for iteratively building and testing large-scale optimization models?
I regularly work on models with millions of variables and equations. ("10s-100 of constraint equations" is considered small-scale). Luckily they all have less than say 50 blocks of similar equations (indexed equations). Obviously just eyeballing solutions is impossible. So we add a lot of checks (also on the data, which can contain errors). For debugging, it is a very good idea to have a very small data set around. Finally, it helps to have good tools, such as modeling systems with type/domain checking, automatic differentiation etc.)
Often we cannot really check equations in isolation because we are dealing with simultaneous equations. The model only makes sense when all equations are present. So "iterative building and testing" is usually not possible for me. Sometimes we keep small stylized models around for documentation and education.
I have a vrp variant that minimizes cost for a set of liquid deliveries. I have been asked to minimize cost per unit instead.
The costs are: hourly vehicle costs from standstill, back to depot (just the time to previous standstill and time to depot from the VRP example, multiplied by the vehicle hourly rate), plus the cost of product.
The amount delivered varies depending on the solution, but can be calculated by doing a sum of the deliveries of each vehicle.
So I have three streams for costs and one for unit count. Is there a way to join them and divide the two sums? Or is a shadow variable the only way to do it?
For a shadow variable method, I would add "cost"to each customer and then have a single constraint that replaces all the soft constraints that looks like:
protected Constraint costPerUnit(ConstraintFactory factory) {
return factory.forEach(Customer.class)
.groupBy( c->sumLong(c.getCost()), sumLong(c.getLitres))
.penalizeLong(
HardSoftLongScore.ONE_SOFT,
(cost, amount) -> cost / amount)
.asConstraint("costOfProduct");
}
It seems like it would be very slow though.
edit: thinking about this some more, is there a performance reason for using constraint streams instead of just calculating the score in listerners and then using one simple constraint stream rule for all soft constraints?
Even though, with a lot of care and attention, you could probably implement a very fast listener to tackle this sort of problem, I doubt it would be as fast as a properly incremental solution.
Now does that solution need to be implemented using Constraint Streams? No. For small problems, EasyScoreCalculator will be, well, easy - but for small problems, you wouldn't need OptaPlanner. For problems large in size but easy in how the score is calculated, you may want to look into IncrementalScoreCalculator - those are tricky to implement, but once you get it right, there is no way you could be any faster. Well-designed incremental calculators routinely beat Constraint Streams in terms of performance.
The main benefit of Constraint Streams is good performance without the need for complex code. And you get constraint justifications, and therefore score explanations. The downside is you have to learn to think in the API, and there is some overhead. It's up to you to weigh these factors against each other and make the choice that is best for your particular problem.
I have a domain model where I penalize a score at multiple levels within a single rule. Consider a cloud scheduling problem where we have to assign processes to computers, but each process can be split amongst several computers. Each process has a threshold (e.g. 75%), and we can only "win" the process if we can schedule up to its threshold. We get some small additional benefit from scheduling the remaining 25% of the process, but our solver is geared to "winning" as many processes as possible, so we should be scheduling as many processes as possible to their threshold before scheduling the remainder of the process.
Our hard rule counts hard constraints (we can't schedule more processes on a computer than it can handle)
Our medium rule is rewarded for how many processes have been scheduled up to the threshold (no additional reward for going above 75%).
Our soft rule is rewarded for how many processes have been scheduled total (here we do get additional reward for going above 75%).
This scoring implementation means that it is more important to schedule all processes up to their threshold than to waste precious computer space scheduling 100% of a process.
When we used a drools implementation, we had a rule which rewarded the medium and soft levels simultaneously.
when
$process : Process()
$percentAllocated : calculatePercentProcessAllocated($process) //uses an accumulator over all computers
then
mediumReward;
if ($percentAllocated > $process.getThreshold()) {
mediumReward = $process.getThreshold();
}
else {
mediumReward = $percentAllocated;
}
softReward = $percentAllocated;
scoreHolder.addMultiConstraintMatch(0, mediumReward, softReward);
The above pseudo-drools is heavily simplified, just want to show how we were rewarded two levels at once.
The problem is that I don't see any good way to apply multi constraint matches using constraint streams. All the examples I see automatically add a terminator after applying a penalize or reward method, so that no further score modifications can be made. The only way I see to implement my rules is to make two rules which are identical outside of their reward calls.
I would very much like to avoid running the same constraint twice if possible, so is there a way to penalize the score at multiple levels at once?
Also to anticipate a possible answer to this question, it is not possible to split our domain model so that each process is two processes (one process from 0% to the threshold, and another from the threshold to 100%). Part of the accumulation that I have glossed over involves linking the two parts and would be too expensive to perform if they were separate objects.
There is currently no way in the constraint streams API to do that. Constraint weights are assumed to be constant.
If you must do this, you can take advantage of the fact that there really is no "medium" score. The medium part in HardMediumSoftScore is just another level of soft. Therefore 0hard/1medium/2soft would in practice behave the same as 0hard/1000002soft. If you pick a sufficiently high constant to multiply the medium part with, you can have HardSoftScore work just like HardMediumSoftScore and implement your use case at the same time.
Is it a hack? Yes. Do I like it? No. But it does solve your issue.
Another way to do that would be to take advantage of node sharing. I'll use an example that will show it better than a thousand words:
UniConstraintStream<Person> stream = constraintFactory.forEach(Person.class);
Constraint one = stream.penalize(HardSoftScore.ONE).asConstraint("Constraint 1")
Constraint two = stream.penalize(HardSoftScore.TEN).asConstraint("Constraint 2")
This looks like two constraints executing twice. And in CS-Drools, it is exactly that. But CS-Bavet can actually optimize this and will only execute the stream once, applying two different penalties at the end.
This is not a very nice programming model, you lose the fluency of the API and you need to switch to the non-default CS implementation. But again - if you absolutely need to do what you want to do, this would be another way.
I am working on a project that attempts to use simulated annealing to organise various items into designated sections while maintaining some stability requirements.
For this problem the neighbourhood configuration should remove two or more of these items and switch their places (there are exactly as many items as there are places for them), but I am unsure of how calculated or random this switch should be.
When testing if the stability requirements are fulfilled, I am able to tell vaguely how items should be repositioned in order to improve the stability, but should the neighbourhood configuration take this into account? Or will that make the algorithm too "greedy"?
Any help/direction is much appreciated!
I have some experience with simulated annealing problems, but I don't know what you mean by 'neighbourhood configuration'. My scenario used the following rules: 1) if making the simulation lower energy, always accept the exchange 2) if making the simulation higher energy, calculate a probability of exchange using the Boltzmann factor and randomise a yes/no for acceptance. "Energy" is abstracted as counting all the non-same-type nearest neighbour interactions and subtracting all the same-type interactions, so that segregating into regions of similar neighbours is energetically favourable.
I was recently talking with someone in Resource Management and we discussed the problem of assigning developers to projects when there are many variables to consider (of possibly different weights), e.g.:
The developer's skills & the technology/domain of the project
The developer's travel preferences & the location of the project
The developer's interests and the nature of the project
The basic problem the RM person had to deal with on a regular basis was this: given X developers where each developers has a unique set of attributes/preferences, assign them to Y projects where each project has its own set of unique attributes/requirements.
It seems clear to me that this is a very mathematical problem; it reminds me of old optimization problems from algebra and/or calculus (I don't remember which) back in high school: you know, find the optimal dimensions for a container to hold the maximum volume given this amount of material—that sort of thing.
My question isn't about the math, but rather whether there are any software projects/libraries out there designed to address this kind of problem. Does anyone know of any?
My question isn't about the math, but rather whether there are any software projects/libraries out there designed to address this kind of problem. Does anyone know of any?
In my humble opinion, I think that this is putting the cart before the horse. You first need to figure out what problem you want to solve. Then, you can look for solutions.
For example, if you formulate the problem by assigning some kind of numerical compatibility score to every developer/project pair with the goal of maximizing the total sum of compatibility scores, then you have a maximum-weight matching problem which can be solved with the Hungarian algorithm. Conveniently, this algorithm is implemented as part of Google's or-tools library.
On the other hand, let's say that you find that computing compatibility scores to be infeasible or unreasonable. Instead, let's say that each developer ranks all the projects from best to worst (e.g.: in terms of preference) and, similarly, each project ranks each developer from best to worst (e.g.: in terms of suitability to the project). In this case, you have an instance of the Stable Marriage problem, which is solved by the Gale-Shapley algorithm. I don't have a pointer to an established library for G-S, but it's simple enough that it seems that lots of people just code their own.
Yes, there are mathematical methods for solving a type of problem which this problem can be shoehorned into. It is the natural consequence of thinking of developers as "resources", like machine parts, largely interchangeable, their individuality easily reduced to simple numerical parameters. You can make up rules such as
The fitness value is equal to the subject skill parameter multiplied by the square root of the reliability index.
and never worry about them again. The same rules can be applied to different developers, different subjects, different scales of projects (with a SLOC scaling factor of, say, 1.5). No insight or real leadership is needed, the equations make everything precise and "assured". The best thing about this approach is that when the resources fail to perform the way your equations say they should, you can just reduce their performance scores to make them fit. And if someone has already written the tool, then you don't even have to worry about the math.
(It is interesting to note that Resource Management people always seem to impose such metrics on others in an organization -- thereby making their own jobs easier-- and never on themselves...)