I know that column generation gives an optimal solution and it can be used with other heuristics. But does that make it an exact algorithm? Thanks in advance.
Traditional CG operates on the relaxed problem. Although it finds the optimal LP solution, this may not translate directly into an optimal MIP solution. For some problems (e.g. 1d cutting stock) there is evidence this gap is small, and we just apply the set of columns found for the relaxed problem to a final MIP knowing this is a good solution but necessarily optimal. So it is a heuristic.
With some effort you can use column generation inside a branch-and-bound algorithm (this is called branch-and-price). This gives proven optimal solutions.
An exact algorithm means that the algorithm can solve the optimization problem globally i.e it has given the global optima.
Column generation technique is conventionally applied to relaxed LP problem and tries to optimize the relaxed LP problem by constantly improving the current solution with the help of dual multipliers. It gives an exact LP solution for the relaxed LP problem. But sometimes in real-world problems, the exact solution of the relaxed Lp problem is not feasible to use, it needs to be translated to an integer solution in order to use it. Now if the problem scale is small, then there are many exact MIP algorithms (such as Branch and Bound) which can solve it exactly and give an integer solution. But if the problem is large-scale, even the exact MIP algorithms can take longer runtimes, hence, we use some special/intelligent heuristics to lower the difficulty of the MIP problem.
Summary: Column generation is an exact technique for solving the relaxed LP problem, not the original IP problem.
First, strictly speaking, all algorithms are heuristic, including Simplex Method.
Second, I think Column generation is a heuristic algorithm, because it solves the LP relaxation of the master problem. It does not guarantee IP optimal. Actually CG does not always converge very well.
Related
I have an optimization problem that is subjected to linear constraints.
How to know which method is better for modelling and solving the problem.
I am generally asking about solving a problem as a satisfiability problem (SAT or SMT) vs. Solving as a linear programming problem (ILP OR MILP).
I don't have much knowledge in both. So, please simplify your answer if you have any.
Generally speaking, the difference is that SAT is only trying for feasible solutions, while ILP is trying to optimize something subject to constraints. I believe some ILP solvers actually use SAT solvers to get an initial feasible solution. The sensor array problem you describe in a comment is formulated as an ILP: "minimize this subject to that." A SAT version of that would instead pick a maximum acceptable number of sensors and use that as a constraint. Now, this is a satisfiability problem, but not one that's easily expressed in conjunctive normal form. I'd recommend using a solver with a theory of integers. My favorite is Z3.
However, before you give up on optimizing, you should try GMPL / GLPK. You might be surprised by how tractable your problem is. If you're not so lucky, turn it into a satisfiability problem and bring out Z3.
Initially I modeled my objective function as follows:
argmin var(f(x),g(x))+var(c(x),d(x))
where f,g,c,d are linear functions
in order to be able to use linear solvers I modeled the problem as follows
argmin abs(f(x),g(x))+abs(c(x),d(x))
is it correct to change variance to absolute value in this context, I'm pretty sure they imply the same meaning as having the least difference between two functions
You haven't given enough context to answer the question. Even though your question doesn't seem to be about regression, in many ways it is similar to the question of choosing between least squares and least absolute deviations approaches to regression. If that term in your objective function is in any sense an error term then the most appropriate way to model the error depends on the nature of the error distribution. Least squares is better if there is normally distributed noise. Least absolute deviations is better in the nonparametric setting and is less sensitive to outliers. If the problem has nothing to do with probability at all then other criteria need to be brought in to decide between the two options.
Having said all this, the two ways of measuring distance are broadly similar. One will be fairly small if and only if the other is -- though they won't be equally small. If they are similar enough for your purposes then the fact that absolute values can be linearized could be a good motivation to use it. On the other hand -- if the variance-based one is really a better expression of what you are interested in then the fact that you can't use LP isn't sufficient justification to adopt absolute values. After all -- quadratic programming is not all that much harder than LP, at least below a certain scale.
To sum up -- they don't imply the same meaning, but they do imply similar meanings; and, whether or not they are similar enough depends upon your purposes.
I have a large MIP problem, and I use GLPSOL in GLPK to solve it. However, solving the LP relaxation problem takes many iterations, and each iteration the obj and infeas value are all the same. I think it has found the optimal solution, but it won't stop and has continued to run for many hours. Will this happen for every large-scale MIP/LP problem? How can I deal with such cases? Can anyone give me any suggestions about this? Thanks!
The problem of solving MIPs is NP-complete in general, which means that there are instances which can't be solved efficiently. But often our problems have enough structure, so that heuristics can help to solve these models. This allowed huge gains in solving-capabilities in the last decades (overview).
For understanding the basic-approach and understanding what exactly is the problem in your case (no progress in upper-bound, no progress in lower-bound, ...), read Practical Guidelines for Solving Difficult Mixed Integer Linear
Programs.
Keep in mind, that there are huge gaps between commercial solvers like Gurobi / Cplex and non-commercial ones in general (especially in MIP-solving). There is a huge amount of benchmarks here.
There are also a lot of parameters to tune. Gurobi for example has different parameter-templates: one targets fast findings of feasible solution; one targets to proof the bounds.
My personal opinion: compared to cbc (open-source) & scip (open-source but non-free for commercial usage), glpk is quite bad.
I have a directed graph which is strongly connected and every node have some price(plus or negative). I would like to find best (highest score) path from node A to node B. My solution is some kind of brutal force so it takes ages to find that path. Is any algorithm for this or any idea how can I do it?
Have you tried the A* algorithm?
It's a fairly popular pathfinding algorithm.
The algorithm itself is not to difficult to implement, but there are plenty of implementations available online.
Dijkstra's algorithm is a special case for the A* (in which the heuristic function h(x) = 0).
There are other algorithms who can outperform it, but they usually require graph pre-processing. If the problem is not to complex and you're looking for a quick solution, give it a try.
EDIT:
For graphs containing negative edges, there's the Bellman–Ford algorithm. Detecting the negative cycles comes at the cost of performance, though (worse than the A*). But it still may be better than what you're currently using.
EDIT 2:
User #templatetypedef is right when he says the Bellman-Ford algorithm may not work in here.
The B-F works with graphs where there are edges with negative weight. However, the algorithm stops upon finding a negative cycle. I believe that is a useful behavior. Optimizing the shortest path in a graph that contains a cycle of negative weights will be like going down a Penrose staircase.
What should happen if there's the possibility of reaching a path with "minus infinity cost" depends on the problem.
This question is related to dynamic programming and specifically rod cutting problem from CLRS Pg 362
The overall optimal solution incorporates optimal solutions to two related subproblems.
The overall optimal solution is obtained by finding optimal solutions to individual subproblems and then somehow clubbing them. I can't understand the intuition and concept. Any links, examples?
thanks
You can compare dynamic programming and greedy approach.
Optimal substructure means that optimal solution can be found by finding optimal solutions to subproblems. If this is not the case, than sum of optimal solution of subproblems doesn't give us optimal global solution.
For example, consider Dijkstra's algorithm. If we know shortest path from node A to node C than we can use this information to find shortest path to another nodes as well.
If this is not the case, we can't compose optimal solutions to subproblems and get global optimal solution, than we can use greedy approach. Look at change making problem for example. Greedy algorithm makes local optimal decisions and find some solution, but this solution is not guaranteed to be optimal.