I would like to prove the following: "For a Linear Programming in standard form with constraint Ax = b and all variables >= 0 show that d is a direction of unboundedness if and only if Ad = 0 and all entries in d >= 0. Please help.
I highly recommend Bertsekas - Introduction to Linear Optimization, since it deals with Linear Programming in a graphical and intuitive way. It also contains the proof you seek.
A few hints:
If Ad = 0, and Ax = b, then A(x + td) = b for t >= 0;
Then, if d >= 0, what does this say about x + td? Does it ever become smaller than 0?
Now, the other way around:
If d is a direction for unboundedness, what happens if any d < 0?
Similarly, what happens if Ad != 0?
Related
I'm struggling a bit finding a fast algorithm that's suitable.
I just want to minimize:
norm2(x-s)
st
G.x <= h
x >= 0
sum(x) = R
G is sparse and contains only 1s (and zeros obviously).
In the case of iterative algorithms, it would be nice to get the interim solutions to show to the user.
The context is that s is a vector of current results, and the user is saying "well the sum of these few entries (entries indicated by a few 1.0's in a row in G) should be less than this value (a row in h). So we have to remove quantities from the entries the user specified (indicated by 1.0 entries in G) in a least-squares optimal way, but since we have a global constraint on the total (R) the values removed need to be allocated in a least-squares optimal way amongst the other entries. The entries can't go negative.
All the algorithms I'm looking at are much more general, and as a result are much more complex. Also, they seem quite slow. I don't see this as a complex problem, although mixes of equality and inequality constraints always seem to make things more complex.
This has to be called from Python, so I'm looking at Python libraries like qpsolvers and scipy.optimize. But I suppose Java or C++ libraries could be used and called from Python, which might be good since multithreading is better in Java and C++.
Any thoughts on what library/package/approach to use to best solve this problem?
The size of the problem is about 150,000 rows in s, and a few dozen rows in G.
Thanks!
Your problem is a linear least squares:
minimize_x norm2(x-s)
such that G x <= h
x >= 0
1^T x = R
Thus it fits the bill of the solve_ls function in qpsolvers.
Here is an instance of how I imagine your problem matrices would look like, given what you specified. Since it is sparse we should use SciPy CSC matrices, and regular NumPy arrays for vectors:
import numpy as np
import scipy.sparse as spa
n = 150_000
# minimize 1/2 || x - s ||^2
R = spa.eye(n, format="csc")
s = np.array(range(n), dtype=float)
# such that G * x <= h
G = spa.diags(
diagonals=[
[1.0 if i % 2 == 0 else 0.0 for i in range(n)],
[1.0 if i % 3 == 0 else 0.0 for i in range(n - 1)],
[1.0 if i % 5 == 0 else 0.0 for i in range(n - 1)],
],
offsets=[0, 1, -1],
)
a_dozen_rows = np.linspace(0, n - 1, 12, dtype=int)
G = G[a_dozen_rows]
h = np.ones(12)
# such that sum(x) == 42
A = spa.csc_matrix(np.ones((1, n)))
b = np.array([42.0]).reshape((1,))
# such that x >= 0
lb = np.zeros(n)
Next, we can solve this problem with:
from qpsolvers import solve_ls
x = solve_ls(R, s, G, h, A, b, lb, solver="osqp", verbose=True)
Here I picked CVXOPT but there are other open-source solvers you can install such as ProxQP, OSQP or SCS. You can install a set of open-source solvers by: pip install qpsolvers[open_source_solvers]. After some solvers are installed, you can list those for sparse matrices by:
print(qpsolvers.sparse_solvers)
Finally, here is some code to check that the solution returned by the solver satisfies our constraints:
tol = 1e-6 # tolerance for checks
print(f"- Objective: {0.5 * (x - s).dot(x - s):.1f}")
print(f"- G * x <= h: {(G.dot(x) <= h + tol).all()}")
print(f"- x >= 0: {(x + tol >= 0.0).all()}")
print(f"- sum(x) = {x.sum():.1f}")
I just tried it with OSQP (adding the eps_rel=1e-5 keyword argument when calling solve_ls, otherwise the returned solution would be less accurate than the tol = 1e-6 tolerance) and it found a solution is 737 milliseconds on my (rather old) CPU with:
- Objective: 562494373088866.8
- G * x <= h: True
- x >= 0: True
- sum(x) = 42.0
Hoping this helps. Happy solving!
I have 4 non negative real variable that are A, B, C and X. Based on the current problem that I have, I notice that the variable X must belong to the interval of [B,C] and the relation will be a bunch of if-else conditions like this:
If A < B:
x = B
elseif A > C:
x = C
elseif B<=A<=C:
x = A
As you can see, it quite difficult to reformulate as a Mixed Integer Programming problem with corresponding decision variable (d1, d2 and d3). I have try reading some instructions regarding if-then formulation using big M method at this site:
https://www.math.cuhk.edu.hk/course_builder/1415/math3220/L2%20(without%20solution).pdf but it seem that this problem is more challenging than their tutorial.
Could you kindly provide me with a formulation for this situation ?
Thank you very much !
I have an arbitrary set of constraints. For example:
A, B, C and D are 8-bit integers.
A + B + C + D = 50
(A + B) = 25
(C + D) = 30
A < 10
I can convert this to a SAT problem that can be solved by picosat (I can't get minisat to compile on my Mac), or to a SMT problem that can be solved by CVC4. To do that I need to:
Map these equations into a Boolean circuit.
Apply Tseitin's transformation to the circuit and convert it into DIMACS format.
My questions:
What tools do I use to do the conversion to the circuit?
What are the file formats for circuits and which ones are commonly used?
What tools do I use to transform the circuit to DIMACS format?
Conceptually
Build a circuit, then apply Tseitin's transform.
You'll need to express the addition and comparison operators as boolean logic. There are standard ways to build a circuit for twos-complement addition and for twos-complement comparison.
Then, use Tseitin's transform to convert this to a SAT instance.
In practice
Use a SAT front-end that will do this conversion for you. Z3 will take care of this for you. So will STP. (The conversion is sometimes known as "bit-blasting".)
In MiniZinc, you could just write a constraint programming model:
set of int: I8 = 0..255;
var I8: A;
var I8: B;
var I8: C;
var I8: D;
constraint A + B + C + D == 50;
constraint (A + B) = 25;
constraint (C + D) = 30;
constraint A < 10;
solve satisfy;
The example constraints cannot be satisfied, because of 25 + 30 > 50.
The Python interface of Z3 would allow the following:
from z3 import *
A, B, C, D = Ints('A B C D')
s = Solver()
s.add(A >= 0, A < 256)
s.add(B >= 0, B < 256)
s.add(C >= 0, C < 256)
s.add(A+B+C+D == 50)
s.add((A+B) == 25)
s.add((C+D) == 30)
s.add(A < 10)
print(s.check())
print(s.model())
So I have an answer. There's a program called Sugar: a SAT-based Constraint Solver which takes a series of constraints as S-expressions, converts the whole thing into a DIMACS file, runs the SAT solver, and then converts the results of the SAT solver back to the results of your constraitns.
Sugar was developed by Naoyuki Tamura to solve math puzzles like Sudoku puzzles. I found that it make it extremely simple to code complex constraints and run them.
For example, to find the square-root of 625, one could do this:
(int X 0 625)
(= (* X X) 625)
The first line says that X ranges from 0 to 625. The second line says that X*X is 625.
This may not be as simple and elegant as Z3, but it worked really well.
Suppose we have N (in this example N = 3) events that can happen depending on some variables. Each of them can generate certain profit or loses (event1 = 300, event2 = -100, event3 = 200), they are constrained by rules when they happen.
event 1 happens only when x > 5,
event 2 happens only when x = 2 and y = 3
event 3 happens only when x is odd.
The problem is to know the maximum profit.
Assume x, y are integer numbers >= 0
In the real problem there are many events and many dimensions.
(the solution should not be specific)
My question is:
Is this linear programming problem? If yes please provide solution to the example problem using this approach. If no please suggest some algorithms to optimize such problem.
This can be formulated as a mixed integer linear program. This is a linear program where some of the variables are constrained to be integer. Contrary to linear programs, solving the general integer program is NP-hard. However, there are many commercial or open source solvers that can solve efficiently large-scale problems. For up to 300 variables and constraints, you can use excel's solver.
Here is a way to formulate the above constraints:
If you go down this route, you might find this document useful.
the last constraint in an interesting one. I am assuming that x has to be integer, but if x can be either integer or continuous I will edit the answer accordingly.
I hope this helps!
Edit: L and U above should be interpreted as L1 and U1.
Edit 2: z2 needs to changed to (1-z2) on the 3rd and 4th constraint.
A specific answer:
seems more like a mathematical calculation than a programming problem, can't you just run a loop for x= 1->1000 to see what results occur?
for the example:
as x = 2 or 3 = -200 then x > 2 or 3, and if x < 5 doesn't get the 300, so all that is really happening is x > 5 and x = odd = maximum results.
x = 7 = 300 + 200 . = maximum profit for x
A general answer:
I don't see how to answer the question without seeing what the events are and how the events effect X ? Weather it's a linear or functional (mathematical) answer seems rather beside the point of finding the desired solution.
I want to investigate whether it is profitable to invest in an additional production facility, and hence I have to account for an capitilization in my objective function.
As such I am wondering if it is possible to, say, if y[t] = 1, then y[g] = 1
for g != t, g > t and where g,t is a subset of the time interval set T.
My first thought was to have:
subject to Constraint1:
y[t] = y[t-1] for all t in T
But that must surely render the solution of y to become the start value in y[0] which is something I obviously do not want.
For clarification. Assume that y[t] is a binary variable whose value is 1 if the investment is undertaken in time t, otherwise 0.
Hope anyone can shed some light into this!
Regards
The constraint y[t] = 1, then y[g] = 1 for g != t, g > t can be represented in AMPL as something like:
s.t. c{t in T: t != t0}: y[t + 1] >= y[t];
where t0 is the first element of set T. Note the use of >= instead of =. If y[t] is 1 for some t, it will drive y for all subsequent values of t to 1.