Finding out reason of Pyomo model infeasibility - variables

I got a pyomo concrete model with lots of variables and constraints.
Somehow, one of the variable inside my model violates one constraint, which makes my model infeasible:
WARNING: Loading a SolverResults object with a warning status into model=xxxx;
message from solver=Model was proven to be infeasible.
Is there a way to ask the solver, the reason of the infeasibility?
So for example, lets assume I got a variable called x, and if I define following 2 constraints, model will be ofc. infeasible.
const1:
x >= 10
const2:
x <= 5
And what I want to achieve that pointing out the constraints and variable which causes this infeasibility, so that I can fix it. Otherwise with my big model it is kinda hard to get what causing this infeasibility.
IN: write_some_comment
OUT: variable "x" cannot fulfill "const1" and "const2" at the same time.

Many solvers (including IPOPT) will hand you back the value of the variables at solver termination, even if the problem was found infeasible. At that point, you do have some options.
There is contributed code in pyomo.util.infeasible that might help you out. https://github.com/Pyomo/pyomo/blob/master/pyomo/util/infeasible.py
Usage:
from pyomo.util.infeasible import log_infeasible_constraints
...
SolverFactory('your_solver').solve(model)
...
log_infeasible_constraints(model)

I would not trust any numbers that the solver loads into the model after reporting "infeasible." I don't think any solvers come w/ guarantees on the validity of those numbers. Further, unless a package can divine the modeler's intent, it isn't clear how it would list the infeasible constraints. Consider 2 constraints:
C1: x <= 5
C2: x >= 10
X ∈ Reals, or Integers, ...
Which is the invalid constraint? Well, it depends! Point being, it seems an impossible task to unwind the mystery based on values the solver tries.
A possible alternate strategy: Load the model with what you believe to be a valid solution, and test the slack on the constraints. This "loaded solution" could even be a null case where everything is zero'ed out (if that makes sense in the context of the model). It could also be a set of known feasible solutions tried via unit test code.
If you can construct what you believe to be a valid solution (forget about optimal, just something valid), you can (1) load those values, (2) iterate through the constraints in the model, (3) evaluate the constraint and look for negative slack, and (4) report the culprits with values and expressions
An example:
import pyomo.environ as pe
test_null_case = True
m = pe.ConcreteModel('sour constraints')
# SETS
m.T = pe.Set(initialize=['foo', 'bar'])
# VARS
m.X = pe.Var(m.T)
m.Y = pe.Var()
# OBJ
m.obj = pe.Objective(expr = sum(m.X[t] for t in m.T) + m.Y)
# Constraints
m.C1 = pe.Constraint(expr=sum(m.X[t] for t in m.T) <= 5)
m.C2 = pe.Constraint(expr=sum(m.X[t] for t in m.T) >= 10)
m.C3 = pe.Constraint(expr=m.Y >= 7)
m.C4 = pe.Constraint(expr=m.Y <= sum(m.X[t] for t in m.T))
if test_null_case:
# set values of all variables to a "known good" solution...
m.X.set_values({'foo':1, 'bar':3}) # index:value
m.Y.set_value(2) # scalar
for c in m.component_objects(ctype=pe.Constraint):
if c.slack() < 0: # constraint is not met
print(f'Constraint {c.name} is not satisfied')
c.display() # show the evaluation of c
c.pprint() # show the construction of c
print()
else:
pass
# instantiate solver & solve, etc...
Reports:
Constraint C2 is not satisfied
C2 : Size=1
Key : Lower : Body : Upper
None : 10.0 : 4 : None
C2 : Size=1, Index=None, Active=True
Key : Lower : Body : Upper : Active
None : 10.0 : X[foo] + X[bar] : +Inf : True
Constraint C3 is not satisfied
C3 : Size=1
Key : Lower : Body : Upper
None : 7.0 : 2 : None
C3 : Size=1, Index=None, Active=True
Key : Lower : Body : Upper : Active
None : 7.0 : Y : +Inf : True

Related

pyomo matrix product

I would like to use pyomo to solve a multiple linear regression under constraint in pyomo.
to do so I have 3 matrices :
X (noted tour1 in the following code) My inputs (600x13)(bureaux*t1 in pyomo)
Y (noted tour2 in the following code) the matrix I want to predict (6003)(bureauxt2 inpyomo)
T (noted transfer In the code) (13x3)(t1*t2 in pyomo)
I would like to do the following
ypred = XT
minimize (ypred-y)**2
subject to
0<T<1
and Sum_i(Tij)=1
To that effect, I started the following code
from pyomo.environ import *
tour1=pd.DataFrame(np.random.random(size=(60,13)),columns=["X"+str(i) for i in range(13)],index=["B"+str(i) for i in range(60)])
tour2=pd.DataFrame(np.random.random(size=(60,3)),columns=["Y"+str(i) for i in range(3)],index=["B"+str(i) for i in range(60)])
def gettour1(model,i,j):
return tour1.loc[i,j]
def gettour2(model,i,j):
return tour2.loc[i,j]
def cost(model):
return sum((sum(model.tour1[i,k] * model.transfer[k,j] for k in model.t1) - model.tour2[i,j] )**2 for i in model.bureaux for j in model.tour2)
model = ConcreteModel()
model.bureaux = Set(initialize=tour1.index.tolist())
model.t1 = Set(initialize=tour1.columns)
model.t2 = Set(initialize=tour2.columns)
model.tour1 = Param(model.bureaux, model.t1,initialize=gettour1)
model.tour2 = Param(model.bureaux, model.t2,initialize=gettour2)
model.transfer = Var(model.t1,model.t2,bounds=[0,1])
model.obj=Objective(rule=cost, sense=minimize)
I unfortunately get an error at this stage :
KeyError: "Index '('X0', 'B0', 'Y0')' is not valid for indexed component 'transfer'"
anyone knows how I can calculate the objective ?
furthermore any help for the constrains would be appreciated :-)
A couple things...
First, the error you are getting. There is information in that error statement that should help identify the problem. The construction appears to be trying to index transfer with a 3-part index (x, b, y). That clearly is outside of t1 x t2. If you look at the sum equation you have, you are mistakenly using model.tour2 instead of model.t2.
Also, your bounds parameter needs to be a tuple.
While building the model, you should be pprint()-ing the model very frequently to look for these types of issues. That only works well if you have "small" data. 60 x 13 may be the normal problem size, but it is a total pain to troubleshoot. So, start with something tiny, maybe 3 x 4. Make a Set, pprint(). Make a Constraint, pprint()... Once the model computes/solves with "tiny" data, just pop in the real stuff.

I use gurobipy to solve a simple problem,but after adding constraints containing exponential terms and solving them, I got a wrong result

I had trouble solving a simple problem with gurobi:
e^x+x=lnP
x=1
In Gurobipy,it transforms into this form:
x+y=temp
y=e^x
lnP=temp
x=1
The result is here:
Variable X
x 1
P 749.103
y 2.71828
Temp 3.71828
The code is as follows:
from gurobipy import *
model = Model('Antoine')
P = model.addVar(vtype=GRB.CONTINUOUS, name='P',lb=0)
x = model.addVar(vtype=GRB.CONTINUOUS, name='x',lb=0)
y = model.addVar(vtype=GRB.CONTINUOUS, name='y',lb=-GRB.INFINITY)
temp = model.addVar(vtype=GRB.CONTINUOUS, name='Temp1',lb=-GRB.INFINITY)
model.addConstr(x == 1)
model.addGenConstrExp(x,y)
model.addConstr(x+y == temp)
model.addGenConstrLog(P,temp)
model.setObjective(P, GRB.MINIMIZE)
model.write("test.lp")
model.optimize()
I don't know why the result of P is wrong
Gurobi represents nonlinear functions by piecewise linear approximations. When I solve the original model on my computer using Gurobi Optimizer 9.5.2, I get the following warning:
Warning: max constraint violation (2.9006e+00) exceeds tolerance
Warning: max general constraint violation (2.9006e+00) exceeds tolerance
Piecewise linearization of function constraints often causes big violation.
Try to adjust the settings of the related parameters, such as FuncPieces.
This means the default automatic linearization is not sufficiently accurate for this model. As suggested in the warning message, adjust the FuncPieces parameter to get a more accurate representation for this model. For example, with model.Params.FuncPieces=-1 on my computer, I get this more accurate result:
Variable X
-------------------------
P 41.29
x 1
y 2.71828
Temp1 3.71828

How to use PySCIPOpt for feasibility-only problem

I have used CVXPY and some of its LP solvers to determine whether a solution to an A*x <= b problem is feasible, and now I would like to try PySCIPOpt. I could not find an example of this in the docs, and I'm having trouble figuring out the right syntax. With CVXPY the code is simply:
def do_cvxpy(A, b, solver):
x = cvxpy.Variable(A.shape[1])
constraints = [A#x <= b] #The # denotes matrix multiplication in CVXPY
obj = cvxpy.Minimize(0)
prob = cvxpy.Problem(obj, constraints)
prob.solve(solver=solver)
return prob.status
I think with PySCIPOpt one cannot use matrix notation as above, but must treat vectors and matrices as collections of scalar variables, each of which has to be added individually, so I tried this:
def do_scip(A, b):
model = Model("XYZ")
x = {}
for i in range(A.shape[1]):
x[i] = model.addVar(vtype="C", name="x(%s)" % i)
model.setObjective(0) #Is this right for a feasibility-only problem?
model.addCons(A*x <= b) #This is certainly the wrong syntax
model.optimize()
return model.getStatus()
Could anyone please help me out with the correct form for the constraint in addCons() for this kind of problem, and confirm that an acceptable way to ask whether a solution is feasible is to simply pass 0 as the objective?
I'm still not positive about the setObjective(0), but at least I can get the code to run without errors by "unpacking" the A matrix and the b vector and adding each element as a constraint:
for i in range(ncols):
for j in range(nrows):
model.addCons(A[j,i]*x[i] <= b[i])
I also discovered that CVXPY actually has an interface to SCIP, but it gives me an error when I try to use it:
getSolObjVal cannot only be called in stage SOLVING without a valid solution
which seems to suggest that the interface cannot be used for feasibility-only problems.

Restricting one variable in my variable vector to integer in Pyomo

I have a minimization problem that currently has only continuous variables
min Cx
s.t. Ax <= b
lb <= x <= ub
Where C is my cost vector, A is my coefficient matrix and b is my fixed vector. X is my variable vector of continuous variables.
A = 24x29, x = 29x1, b = 24x1
I want to force one of the x variables to be an integer, how can that be done in Pyomo?
I am new to the package, any help is appreciated
Pyomo offers a way to set the domain of your variables. Let me prove it to you in this example that you can entirely run in a Python console with copy/paste.
Suppose you want to change x[1] to integers, you can use (1 being part of set S = {1,2,3}):
from pyomo.environ import ConcreteModel, Set, Var, Integers
# Create your model here (set params, vars, constraint...)
model = ConcreteModel()
model.S = Set(initialize={1,2,3})
model.x = Var(model.S)
Let's pause the example here and type model.x.display() in the Python console. You should see that the domain of all elements in model.x is set by default to Real. Let's continue.
# Here, we suppose your model is created.
# You can change the domain of one lement of your var using this line:
model.x[1].domain = Integers
Now, type model.x.display() in your Python console and you should see this:
x : Size=3, Index=S
Key : Lower : Value : Upper : Fixed : Stale : Domain
1 : None : None : None : False : True : Integers
2 : None : None : None : False : True : Reals
3 : None : None : None : False : True : Reals
So, only x[1] is integer.
In practice, we never mix continuous variables with integer variables. So for practical models, it is not a real restriction that we declare variables to be continuous or integer by their name. This rule is not exclusive to Pyomo. Modeling tools like AMPL and GAMS use the same paradigm.
Having said that, let's focus on your problem.
One way to tackle this, would be:
Create a single, scalar integer variable y.
Add the constraint y = x[342] (or whatever the x variable you want to be integer valued)
This is not as crazy as it sounds. A good MIP solver will presolve this away, so the performance impact should be minimal.

Difference of building constraints with BuildAction() vs normal way

As referred in pyomo documentation, BuildAction() is an advance topic and additionally according to the documentation, it is somewhat more efficient to build/solve? constraints with BuildAction().
An example constraint generated with BuildAction():
m.const1 = Constraint([(t, a) for t in m.TIME for a in m.AREA],
noruleinit=True)
def const1_rule(m):
for t in m.TIME:
for a in m.AREA:
lhs = (some_vars_1[t,a])
rhs = (some_vars_2[t,a])
m.const1.add((t,a), (lhs == rhs))
m.const1_build = BuildAction(rule=const1_rule)
So m.const1 = Constraint() builds the pyomo constraint without any rule via noruleinit=True.
Then m.const1_build = BuildAction() runs the function const1_rule, and in this function the constraints are added to m.const1 via .add().
An example constraint generated with normal way:
def const1_rule(m, t, a):
lhs = (some_vars_1[t,a])
rhs = (some_vars_2[t,a])
return lhs == rhs
m.const1 = Constraint(m.TIME, m.AREA, rule=const1_rule)
QUESTIONS:
Another application of BuildAction can be intialization of Pyomo model data from Python data structures, or efficient initialization of Pyomo model data from other Pyomo model data.
1) In what way it is more efficient?
2) What is the difference building the constraints with BuildAction()?
3) Should I use it, yes-no-why?
4) If BuildAction() is better, how would I take advantage of it? (for example; assume there is a different working principle of BuildAction(), and maybe because of that I don't need to create some pyomo Sets or Params.)