I am new to Gurobi (and solvers in general) and have set up a quadratic non-convex optimization problem by hand in LP file format. The following is a small example that is representative of much larger problems that I will be solving. Gurobi gives "Model is infeasible or unbounded" for this problem. I am setting params.NonConvex = 2 in the model to select the non-convex quadratic solver.
I do understand that having product terms of binary variables is bad form and will be fixing that once I can solve this issue.
Minimize
64 x_1_1 + 100 x_2_1 + 169 x_3_1 + 196 x_4_1 +
64 x_1_2 + 100 x_2_2 + 169 x_3_2 + 196 x_4_2 - r_1 - r_2
Subject To
denom1: x_1_1 + x_2_1 + x_3_1 + x_4_1 - d_1 = 0
denom2: x_1_2 + x_2_2 + x_3_2 + x_4_2 - d_2 = 0
num1: 8 x_1_1 + 10 x_2_1 + 13 x_3_1 + 14 x_4_1 - n_1 = 0
num2: 8 x_1_2 + 10 x_2_2 + 13 x_3_2 + 14 x_4_2 - n_2 = 0
ratio1: [ r_1 * d_1 - n_1 ^ 2 ] = 0
ratio2: [ r_2 * d_2 - n_2 ^ 2 ] = 0
item1: x_1_1 + x_1_2 = 1
item2: x_2_1 + x_2_2 = 1
item3: x_3_1 + x_3_2 = 1
item4: x_4_1 + x_4_2 = 1
bin1: -2 y_1 + [ x_1_1 * y_1 + x_2_1 * y_1 + x_3_1 * y_1 + x_4_1 * y_1 ] >= 0
bin2: -2 y_2 + [ x_1_2 * y_2 + x_2_2 * y_2 + x_3_2 * y_2 + x_4_2 * y_2 ] >= 0
empty1: x_1_1 + x_2_1 + x_3_1 + x_4_1 -
[ x_1_1 * y_1 + x_2_1 * y_1 + x_3_1 * y_1 + x_4_1 * y_1 ] = 0
empty2: x_1_2 + x_2_2 + x_3_2 + x_4_2 -
[ x_1_2 * y_2 + x_2_2 * y_2 + x_3_2 * y_2 + x_4_2 * y_2 ] = 0
Binary
x_1_1 x_2_1 x_3_1 x_4_1
x_1_2 x_2_2 x_3_2 x_4_2
y_1 y_2
End
All the actual decision variables are binary (the rest, r_1, r_2, n_1, n_2, d_1, d_2 are are computed values from the decision variables) and so the problem cannot be unbounded. The following solution is feasible (and also optimal, based on hand calculation):
x_1_1 = 1, x_1_2 = 1, x_1_3 = 0 x_1_4 = 0
x_2_1 = 0, x_2_2 = 0, x_3_2 = 1, x_4_2 = 1
y_1 = 1
y_2 = 1
I am wondering why I am getting "Model is infeasible or unbounded" in Gurobi.
"Model is infeasible or unbounded" is exactly what it sounds like: presolve cannot determine whether the model is infeasible or unbounded. To understand your model better, first set DualReductions=0, then solve it again to determine which case it is:
If it is infeasible, call computeIIS() to solve the IIS, then find the infeasibilities via the IISConstr.
If it is unbounded, then set InfUnbdInfo=1, solve, then retrieve the UnbdRay attribute to find the unbounded ray.
I found the problem. When I ran with presolve = 0, I found that the model was unbounded. From the objective function, it is clear that only r_1 and r_2 could possibly cause this. To confirm, I artificially constrained r_1 and r_2 and Gurobi was able to solve the problem.
Examining the solution that Gurobi gave when I artificially constrained r_1 and r_2 helped me to debug the model. In this formulation, if d_1 and n_1 are 0, then r_1 becomes unbounded (and analogously for d_1 and n_1). Technically it results in division by 0, but Gurobi does not seem to flag this.
Now I have to change the model so that the constraints ratio1 and ratio2 only apply when d_1 and d_2 are respectively non-zero.
Related
import numpy as np
import time
features, labels = d2l.get_data_ch7()
def init_adam_states():
v_w, v_b = torch.zeros((features.shape[1], 1),dtype=torch.float32), torch.zeros(1, dtype=torch.float32)
s_w, s_b = torch.zeros((features.shape[1], 1),dtype=torch.float32), torch.zeros(1, dtype=torch.float32)
return ((v_w, s_w), (v_b, s_b))
def adam(params, states, hyperparams):
beta1, beta2, eps = 0.9, 0.999, 1e-6
for p, (v, s) in zip(params, states):
v[:] = beta1 * v + (1 - beta1) * p.grad.data
s = beta2 * s + (1 - beta2) * p.grad.data**2
v_bias_corr = v / (1 - beta1 ** hyperparams['t'])
s_bias_corr = s / (1 - beta2 ** hyperparams['t'])
p.data -= hyperparams['lr'] * v_bias_corr / (torch.sqrt(s_bias_corr) + eps)
hyperparams['t'] += 1
def train_ch7(optimizer_fn, states, hyperparams, features, labels, batch_size=10, num_epochs=2):
# 初始化模型
net, loss = d2l.linreg, d2l.squared_loss
w = torch.nn.Parameter(torch.tensor(np.random.normal(0, 0.01, size=(features.shape[1], 1)), dtype=torch.float32),
requires_grad=True)
b = torch.nn.Parameter(torch.zeros(1, dtype=torch.float32), requires_grad=True)
def eval_loss():
return loss(net(features, w, b), labels).mean().item()
ls = [eval_loss()]
data_iter = torch.utils.data.DataLoader(torch.utils.data.TensorDataset(features, labels), batch_size, shuffle=True)
for _ in range(num_epochs):
start = time.time()
print(w)
print(b)
for batch_i, (X, y) in enumerate(data_iter):
l = loss(net(X, w, b), y).mean() # 使⽤平均损失
# 梯度清零
if w.grad is not None:
w.grad.data.zero_()
b.grad.data.zero_()
l.backward()
optimizer_fn([w, b], states, hyperparams) # 迭代模型参数
if (batch_i + 1) * batch_size % 100 == 0:
ls.append(eval_loss()) # 每100个样本记录下当前训练误差
# 打印结果和作图
print('loss: %f, %f sec per epoch' % (ls[-1], time.time() - start))
d2l.set_figsize()
d2l.plt.plot(np.linspace(0, num_epochs, len(ls)), ls)
d2l.plt.xlabel('epoch')
d2l.plt.ylabel('loss')
train_ch7(adam, init_adam_states(), {'lr': 0.01, 't': 1}, features, labels)
I want to implement the Adam algorithm in the follow code and I feel confused in the function named adam.
v = beta1 * v + (1 - beta1) * p.grad.data
s = beta2 * s + (1 - beta2) * p.grad.data**2
when I use the follow code, the loss function curve is figure 1.
figure 1
v[:] = beta1 * v + (1 - beta1) * p.grad.data
s = beta2 * s + (1 - beta2) * p.grad.data**2
or
v = beta1 * v + (1 - beta1) * p.grad.data
s[:] = beta2 * s + (1 - beta2) * p.grad.data**2
when I use the follow code, the loss function curve is figure 2.
figure 2
v[:] = beta1 * v + (1 - beta1) * p.grad.data
s[:] = beta2 * s + (1 - beta2) * p.grad.data**2
when I use the follow code, the loss function curve is figure 3.
figure 3
The loss function curve in case 3 has always been smoother than that in case 1.
The loss function curve in case 2 sometimes can't converge.
Why is different?
To answer the first question,
v = beta1 * v + (1 - beta1) * p.grad.data
is an out-of-place operation. Remember that python variables are references to objects. By assigning a new value to variable v, the underlying object which v referred to before this assignment will not be changed. Instead the expression beta1 * v + (1 - beta1) * p.grad.data results in a new tensor which is then referred to by v.
On the other hand
v[:] = beta1 * v + (1 - beta1) * p.grad.data
is an in-place operation. After this operation v still refers to the same underlying object, and the elements of that tensor are modified and replaced with the values of the new tensor beta1 * v + (1 - beta1) * p.grad.data.
Take a look at the following 3 lines to see why this matters
for p, (v, s) in zip(params, states):
v[:] = beta1 * v + (1 - beta1) * p.grad.data
s[:] = beta2 * s + (1 - beta2) * p.grad.data**2
v and s are actually referring to tensors which are stored in states. If we do in-place operations then the values in states are changed to reflect the value assigned to v[:] and s[:].
If out-of-place operations are used then the values in states remain unchanged.
I have the following MIP problem. Upper bound for pre_6_0 should not be infinite because it is calculated from inp1, inp2, inp3, and inp4, all of which are bounded on both sides.
Maximize
obj: pre_6_0
Subject To
c1: inp0 >= -84
c2: inp0 <= 174
c3: inp1 >= -128
c4: inp1 <= 128
c5: inp2 >= -128
c6: inp2 <= 128
c7: inp3 >= -128
c8: inp3 <= 128
c9: inp4 >= -128
c10: inp4 <= 128
c11: pre_6_0 + 0.03125 inp1 - 0.0078125 inp2 - 0.00390625 inp3
+ 0.00390625 inp4 = -2.5
c12: - 0.0078125 inp0 + pre_6_1 = -2.5
c13: - 0.00390625 inp0 - 0.01171875 inp3 + pre_6_2 = 6.5
c14: - 0.0078125 inp0 + pre_6_3 = -1.5
c15: - 0.00390625 inp0 - 0.0078125 inp3 + pre_6_4 = 6.5
Bounds
pre_6_0 Free
inp0 Free
inp1 Free
inp2 Free
inp3 Free
inp4 Free
pre_6_1 Free
pre_6_2 Free
pre_6_3 Free
pre_6_4 Free
Generals
pre_6_0 inp0 inp1 inp2 inp3 inp4 pre_6_1 pre_6_2 pre_6_3 pre_6_4
The MIP best bound is infinite because no feasible integer solution exists.
Indeed, all the variables in your ILP have been restricted to general integer values (Generals section).
Here an example by using GLPK to solve the ILP.
15 rows, 10 columns, 25 non-zeros
10 integer variables, none of which are binary
...
Solving LP relaxation...
GLPK Simplex Optimizer, v4.65
5 rows, 10 columns, 15 non-zeros
0: obj = -8.000000000e+00 inf = 1.631e+01 (5)
5: obj = -3.750000000e-01 inf = 0.000e+00 (0)
* 8: obj = 3.000000000e+00 inf = 0.000e+00 (0)
OPTIMAL LP SOLUTION FOUND
Integer optimization begins...
Long-step dual simplex will be used
+ 8: mip = not found yet <= +inf (1; 0)
+ 8: mip = not found yet <= tree is empty (0; 3)
PROBLEM HAS NO INTEGER FEASIBLE SOLUTION
Time used: 0.0 secs
Memory used: 0.1 Mb (63069 bytes)
I want to solve the following portfolio optimization problem by means of the Python API of Gurobi:
enter image description here
I have implemented the problem in the following code:
subassets = ptf_optimization.ptf.index
max_rebalancing = 0.50
max_number_rebalancing = 10
rebalancing_threshold = 0.02
J = np.array(Chosen_new.tolist())
# Define model and set parameters
m2 = gp.Model('eff_ret_portfolio')
m2.setParam("NonConvex", 2)
m2.setParam("MIPGapAbs", 5e-3)
m2.setParam("TimeLimit", 100.0)
# Define variables
vars = pd.Series(m2.addVars(subassets), index=subassets)
vars_diff = pd.Series(m2.addVars(subassets, lb=(-1)*np.ones(len(subassets)), ub=2*np.ones(len(subassets))), index=subassets)
vars_abs = pd.Series(m2.addVars(subassets, lb=np.zeros(len(subassets)), ub=2*np.ones(len(subassets))), index=subassets)
vars_ind = pd.Series(m2.addVars(subassets, vtype=GRB.BINARY), index=subassets)
m2.update()
# Set objective
m2.setObjective(vars.T.dot(BL_returns), GRB.MAXIMIZE)
## Set constraints
portfolio_variance = vars.T.dot(ptf_optimization.cov.dot(vars))
m2.addConstr(portfolio_variance == target_std**2)
m2.addConstr(vars.sum() == 1-cash, 'budget')
m2.addConstrs((vars[j] == 0 for j in range(len(J)) if J[j] == True), name='c1') #J defined in min vol optimization
m2.addConstrs((vars_diff[item] == vars[item]-w_start[item] for item in subassets), name='diff')
m2.addConstrs((vars_abs[item] == gp.abs_(vars_diff[item]) for item in subassets), name='abs')
for item in subassets:
m2.addConstr((vars_ind[item]==1) >> (vars_abs[item] >= 1e-5))
m2.addConstr((vars_ind[item]==0) >> (vars_abs[item] <= 1e-5))
m2.addConstr(quicksum(vars_ind) <= max_number_rebalancing, 'max_number_rebalancing')
m2.addConstrs((vars_abs[item]*vars_ind[item] >= rebalancing_threshold for item in subassets), name='rebalancing_threshold')
m2.addConstr(quicksum(vars_abs) <= max_rebalancing, 'max_rebalancing')
# Add absolute constraints
for item in subassets:
l = ptf_optimization.absolute_constraints[item]['Min']
u = ptf_optimization.absolute_constraints[item]['Max']
m2.addConstr(l <= vars[item], f'constraint_abs_{item}_lower')
m2.addConstr(vars[item] <= u, f'constraint_abs_{item}_upper')
# Add group constraints
for j, item in enumerate(ptf_optimization.group_constraints):
constraint = ptf_optimization.group_constraints[item]
l = constraint['Min']
u = constraint['Max']
vars_list = (vars[item] for item in constraint['Group'])
m2.addConstr(l <= quicksum(vars_list), f'constraint_group_{j}_lower')
m2.addConstr(quicksum(vars_list) <= u, f'constraint_group_{j}_upper')
m2.setParam('OutputFlag', 1)
m2.optimize()
However, by inserting the constraint that rebalancing must be above a certain threshold, I obtained that the problem becomes infeasible. I computed an IIS and obtain the following .ilp file:
\ Model eff_ret_portfolio_copy
\ LP format - for model browsing. Use MPS format to capture full model detail.
Maximize
Subject To
max_number_rebalancing: C96 + C97 + C98 + C99 + C100 + C101 + C102 + C103
+ C104 + C105 + C106 + C107 + C108 + C109 + C110 + C111 + C112 + C113
+ C114 + C115 + C116 + C117 + C118 + C119 + C120 + C121 + C122 + C123
+ C124 + C125 + C126 + C127 <= 10
rebalancing_threshold[Obbligazionari_Governativi_Dollari]: [ C64 * C96 ]
>= 0.02
rebalancing_threshold[Obbligazionari_High_Yield_Euro]: [ C70 * C102 ]
>= 0.02
rebalancing_threshold[Obbligazionari_Emergenti_Hard_Currency]: [
C71 * C103 ] >= 0.02
rebalancing_threshold[Azionari_Euro]: [ C73 * C105 ] >= 0.02
rebalancing_threshold[Obbligazionari_Governativi_Breve_Termine_Europe_ex_Euro]:
[ C84 * C116 ] >= 0.02
rebalancing_threshold[Obbligazionari_Governativi_Breve_Termine_Yen]: [
C85 * C117 ] >= 0.02
rebalancing_threshold[Obbligazionari_Inflation_Linked_Dollari]: [
C87 * C119 ] >= 0.02
rebalancing_threshold[Obbligazionari_Corporate_Finanziari]: [ C89 * C121 ]
>= 0.02
rebalancing_threshold[Obbligazionari_Corporate_Dollari_Breve_Termine]: [
C91 * C123 ] >= 0.02
rebalancing_threshold[Obbligazionari_ABS]: [ C92 * C124 ] >= 0.02
rebalancing_threshold[Obbligazionari_Convertible_Euro]: [ C94 * C126 ]
>= 0.02
Bounds
C64 free
C70 free
C71 free
C73 free
C84 free
C85 free
C87 free
C89 free
C91 free
C92 free
C94 free
Binaries
C96 C97 C98 C99 C100 C101 C102 C103 C104 C105 C106 C107 C108 C109 C110
C111 C112 C113 C114 C115 C116 C117 C118 C119 C120 C121 C122 C123 C124 C125
C126 C127
End
Can you please tell me if I coded the model correctly and where the source of infeasibility is? Thanks a lot in advance.
Paolo
I have two matrices: quantities and displacements.
The problem is as follows:
[0.011 * x + 0.0295 * y + 0.080 * w + 0.182 * z] = [-4.31, 8.15, 0.83]
[0.011 * x + 0.0220 * y + 0.098 * w + 0.180 * z] = [-3.70, 6.30, 1.03]
[0.013 * x + 0.0230 * y + 0.108 * w + 0.172 * z] = [-3.89, 6.33, 0.52]
[0.013 * x + 0.0230 * y + 0.105 * w + 0.175 * z] = [-3.38, 5.55, 0.54]
In numpy:
quantities = np matrix ([[0.011, 0.0295, 0.080, 0.182], [0.011, 0.022, 0.098, 0.180], [0.013, 0.023, 0.108, 0.172], [0.013, 0.023, 0.105, 0.175]))
displacements = np matrix ([[-4.31, 8.15, 0.83], [-3.7, 6.3, 1.03], [-3.89, 6.33, 0.52] , [-3.38, 5.55, 0.54]])
To obtain the displacement [-4.37, 7.44, 1.01], what are the values of x, y, w, z used?
That is:
[ax + by + cw + dz] = [-4.37, 7.44, 1.01]
What are the values of a, b, c and d?
I tried to use Scipy's fsolve to find the answers to a system of two nonlinear equations.
The two equations are:
f1 = math.log(x) + 1. - ((1. + (m - 1)*x) / m) + chi * (1 - x)**2
f2 = math.log(1 - x) - (m - 1)*x + chi*m*x**2
m and chi are constants in this case. The essential goal is to find x, y that satisfies simultaneously f1(x) = f1(y) and f2(x) = f2(y). I know the initial guess for x, y are 0.3 and 0.99. Below is my code.
from scipy.optimize import fsolve
import math
# some global variables
m = 46.663
chi = 1.1500799949128826
def binodal_fsolve():
def equations(p):
x, y = p
out = []
out.append(math.log(x) + 1. - ((1. + (m - 1)*x) / m) + chi * (1 - x)**2 - (math.log(y) + 1. - ((1. + (m - 1)*y) / m) + chi * (1 - y)**2))
out.append(math.log(1 - x) - (m - 1)*x + chi*m*x**2 - (math.log(1 - y) - (m - 1)*y + chi*m*y**2))
return out
initial_guess = [0.3, 0.99]
ans = fsolve(equations, initial_guess)
return ans
def test_answers(phiL, phiR):
def functions(x):
return math.log(x) + 1. - ((1. + (m - 1)*x) / m) + chi * (1 - x)**2, math.log(1 - x) - (m - 1)*x + chi*m*x**2
return functions(phiL)[0], functions(phiR)[0], functions(phiL)[1], functions(phiR)[1]
print (test_answers(0.2542983070, 0.9999999274))
# (1.3598772108380786e-09, -1.5558330624053502e-09, -8.434988430355375, -8.435122589529684)
res = binodal_fsolve()
print (res)
When I executed the code, I always encountered the math domain error.
However, if I tried to solve it using MAPLE fsolve. I can get the answers (0.2542983070, 0.9999999274).
By plugging these back to the equations, I get (1.3598772108380786e-09, -1.5558330624053502e-09, -8.434988430355375, -8.435122589529684) which suggests the answers are correct.
I don't know how to make scipy fsolve work. Any suggestions will be greatly appreciated.
In this case you can use the log function from numpy.lib.scimath that returns a complex number when its argument is negative.
Instead of using scipy.optimize.fsolve, use scipy.optimize.root and change the method to lm which solves the system of nonlinear equations in a least squares sense using a modification of the Levenberg-Marquardt algorithm. For more methods, see the documentation.
from scipy.optimize import root
import numpy.lib.scimath as math
# some global variables
m = 46.663
chi = 1.1500799949128826
def binodal_fsolve():
def equations(p):
x, y = p
out = []
out.append(math.log(x) + 1. - ((1. + (m - 1)*x) / m) + chi * (1 - x)**2 - (math.log(y) + 1. - ((1. + (m - 1)*y) / m) + chi * (1 - y)**2))
out.append(math.log(1 - x) - (m - 1)*x + chi*m*x**2 - (math.log(1 - y) - (m - 1)*y + chi*m*y**2))
return out
initial_guess = [0.3, 0.99]
#ans = fsolve(equations, initial_guess)
ans = root(equations, initial_guess, method='lm')
return ans
def test_answers(phiL, phiR):
def functions(x):
return math.log(x) + 1. - ((1. + (m - 1)*x) / m) + chi * (1 - x)**2, math.log(1 - x) - (m - 1)*x + chi*m*x**2
return functions(phiL)[0], functions(phiR)[0], functions(phiL)[1], functions(phiR)[1]
print (test_answers(0.2542983070, 0.9999999274))
# (1.3598772108380786e-09, -1.5558330624053502e-09, -8.434988430355375, -8.435122589529684)
res = binodal_fsolve()
print (res)
Which gives the following roots x and y: : array([0.25429812, 0.99999993]).
The full output:
(1.3598772108380786e-09, -1.5558330624053502e-09, -8.434988430355375, -8.435122589529684)
/home/user/.local/lib/python3.6/site-packages/scipy/optimize/minpack.py:401: ComplexWarning: Casting complex values to real discards the imaginary part
gtol, maxfev, epsfcn, factor, diag)
cov_x: array([[6.49303571e-01, 8.37627537e-07],
[8.37627537e-07, 1.08484856e-12]])
fjac: array([[ 1.52933340e+07, -1.00000000e+00],
[-1.97290115e+01, -1.24101235e+00]])
fun: array([-2.22945317e-07, -7.20367503e-04])
ipvt: array([2, 1], dtype=int32)
message: 'The relative error between two consecutive iterates is at most 0.000000'
nfev: 84
qtf: array([-0.00338589, 0.00022828])
status: 2
success: True
x: array([0.25429812, 0.99999993])