I am trying to solve a portfolio optimisation problem with the constraint that weights can be either zero or at least min (a Nx1 vector).
import cvxpy as cp
w = cp.Variable(len(mu))
mins = np.ones(len(mu)) * 0.03
risk = cp.quad_form(w, S)
prob = cp.Problem(cp.Minimize(risk),
[cp.sum(w) == 1,
w >= 0,
w >= min OR w == 0 # pseudocode for my desired constraint]
This is equivalent to a constraint that the weights are NOT 0 < w <= min, but I cannot find a way to express this in CVXPY (I have googled things like "cvxpy OR constraint" to no avail).
It feels like I'm missing something obvious. Perhaps there is a solution involving some boolean vector?
This is called w being a semi-continuous variable. Most advanced solvers support this type of variable directly. As CVXPY does not understand semi-continuous variables, we can use binary variables δ ∈ {0,1} and form the constraints:
δ⋅min ≤ w ≤ δ⋅max
where we can set max=1.
This makes the problem a MIQP (Mixed-Integer Quadratic Programming) problem. This usually means that you need to use a high-end solver that supports this type of model.
Based on Erwin's answer, this is the working code.
import cvxpy as cp
w = cp.Variable(n)
mins = np.ones(n) * 0.03
maxs = np.ones(n)
risk = cp.quad_form(w, S)
prob = cp.Problem(cp.Minimize(risk),
[cp.sum(w) == 1,
w >= 0,
w >= cp.multiply(k, mins),
w <= cp.multiply(k, maxs)])
prob.solve(solver="ECOS_BB")
EDIT: changed k # mins to cp.multiply(k, mins) as per comment
Related
I'm struggling a bit finding a fast algorithm that's suitable.
I just want to minimize:
norm2(x-s)
st
G.x <= h
x >= 0
sum(x) = R
G is sparse and contains only 1s (and zeros obviously).
In the case of iterative algorithms, it would be nice to get the interim solutions to show to the user.
The context is that s is a vector of current results, and the user is saying "well the sum of these few entries (entries indicated by a few 1.0's in a row in G) should be less than this value (a row in h). So we have to remove quantities from the entries the user specified (indicated by 1.0 entries in G) in a least-squares optimal way, but since we have a global constraint on the total (R) the values removed need to be allocated in a least-squares optimal way amongst the other entries. The entries can't go negative.
All the algorithms I'm looking at are much more general, and as a result are much more complex. Also, they seem quite slow. I don't see this as a complex problem, although mixes of equality and inequality constraints always seem to make things more complex.
This has to be called from Python, so I'm looking at Python libraries like qpsolvers and scipy.optimize. But I suppose Java or C++ libraries could be used and called from Python, which might be good since multithreading is better in Java and C++.
Any thoughts on what library/package/approach to use to best solve this problem?
The size of the problem is about 150,000 rows in s, and a few dozen rows in G.
Thanks!
Your problem is a linear least squares:
minimize_x norm2(x-s)
such that G x <= h
x >= 0
1^T x = R
Thus it fits the bill of the solve_ls function in qpsolvers.
Here is an instance of how I imagine your problem matrices would look like, given what you specified. Since it is sparse we should use SciPy CSC matrices, and regular NumPy arrays for vectors:
import numpy as np
import scipy.sparse as spa
n = 150_000
# minimize 1/2 || x - s ||^2
R = spa.eye(n, format="csc")
s = np.array(range(n), dtype=float)
# such that G * x <= h
G = spa.diags(
diagonals=[
[1.0 if i % 2 == 0 else 0.0 for i in range(n)],
[1.0 if i % 3 == 0 else 0.0 for i in range(n - 1)],
[1.0 if i % 5 == 0 else 0.0 for i in range(n - 1)],
],
offsets=[0, 1, -1],
)
a_dozen_rows = np.linspace(0, n - 1, 12, dtype=int)
G = G[a_dozen_rows]
h = np.ones(12)
# such that sum(x) == 42
A = spa.csc_matrix(np.ones((1, n)))
b = np.array([42.0]).reshape((1,))
# such that x >= 0
lb = np.zeros(n)
Next, we can solve this problem with:
from qpsolvers import solve_ls
x = solve_ls(R, s, G, h, A, b, lb, solver="osqp", verbose=True)
Here I picked CVXOPT but there are other open-source solvers you can install such as ProxQP, OSQP or SCS. You can install a set of open-source solvers by: pip install qpsolvers[open_source_solvers]. After some solvers are installed, you can list those for sparse matrices by:
print(qpsolvers.sparse_solvers)
Finally, here is some code to check that the solution returned by the solver satisfies our constraints:
tol = 1e-6 # tolerance for checks
print(f"- Objective: {0.5 * (x - s).dot(x - s):.1f}")
print(f"- G * x <= h: {(G.dot(x) <= h + tol).all()}")
print(f"- x >= 0: {(x + tol >= 0.0).all()}")
print(f"- sum(x) = {x.sum():.1f}")
I just tried it with OSQP (adding the eps_rel=1e-5 keyword argument when calling solve_ls, otherwise the returned solution would be less accurate than the tol = 1e-6 tolerance) and it found a solution is 737 milliseconds on my (rather old) CPU with:
- Objective: 562494373088866.8
- G * x <= h: True
- x >= 0: True
- sum(x) = 42.0
Hoping this helps. Happy solving!
The input vector is x, which contains 5 elements and I want to optimize (minimize) a function with a constraint that all elements of x should be greater than or equal to 0.
ie., x[i] >= 0 for 1 <= i <= 5
In one of the answer I saw this and used in my code, but the answer is returning negative values also
def constraint2(x):
"""constrain all elements of a to be >= 0"""
return x
cons2 = {'type': 'ineq', 'fun': constraint2}
Where am I going wrong? How to enforce the constraint?
all(ele >= 0 for ele in x)
Try this
I wrote a Gorubi optimization code, but because of some issues, I need to convert to Scipy code. Still have difficulties to convert it. Here is a part of code related to Gorubi:
m = Model()
#x is charging, discharging variable
x = m.addVars(n,lb=-1.5,ub=1.5,vtype=GRB.INTEGER, name="x")
#Y is SOC variable
Y = m.addVars(n+1,lb=0,ub=100,vtype=GRB.CONTINUOUS, name="Y")
# Add constraint: SOC[start]=50, initial SOC
m.addConstr(Y[0]==initialsoc,name='c1')
#Final targeted SOC
m.addConstr(Y[n]>=65,name='c2')
m.addConstrs((Y[i+1]-Y[i] == 3.75*x[i] for i in range(n)), name='c0')
#Objective function. 6 comes from capacity of inverter.
obj1=sum(((load[i+1]-(6*x[i]))*(load[i+1]-(6*x[i])) for i in range (n)))
m.setObjective(obj1,GRB.MINIMIZE)
m.optimize()
My x constraint can only have -1 or 0 or 1 values. The other constraint is Y where at each step y(i+1)-y(i) equals to 3.75*X(i).
Is it possible to convert this code to Scipy? Or do you recommend any other libraries?
I am trying to use Pytorch for non-convex optimisation, trying to maximise my objective (so minimise in SGD). I would like to bound my dependent variable x > 0, and also have the sum of my x values be less than 1000.
I think I have the penalty implemented correctly in the form of a ramp penalty, but am struggling with the bounding of the x variable. In Pytorch you can set the bounds using clamp but it doesn't seem appropriate in this case. I think this is because optim needs the gradients free under the hood. Full working example:
import torch
from torch.autograd import Variable
import numpy as np
def objective(x, a, b, c): # Want to maximise this quantity (so minimise in SGD)
d = 1 / (1 + torch.exp(-a * (x)))
# Checking constraint
exceeded_limit = constraint(x).item()
#print(exceeded_limit)
obj = torch.sum(d * (b * c - x))
# If overlimit add ramp penalty
if exceeded_limit < 0:
obj = obj - (exceeded_limit * 10)
print("Exceeded limit")
return - obj
def constraint(x, limit = 1000): # Must be > 0
return limit - x.sum()
N = 1000
# x is variable to optimise for
x = Variable(torch.Tensor([1 for ii in range(N)]), requires_grad=True)
a = Variable(torch.Tensor(np.random.uniform(0,100,N)), requires_grad=True)
b = Variable(torch.Tensor(np.random.rand(N)), requires_grad=True)
c = Variable(torch.Tensor(np.random.rand(N)), requires_grad=True)
# Would like to include the clamp
# x = torch.clamp(x, min=0)
# Non-convex methodf
opt = torch.optim.SGD([x], lr=.01)
for i in range(10000):
# Zeroing gradients
opt.zero_grad()
# Evaluating the objective
obj = objective(x, a, b, c)
# Calculate gradients
obj.backward()
opt.step()
if i%1000==0: print("Objective: %.1f" % -obj.item())
print("\nObjective: {}".format(-obj))
print("Limit: {}".format(constraint(x).item()))
if torch.sum(x<0) > 0: print("Bounds not met")
if constraint(x).item() < 0: print("Constraint not met")
Any suggestions as to how to impose the bounds would be appreciated, either using clamp or otherwise. Or generally advice on non-convex optimisation using Pytorch. This is a much simpler and scaled down version of the problem I'm working so am trying to find a lightweight solution if possible. I am considering using a workaround such as transforming the x variable using an exponential function but then you'd have to scale the function to avoid the positive values becoming infinite, and I want some flexibility with being able to set the constraint.
I meet the same problem with you.
I want to apply bounds on a variable in PyTorch, too.
And I solved this problem by the below Way3.
Your example is a little compliex but I am still learning English.
So I give a simpler example below.
For example, there is a trainable variable v, its bounds is (-1, 1)
v = torch.tensor((0.5, ）, require_grad=True)
v_loss = xxxx
optimizer.zero_grad()
v_loss.backward()
optimizer.step()
Way1. RuntimeError: a leaf Variable that requires grad has been used in an in-place operation.
v.clamp_(-1, 1)
Way2. RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed.
v = torch.clamp(v, -1, +1) # equal to v = v.clamp(-1, +1)
Way3. NotError. I solved this problem in Way3.
with torch.no_grad():
v[:] = v.clamp(-1, +1) # You must use v[:]=xxx instead of v=xxx
I'm trying to solve a model using Julia-JuMP. The following is the outline of the model that I created. Here, z[i,j] is a binary variable and d[i,j] is the cost for which z[i,j]=1.
My constraint creates an infinite number of constraint and hence I need to use a separation algorithm to solve it.
First, I solve the model without any constraint, so the answer to all variables z[i,j] and d[i,j] are zero.
Then, I'm including the separation algorithm (which is given inside the if condition). Even though I'm including if z_value == 0, z_values are not passing to it.
Am I missing something in the format of this model?
m = Model(solver=GurobiSolver())
#variable(m, z[N,N], Bin)
#variable(m, d[N,N]>=0)
#objective(m, Min, sum{ d[i,j]*z[i,j], i in N, j in N} )
z_value = getvalue(z)
d_value = getvalue(d)
if z_value == 0
statement
elseif z_value == 1
statement
end
#constraint(m, sum{z[i,j], i in N, j in N}>=2)
solve(m)
println("Final solution: [ $(getvalue(z)), $(getvalue(d)) ]")
You're multiplying z by d which both are variables, hence your model is non-linear,
Are the costs d[i,j] constant or really a variable of the problem ?
If so you need to use a non-linear solver