I am currently working with a JuMP model where I define the following example variables:
using JuMP
N = 3
outN = [[4,5],[1,3],[5,7]]
m = Model()
#variable(m, x[i=1:N,j in outN[i]] >=0)
At some point, I want to add, for example, a variable x[1,7]. How can I do that in an effective way? Likewise, how can I remove it afterwards? Is there an alternative to just fixing it to 0?
Thanks in advance
You're probably better off just using a dictionary:
using JuMP
N = 3
outN = [[4,5],[1,3],[5,7]]
model = Model()
x = Dict(
(i, j) => #variable(model, lower_bound = 0, base_name = "x[$i, $j]")
for i in 1:N for j in outN[i]
)
x[1, 7] = #variable(model, lower_bound = 0)
delete(model, x[1, 4])
delete!(x, (1, 4))
Nothing about JuMP restricts you to using only the built-in variable containers: https://jump.dev/JuMP.jl/stable/variables/#User-defined-containers-1
Related
I am very new to R, so I am sorry if this question is obvious. I would like to add multiple labels to branches in a phylogenetic tree, but I can only figure out how to add one label per branch. I am using the following code:
treetext = "(Japon[&&NHX:S=2],(((Al,Luteo),(Loam,(Niet,Cal))),(((Car,Bar),(Aph,Long[&&NHX:S=1],((Yam,Stig),((Zey,Semp),(A,(Hap,(This,That))))))));"
mytree <- read.nhx(textConnection(treetext))
ggtree(mytree) + geom_tiplab() +
geom_label(aes(x=branch, label=S))
I can add multiple symbols to a branch using the code below, but is so labor-intensive that I may as well do it by hand:
ggtree(mytree) +
geom_tiplab()+
geom_nodepoint(aes(subset = node == 32, x = x - .5),
size = 5, colour = "black", shape = 15) +
geom_nodepoint(aes(subset = node == 32, x = x - 2),
size = 5, colour = "gray", shape = 15)
A solution using the "ape" package would be:
library(ape)
mytree <- rtree(7) # A random tree
labels1 <- letters[1:6]
labels2 <- LETTERS[1:6]
plot(mytree)
# Setting location of label with `adj`
nodelabels(labels1, adj = c(1, -1), frame = 'none')
# We could also use `pos =` 1: below node; 3: above node
nodelabels(labels2, pos = 1, frame = 'n')
You might want to tweak the adj parameter to set the location as you desire it.
As I couldn't parse the treetext object you provided as an example (unbalanced braces), and I'm not familiar with how read.nhx() stores node labels, you might need a little R code to extract the labels elements; you can use a bare nodelabels() to plot the numbers of the nodes on the tree to be sure that your vectors are in the correct sequence.
If you wanted labels to appear on edges rather than at nodes, the function is ape::edgelabels().
cvxpy has a very neat way to write out the optimisation form without worrying too much about converting it into a "standard" matrix form as this is done internally somehow. Best to explain with an example:
def cvxpy_implementation():
var1 = cp.Variable()
var2 = cp.Variable()
constraints = [
var1 <= 3,
var2 >= 2
]
obj_fun = cp.Minimize(var1**2 + var2**2)
problem = cp.Problem(obj_fun, constraints)
problem.solve()
return var1.value, var2.value
def scipy_implementation1():
A = np.diag(np.ones(2))
lb = np.array([-np.inf, 2])
ub = np.array([3, np.inf])
con = LinearConstraint(A, lb, ub)
def obj_fun(x):
return (x**2).sum()
result = minimize(obj_fun, [0, 0], constraints=con)
return result.x
def scipy_implementation2():
con = [
{'type': 'ineq', 'fun': lambda x: 3 - x[0]},
{'type': 'ineq', 'fun': lambda x: x[1] - 2},]
def obj_fun(x):
return (x**2).sum()
result = minimize(obj_fun, [0, 0], constraints=con)
return result.x
All of the above give the correct result but the cvxpy implementation is much "easier" to write out, specifically I don't have to worry about the inequalities and can name variables useful thinks when writing out the inequalities. Compare that to the scipy1 and scipy2 implementations where in the first case I have to write out these extra infs and in the second case I have to remember which variable is which. You can imagine a case where I have 100 variables and while concatenating them will ultimately need to be done I'd like to be able to write it out like in cvxpy.
Question:
Has anyone implemented this for scipy? or is there an alternative library that could make this work?
thank you
Wrote something up that would do this and seems to cover the main issues I had in mind.
The general idea is you define variables and then create a simple expression as you would normally write it out and then the solver class optimises over the defined variables
https://github.com/evan54/optimisation/blob/master/var.py
The example below illustrates a simple use case
# fake data
a = 2
m = 3
x = np.linspace(0, 10)
y = a * x + m + np.random.randn(len(x))
a_ = Variable()
m_ = Variable()
y_ = a_ * x + m_
error = y_ - y
prob = Problem((error**2).sum(), None)
prob.minimize() print(f'a = {a}, a_ = {a_}') print(f'm = {m}, m_ = {m_}')
I am building machine learning models for a certain data set. Then, based on the constraints and bounds for the outputs and inputs, I am trying to find the input parameters for the most minimized answer.
The problem which I am facing is that, when the model is a linear regression model or something like lasso, the minimization works perfectly fine.
However, when the model is "Decision Tree", it constantly returns the very initial value that is given to it. So basically, it does not enforce the constraints.
import numpy as np
import pandas as pd
from scipy.optimize import minimize
I am using the very first sample from the input data set for the optimization. As it is only one sample, I need to reshape it to (1,-1) as well.
x = df_in.iloc[0,:]
x = np.array(x)
x = x.reshape(1,-1)
This is my Objective function:
def objective(x):
x = np.array(x)
x = x.reshape(1,-1)
y = 0
for n in range(df_out.shape[1]):
y = Model[n].predict(x)
Y = y[0]
return Y
Here I am defining the bounds of inputs:
range_max = pd.DataFrame(range_max)
range_min = pd.DataFrame(range_min)
B_max=[]
B_min =[]
for i in range(range_max.shape[0]):
b_max = range_max.iloc[i]
b_min = range_min.iloc[i]
B_max.append(b_max)
B_min.append(b_min)
B_max = pd.DataFrame(B_max)
B_min = pd.DataFrame(B_min)
bnds = pd.concat([B_min, B_max], axis=1)
These are my constraints:
con_min = pd.DataFrame(c_min)
con_max = pd.DataFrame(c_max)
Here I am defining the constraint function:
def const(x):
x = np.array(x)
x = x.reshape(1,-1)
Y = []
for n in range(df_out.shape[1]):
y = Model[n].predict(x)[0]
Y.append(y)
Y = pd.DataFrame(Y)
a4 =[]
for k in range(Y.shape[0]):
a1 = Y.iloc[k,0] - con_min.iloc[k,0]
a2 = con_max.iloc[k, 0] - Y.iloc[k,0]
a3 = [a2,a1]
a4 = np.concatenate([a4, a3])
return a4
c = const(x)
con = {'type': 'ineq', 'fun': const}
This is where I try to minimize. I do not pick a method as the automatically picked model has worked so far.
sol = minimize(fun = objective, x0=x,constraints=con, bounds=bnds)
So the actual constraints are:
c_min = [0.20,1000]
c_max = [0.3,1600]
and the max and min range for the boundaries are:
range_max = [285,200,8,85,0.04,1.6,10,3.5,20,-5]
range_min = [215,170,-1,60,0,1,6,2.5,16,-18]
I think you should check the output of 'sol'. At times, the algorithm is not able to perform line search completely. To check for this, you should check message associated with 'sol'. In such a case, the optimizer returns initial parameters itself. There may be various reasons of this behavior. In a nutshell, please check the output of sol and act accordingly.
Arad,
If you have not yet resolved your issue, try using scipy.optimize.differential_evolution instead of scipy.optimize.minimize. I ran into similar issues, particularly with decision trees because of their step-like behavior resulting in infinite gradients.
I am using cvxpy to work on some simple portfolio optimisation problem. The only constraint I can't get my head around is the cardinality constraint for the number non-zero portfolio holdings. I tried two approaches, a MIP approach and a traditional convex one.
here is some dummy code for a working traditional example.
import numpy as np
import cvxpy as cvx
np.random.seed(12345)
n = 10
k = 6
mu = np.abs(np.random.randn(n, 1))
Sigma = np.random.randn(n, n)
Sigma = Sigma.T.dot(Sigma)
w = cvx.Variable(n)
ret = mu.T*w
risk = cvx.quad_form(w, Sigma)
objective = cvx.Maximize(ret - risk)
constraints = [cvx.sum_entries(w) == 1, w>= 0, cvx.sum_smallest(w, n-k) >= 0, cvx.sum_largest(w, k) <=1 ]
prob = cvx.Problem(objective, constraints)
prob.solve()
print prob.status
output = []
for i in range(len(w.value)):
output.append(round(w[i].value,2))
print 'Number of non-zero elements : ',sum(1 for i in output if i > 0)
I had the idea to use, sum_smallest and sum_largest (cvxpy manual) my thought was to constraint the smallest n-k entries to 0 and let my target range k sum up to one, I know I can't change the direction of the inequality in order to stay convex, but maybe anyone knows about a clever way of constraining the problem while still keeping it simple.
My second idea was to make this a mixed integer problem, s.th along the lines of
import numpy as np
import cvxpy as cvx
np.random.seed(12345)
n = 10
k = 6
mu = np.abs(np.random.randn(n, 1))
Sigma = np.random.randn(n, n)
Sigma = Sigma.T.dot(Sigma)
w = cvx.Variable(n)
binary = cvx.Bool(n)
integer = cvx.Int(n)
ret = mu.T*w
risk = cvx.quad_form(w, Sigma)
objective = cvx.Maximize(ret - risk)
constraints = [cvx.sum_entries(w) == 1, w>= 0, cvx.sum_entries(binary) == k ]
prob = cvx.Problem(objective, constraints)
prob.solve()
print prob.status
output = []
for i in range(len(w.value)):
output.append(round(w[i].value,2))
print sum(1 for i in output if i > 0)
for i in range(len(w.value)):
print round(binary[i].value,2)
print output
looking at my binary vector it seems to be doing the right thing but the sum_entries constraint doesn't work, looking into the binary vector values I noticed that 0 isn't 0 it's very small e.g xxe^-20 I assume this will mess things up. Anyone can give me any guidance if this is the right way to go? I can use the standard solvers, as well as Mosek if that helps. I would prefer to have a non MIP implementation as I understand this is a combinatorial problem and will get very slow for larger problems. Ultimately I would like to either constraint on exact number of target holdings or a range e.g. 20-30.
Also the documentation in cvxpy around MIP is very short. thanks
A bit chaotic, this question.
So first: this kind of cardinality-constraint is NP-hard. This means, you can't express it using cvxpy without using Integer-programming (or else it would implicate P=NP)!
That beeing said, it would have been nicer, if there would be a pure version of the code without trying to formulate this constraint. I just assume it's the first code without the sum_smallest and sum_largest constraints.
So let's tackle the MIP-approach:
Your code trying to do this makes no sense at all
You introduce some binary-vars, but they have no connection to any other variable at all (so a constraint on it's sum is useless)!
You introduce some integer-vars, but they don't have any use at all!
So here is a MIP-approach:
import numpy as np
import cvxpy as cvx
np.random.seed(12345)
n = 10
k = 6
mu = np.abs(np.random.randn(n, 1))
Sigma = np.random.randn(n, n)
Sigma = Sigma.T.dot(Sigma)
w = cvx.Variable(n)
ret = mu.T*w
risk = cvx.quad_form(w, Sigma)
objective = cvx.Maximize(ret - risk)
binary = cvx.Bool(n) # !!!
constraints = [cvx.sum_entries(w) == 1, w>= 0, w - binary <= 0., cvx.sum_entries(binary) == k] # !!!
prob = cvx.Problem(objective, constraints)
prob.solve(verbose=True)
print(prob.status)
output = []
for i in range(len(w.value)):
output.append(round(w[i].value,2))
print('Number of non-zero elements : ',sum(1 for i in output if i > 0))
So we just added some binary-variables and connected them to w to indicate if w is nonzero or not.
If w is nonzero:
w will be > 0 because of constraint w>= 0
binary needs to be 1, or else constraint w - binary <= 0. is not fulfilled
So it's just introducing these binaries and this one indicator-constraint.
Now the cvx.sum_entries(binary) == k does what it should do.
Be careful with the implication-direction we used here. It might be relevant when chaging the constraint on k (like <=).
Keep in mind, that the default MIP-solver is awful. I also fear that Mosek's interface (sub-optimal within cvxpy) won't solve this, but i might be wrong.
Edit: Your in-range can easily be formulated using two more indicators for:
(k >= a) <= ind_0
(k <= b) <= ind_1
and adding a constraint which equals a logical_and:
ind_0 + ind_1 >= 2
I've had a similar problem where my weights could be negative and did not need to sum to 1 (but still need to be bounded), so I've modified sascha's example to accommodate relaxing these constraints using the CVXpy absolute value function. This should allow for a more general approach to tackling cardinality constraints with MIP
import numpy as np
import cvxpy as cvx
np.random.seed(12345)
n = 10
k = 6
mu = np.abs(np.random.randn(n, 1))
Sigma = np.random.randn(n, n)
Sigma = Sigma.T.dot(Sigma)
w = cvx.Variable(n)
ret = mu.T*w
risk = cvx.quad_form(w, Sigma)
objective = cvx.Maximize(ret - risk)
binary = cvx.Variable(n,boolean=True) # !!!
maxabsw=2
constraints = [ w>= -maxabsw,w<=maxabsw, cvx.abs(w)/maxabsw - binary <= 0., cvx.sum(binary) == k] # !!!
prob = cvx.Problem(objective, constraints)
prob.solve(verbose=True)
print(prob.status)
output = []
for i in range(len(w.value)):
output.append(round(w[i].value,2))
print('Number of non-zero elements : ',sum(1 for i in output if i > 0))
In Julia, the recommended way to iterate over all indices of an AbstractArray is to use eachindex, e.g.,
for i in eachindex(a)
do_something(a[i], i)
end
In contrast to 1:length(a), eachindex(a) supports arrays with unconventional indexing, i.e., indices not starting at 1. Additionally, it is more efficient for arrays with slow linear indexing.
If I want to skip the first index I can use Iterators.drop(eachindex(a), 1) (is there a better way?) but how do I skip the last one in a generic way?
A "front" iterator is relatively simple and generally useful. Edit: it's also totally overkill to define it in full generality just for this case. It's much easier and simpler to rely on Base's builtins with a definition like:
front(itr, n=1) = Iterators.take(itr, length(itr)-n)
This will work for all iterators with length defined — which will include everything that eachindex will return.
Alternatively, you can define a specialized iterator from first principles that doesn't depend upon length being defined. I'm not aware of such a structure in any existing packages. Using Julia 0.6, an implementation could look like:
struct Front{T}
itr::T
end
# Basic iterator definition
function Base.start(f::Front)
s = start(f.itr)
done(f.itr, s) && throw(ArgumentError("cannot take the front of an empty iterator"))
return next(f.itr, s)
end
function Base.next(f::Front, state)
val, s = state
return val, next(f.itr, s)
end
Base.done(f::Front, state) = done(f.itr, state[2])
# Inherit traits as appropriate
Base.iteratorsize(::Type{Front{T}}) where {T} = _dropshape(Base.iteratorsize(T))
_dropshape(x) = x
_dropshape(::Base.HasShape) = Base.HasLength()
Base.iteratoreltype(::Type{Front{T}}) where {T} = Base.iteratoreltype(T)
Base.length(f::Front) = length(f.itr) - 1
Base.eltype(f::Front{T}) where {T} = eltype(T)
Now:
julia> collect(Front(eachindex(rand(5))))
4-element Array{Int64,1}:
1
2
3
4
julia> collect(Front(eachindex(sprand(3, 2, .2))))
5-element Array{CartesianIndex{2},1}:
CartesianIndex{2}((1, 1))
CartesianIndex{2}((2, 1))
CartesianIndex{2}((3, 1))
CartesianIndex{2}((1, 2))
CartesianIndex{2}((2, 2))
Another way to define #MattB.'s Front is
front(itr,n=1) = (first(x) for x in Iterators.partition(itr,n+1,1))
This also gives:
julia> front(eachindex([1,2,3,4,5]))|>collect
4-element Array{Int64,1}:
1
2
3
4
and as a bonus:
julia> front(eachindex([1,2,3,4,5]),2)|>collect
3-element Array{Int64,1}:
1
2
3
the corresponding iterator to drop(eachindex([1,2,3,4,5]),2).
There's also the following:
for I in CartesianRange(Base.front(indices(A)))
#show I A[I, :]
end
On A = reshape(1:27, 3, 3, 3) this yields
I = CartesianIndex{2}((1,1))
A[I,:] = [1,10,19]
I = CartesianIndex{2}((2,1))
A[I,:] = [2,11,20]
I = CartesianIndex{2}((3,1))
A[I,:] = [3,12,21]
I = CartesianIndex{2}((1,2))
A[I,:] = [4,13,22]
I = CartesianIndex{2}((2,2))
A[I,:] = [5,14,23]
I = CartesianIndex{2}((3,2))
A[I,:] = [6,15,24]
I = CartesianIndex{2}((1,3))
A[I,:] = [7,16,25]
I = CartesianIndex{2}((2,3))
A[I,:] = [8,17,26]
I = CartesianIndex{2}((3,3))
A[I,:] = [9,18,27]