It is my first time implementing an optimization model in python with Gurobi, and I run into issues building up a decision variable.
I tried at first to use the following method with a defaultdict:
from gurobipy import *
from collections import defaultdict
def make_dict():
return defaultdict(make_dict)
decvary = defaultdict(make_dict)
for k in K:
for d in D:
for i in V_L:
for w in V_D:
if (w != i):
for j in V:
if (w != j) and (i != j):
decvary[k][d][i][w][j] = m.addVar(lb=0, ub=1, obj=0, vtype=GRB.BINARY, name="y.%d.%d.%d.%d.%d" % (k,d,i,w,j))
But later when I try to add constraints in the optimization model, the variable decvary[k][d][i][w][j] is of type <class 'collections.defaultdict'> but it should actually be 1 or 0 (binary).
So then I tried the old silly way to build the nested dictionary:
for k in K:
decvary[k]={}
for d in D:
decvary[k][d]={}
for i in V_L:
decvary[k][d][i]={}
for w in V_D:
if (w != i):
decvary[k][d][i][w]={}
for j in V:
if (w != j) and (i != j):
decvary[k][d][i][w][j] = m.addVar(lb=0, ub=1, obj=0, vtype=GRB.BINARY, name="y.%d.%d.%d.%d.%d" % (k,d,i,w,j))
But this time, I am getting a KeyError when adding constraints, and the KeyError always happens at the last key [j]
Does anyone have any idea of what's going on? Many thanks!
Gurobi's Python API has a built-in method to create a dictionary very easily: the model's addVars method. E.g. you could do
decvary = m.addVars(K, D, V_L, V_D, V, ub=1, vtype=GRB.BINARY, name="y")
or (to also respect your exceptions)
decvary = m.addVars(((k, d, i, w, j) for k in K for d in D for i in V_L for w in V_D for j in V if w!=i and w!=j and i!=j), ub=1, vtype=GRB.BINARY, name="y")
to create that dictionary.
Related
I have an B x M x N tensor, X, and I have and B x 1 tensor, Y, which corresponds to the index of tensor X at dimension=1 that I want to keep. What is the shorthand for this slice so that I can avoid a loop?
Essentially I want to do this:
Z = torch.zeros(B,N)
for i in range(B):
Z[i] = X[i][Y[i]]
the following code is similar to the code in the loop. the difference is that instead of sequentially indexing the array Z,X and Y we are indexing them in parallel using the array i
B, M, N = 13, 7, 19
X = np.random.randint(100, size= [B,M,N])
Y = np.random.randint(M , size= [B,1])
Z = np.random.randint(100, size= [B,N])
i = np.arange(B)
Y = Y.ravel() # reducing array to rank-1, for easy indexing
Z[i] = X[i,Y[i],:]
this code can be further simplified as
-> Z[i] = X[i,Y[i],:]
-> Z[i] = X[i,Y[i]]
-> Z[i] = X[i,Y]
-> Z = X[i,Y]
pytorch equivalent code
B, M, N = 5, 7, 3
X = torch.randint(100, size= [B,M,N])
Y = torch.randint(M , size= [B,1])
Z = torch.randint(100, size= [B,N])
i = torch.arange(B)
Y = Y.ravel()
Z = X[i,Y]
The answer provided by #Hammad is short and perfect for the job. Here's an alternative solution if you're interested in using some less known Pytorch built-ins. We will use torch.gather (similarly you can achieve this with numpy.take).
The idea behind torch.gather is to construct a new tensor-based on two identically shaped tensors containing the indices (here ~ Y) and the values (here ~ X).
The operation performed is Z[i][j][k] = X[i][Y[i][j][k]][k].
Since X's shape is (B, M, N) and Y shape is (B, 1) we are looking to fill in the blanks inside Y such that Y's shape becomes (B, 1, N).
This can be achieved with some axis manipulation:
>>> Y.expand(-1, N)[:, None] # expand to dim=1 to N and unsqueeze dim=1
The actual call to torch.gather will be:
>>> X.gather(dim=1, index=Y.expand(-1, N)[:, None])
Which you can reshape to (B, N) by adding in [:, 0].
This function can be very effective in tricky scenarios...
I wander, is it possible to index several dimensions at once ? With some broadcasting. Example :
Suppose i have an array A, shaped (n,d). Suppose i have a indexing array, say I with integer values between 0 and d-1. Set B = A[:,I].
If shape(I) == (k,), for whaterver k, then B has shape (n,k) and B[x,y] = A[x,I[y]].
But if shape(I) == (k,p) for whatever (k,p), then i wanted B to be shaped (n,k,p) with B[x,y,z] = A[x,I[y,z]].
1° How can i get this behavior ?
2° Does it have a drawback i did not see ?
You can do it exactly as you described it:
import numpy as np
n = 100
d = 20
k = 10
p = 17
A = np.random.random((n, d))
I = np.random.randint(low=0, high=d, size=(k, p))
B = A[:, I]
print(B.shape) # (n, k, p)
# Testing if the new array B is constructed as expected
x = 3
y = 5
z = 7
print(B[x, y, z])
print(A[x, I[y, z]])
print(B[x, y, z] == A[x, I[y, z]])
Its hard to answer if this is a good implementation or not, without context. But in general it is a good idea to use numpy and vectorization if you have speed in mind.
I am trying to solve a differential equation using scipy.integrate.solve_ivp
L*Q'' + R*Q' + (1/C)*Q = E(t), E(t) = 230*sin(50*t)
for Q(t) and Q'(t)
C = 0.0014 #F
dQ_0 = 2.6 #A
L = 1.8 #H
n = 575 #/
Q_0 = 1e-06 #C
R = 43 #Ohm
t_f = 2.8 #s
import numpy as np
from scipy.integrate import solve_ivp
t = np.linspace(0, t_f, n)
def E(x):
return 230*np.sin(50*x)
y = E(y)
def Q(t, y, R, L, C):
return (y - L*Q'' - R*Q')*C
init_cond = [Q_0, dQ_0]
y_ivp = solve_ivp(Q, t_span=(0, t_f), y0=init_cond)
I am only trying to understand how to correctly define a function that is passed as an argument 'fun' in scipy.integrate.solve_ivp
The answer below is no longer valid, solve_ode has since implemented the args parameter similar to odeint. So indeed
y_ivp = solve_ivp(Q, t_span=(0, t_f), y0=init_cond, args=(R, L, C))
is now valid (do not forget to set appropriate error tolerances, or at least check that the default values arol=1e-3, rtol=1e-6 are appropriate).
Always available was the use of semi-global variables in a closure or lambda expression
y_ivp = solve_ivp(lambda t,y: Q(t,y,R, L, C), t_span=(0, t_f), y0=init_cond, args=(R, L, C))
(obsolete part) solve_ivp has no parameter passing mechanism, so treat the parameters as global variables. You are formulating an ODE for Q, as it is a second order ODE, the state also contains the first derivative, as you somehow recognized in the composition of the initial state. The ODE function then needs to produce the derivative values at a given state. Identify Q(t)=Q[0] and Q'(t)=Q[1], then
def Q_ode(t, Q):
return [ Q[1], (E(t) - R*Q[1] - (1/C)*Q[0])/L ]
I would continue to name the variables containing Q values with the letter Q.
Link is here : https://www.csie.ntu.edu.tw/~r01922136/slides/ffm.pdf (slides 5-6)
Given the following matrices:
X : n * d
W : d * k
Is there an efficient way to calculate the n x 1 matrix using only matrix operations (eg. numpy, tensorflow), where the jth element is :
EDIT:
Current attempt is this, but obviously it's not very space efficient, as it requires storing matrices of size n*d*d :
n = 1000
d = 256
k = 32
x = np.random.normal(size=[n,d])
w = np.random.normal(size=[d,k])
xxt = np.matmul(x.reshape([n,d,1]),x.reshape([n,1,d]))
wwt = np.matmul(w.reshape([1,d,k]),w.reshape([1,k,d]))
output = xxt*wwt
output = np.sum(output,(1,2))
Avoid large temporary arrays
Not all types of algorithms are that easily or obviously to vectorize. The np.sum(xxt*wwt) can be rewritten using np.einsum. This should be faster than your solution, but has some other limitations (eg. no multithreading).
I would therefor suggest using a compiler like Numba.
Example
import numpy as np
import numba as nb
import time
#nb.njit(fastmath=True,parallel=True)
def factorization_nb(w,x):
n = x.shape[0]
d = x.shape[1]
k = w.shape[1]
output=np.empty(n,dtype=w.dtype)
wwt=np.dot(w.reshape((d,k)),w.reshape((k,d)))
for i in nb.prange(n):
sum=0.
for j in range(d):
for jj in range(d):
sum+=x[i,j]*x[i,jj]*wwt[j,jj]
output[i]=sum
return output
def factorization_orig(w,x):
n = x.shape[0]
d = x.shape[1]
k = w.shape[1]
xxt = np.matmul(x.reshape([n,d,1]),x.reshape([n,1,d]))
wwt = np.matmul(w.reshape([1,d,k]),w.reshape([1,k,d]))
output = xxt*wwt
output = np.sum(output,(1,2))
return output
Mesuring Performance
n = 1000
d = 256
k = 32
x = np.random.normal(size=[n,d])
w = np.random.normal(size=[d,k])
#first call has some compilation overhead
res_1=factorization_nb(w,x)
t1=time.time()
for i in range(100):
res_1=factorization_nb(w,x)
#res_2=factorization_orig(w,x)
print(time.time()-t1)
Timings
factorization_nb: 4.2 ms per iteration
factorization_orig: 460 ms per iteration (110x speedup)
For an einsum implemtnation in pytorch, it would be something like
V = torch.randn([50, 10])
x = torch.randn([50])
result = (torch.einsum('ik,jk,i,j->', V, V, x, x)-torch.einsum('ik,ik,i,i->', V, V, x, x))/2
where we subtract the contribution from the feature weight being dotted with itself.
I am using cvxpy to work on some simple portfolio optimisation problem. The only constraint I can't get my head around is the cardinality constraint for the number non-zero portfolio holdings. I tried two approaches, a MIP approach and a traditional convex one.
here is some dummy code for a working traditional example.
import numpy as np
import cvxpy as cvx
np.random.seed(12345)
n = 10
k = 6
mu = np.abs(np.random.randn(n, 1))
Sigma = np.random.randn(n, n)
Sigma = Sigma.T.dot(Sigma)
w = cvx.Variable(n)
ret = mu.T*w
risk = cvx.quad_form(w, Sigma)
objective = cvx.Maximize(ret - risk)
constraints = [cvx.sum_entries(w) == 1, w>= 0, cvx.sum_smallest(w, n-k) >= 0, cvx.sum_largest(w, k) <=1 ]
prob = cvx.Problem(objective, constraints)
prob.solve()
print prob.status
output = []
for i in range(len(w.value)):
output.append(round(w[i].value,2))
print 'Number of non-zero elements : ',sum(1 for i in output if i > 0)
I had the idea to use, sum_smallest and sum_largest (cvxpy manual) my thought was to constraint the smallest n-k entries to 0 and let my target range k sum up to one, I know I can't change the direction of the inequality in order to stay convex, but maybe anyone knows about a clever way of constraining the problem while still keeping it simple.
My second idea was to make this a mixed integer problem, s.th along the lines of
import numpy as np
import cvxpy as cvx
np.random.seed(12345)
n = 10
k = 6
mu = np.abs(np.random.randn(n, 1))
Sigma = np.random.randn(n, n)
Sigma = Sigma.T.dot(Sigma)
w = cvx.Variable(n)
binary = cvx.Bool(n)
integer = cvx.Int(n)
ret = mu.T*w
risk = cvx.quad_form(w, Sigma)
objective = cvx.Maximize(ret - risk)
constraints = [cvx.sum_entries(w) == 1, w>= 0, cvx.sum_entries(binary) == k ]
prob = cvx.Problem(objective, constraints)
prob.solve()
print prob.status
output = []
for i in range(len(w.value)):
output.append(round(w[i].value,2))
print sum(1 for i in output if i > 0)
for i in range(len(w.value)):
print round(binary[i].value,2)
print output
looking at my binary vector it seems to be doing the right thing but the sum_entries constraint doesn't work, looking into the binary vector values I noticed that 0 isn't 0 it's very small e.g xxe^-20 I assume this will mess things up. Anyone can give me any guidance if this is the right way to go? I can use the standard solvers, as well as Mosek if that helps. I would prefer to have a non MIP implementation as I understand this is a combinatorial problem and will get very slow for larger problems. Ultimately I would like to either constraint on exact number of target holdings or a range e.g. 20-30.
Also the documentation in cvxpy around MIP is very short. thanks
A bit chaotic, this question.
So first: this kind of cardinality-constraint is NP-hard. This means, you can't express it using cvxpy without using Integer-programming (or else it would implicate P=NP)!
That beeing said, it would have been nicer, if there would be a pure version of the code without trying to formulate this constraint. I just assume it's the first code without the sum_smallest and sum_largest constraints.
So let's tackle the MIP-approach:
Your code trying to do this makes no sense at all
You introduce some binary-vars, but they have no connection to any other variable at all (so a constraint on it's sum is useless)!
You introduce some integer-vars, but they don't have any use at all!
So here is a MIP-approach:
import numpy as np
import cvxpy as cvx
np.random.seed(12345)
n = 10
k = 6
mu = np.abs(np.random.randn(n, 1))
Sigma = np.random.randn(n, n)
Sigma = Sigma.T.dot(Sigma)
w = cvx.Variable(n)
ret = mu.T*w
risk = cvx.quad_form(w, Sigma)
objective = cvx.Maximize(ret - risk)
binary = cvx.Bool(n) # !!!
constraints = [cvx.sum_entries(w) == 1, w>= 0, w - binary <= 0., cvx.sum_entries(binary) == k] # !!!
prob = cvx.Problem(objective, constraints)
prob.solve(verbose=True)
print(prob.status)
output = []
for i in range(len(w.value)):
output.append(round(w[i].value,2))
print('Number of non-zero elements : ',sum(1 for i in output if i > 0))
So we just added some binary-variables and connected them to w to indicate if w is nonzero or not.
If w is nonzero:
w will be > 0 because of constraint w>= 0
binary needs to be 1, or else constraint w - binary <= 0. is not fulfilled
So it's just introducing these binaries and this one indicator-constraint.
Now the cvx.sum_entries(binary) == k does what it should do.
Be careful with the implication-direction we used here. It might be relevant when chaging the constraint on k (like <=).
Keep in mind, that the default MIP-solver is awful. I also fear that Mosek's interface (sub-optimal within cvxpy) won't solve this, but i might be wrong.
Edit: Your in-range can easily be formulated using two more indicators for:
(k >= a) <= ind_0
(k <= b) <= ind_1
and adding a constraint which equals a logical_and:
ind_0 + ind_1 >= 2
I've had a similar problem where my weights could be negative and did not need to sum to 1 (but still need to be bounded), so I've modified sascha's example to accommodate relaxing these constraints using the CVXpy absolute value function. This should allow for a more general approach to tackling cardinality constraints with MIP
import numpy as np
import cvxpy as cvx
np.random.seed(12345)
n = 10
k = 6
mu = np.abs(np.random.randn(n, 1))
Sigma = np.random.randn(n, n)
Sigma = Sigma.T.dot(Sigma)
w = cvx.Variable(n)
ret = mu.T*w
risk = cvx.quad_form(w, Sigma)
objective = cvx.Maximize(ret - risk)
binary = cvx.Variable(n,boolean=True) # !!!
maxabsw=2
constraints = [ w>= -maxabsw,w<=maxabsw, cvx.abs(w)/maxabsw - binary <= 0., cvx.sum(binary) == k] # !!!
prob = cvx.Problem(objective, constraints)
prob.solve(verbose=True)
print(prob.status)
output = []
for i in range(len(w.value)):
output.append(round(w[i].value,2))
print('Number of non-zero elements : ',sum(1 for i in output if i > 0))