I'd like to write a LP problem in the standard format with MatOptInterface, e.i.:
min c'*x
S.t A*x .== b
x >= 0
Now, how can one write this problem with MathOptInterface? I'm having many issues, one of them is how to define the variable "model". For example, if I try to run:
x = add_variables(model,3)
I first would need to declare this model variable. But I don't know how one is supposed to do this on MathOptInterface.
IIUC in your situation model has to be an argument to be specified by the user of your function.
The user can then pass GLPK.Optimizer(), Tulip.Optimizer() or any other optimizer inheriting from MathOptInterface.AbstractOptimizer.
See e.g. Manual#A complete example.
Alternatively you can look at MOI.Utilities.Model but I don't know how to get an optimizer to solve that model.
Here is how to implement the LP solver for standard Simplex format:
function SolveLP(c,A,b,model::MOI.ModelLike)
x = MOI.add_variables(model, length(c));
MOI.set(model, MOI.ObjectiveFunction{MOI.ScalarAffineFunction{Float64}}(),
MOI.ScalarAffineFunction(MOI.ScalarAffineTerm.(c, x), 0.0))
MOI.set(model, MOI.ObjectiveSense(), MOI.MIN_SENSE)
for xi in x
MOI.add_constraint(model, MOI.SingleVariable(xi), MOI.GreaterThan(0.0))
end
for (i,row) in enumerate(eachrow(A))
row_function = MOI.ScalarAffineFunction(MOI.ScalarAffineTerm.(row, x), 0.0);
MOI.add_constraint(model, row_function, MOI.EqualTo(b[i]))
end
MOI.optimize!(model)
p = MOI.get(model, MOI.VariablePrimal(), x);
return p
end
For the model, just choose something like GLPK.Optimizer()
Related
Suppose there are three variables that take on discrete integer values, say w1 = {1,2,3,4,5,6,7,8,9,10,11,12}, w2 = {1,2,3,4,5,6,7,8,9,10,11,12}, and w3 = {1,2,3,4,5,6,7,8,9,10,11,12}. The task is to pick one value from each set such that the resulting triplet minimizes some (black box, computationally expensive) cost function.
I've tried the surrogate optimization in Matlab but I'm not sure it is appropriate. I've also heard about simulated annealing but found no implementation applied to this instance.
Which algorithm, apart from exhaustive search, can solve this combinatorial optimization problem?
Any help would be much appreciated.
The requirement/benefit of Simulated Annealing (SA), is that the objective surface is somewhat smooth, that is, we can be close to a solution.
For a completely random spiky surface- you might as well do a random search
If it is anything smooth, or even sometimes, it makes sense to try SA.
The idea is that (sometimes) changing only 1 of the 3 values, we have little effect on out blackbox function.
Here is a basic example to do this with Simulated Annealing, using frigidum in Python
import numpy as np
w1 = np.array( [1,2,3,4,5,6,7,8,9,10,11,12] )
w2 = np.array( [1,2,3,4,5,6,7,8,9,10,11,12] )
w3 = np.array( [1,2,3,4,5,6,7,8,9,10,11,12] )
W = np.array([w1,w2,w3])
LENGTH = 12
I define a black-box using the Rastrigin function.
def rastrigin_function_n( x ):
"""
N-dimensional Rastrigin
https://en.wikipedia.org/wiki/Rastrigin_function
x_i is in [-5.12, 5.12]
"""
A = 10
n = x.shape[0]
return A*n + np.sum( x**2- A*np.cos(2*np.pi * x) )
def black_box( x ):
"""
Transform from domain [1,12] to [-5,5]
to be able to push to rastrigin
"""
x = (x - 6.5) * (5/5.5)
return rastrigin_function_n(x)
Simulated Annealing needs to modify state X. Instead of taking/modifying values directly, we keep track of indices. This simplifies creating new proposals as an index is always an integer we can simply add/subtract 1 modulo LENGTH.
def random_start():
"""
returns 3 random indices
"""
return np.random.randint(0, LENGTH, size=3)
def random_small_step(x):
"""
change only 1 index
"""
d = np.array( [1,0,0] )
if np.random.random() < .5:
d = np.array( [-1,0,0] )
np.random.shuffle(d)
return (x+d) % LENGTH
def random_big_step(x):
"""
change 2 indici
"""
d = np.array( [1,-1,0] )
np.random.shuffle(d)
return (x+d) % LENGTH
def obj(x):
"""
We have a triplet of indici,
1. Calculate corresponding values in W = [w1,w2,w3]
2. Push the values in out black-box function
"""
indices = x
values = W[np.array([0,1,2]), indices]
return black_box(values)
And throw a SA Scheme at it
import frigidum
local_opt = frigidum.sa(random_start=random_start,
neighbours=[random_small_step, random_big_step],
objective_function=obj,
T_start=10**4,
T_stop=0.000001,
repeats=10**3,
copy_state=frigidum.annealing.naked)
I am not sure what the minimum for this function should be, but it found a objective with 47.9095 with indicis np.array([9, 2, 2])
Edit:
For frigidum to change the cooling schedule, use alpha=.9. My experience is that all the work of experiment which cooling scheme works best doesn't out-weight simply let it run a little longer. The multiplication you proposed, (sometimes called geometric) is the standard one, also implemented in frigidum. So to implement Tn+1 = 0.9*Tn you need a alpha=.9. Be aware this cooling step is done after N repeats, so if repeats=100, it will first do 100 proposals before lowering the temperature with factor alpha
Simple variations on current state often works best. Since its best practice to set the initial temperature high enough to make most proposals (>90%) accepted, it doesn't matter the steps are small. But if you fear its soo small, try 2 or 3 variations. Frigidum accepts a list of proposal functions, and combinations can enforce each other.
I have no experience with MINLP. But even if, so many times experiments can surprise us. So if time/cost is small to bring another competitor to the table, yes!
Try every possible combination of the three values and see which has the lowest cost.
So I am trying to calculate the value of a parameter, beta, by minimizing the chi-square function. To do this, I'm using the scipy.optimize.minimize() function. I can't seem to get the code to do what I want. Is there a way to do this? I am open to other ways of approaching the problem.
For some background, the variables vr, rms and delta are both 1D tuples of the same length, and zeff, H and beta are parameters. I am trying to calculate an optimized beta value.
def chisq(beta,vr, delta,rvs,rms,zeff,H):
c = -(H/(1+zeff))*(beta/3)
model = c*np.multiply(rms,delta)
q = (vr-model)**2
p = model**-1
ratio = np.multiply(p,q)
chisq = np.sum(ratio)
return chisq
initial_guess = 0.47663662075855323
res = opt.minimize(chisq,initial_guess,args = (beta,delta,rvs,rms,zeff,H))
I usually get an error saying the dimensions of the function don't match the syntax for the minimize() function.
In your case beta is the optimization variable, so you don't need to pass it as an extra argument to the function chisq:
res = opt.minimize(chisq, x0=initial_guess, args=(vr, delta, rvs, rms, zeff, H))
How do I plot e.g. a function f(x) = x for 0 < x < 1 and f(x) = 1 for x >= 1?
Thanks in advance!
EDIT:
Okay, I have thought for a while and found a solution for the given function, but I'd really like to find a more generic solution. Maybe like f=f1+f2+...fn, where fi is the function in domain i and then plot f alltogehter.
f = 0.5*(1*(1-np.sign(1-x))+x*(1-np.sign(x-1)))
Matplotlib doesn't care where your data comes from: you can either make lists from two different functions and combine them, or call a function with a conditional in it. The most mathematically appealing choice is probably
def f(x):
return 1 if x>=1 else 0 if x>0 else ...
Of course, if you care only about x>0, your function can be computed just as x>=1, which can be used as a number already.
I am new to GNUPLOT. I am trying to plot 3d vector fields. However I am having trouble defining a function of three variables f(x,y,z). Can anyone show me how to do this correctly?
Defining your own functions in gnuplot is pretty intuitive. According to the gnuplot documentation the syntax is as follows
<func-name>( <dummy1> {,<dummy2>} ... {,<dummy5>} ) = <expression>
Examples:
w = 2
q = floor(tan(pi/2 - 0.1))
f(x) = sin(w*x)
sinc(x) = sin(pi*x)/(pi*x)
delta(t) = (t == 0)
ramp(t) = (t > 0) ? t : 0
min(a,b) = (a < b) ? a : b
comb(n,k) = n!/(k!*(n-k)!)
len3d(x,y,z) = sqrt(x*x+y*y+z*z)
plot f(x) = sin(x*a), a = 0.2, f(x), a = 0.4, f(x)
There is also a large set of built-in mathematical functions which you can use (in the definition of your own function).
For piecewise defined functions you can use the fact that undefined values are ignored. Therefore, the function
y(x) = x < 0 ? 1/0 : x
is only defined for positive arguments.
Powers are defined by **. Hence f(x)=x*x is identical to f(x)=x**2
If you have still problems in defining your own function, please feel free to ask. (Shouldn't a 3d-function only depend on x and y, i.e., f(x,y)=...?)
For examples of 3d plots, also see the gnuplot demo site.
I am trying to follow the tutorial of using the optimization tool box in MATLAB. Specifically, I have a function
f = exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1)+b
subject to the constraint:
(x(1))^2+x(2)-1=0,
-x(1)*x(2)-10<=0.
and I want to minimize this function for a range of b=[0,20]. (That is, I want to minimize this function for b=0, b=1,b=2 ... and so on).
Below is the steps taken from the MATLAB's tutorial webpage(http://www.mathworks.com/help/optim/ug/nonlinear-equality-and-inequality-constraints.html), how should I change the code so that, the optimization will run for 20 times, and save the optimal values for each b?
Step 1: Write a file objfun.m.
function f = objfun(x)
f = exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1)+b;
Step 2: Write a file confuneq.m for the nonlinear constraints.
function [c, ceq] = confuneq(x)
% Nonlinear inequality constraints
c = -x(1)*x(2) - 10;
% Nonlinear equality constraints
ceq = x(1)^2 + x(2) - 1;
Step 3: Invoke constrained optimization routine.
x0 = [-1,1]; % Make a starting guess at the solution
options = optimoptions(#fmincon,'Algorithm','sqp');
[x,fval] = fmincon(#objfun,x0,[],[],[],[],[],[],...
#confuneq,options);
After 21 function evaluations, the solution produced is
x, fval
x =
-0.7529 0.4332
fval =
1.5093
Update:
I tried your answer, but I am encountering problem with your step 2. Bascially, I just fill the my step 2 to your step 2 (below the comment "optimization just like before").
%initialize list of targets
b = 0:1:20;
%preallocate/initialize result vectors using zeros (increases speed)
opt_x = zeros(length(b));
opt_fval = zeros(length(b));
>> for idx = 1, length(b)
objfun = #(x)objfun_builder(x,b)
%optimization just like before
x0 = [-1,1]; % Make a starting guess at the solution
options = optimoptions(#fmincon,'Algorithm','sqp');
[x,fval] = fmincon(#objfun,x0,[],[],[],[],[],[],...
#confuneq,options);
%end the stuff I fill in
opt_x(idx) = x
opt_fval(idx) = fval
end
However, it gave me the output is:
Error: "objfun" was previously used as a variable, conflicting
with its use here as the name of a function or command.
See "How MATLAB Recognizes Command Syntax" in the MATLAB
documentation for details.
There are two things you need to change about your code:
Creation of the objective function.
Multiple optimizations using a loop.
1st Step
For more flexibility with regard to b, you need to set up another function that returns a handle to the desired objective function, e.g.
function h = objfun_builder(x, b)
h = #(x)(objfun(x));
function f = objfun(x)
f = exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1) + b;
end
end
A more elegant and shorter approach are anonymous functions, e.g.
objfun_builder = #(x,b)(exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1) + b);
After all, this works out to be the same as above. It might be less intuitive for a Matlab-beginner, though.
2nd Step
Instead of placing an .m-file objfun.m in your path, you will need to call
objfun = #(x)(objfun_builder(x,myB));
to create an objective function in your workspace. In order to loop over the interval b=[0,20], use the following loop
%initialize list of targets
b = 0:1:20;
%preallocate/initialize result vectors using zeros (increases speed)
opt_x = zeros(length(b))
opt_fval = zeros(length(b))
%start optimization of list of targets (`b`s)
for idx = 1, length(b)
objfun = #(x)objfun_builder(x,b)
%optimization just like before
opt_x(idx) = x
opt_fval(idx) = fval
end