I am trying to learn Julia for its potential use in parameter estimation. I am interested in estimating kinetic parameters of chemical reactions, which usually involves optimizing reaction parameters with multiple independent batches of experiments. I have successfully optimized a single batch, but need to expand the problem to use many different batches. In developing a sample problem, I am trying to optimize using two toy batches. I know there are probably smarter ways to do this (subject of a future question), but my current workflow involves calling an ODEProblem for each batch, calculating its loss against the data, and minimizing the sum of the residuals for the two batches. Unfortunately, I get an error when initiating the optimization with Optimization.jl. The current code and error are shown below:
using DifferentialEquations, Plots, DiffEqParamEstim
using Optimization, ForwardDiff, OptimizationOptimJL, OptimizationNLopt
using Ipopt, OptimizationGCMAES, Optimisers
using Random
#Experimental data, species B is NOT observed in the data
times = [0.0, 0.071875, 0.143750, 0.215625, 0.287500, 0.359375, 0.431250,
0.503125, 0.575000, 0.646875, 0.718750, 0.790625, 0.862500,
0.934375, 1.006250, 1.078125, 1.150000]
A_obs = [1.0, 0.552208, 0.300598, 0.196879, 0.101175, 0.065684, 0.045096,
0.028880, 0.018433, 0.011509, 0.006215, 0.004278, 0.002698,
0.001944, 0.001116, 0.000732, 0.000426]
C_obs = [0.0, 0.187768, 0.262406, 0.350412, 0.325110, 0.367181, 0.348264,
0.325085, 0.355673, 0.361805, 0.363117, 0.327266, 0.330211,
0.385798, 0.358132, 0.380497, 0.383051]
P_obs = [0.0, 0.117684, 0.175074, 0.236679, 0.234442, 0.270303, 0.272637,
0.274075, 0.278981, 0.297151, 0.297797, 0.298722, 0.326645,
0.303198, 0.277822, 0.284194, 0.301471]
#Create additional data sets for a multi data set optimization
#Simple noise added to data for testing
times_2 = times[2:end] .+ rand(range(-0.05,0.05,100))
P_obs_2 = P_obs[2:end] .+ rand(range(-0.05,0.05,100))
A_obs_2 = A_obs[2:end].+ rand(range(-0.05,0.05,100))
C_obs_2 = C_obs[2:end].+ rand(range(-0.05,0.05,100))
#ki = [2.78E+00, 1.00E-09, 1.97E-01, 3.04E+00, 2.15E+00, 5.27E-01] #Target optimized parameters
ki = [0.1, 0.1, 0.1, 0.1, 0.1, 0.1] #Initial guess of parameters
IC = [1.0, 0.0, 0.0, 0.0] #Initial condition for each species
tspan1 = (minimum(times),maximum(times)) #tuple timespan of data set 1
tspan2 = (minimum(times_2),maximum(times_2)) #tuple timespan of data set 2
# data = VectorOfArray([A_obs,C_obs,P_obs])'
data = vcat(A_obs',C_obs',P_obs') #Make multidimensional array containing all observed data for dataset1, transpose to match shape of ODEProblem output
data2 = vcat(A_obs_2',C_obs_2',P_obs_2') #Make multidimensional array containing all observed data for dataset2, transpose to match shape of ODEProblem output
#make dictionary containing data, time, and initial conditions
keys1 = ["A","B"]
keys2 = ["time","obs","IC"]
entryA =[times,data,IC]
entryB = [times_2, data2,IC]
nest=[Dict(zip(keys2,entryA)),Dict(zip(keys2,entryB))]
exp_dict = Dict(zip(keys1,nest)) #data dictionary
#rate equations in power law form r = k [A][B]
function rxn(x, k)
A = x[1]
B = x[2]
C = x[3]
P = x[4]
k1 = k[1]
k2 = k[2]
k3 = k[3]
k4 = k[4]
k5 = k[5]
k6 = k[6]
r1 = k1 * A
r2 = k2 * A * B
r3 = k3 * C * B
r4 = k4 * A
r5 = k5 * A
r6 = k6 * A * B
return [r1, r2, r3, r4, r5, r6] #returns reaction rate of each equation
end
#Mass balance differential equations
function mass_balances(di,x,args,t)
k = args
r = rxn(x, k)
di[1] = - r[1] - r[2] - r[4] - r[5] - r[6] #Species A
di[2] = + r[1] - r[2] - r[3] - r[6] #Species B
di[3] = + r[2] - r[3] + r[4] #Species C
di[4] = + r[3] + r[5] + r[6] #Species P
end
function ODESols(time,uo,parms)
time_init = (minimum(time),maximum(time))
prob = ODEProblem(mass_balances,uo,time_init,parms)
sol = solve(prob, Tsit5(), reltol=1e-8, abstol=1e-8,save_idxs = [1,3,4],saveat=time) #Integrate prob
return sol
end
function cost_function(data_dict,parms)
res_dict = Dict(zip(keys(data_dict),[0.0,0.0]))
for key in keys(data_dict)
pred = ODESols(data_dict[key]["time"],data_dict[key]["IC"],parms)
loss = L2Loss(data_dict[key]["time"],data_dict[key]["obs"])
err = loss(pred)
res_dict[key] = err
end
residual = sum(res_dict[key] for key in keys(res_dict))
#show typeof(residual)
return residual
end
lb = [0.0,0.0,0.0,0.0,0.0,0.0] #parameter lower bounds
ub = [10.0,10.0,10.0,10.0,10.0,10.0] #parameter upper bounds
optfun = Optimization.OptimizationFunction(cost_function,Optimization.AutoForwardDiff())
optprob = Optimization.OptimizationProblem(optfun,exp_dict, ki,lb=lb,ub=ub,reltol=1E-8) #Set up optimization problem
optsol=solve(optprob, BFGS(),maxiters=10000) #Solve optimization problem
println(optsol.u) #print solution
when I call optsol I get the error:
ERROR: MethodError: no method matching ForwardDiff.GradientConfig(::Optimization.var"#89#106"{OptimizationFunction{true, Optimization.AutoForwardDiff{nothing}, typeof(cost_function), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, typeof(SciMLBase.DEFAULT_OBSERVED_NO_TIME), Nothing, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Vector{Float64}}, ::Dict{String, Dict{String, Array{Float64}}}, ::ForwardDiff.Chunk{2})
Searching online suggests that the issue may be that my cost_function function is not generic enough for ForwardDiff to handle, however I am not sure how to identify where the issue is in this function, or whether it is related to the functions (mass_balances and rxn) that are called within cost_function. Another potential issue is that I am not calling the functions appropriately when building the OptimizationFunction or the OpptimizationProblem, but I cannot identify the issue here either.
Thank you for any suggestions and your help in troubleshooting this application!
res_dict = Dict(zip(keys(data_dict),[0.0,0.0]))
This dictionary is declared to the wrong type.
zerotype = zero(params[1])
res_dict = Dict(zip(keys(data_dict),[zerotype ,zerotype]))
or
res_dict = Dict(zip(keys(data_dict),zeros(eltype(params),2)))
Either way, you want your intermediate calculations to match the type of params when using AutoForwardDiff().
In addition to the variable type specification suggested by Chris, my model also had an issue with the order of the arguments of cost_function and how I passed the arguments to the problem in optprob. This solution was shown by Contradict here
I am trying to update my mutable parameter Nc in my Abstract model
the initial value is 4
I constructed the instance then change instance.Nc to 5 and solve it but it is still 4 (initial value) , can any body help ?
from pyomo.environ import *
import random
model = AbstractModel()
model.i = RangeSet(40)
model.j = Set(initialize=model.i)
model.x = Var(model.i,model.j, initialize=0,within=Binary)
model.y = Var(model.i, within=Binary)
model.Nc=Param(initialize=5,mutable=True)
def Ninit(model,i):
return random.randint(0,1)
model.N=Param(model.i,initialize=Ninit,mutable=True)
def Dinit(model,i,j):
return random.random()
model.D=Param(model.i,model.j,initialize=Dinit,mutable=True)
def rule_C1(model,i,j):
return model.x[i,j]<=model.N[i]*model.y[j]
model.C1 = Constraint(model.i,model.j,rule=rule_C1)
def rule_C2(model):
return sum(model.y[i] for i in model.i )==model.Nc
model.C2 = Constraint(rule=rule_C2)
def rule_C3(model,i):
return sum(model.x[i,j] for j in model.j)==model.N[i]
model.C3 = Constraint(model.i,rule=rule_C3)
def rule_OF(model):
return sum( model.x[i,j]*model.D[i,j] for i in model.i for j in model.j )
model.obj = Objective(rule=rule_OF, sense=minimize)
opt = SolverFactory('glpk')
#model.NC=4
instance = model.create_instance()
instance.NC=4
results = opt.solve(instance) # solves and updates instance
print('NC= ',value(instance.Nc))
print('OF= ',value(instance.obj))
It seems you are actually initializing your parmeter Nc to 5 (model.Nc=Param(initialize=5,mutable=True)) and then changing it to 4 once you create the instance (instance.Nc=4), so you might want to do the opposite (model.Nc=Param(initialize=4,mutable=True) then instance.Nc=4)
Also, note that you are inconsistantly addressing the Nc parameter throughout the code. When you declare the parameter you name it "Nc" (model.Nc=Param(initialize=5,mutable=True)), which is the actual python variable that Pyomo will use in the model, but later you try to change it with capital letters "NC", which is not a parameter (instance.NC=4). Minor typos like these can cause confusion and give you errors. Make sure to fix them and give it a try again
I am currently working with a JuMP model where I define the following example variables:
using JuMP
N = 3
outN = [[4,5],[1,3],[5,7]]
m = Model()
#variable(m, x[i=1:N,j in outN[i]] >=0)
At some point, I want to add, for example, a variable x[1,7]. How can I do that in an effective way? Likewise, how can I remove it afterwards? Is there an alternative to just fixing it to 0?
Thanks in advance
You're probably better off just using a dictionary:
using JuMP
N = 3
outN = [[4,5],[1,3],[5,7]]
model = Model()
x = Dict(
(i, j) => #variable(model, lower_bound = 0, base_name = "x[$i, $j]")
for i in 1:N for j in outN[i]
)
x[1, 7] = #variable(model, lower_bound = 0)
delete(model, x[1, 4])
delete!(x, (1, 4))
Nothing about JuMP restricts you to using only the built-in variable containers: https://jump.dev/JuMP.jl/stable/variables/#User-defined-containers-1
I need to transform a AMPL code to JuMP.
param f;
set R := 1..N;
set R_OK := 1..M;
set V := 1..N;
param tMax;
set T := 1..tMax;
var primary{R,V}, binary;
var SendPrepReq{T,R,V}, binary;
"param f" would be an int. The varibles I know how to do. But what about the sets? What is its equivalent in JuMP?
One of the most relevant pieces of documentation may be the Quickstart guide to get the basics of how JuMP works.
For your example, you can just declare your parameters directly:
using JuMP
# declare some parameters
f = 3
N = 10
M = 5
R = 1:N
V = 1:N
R_OK = 1:M
Tmax = 33
T = 1:Tmax
# create the model
m = Model()
# add variables
#variable(m, primary[R,V], Bin)
#variable(m, SendPrepReq[T,R,V], Bin)
EDIT
One might want to provide parameters independently from the model declaration as in AMLP. The most straightforward way in Julia will be to build and solve the model in a function taking the problem parameters in argument:
function build_model(f, N, M, Tmax)
R = 1:N
V = 1:N
R_OK = 1:M
T = 1:Tmax
# create the model
m = Model()
# add variables
#variable(m, primary[R,V], Bin)
#variable(m, SendPrepReq[T,R,V], Bin)
return (m, primary, SendPrepReq)
end