I'm attempting to write a Newton-Raphson solver in Julia. The Newton-Raphson method is shown in this image.
f(x) = x^2.5 - 3x^1.5 - 10
fprime(x) = 2.5x^1.5 - 4.5x^0.5
x = zeros(1000)
x[1] = 10
δ = 1 # a relatively large number compared to what we want the error to be
iter = 1
while δ > 1e-6
x[iter + 1] = x[iter] - f(x[iter])/fprime(x[iter])
iter += 1
δ = abs(x[iter] - x[iter + 1])
if iter == 100
break
end
end
println("The solution is ")
show(x[iter])
However, when I run the code, I get an error saying iter is not defined, even though I defined it just before the start of the loop. Is there some scoping problem I'm completely missing?
ERROR: LoadError: UndefVarError: iter not defined
Stacktrace:
[1] top-level scope at /Users/natemcintosh/Documents/Julia/Learning_julia.jl:11 [inlined]
[2] top-level scope at ./none:0
[3] include_string(::Module, ::String, ::String) at ./loading.jl:1002
[4] (::getfield(Atom, Symbol("##120#125")){String,String,Module})() at /Users/natemcintosh/.julia/packages/Atom/Pab0Z/src/eval.jl:120
[5] withpath(::getfield(Atom, Symbol("##120#125")){String,String,Module}, ::String) at /Users/natemcintosh/.julia/packages/CodeTools/8CjYJ/src/utils.jl:30
[6] withpath at /Users/natemcintosh/.julia/packages/Atom/Pab0Z/src/eval.jl:46 [inlined]
[7] #119 at /Users/natemcintosh/.julia/packages/Atom/Pab0Z/src/eval.jl:117 [inlined]
[8] hideprompt(::getfield(Atom, Symbol("##119#124")){String,String,Module}) at /Users/natemcintosh/.julia/packages/Atom/Pab0Z/src/repl.jl:76
[9] macro expansion at /Users/natemcintosh/.julia/packages/Atom/Pab0Z/src/eval.jl:116 [inlined]
[10] (::getfield(Atom, Symbol("##118#123")){Dict{String,Any}})() at ./task.jl:85
in expression starting at /Users/natemcintosh/Documents/Julia/Learning_julia.jl:10
I've tried printing x at the beginning of the while loop and it knows what x is, but thinks iter is undefined.
First let me give the solution:
There are three possible approaches
Approach 1. Prepend global before iter += 1 and change it to global iter += 1 and all will work (note however the comment below about δ - because it will not work correctly unless you also prepend global before δ = abs(x[iter] - x[iter + 1]), i.e. the code will run but will produce wrong results - approaches 2 and 3 do not have this problem).
Approach 2. Wrap your code inside a function like this:
f(x) = x^2.5 - 3x^1.5 - 10
fprime(x) = 2.5x^1.5 - 4.5x^0.5
function sol(f, fprime)
x = zeros(1000)
x[1] = 10
δ = 1 # a relatively large number compared to what we want the error to be
iter = 1
while δ > 1e-6
x[iter + 1] = x[iter] - f(x[iter])/fprime(x[iter])
iter += 1
δ = abs(x[iter] - x[iter + 1])
if iter == 100
break
end
end
println("The solution is ")
show(x[iter])
end
sol(f, fprime) # now we call it
Solution 3. Wrap your code in a let block by changing line function sol(f, fprime) in solution 2 to simply say let (you do not need to call sol then).
Now the reason why you have to do it.
In Julia 1.0 while introduces a new scope. The scoping rules in Julia 1.0 are that each variable that is assigned to inside a while loop is considered a local variable (this has changed, because Julia 0.6 distinguished hard and soft local scope, in Julia 1.0 this distinction is gone - all local scopes are the same).
In your code you assign values to two variables: iter and δ inside the loop. This means that they are treated by Julia as local so you are not allowed to access their value before they have a value assigned inside the loop.
You want to read iter in line x[iter + 1] = x[iter] - f(x[iter])/fprime(x[iter]) but assign a value to it only in the following line.
As for δ the thing is more tricky. You assign a value to it, but it is used in a loop condition while δ > 1e-6. However, this condition operates on variables defined in outer scope (global in the original case). So all will work, but the condition while δ > 1e-6 will always see that δ is equal to 1 as it looks at the value of the variable outside of the loop. So this condition will never trigger (and you will always run 100 iterations). In summary the code that does what you want is (although if you did not fix δ assignment you would not get a warning):
f(x) = x^2.5 - 3x^1.5 - 10
fprime(x) = 2.5x^1.5 - 4.5x^0.5
x = zeros(1000)
x[1] = 10
δ = 1 # a relatively large number compared to what we want the error to be
iter = 1
while δ > 1e-6
x[iter + 1] = x[iter] - f(x[iter])/fprime(x[iter])
global iter += 1
global δ = abs(x[iter] - x[iter + 1])
if iter == 100
break
end
end
println("The solution is ")
show(x[iter])
Finally notice that the line x[iter + 1] = x[iter] - f(x[iter])/fprime(x[iter]) works fine even if there is an assignment in it, because you do not rebind variable x in it, but only change one element of an array (so x points to the same address in memory and Julia treats it as a global variable all the time).
Also you might want to read this https://docs.julialang.org/en/latest/manual/variables-and-scoping/ in the Julia manual or the answer to this question Julia Variable scope is similar
Related
Extremely new to Julia, so please pardon any obvious oversights on my end
I am trying to estimate a piecewise likelihood function through optimization. I have the code functional in R, but have begun translating it to Julia in the hopes of faster estimation, for eventual bootstrapping
Here is the current block of code that I am trying (v and x are already as 1000x1 vectors elsewhere defined elsewhere):
function est(a,b)
function pwll(v,x)
if v>4
ILL=pdf(Poisson(exp(a+b*x)), v)
elseif v==4
ILL=pdf(Poisson(exp(a+b*x)), 4)+pdf(Poisson(exp(a+b*x)),3)+pdf(Poisson(exp(a+b*x)),2)
else v==0
ILL=pdf(Poisson(exp(a+b*x)), 1)+pdf(Poisson(exp(a+b*x)), 0)
end
return(ILL)
end
ILL=pwll.(v, x)
function fixILL(x)
if x==0
x=0.00000000000000001
else
x=x
end
end
ILL=fixILL.(ILL)
LILL=log10.(ILL)
LL=-1*LILL
return(sum(LL))
end
using Optim
params0=[1,1]
optimize(est, params0)
And the error message(s) I am getting are:
ERROR: InexactError: Int64(NaN)
Stacktrace:
[1] Int64(x::Float64)
# Base ./float.jl:788
[2] x_of_nans(x::Vector{Int64}, Tf::Type{Int64}) (repeats 2 times)
# NLSolversBase ~/.julia/packages/NLSolversBase/kavn7/src/NLSolversBase.jl:60
[3] NonDifferentiable(f::Function, x::Vector{Int64}, F::Int64; inplace::Bool)
# NLSolversBase ~/.julia/packages/NLSolversBase/kavn7/src/objective_types/nondifferentiable.jl:11
[4] NonDifferentiable(f::Function, x::Vector{Int64}, F::Int64)
# NLSolversBase ~/.julia/packages/NLSolversBase/kavn7/src/objective_types/nondifferentiable.jl:10
[5] promote_objtype(method::NelderMead{Optim.AffineSimplexer, Optim.AdaptiveParameters}, x::Vector{Int64}, autodiff::Symbol, inplace::Bool, args::Function)
# Optim ~/.julia/packages/Optim/tP8PJ/src/multivariate/optimize/interface.jl:63
[6] optimize(f::Function, initial_x::Vector{Int64}; inplace::Bool, autodiff::Symbol, kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
# Optim ~/.julia/packages/Optim/tP8PJ/src/multivariate/optimize/interface.jl:86
[7] optimize(f::Function, initial_x::Vector{Int64})
# Optim ~/.julia/packages/Optim/tP8PJ/src/multivariate/optimize/interface.jl:83
[8] top-level scope
# ~/Documents/Projects/ki_new/peicewise_ll.jl:120
I understand that it seems the error is coming from the function to be optimized being non-differentiable. A fairly direct translation works well in R, using the built in optim() function.
Can anyone provide any insight?
I have tried the above code displayed above, with multiple variations. The function to be optimized is functional, I am struggling with the optimization (the issues of which may stem from function being inefficiently written)
Here's an adapted version of your code which produces a solution:
using Distributions, Optim
function pwll(v, x, a, b)
d = Poisson(exp(a+b*x))
if v > 4
return pdf(d, v)
elseif v == 4
return pdf(d, 4) + pdf(d, 3) + pdf(d, 2)
else
return pdf(d, 1) + pdf(d, 0)
end
end
fixILL(x) = iszero(x) ? 1e-17 : x
est(a, b, v, x) = sum(-1 .* log10.(fixILL.(pwll.(v, x, a, b))))
v = 4; x = 0.5 # Defining these here as they are not given in your post
obj(input; v = v, x = x) = est(input[1], input[2], v, x)
optimize(obj, [1.0, 1.0])
I have no idea whether this is correct of course, check this against some sort of known result if you can.
I try to add the following constraints to my model. my problem: the function g() expects x as a binary numpy array. So the result arr_a depends on the current value of x in every step of the optimization!
Afterwards, I want the max of this array times x to be smaller than 50.
How can I add this constraint dynamically so that arr_a is always rightfully calculated with the value of x at each iteration while telling the model to keep the constraint arr_a * x <= 50 ? Currently I am getting an error when adding the constraint to the model because g() expects x as numpy array to calculate arr_a, arr_b, arr_c ( g uses np.where(x == 1) within its calculation).
#Init model
from ortools.sat.python import cp_model
model = cp_model.CpModel()
# Declare the variables
x = []
for i in range(self.ds.n_banks):
x.append(model.NewIntVar(0, 1, "x[%i]" % (i)))
#add bool vars
a = model.NewBoolVar('a')
arr_a, arr_b, arr_c = g(df1,df2,df3,x)
model.Add((arr_a.astype('int32') * x).max() <= 50).OnlyEnforceIf(a)
model.Add((arr_a.astype('int32') * x).max() > 50).OnlyEnforceIf(a.Not())
Afterwards i add the target function that naturally also depends on x.
model.Minimize(target(x))
def target(x):
arr_a, arr_b, arr_c = g(df1,df2,df3,x)
return (3 * arr_b * x + 2 * arr_c * x).sum()
EDIT:
My problem changed a bit and i managed to get it work without issues. Nevertheless, I experienced that the constraint is never actually met! self-defined-function is a highly non-linear function that expects the indices where x==1 and where x == 0 and returns a numpy array. Also it is not possible to re-build it with pre-defined functions of the sat.solver.
#Init model
model = cp_model.CpModel()
# Declare the variables
x = [model.NewIntVar(0, 1, "x[%i]" % (i)) for i in range(66)]
# add hints
[model.AddHint(x[i],np.random.choice(2, 1, p=[0.4, 0.6])[0]) for i in range(66)]
open_elements = [model.NewBoolVar("open_elements[%i]" % (i)) for i in range(66)]
closed_elements = [model.NewBoolVar("closed_elements[%i]" % (i)) for i in range(6)]
# open indices as bool vars
for i in range(66):
model.Add(x[i] == 1).OnlyEnforceIf(open_elements[i])
model.Add(x[i] != 1).OnlyEnforceIf(open_elements[i].Not())
model.Add(x[i] != 1).OnlyEnforceIf(closed_elements[i])
model.Add(x[i] == 1).OnlyEnforceIf(closed_elements[i].Not())
model.Add((self-defined-function(np.where(open_elements), np.where(closed_elements), some_array).astype('int32') * x - some_vector).all() <= 0)
Even when I apply a simpler function, it will not work properly.
model.Add((self-defined-function(x, some_array).astype('int32') * x - some_vector).all() <= 0)
I also tried the following:
arr_indices_open = []
arr_indices_closed = []
for i in range(66):
if open_elements[i] == True:
arr_indices_open.append(i)
else:
arr_indices_closed.append(i)
# final Constraint
arr_ = self-defined-function(arr_indices_open, arr_indices_closed, some_array)[0].astype('int32')
for i in range(66):
model.Add(arr_[i] * x[i] <= some_other_vector[i])
Some minimal example for the self-defined-function, with which I simply try to say that n_closed shall be smaller than 10. Even that condition is not met by the solver:
def self_defined_function(arr_indices_closed)
return len(arr_indices_closed)
arr_ = self-defined-function(arr_indices_closed)
for i in range(66):
model.Add(arr_ < 10)
I'm not sure I fully understand the question, but generally, if you want to optimize a function g(x), you'll have to implement it in using the solver's primitives (docs).
It's easier to do when your calculation coincides with an existing solver function, e.g.: if you're trying to calculate a linear expression; but could get harder to do when trying to calculate something more complex. However, I believe that's the only way.
There is a error in my code, can anyone help me please?
My code:
function funP(u, τ::Float64)
w = (τ*max(u, 0)) + ((1-τ)*max(-u, 0))
return w
end
τ = 0.2
modelquant = Model(with_optimizer(OSQP.Optimizer))
#variable(modelquant, β[i=0:1])
#variable(modelquant, erro[1:T])
#constraint(modelquant,[i=1:T], erro[i] >= contratos[i] - β[0] - β[1]*spot[i])
#constraint(modelquant,[i=1:T], erro[i] >= -contratos[i] + β[0] + β[1]*spot[i])
#objective(modelquant, Min, sum(funP(erro[i], τ) for i=1:T))
optimize!(modelquant)
objective_value(modelquant)
𝐁 = JuMP.value.(β)
The error is:
julia> #objective(modelquant, Min, sum(funP(erro[i], τ) for i=1:T))
ERROR: MethodError: no method matching isless(::Int64, ::VariableRef)
Closest candidates are:
isless(::Missing, ::Any) at missing.jl:87
isless(::Real, ::AbstractFloat) at operators.jl:166
isless(::Integer, ::ForwardDiff.Dual{Ty,V,N} where N where V) where Ty at C:\Users\Fernanda.julia\packages\ForwardDiff\kU1ce\src\dual.jl:140
...
Stacktrace:
[1] max(::VariableRef, ::Int64) at .\operators.jl:417
[2] funP(::VariableRef, ::Float64) at .\REPL[8]:2
[3] macro expansion at C:\Users\Fernanda.julia\packages\MutableArithmetics\bPWR4\src\rewrite.jl:276 [inlined]
[4] macro expansion at C:\Users\Fernanda.julia\packages\JuMP\qhoVb\src\macros.jl:830 [inlined]
[5] top-level scope at .\REPL[20]:1
Thank you so much!
You need to re-engineer your model to replace the max function with a binary variable.
In your case the code will look like this (check for typos):
#variable(modelquant, erro_neg[1:T], Bin)
#variable(modelquant, erro_pos[1:T], Bin)
#constraint(modelquant,for i=1:T,erro_neg[i]+erro_pos[i]==1)
#constraint(modelquant,for i=1:T,erro[i]*erro_pos[i] >= 0)
#constraint(modelquant,for i=1:T,erro[i]*erro_neg[i] <= 0)
#objective(modelquant, Min, sum( τ*erro_neg[i]*erro[i]+ (1-τ)*erro[i]*erro_npos[i] for i=1:T))
Please note that in my version you could actually safely remove the Bin condition from erro_neg and erro_pos and the model will still work (you need to test empirically what your solver prefers)
I am trying to filter evident measurement mistakes from my data using the 3-sigma rule. x is a numpy array of measurement points and y is an arrray of measured values. To remove wrong points from my data, I zip x.tolist() and y.tolist(), then filter by the second element of each tuple, then I need to convert my zip back into two lists. I tried to first covert my list of tuples into a list of lists, then convert it to numpy 2D array and then take two 1D-slices of it. It looks like the first slice is correct, but then it outputs the following:
x = np.array(list(map(list, list(filter(flt, list(zap))))))[:, 0]
IndexError: too many indices for array
I don't understand what am I doing wrong. Here's the code:
x = np.array(readCol(0, l))
y = np.array(readCol(1, l))
n = len(y)
stdev = np.std(y)
mean = np.mean(y)
print("Stdev is: " + str(stdev))
print("Mean is: " + str(mean))
def flt(n):
global mean
global stdev
global x
if abs(n[1] - mean) < 3*stdev:
return True
else:
print('flt function finds an error: ' + str(n[1]))
return False
def filtration(N):
print(Fore.RED + 'Filtration function launched')
global y
global x
global stdev
global mean
zap = zip(x.tolist(), y.tolist())
for i in range(N):
print(Fore.RED + ' Filtration step number ' + str(i) + Style.RESET_ALL)
y = np.array(list(map(list, list(filter(flt, list(zap))))))[:, 1]
print(Back.GREEN + 'This is y: \n' + Style.RESET_ALL)
print(y)
x = np.array(list(map(list, list(filter(flt, list(zap))))))[:, 0]
print(Back.GREEN + 'This is x: \n' + Style.RESET_ALL)
print(x)
print('filtration fuction main step')
stdev = np.std(y)
print('second step')
mean = np.mean(y)
print('third step')
Have you tried to test the problem line step by step?
x = np.array(list(map(list, list(filter(flt, list(zap))))))[:, 0]
for example:
temp = np.array(list(map(list, list(filter(flt, list(zap))))))
print(temp.shape, temp.dtype)
x = temp[:, 0]
Further break down might be needed, but since [:,0] is the only indexing operation in this line, I'd start there.
Without further study of the code and/or some examples, I'm not going to try to speculate what the nested lists are doing.
The error sounds like temp is not 2d, contrary to your expectations. That could be because temp is object dtype, and composed of lists the vary in length. That seems to be common problem when people make arrays from downloaded databases.
I have to optimize the coefficients for three numpy arrays which maximizes my evaluation function.
I have a target array called train['target'] and three predictions arrays named array1, array2 and array3.
I want to put the best linear coefficients i.e., x,y,z for these three arrays which will maximize the function
roc_aoc_curve(train['target'], xarray1 + yarray2 +z*array3)
the above function would be maximum when prediction is closer to the target.
i.e, xarray1 + yarray2 + z*array3 should be closer to train['target'].
The range of x,y,z >=0 and x,y,z <= 1
Basically I am trying to put the weights x,y,z for each of the three arrays which would make the function
xarray1 + yarray2 +z*array3 closer to the train['target']
Any help in getting this would be appreciated.
I used pulp.LpProblem('Giapetto', pulp.LpMaximize) to do the maximization. It works for normal numbers, integers etc, however failing while trying to do with arrays.
import numpy as np
import pulp
# create the LP object, set up as a maximization problem
prob = pulp.LpProblem('Giapetto', pulp.LpMaximize)
# set up decision variables
x = pulp.LpVariable('x', lowBound=0)
y = pulp.LpVariable('y', lowBound=0)
z = pulp.LpVariable('z', lowBound=0)
score = roc_auc_score(train['target'],x*array1+ y*array2 + z*array3)
prob += score
coef = x+y+z
prob += (coef==1)
# solve the LP using the default solver
optimization_result = prob.solve()
# make sure we got an optimal solution
assert optimization_result == pulp.LpStatusOptimal
# display the results
for var in (x, y,z):
print('Optimal weekly number of {} to produce: {:1.0f}'.format(var.name, var.value()))
Getting error at the line
score = roc_auc_score(train['target'],x*array1+ y*array2 + z*array3)
TypeError: unsupported operand type(s) for /: 'int' and 'LpVariable'
Can't progress beyond this line when using arrays. Not sure if my approach is correct. Any help in optimizing the function would be appreciated.
When you add sums of array elements to a PuLP model, you have to use built-in PuLP constructs like lpSum to do it -- you can't just add arrays together (as you discovered).
So your score definition should look something like this:
score = pulp.lpSum([train['target'][i] - (x * array1[i] + y * array2[i] + z * array3[i]) for i in arr_ind])
A few notes about this:
[+] You didn't provide the definition of roc_auc_score so I just pretended that it equals the sum of the element-wise difference between the target array and the weighted sum of the other 3 arrays.
[+] I suspect your actual calculation for roc_auc_score is nonlinear; more on this below.
[+] arr_ind is a list of the indices of the arrays, which I created like this:
# build array index
arr_ind = range(len(array1))
[+] You also didn't include the arrays, so I created them like this:
array1 = np.random.rand(10, 1)
array2 = np.random.rand(10, 1)
array3 = np.random.rand(10, 1)
train = {}
train['target'] = np.ones((10, 1))
Here is my complete code, which compiles and executes, though I'm sure it doesn't give you the result you are hoping for, since I just guessed about target and roc_auc_score:
import numpy as np
import pulp
# create the LP object, set up as a maximization problem
prob = pulp.LpProblem('Giapetto', pulp.LpMaximize)
# dummy arrays since arrays weren't in OP code
array1 = np.random.rand(10, 1)
array2 = np.random.rand(10, 1)
array3 = np.random.rand(10, 1)
# build array index
arr_ind = range(len(array1))
# set up decision variables
x = pulp.LpVariable('x', lowBound=0)
y = pulp.LpVariable('y', lowBound=0)
z = pulp.LpVariable('z', lowBound=0)
# dummy roc_auc_score since roc_auc_score wasn't in OP code
train = {}
train['target'] = np.ones((10, 1))
score = pulp.lpSum([train['target'][i] - (x * array1[i] + y * array2[i] + z * array3[i]) for i in arr_ind])
prob += score
coef = x + y + z
prob += coef == 1
# solve the LP using the default solver
optimization_result = prob.solve()
# make sure we got an optimal solution
assert optimization_result == pulp.LpStatusOptimal
# display the results
for var in (x, y,z):
print('Optimal weekly number of {} to produce: {:1.0f}'.format(var.name, var.value()))
Output:
Optimal weekly number of x to produce: 0
Optimal weekly number of y to produce: 0
Optimal weekly number of z to produce: 1
Process finished with exit code 0
Now, if your roc_auc_score function is nonlinear, you will have additional troubles. I would encourage you to try to formulate the score in a way that is linear, possibly using additional variables (for example, if you want the score to be an absolute value).