I am trying to find out the parameters for the function below:
$$
\log L(\alpha,\beta,v) = v/\beta(e^{-\beta T} -1) + \alpha/\beta \sum_{i=1}^{n}(e^{-\beta(T-t_i)} -1) + \sum_{i=1}^{N}log(v e^{-\beta t_i} + \alpha \sum_{j=1}^{jmax(t_i)} e^{-\beta(t_i - t_j)}).
$$
However, the conventional methods like fmin, fminsearch are not converging properly. Any suggestions on any other methods or open libraries which I can use?
I was trying CVXPY, but they don't support the division by a variable in the expression.
The problem may not be convex (I have not verified this but it could be why CVXPY refused it). We don't have the data so we cannot try things out, but I can give some general advice:
Provide exact gradients (and 2nd derivatives if needed) or use a modeling system with automatic differentiation. Especially first derivatives should be preferably quite precise. With finite differences you may lose half the precision.
Provide a good starting point. May be using an alternative estimation method.
Some solvers can use bounds on the variables to restrict the feasible region where functions will be evaluated. This can be used to restrict the search to interesting areas only and also to protect operations like division and log functions.
Related
Regarding the following coda.samples in rjags,
samples<-coda.samples(jags,
c('B', 'A'),
5000,thin=5)
The iteration is setup as 5000 and the thin parameter is setup as 5. Are there any mechanisms or rules for setting up these two parameters in Bayesian analysis.
Do you mean "is there a way to define 'correct' or 'optimum' values for these parameters?
If so, then the answer is "No". Not in R or any language. You need to look at diagnostics for your model fit (such as trace plots, Gelman-Rubin, Raftery-Lewis and Heidelberger-Welch statistics) and adjusting the parameters of your MCMC process until you get "acceptable" results. It's an art, not a deterministic algorithm.
You should also look at the number of chains you use, as well as the burn-in, thinning and other parameters of each chain.
I would like to use z3 optimize class to get sub-optimal results, while still being able to control how far am I from the optimum result. I am using the C++ API.
As an example, CPLEX has the parameters epgap and epagap for relative and absolute tolerance respectively. It uses the current lower or upper bounds (depending if it is a minimization or maximization) to assess how far (at most) the current solution is from the optimal one.
This leads to shorter run-times for when an approximate solution is already good enough.
Is this possible using the optimize class, or is this something I would need to implement using a solver instance and control the bounds myself?
I am not absolutely certain about this, but I doubt that z3 has such parameters.
For sure, nothing like that appears to be exposed in the command-line interface:
~$ z3 -p
...
[module] opt, description: optimization parameters
dump_benchmarks (bool) (default: false)
dump_models (bool) (default: false)
elim_01 (bool) (default: true)
enable_sat (bool) (default: true)
enable_sls (bool) (default: false)
maxlex.enable (bool) (default: true)
maxres.add_upper_bound_block (bool) (default: false)
maxres.hill_climb (bool) (default: true)
maxres.max_core_size (unsigned int) (default: 3)
maxres.max_correction_set_size (unsigned int) (default: 3)
maxres.max_num_cores (unsigned int) (default: 4294967295)
maxres.maximize_assignment (bool) (default: false)
maxres.pivot_on_correction_set (bool) (default: true)
maxres.wmax (bool) (default: false)
maxsat_engine (symbol) (default: maxres)
optsmt_engine (symbol) (default: basic)
pb.compile_equality (bool) (default: false)
pp.neat (bool) (default: true)
priority (symbol) (default: lex)
rlimit (unsigned int) (default: 0)
solution_prefix (symbol) (default: )
timeout (unsigned int) (default: 4294967295)
...
Alternative #01:
An option is to implement this yourself on top of z3.
I would suggest using the binary search schema (see Optimization in SMT with LA(Q) Cost Functions), otherwise the OMT solver is going to refine only one end of the optimization search interval and this may defeat the intended purpose of your search-termination criteria.
Notice that in order for this approach to be effective, it is important that the internal T-optimizer is invoked over the Boolean assignment of each intermediate model encountered along the search. (I am not sure whether this functionality is exposed at the API level with z3).
You may also want to take a look at the approach based on linear regression used in Puli - A Problem-Specific OMT Solver. If applicable, it may speed-up the optimization search and improve the estimate of the relative distance from the optimal solution.
Alternative #02:
OptiMathSAT may be exposing the functionality you are looking for, both at the API and the command-line level:
~$ optimathsat -help
Optimization search options:
-opt.abort_interval=FLOAT
If greater than zero, an objective is no longer actively optimized as
soon as the current search interval size is smaller than the given
value. Applies to all objective functions. (default: 0)
-opt.abort_tolerance=FLOAT
If greater than zero, an objective is no longer actively optimized as
soon as the ratio among the current search interval size wrt. its
initial size is smaller than the given value. Applies to all
objective functions. (default: 0)
The abort interval is a termination criterion based on the absolute size of the current optimization search interval, while the abort tolerance is a termination criterion based on the relative size of the current optimization search interval with respect to the initial search interval.
Notice that in order to use these termination criteria, the user is expected to:
provide (at least) an initial lower-bound for any minimization objective:
(minimize ... :lower ...)
provide (at least) an initial upper-bound for any maximization objective:
(maximize ... :upper ...)
Furthermore, the tool must be configured to use either Binary or Adaptive search:
-opt.strategy=STR
Sets the optimization search strategy:
- lin : linear search (default)
- bin : binary search
- ada : adaptive search
A lower bound is required to minimize an objective with bin/ada
search strategy. Dual for maximization.
In case neither of these termination criterion is satisfactory to you, you can also implement your own algorithm on top of OptiMathSAT. It is relatively easy to do, thanks to the following option that can be set both via API and command-line:
-opt.no_optimization=BOOL
If true, the optimization search stops at the first (not optimal)
satisfiable solution. (default: false)
When enabled, it makes OptiMathSAT behave like a regular SMT solver, except that when it finds a complete Boolean assignment for which there exists a Model of the input formula, it ensures that the Model is optimal wrt. the objective function and the given Boolean assignment (in other words, it invokes the internal T-optimizer procedure for you).
Some Thoughts.
OMT solvers work differently from most CP solvers. They use infinite-precision arithmetic and the optimization search is guided by the SAT engine. Improving the value of the objective function becomes increasingly difficult because the OMT solver is forced to enumerate a progressively larger number of (possibly total) Boolean assignments while resolving conflicts and back-jumping along the way.
In my opinion, the size of the current search interval is not always a good indicator of the relative difficulty of making progress with the optimization search. There are far too many factors to take into consideration, e.g. the pruning power of conflict clauses involving the objective function, the encoding of the input formula, and so on. This is also one of the reasons why, as far as I have seen, most people in the OMT community simply use a fixed timeout rather than bothering to use any other termination criteria. The only situation in which I have found it to be particularly useful, is when dealing with non-linear optimization (which, however, is not yet publicly available with OptiMathSAT).
I'm trying to fit the lppl model to KLSE index to predict the most probable crash time. Many papers suggested tabuSearch to identify the initial value for non-linear parameters but none of them publish their code. I have tried to fit the mentioned index with the help of NLS And Log-Periodic Power Law (LPPL) in R. But the obtained error and p values are not significant. I believe that the initial values are not accurate. Can anyone help me on how to find the proper initial values?
library(tseries)
library(zoo)
ts<-get.hist.quote(instrument="^KLSE",start="2003-04-18",end="2008-01-30",quote="Close",provider="yahoo",origin="1970-01-01",compression="d",retclass="zoo")
df<-data.frame(ts)
df<-data.frame(Date=as.Date(rownames(df)),Y=df$Close)
df<-df[!is.na(df$Y),]
library(minpack.lm)
library(ggplot2)
df$days<-as.numeric(df$Date-df[1,]$Date)
f<-function(pars,xx){pars$a + (pars$tc - xx)^pars$m *(pars$b+ pars$c * cos(pars$omega*log(pars$tc - xx) + pars$phi))}
resids<-function(p,observed,xx){df$Y-f(p,xx)}
nls.out <- nls.lm(par=list(a=600,b=-266,tc=3000, m=.5,omega=7.8,phi=-4,c=-14),fn = resids, observed = df$Y, xx = df$days, control= nls.lm.control (maxiter =1024, ftol=1e-6, maxfev=1e6))
par<-nls.out$par
nls.final<-nls(Y~(a+(tc-days)^m*(b+c*cos(omega*log(tc-days)+phi))),data=df,start=par,algorithm="plinear",control=nls.control(maxiter=10024,minFactor=1e-8))
summary(nls.final)
I would look at some of the newer research on this topic, there is a good trig modification that will practically guarantee a singular optimization. Additionally, you can use r's built in linear equation solver, to find the linearizable parameters, ergo you will only need to optimize in 3 dimensions. The link below should get you started. I would cite recent literature and personal experience to strongly advise against a tabu search.
https://www.ethz.ch/content/dam/ethz/special-interest/mtec/chair-of-entrepreneurial-risks-dam/documents/dissertation/master%20thesis/MAS_final_Tuncay.pdf
I am currently trying to solve a complementarity problem with a function that features a downward discontinuity, using the mcpsolve() function of the NLsolve package in Julia. The function is reproduced here for specific parameters, and the numbers below refer to the three panels of the figure.
Unfortunately, the algorithm does not always return the interior solution, even though it exists:
In (1), when starting at 0, the algorithm stays at 0, thinking that the boundary constraint binds,
In (2), when starting at 0, the algorithm stops right before the downward jump, even though the solution lies to the right of this point.
These problems occur regardless of the method used - trust region or Newton's method. Ideally, the algorithm would look for potential solutions in the entire set before stopping.
I was wondering if some of you had worked with similar functions, and had found a clever solution to bypass these issues. Note that
Starting to the right of the solution would not solve these problems, as they would also occur for other parametrization - see (3) this time,
It is not known a priori where in the parameter space the particular cases occur.
As an illustrative example, consider the following piece of code. Note that the function is smoother, and yet here as well the algorithm cannot find the solution.
function f!(x,fvec)
if x[1] <= 1.8
fvec[1] = 0.1 * (sin(3*x[1]) - 3)
else
fvec[1] = 0.1 * (x[1]^2 - 7)
end
end
NLsolve.mcpsolve(f!,[0.], [Inf], [0.], reformulation = :smooth, autodiff = true)
Once more, setting the initial value to something else than 0 would only postpone the problem. Also, as pointed out by Halirutan, fzero from the Roots package would probably work, but I'd rather use mcpsolve() as the problem is initially a complementarity problem.
Thank you very much in advance for your help.
I was profiling an application that does a lot of math operations on NMatrix matrices.
The application spends most of it's time in in the code below.
{add: :+, sub: :-, mul: :*, div: :/, pow: :**, mod: :%}.each_pair do |ewop, op|
define_method("__list_elementwise_#{ewop}__") do |rhs|
self.__list_map_merged_stored__(rhs, nil) { |l,r| l.send(op,r) }.cast(stype, NMatrix.upcast(dtype, rhs.dtype))
end
define_method("__dense_elementwise_#{ewop}__") do |rhs|
self.__dense_map_pair__(rhs) { |l,r| l.send(op,r) }.cast(stype, NMatrix.upcast(dtype, rhs.dtype))
end
define_method("__yale_elementwise_#{ewop}__") do |rhs|
self.__yale_map_merged_stored__(rhs, nil) { |l,r| l.send(op,r) }.cast(stype, NMatrix.upcast(dtype, rhs.dtype))
end
end
In the commets above the code it says:
# Define the element-wise operations for lists. Note that the __list_map_merged_stored__ iterator returns a Ruby Object
# matrix, which we then cast back to the appropriate type. If you don't want that, you can redefine these functions in
# your own code.
I am not that familiar with the internals of NMatrix but it seems as though the math operations are being executed in Ruby. Is there anyway to speed up these methods?
We had written them in C/C++ originally, but it required some really complicated macros which were basically unmaintainable and buggy, and substantially increased compile time.
If you look in History.txt, you'll be able to find at what version we started writing the math operations in Ruby. You could use the prior code to override and put the element-wise operations (where you need speed) exclusively in C/C++.
However, you may run into problems getting those to work properly (without a crash) on matrices of dtype :object.
As a side note, the sciruby-dev Google Group (or the nmatrix issue tracker) might be a more appropriate place for a question like this one.