I am trying to get the theory behind LabVIEW's Constrained Nonlinear Optimization VI. There description provides how to use it but not which optimization algorithms works behind it.
Here is an overview of the optimization algorithms but it simply states
Solves a general nonlinear optimization problem with nonlinear equality constraint and nonlinear inequality constraint bounds using a sequential quadratic programming method.
I suspect that it is a wrapper for multiple algorithms depending on the inputs... I want to know if it uses a Levenberg-Marquardt or a Downhill-Simplex or other theory. It is not even said if it is trust-region or line search and how the bounds are ensured (e.g. by reflection)... In other languages, the documentation often refers to a paper from which I can take the original theory. This is what I am looking for. Can anyone help (or do I have to contact the NI support)? Thx
(using LabVIEW 2017 and 2018 32bit)
Related
Which of the following optimisation methods can't be done in an optimisation software such as CPLEX? Why not?
Dynamic programming
Integer programming
Combinatorial optimisation
Nonlinear programming
Graph theory
Precedence diagram method
Simulation
Queueing theory
Can anyone point me in the right direction? I didn't find too much information regarding the limitations of CPLEX on the IBM website.
Thank you!
That's kind-of a big shopping list, and most of the things on it are not optimisation methods.
For sure CPLEX does integer programming, non-linear programming (just quadratic, SOCP, and similar but not general non-linear) and combinatoric optimisation out of the box.
It is usually possible to re-cast things like DP as MILP models, but will obviously require a bit of work. Lots of MILP models are also based on graphs, so yes it is certainly possible to solve a lot of graph problems using a MILP solver such as CPLEX.
Looking wider at topics like simulation, then that is quite a different approach. Simulation really is NOT an optimisation method, but it can be used alongside optimisation to get extra insights which may be useful in a business context. Might be used for example to discover some empirical relationships that could be used in an optimisation model by CPLEX.
The same can probably also be said for things like queuing theory, precedence, etc. Basically, use CPLEX as an optimisation tool to solve part or all of your problem once you have structured and analysed it via one of these other approaches.
Hope that helps.
I have read that Nelder Mead algorithm is working for unconstrained optimization.
http://www.scholarpedia.org/article/Nelder-Mead_algorithm
I think in Matlab Nelder Mead is used also for unconstrained optimization.
However, I am a little bit confused, since I found a Java API for optimization
http://www.ee.ucl.ac.uk/~mflanaga/java/Minimisation.html
(Flanagan's Scientific Library)
that has a class that implements Nelder Mead simplex and allows for defining constraints and bounds.
So, is the version implemented in Flanagan's API a modified variation of the "classical" Nelder Mead algorithm?
It looks like the API is implementing a simple "soft" constraint system, where constraints are transformed into penalty functions which severely penalize regions outside the constraints. It's a cheap-and-cheerful way of adding constraints to an unconstrained solver, but there'll be a tradeoff between optimality, convergence, and the degree to which the constraints are satisfied.
I am learning about optimization algorithms for automatic grouping of users. However, I am completely new to these algorithms and I have heard about them as I reviewed the related literature. And, differently, in one of the articles, the authors implemented their own algorithm (based on their own logic) using Integer Programming (this is how I heard about IP).
I am wondering if one needs to implement a genetic/particle swarm (or any other optimization) algorithm using mixed integer linear programming, or is this just one of the options. At the end, I will need to build a web-based system that groups users automatically. I appreciate any help.
I think you are confusing the terms a bit. These are all different optimization techniques. You can surely represent a problem using Mixed Integer Programming (MIP) notation but you can solve it with a MIP solver or genetic algorithms (GA) or Particle Swarm Optimization (PSO).
Integer Programming is part of a more traditional paradigm called mathematical programming, in which a problem is modelled based on a set of somewhat rigid equations. There are different types of mathematical programming models: linear programming (where all variables are continuous), integer programming, mixed integer programming (a mix of continuous and discrete variables), nonlinear programming (some of the equations are not linear).
Mathematical programming models are good and robust, depending on the model, you can tell how far you are from an ideal solution, for example. But these models often struggle in problems with many variables.
On the other hand, genetic algorithms and PSO belong to a younger branch of optimization techniques, one that it is often called metaheuristics. These techniques often find good or at least reasonable solutions even for large and complex problems, many practical applications
There are some hybrid algorithms that combine both mathematical models and metaheuristics and in this case, yes, you would use both MIP and GA/PSO. Choosing which approach (MIP, metaheuristics or hybrid) is very problem-dependent, you have to test what works better for you. I would usually prefer mathematical models if the focus is on the accuracy of the solution and I would prefer metaheuristics if my objective function is very complex and I need a quick, although poorer, solution.
I am currently working on a support vector machine (SVM) project. The version of SVM that I am working on is Linear SVM in Primal Form and I am having hard time understanding where to start.
In general, I think I understand the theory; basically I need to minimize norm of w under certain constraint. And the Lagrangian function will be my objective function to be minimized (after Lagrange multiplier is applied).
The things that I don't understand is that I was told from my professor that we will be using Quasi-Newton method along with BFGS update. I have tried 2D and 3D case for Newton's method and I think I have good grasp of the algorithm, but I don't see how Quasi-Newton method is applied to find the coefficients alpha. Also, many literature that I read so far tells to apply Quadratic programming to find the coefficients.
How is the iterative algorithm of Quasi-Newton related to finding coefficients of w...? And how is quadratic programming related to Quasi-Newton? Can anyone please walk me through what is going on?
You are cunfusing many things here
"alpha coefficients" are only in the dual form, so you do not find them in your case
"apply Quadratic programming", quadratic programming is a problem, not a solution. you cannot "apply QP", you can only solve a QP, which in your case will be solved using quasi-newton method
"how is (...) related to finding coefficientss of w" exactly the same way, as this optimization technique is related to finding the optimal coefficients of any function. You are going to minimize the function of w, so applying any optimization technique (in particular quasi-netwton) will lead to solution expressed as w coefficients
This is for self-study of N-dimensional system of linear homogeneous ordinary differential equations of the form:
dx/dt=Ax
where A is the coefficient matrix of the system.
I have learned that you can check for stability by determining if the real parts of all the eigenvalues of A are negative. You can check for oscillations if there are any purely imaginary eigenvalues of A.
The author in the book I'm reading then introduces the Routh-Hurwitz criterion for detecting stability and oscillations of the system. This seems to be a more efficient computational short-cut than calculating eigenvalues.
What are the advantages of using Routh-Hurwitz criteria for stability and oscillations, when you can just find the eigenvalues quickly nowadays? For instance, will it be useful when I start to study nonlinear dynamics? Is there some additional use that I am completely missing?
Wikipedia entry on RH stability analysis has stuff about control systems, and ends up with a lot of equations in the s-domain (Laplace transforms), but for my applications I will be staying in the time-domain for the most part, and just focusing fairly narrowly on stability and oscillations in linear (or linearized) systems.
My motivation: it seems easy to calculate eigenvalues on my computer, and the Routh-Hurwitz criterion comes off as sort of anachronistic, the sort of thing that might save me a lot of time if I were doing this by hand, but not very helpful for doing analysis of small-fry systems via Matlab.
Edit: I've asked this at Math Exchange, which seems more appropriate:
https://math.stackexchange.com/questions/690634/use-of-routh-hurwitz-if-you-have-the-eigenvalues
There is an accepted answer there.
This is just legacy educational curriculum which fell way behind of the actual computational age. Routh-Hurwitz gives a very nice theoretical basis for parametrization of root positions and linked to much more abstract math.
However, for control purposes it is just a nice trick that has no practical value except maybe simple transfer functions with one or two unknown parameters. It had real value when computing the roots of the polynomials were expensive or even manual. Today, even root finding of polynomials is based on forming the companion matrix and computing the eigenvalues. In fact you can basically form a meshgrid and check the stability surface by plotting the largest real part in a few minutes.