Is the Pearson correlation coefficient -- with one vector, x, exogenous and another vector, y, as a choice variable -- a suitable quadratic objective function for quadratic programming solvers like Gurobi?
A quick Google search for "Gurobi objective function" shows that Gurobi has an API to set an objective function that accepts a linear or quadratic expression. That is quite expected because quadratic programming is, by definition, an optimization of a quadratic function, with the math behind the methods specifically designed for this class (like, working directly with the Q coefficient matrix and the c vector rather than the raw function).
I didn't look into details too much, but I can see that Pearson product-moment correlation coefficient appears to be not a quadratic but a rational function. So, if your specific case can't be simplified to that, no.
I cannot say anything about other solvers because each one is an independent product and has to be considered separately.
Since your function appears to be piecewise continuous and infinitely differentiable, you're probably interested in general-purpose gradient methods instead.
Related
when I set ‘cov=True’，the function ‘numpy.polyfit’ will return a covariance matrix，I wonder what does it mean and how to calculate it，Thanks a lot ！！！
the author of ‘ZSSR’ super resolution algorithm adopted a learning rate adjustment strategy，which first take a linear fit of reconstruction error，and then compare the standard deviation and the slope to guide the adjustment of learning rate。I can‘t see the relationship between the learning rate and the standard deviation（in fact I don’t know what does the covariance matrix returned by the funciton mean），so I wonder know the detail of this covariance matrix。
I have two - likely simple - questions that are bothering me, both related to quadratic programming:
1). There are two "standard" forms of the objective function I have found, differing by multiplication of negative 1.
In the R package quadprog, the objective function to be minimized is given as −dTb+12bTDb and in Matlab the objective is given as dTb+12bTDb. How can these be the same? It seems that one has been multiplied through by a negative 1 (which as I understand it would change from a min problem to a max problem.
2). Related to the first question, in the case of using quadprog for minimizing least squares, in order to get the objective function to match the standard form, it is necessary to multiply the objective by a positive 2. Does multiplication by a positive number not change the solution?
EDIT: I had the wrong sign for the Matlab objective function.
Function f(b)=dTb is a linear function thus it is both convex and concave. From optimization standpoint it means you can maximize or minimize it. Nevertheless minimizer of −dTb+12bTDb will be different from dTb+12bTDb, because there is additional quadratic term. Matlab implementation will find the one with plus sign. So if you are using different optimization software you will need to change d→−d to get the same result.
The function −dTb+12bTDb where D is symmetric and convex and thus has unique minimum. In general that is called standard quadratic programming form, but that doesn't really matter. The other function dTb−12bTDb is concave function which has unique maximum. It is easy to show that for, say, bounded function f(x) from above the following holds:
argmaxxf=argminx−f
Using the identity above value b∗1 where −dTb+12bTDb achieves minimum is the same as the value b∗2 which achieves maximum at dTb−12bTDb, that is b∗1=b∗2.
Programmatically it doesn't matter if you are minimizing −dTb+12bTDb or maximizing the other one. These are implementation-dependent details.
No it does not. ∀α>0 if x∗=argmaxxf(x), then x∗=argmaxxαf(x). This can be showed by contradiction.
I am new to the optimization problem of Quadratic programming.
In equation 8 of the following paper: here , there is an equation:
The authors state that this is an 'Integer Quadratic Programming (IQP)' formula.
Alternatively, in another website: here , there is the following equation which is described as a 'Mixed Integer Quadratic Programming (MIQP)' formulation:
From my perspective, both of the equations shown above are similar, with the only difference being that the MIQP formula has '1/2' included in it.
1) I am looking for an explanation on the differences between the IQP and MIQP
2) In addition, I am interested to apply quadratic programming to the assignment problem, thus, looking for any insight into which should be used (i.e., IQP vs. MIQP) and when.
Integer Quadratic Programming (IQP) implies there are no continuous variables in the model: all variables are discrete. Mixed Integer Quadratic Programming (MIQP) allows both discrete and continuous variables. If your model only has discrete variables it is both an MIQP and an IQP. All popular solvers are of the MIQP type, so I tend to use MIQP even if I don't have continuous variables. IQP as model type is not often used. I don't think this is really something to worry about.
I got a basic idea of Big-O notation from Big-O notation's definition.
In my problem, a 2-D surface is divided into uniform M grids. Each grid (m) is assigned with a posterior probability based on A features.
The posterior probability of m grid is calculated as follows:
and the marginal likelihood is given as:
Here, A features are independent of each other and sigma and mean symbol represent the standard deviation and mean value of each a feature at each grid. I need to calculate the Posterior probability of all M grids.
What will be the time complexity of the above operation in terms of Big-O notation?
My guess is O(M) or O(M+A). Am I correct? I'm expecting an authenticate answer to present at the formal forum.
Also, what will be the time complexity if M grids are divided into T clusters where every cluster has Q grids (Q << M) (calculating Posterior Probability only on Q grids out of M grids) ?
Thank you very much.
Discrete sum and product
can be understood as loops. If you are happy with floating point approximation most other operators are typically O(1), conditional probability looks like a function call. Just inject constants and variables in your equation and you'll get the expected Big-O, the details of formula are irrelevant. Also be aware that these "loops" can often be simplified using mathematical properties.
If the result is not obvious, please convert your above mathematical formula in actual programming code in a programming language. Computer Science Big-O is never about a formula but about an actual translation of it in programming steps, depending on the implementation the same formula can lead to very different execution complexities. As different as adding integers by actually performing sum O(n) or applying Gauss formula O(1) for instance.
By the way why are you doing a discrete sum on a discrete domaine N ? Shouldn't it be M ?
I'm using
R=QQ['x'];
to declare x as variable such that I can calculate with polynomials in x. What I need is another variable, for example t, to represent an integer which can also be used as exponent. For example I want to consider the polynomial (t+1)x^t. Is that somehow possible in SageMath?
EDIT: I want to explain a little bit the reason why I'm looking for such a feature. I've got a couple of really complex rational functions in a few variables and want SageMath to help me to show that they are identical. They are written down in a different way and if you would do it with pen and paper you would need hours and have a huge change of making mistakes. Actually it are not only a few rational functions but infinitely many. But using variables in the coefficients you can cover infinitely many with only one rational function. A simple example is the polynomial 1+x+x^2+...+x^t. For every non-negative integer you get a different polynomial. But you can write (x^(t+1)-1)/(x-1) as rational function instead. With taking t as a variable you cover infinitely many cases with just one rational function.
Is there a way to do such stuff in SageMath?
To create the polynomial x^t as an element of a polynomial ring, Sage needs to know what integer t is equal to. If polynomials of undetermined degree were introduced, most of the Sage methods for polynomials would not work for them: no way to get the list of coefficients with coefficients(), or to find the factors, or the GCD of two polynomials, etc.
However, you can manipulate and simplify polynomials and rational functions just like any other symbolic expressions. For example:
x,t,k = var('x,t,k')
sum(x^k, k, 1, t)
returns (x^(t + 1) - x)/(x - 1).
The relevant articles are Symbolic Computation and Symbolic Expressions.