I have experimental scattered data (in green, in the picture) in a 2D domain (x,y), that I want to fit with a two-dimensional polynomial, such as:
f(x,y) = c0 + c1*x + c2*y + c3*x*y + c4 * x ** 2 * y ** 2
where c0, c1,... are the coefficients of the polynomial. On top of this, I have equality and inequality constraints:
f(x=0,y) = 0
f(x,y) > 0, for 0 < x < 90
How can I do this? Can I express my inequality in f(x,y), by inequalities in the c0, c1, c2,... coefficients?
I used scipy.optimize.minimize to minimize the least squares of ||Ax-B||, where Ax is the polynomial expression evaluated at the experimental points, x is the vector of the coefficients c0, c1, c2,... to be optimized, and B is my experimental data. I really need some guidance on how to apply the inequality constraint.
What I tried so far:
I was able to implement the equality constraint, manually simplifying f(x,y) and f(x=0,y)=0, by substitution, and reformulating ||Ax-B||, but I cannot do that for the inequality constraint. See the picture,
where f(x=0,y) = 0 is satisfied, but not f(x,y) > 0.
I tried using the constraints parameter, but I could only apply inequality constraints on the c0,c1,c2,... coefficients, instead of applying the constraint on the desired f(x,y).
I have read on Lagrange multipliers and non-linear programming but I'm still lost.
Two solutions:
With scipy.optimize.minimize the function to be minimized is some kind of chi^2, but additionally, it your constraints are not met, then it returns np.inf, which provides hard boundary.
Use Monte-Carlo Markov Chain method. There are many implementations in python.
Related
I would like to fit 2D data where my function is a product of two 1D polynomials of 4th order.
f(x,y) = (a0 + a1*x + ... + a4*x^4)*(b0 + b1*y ... + b4*y^4)
I know that I can solve this using any optimization routine that uses non-linear least squares.
Is there a way to use a linear least square method to fit such a factorizable polynomial?
I could expand the polynomial and formulate the problem as
(1, x1, y1, x1*y1, x1^2, y1^2, x1^2*y1, ...) * X = f(x1, y1)
. .
. .
. .
and solve for X = (a0*b0, a1*b0, a0*b1, ...)^T.
Then I have the problem, that the system of equations that define a0, a1, ... and b0, b1, ... is overdetermined (X has length 5*5 = 25, but I have only 5+5=10 independent variables).
Constrained least squares (using e.g. L*X = d) also does not seem to work, since the constraints are non-linear.
Is there any (tricky?) way to use linear least squares for this type of problem?
I'm trying to write fast, optimized code based on matrices, and have recently discovered einsum as a tool for achieving significant speed-up.
Is it possible to use this to set the diagonals of a multidimensional array efficiently, or can it only return data?
In my problem, I'm trying to set the diagonals for an array of square matrices (shape: M x N x N) by summing the columns in each square (N x N) matrix.
My current (slow, loop-based) solution is:
# Build dummy array
dimx = 2 # Dimension x (likely to be < 100)
dimy = 3 # Dimension y (likely to be between 2 and 10)
M = np.random.randint(low=1, high=9, size=[dimx, dimy, dimy])
# Blank the diagonals so we can see the intended effect
np.fill_diagonal(M[0], 0)
np.fill_diagonal(M[1], 0)
# Compute diagonals based on summing columns
diags = np.einsum('ijk->ik', M)
# Set the diagonal for each matrix
# THIS IS LOW. CAN IT BE IMPROVED?
for i in range(len(M)):
np.fill_diagonal(M[i], diags[i])
# Print result
M
Can this be improved at all please? It seems np.fill_diagonal doesn't accepted non-square matrices (hence forcing my loop based solution). Perhaps einsum can help here too?
One approach would be to reshape to 2D, set the columns at steps of ncols+1 with the diagonal values. Reshaping creates a view and as such allows us to directly access those diagonal positions. Thus, the implementation would be -
s0,s1,s2 = M.shape
M.reshape(s0,-1)[:,::s2+1] = diags
If you do np.source(np.fill_diagonal) you'll see that in the 2d case it uses a 'strided' approach
if a.ndim == 2:
step = a.shape[1] + 1
end = a.shape[1] * a.shape[1]
a.flat[:end:step] = val
#Divakar's solution applies this to your 3d case by 'flattening' on 2 dimensions.
You could sum the columns with M.sum(axis=1). Though I vaguely recall some timings that found that einsum was actually a bit faster. sum is a little more conventional.
Someone has has asked for an ability to expand dimensions in einsum, but I don't think that will happen.
Consider the following pseudo code:
a <- [0,0,0] (initializing a 3d vector to zeros)
b <- [0,0,0] (initializing a 3d vector to zeros)
c <- a . b (Dot product of two vectors)
In the above pseudo code, what is the flop count (i.e. number floating point operations)?
More generally, what I want to know is whether initialization of variables counts towards the total floating point operations or not, when looking at an algorithm's complexity.
In your case, both a and b vectors are zeros and I don't think that it is a good idea to use zeros to describe or explain the flops operation.
I would say that given vector a with entries a1,a2 and a3, and also given vector b with entries b1, b2, b3. The dot product of the two vectors is equal to aTb that gives
aTb = a1*b1+a2*b2+a3*b3
Here we have 3 multiplication operations
(i.e: a1*b1, a2*b2, a3*b3) and 2 addition operations. In total we have 5 operations or 5 flops.
If we want to generalize this example for n dimensional vectors a_n and b_n, we would have n times multiplication operations and n-1 times addition operations. In total we would end up with n+n-1 = 2n-1 operations or flops.
I hope the example I used above gives you the intuition.
According to slide 5 here the condition for submodularity for a multilabel pairwise energy function is
for $\alpha < \beta$ and $\alpha' < \beta'$,
$f(\alpha, \alpha') + f(\beta, \beta') <= f(\alpha, \beta') + f(\alpha', \beta)$
How does that inequality imply submodularity? I can understand how the inequality for the binary label case implies submodularity (marginal gain of picking an element in a set is higher when we didn't have any element initially), but how should one interpret the multilabel case?
I was wondering about this problem concerning Katatsuba's algorithm.
When you apply Karatsuba you basically have to do 3 multiplications per one run of the loop
Those are (let's say ab and cd are 2-digit numbers with digits respectively a, b, c and d):
X = bd
Y = ac
Z = (a+c)(c+d)
and then the sums we were looking for are:
bd = X
ac = Y
(bc + ad) = Z - X - Y
My question is: let's say we have two 3-digit numbers: abc, def. I found out that we will have to perfom only 5 multiplications to do so. I also found this Toom-3 algorithm, but it uses polynomials I can;t quite get. Could someone write down those multiplications and how to calculate the interesting sums bd + ae, ce+ bf, cd + be + af
The basic idea is this: The number 237 is the polynomial p(x)=2x2+3x+7 evaluated at the point x=10. So, we can think of each integer corresponding to a polynomial whose coefficients are the digits of the number. When we evaluate the polynomial at x=10, we get our number back.
What is interesting is that to fully specify a polynomial of degree 2, we need its value at just 3 distinct points. We need 5 values to fully specify a polynomial of degree 4.
So, if we want to multiply two 3 digit numbers, we can do so by:
Evaluating the corresponding polynomials at 5 distinct points.
Multiplying the 5 values. We now have 5 function values of the polynomial of the product.
Finding the coefficients of this polynomial from the five values we computed in step 2.
Karatsuba multiplication works the same way, except that we only need 3 distinct points. Instead of at 10, we evaluate the polynomial at 0, 1, and "infinity", which gives us b,a+b,a and d,d+c,c which multiplied together give you your X,Z,Y.
Now, to write this all out in terms of abc and def is quite involved. In the Wikipedia article, it's actually done quite nicely:
In the Evaluation section, the polynomials are evaluated to give, for example, c,a+b+c,a-b+c,4a+2b+c,a for the first number.
In Pointwise products, the corresponding values for each number are multiplied, which gives:
X = cf
Y = (a+b+c)(d+e+f)
Z = (a+b-c)(d-e+f)
U = (4a+2b+c)(4d+2e+f)
V = ad
In the Interpolation section, these values are combined to give you the digits in the product. This involves solving a 5x5 system of linear equations, so again it's a bit more complicated than the Karatsuba case.