Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 3 years ago.
Improve this question
I'm reading an article about machine learning theory, there are has a step as below to calculate partial derivative:
∂(w5 * h1 + w6 * h2 + b3) / ∂h1 = w5 * f′(w5 * h1 + w6 * h2 + b3)
As I understand the caclulation method of partial derivative, the result of ∂(w5 * h1 + w6 * h2 + b3) / ∂h1 should be w5, should not be w5 * f′(w5 * h1 + w6 * h2 + b3), I'm very confused this step, could you tell me some explain? Thank you.
I believe there is a typo in the formula that you are reading.
∂f(w5 * h1 + w6 * h2 + b3) / ∂h1 = w5 * f′(w5 * h1 + w6 * h2 + b3)
I believe there should be an f on the LHS of the equation, after which, chain rule is being applied.
If f is absent, then you are right.
Related
There is a closed form solution to the arc length of a quadratic Bézier curve, namely:
Assuming the Bézier is of the equation A + Bt + Ct^2 where A,B,C are vectors
a = (B dot B), b = 2*(B dot C), and c = 4*(C dot C)
(2 * c * t + b) * sqrt(a + b * t + c * t ^ 2) / (4 * c) + log(2 * sqrt(c) * sqrt(a + b * t + c * t ^ 2) + 2 * c * t + b) * (4 * a * c - b ^ 2) / (8 * c ^ (3 / 2))
It's hairy, but does provide a precise answer as to the arc length of a given quadratic Bézier. I'm currently working with Bézier curves and need a way to create a function that, given a distance on the Bézier curve, returns a precise t value that can be used to place that point properly.
I'm aware that there are ways to approximate the arc parameterization of a Bézier curve, as for cubics and higher there is no closed form solution to the elliptic integral that returns their arc length, but since there is an exact equation that gives the length of a quadratic Bézier, would it be possible to find the inverse and get an equation that given a distance returns a t value? Putting that equation into Wolfram Alpha and asking it to solve for t has given me no luck.
In particle physics, we have to compute the invariant mass a lot, which is for a two-body decay
When the momenta (p1, p2) are sometimes very large (up to a factor 1000 or more) compared to the masses (m1, m2). In that case, there is large cancellation happening between the last two terms when the calculation is carried out with floating point numbers on a computer.
What kind of numerical tricks can be used to compute this accurately for any inputs?
The question is about suitable numerical tricks to improve the accuracy of the calculation with floating point numbers, so the solution should be language-agnostic. For demonstration purposes, implementations in Python are preferred. Solutions which reformulate the problem and increase the amount of elementary operations are acceptable, but solutions which suggest to use other number types like decimal or multi-precision floating point numbers are not.
Note: The original question presented a simplified 1D dimensional problem in form of a Python expression, but the question is for the general case where the momenta are given in 3D dimensions. The question was reformulated in this way.
With a few tricks listed on Stackoverflow and the transformation described by Jakob Stark in his answer, it is possible to rewrite the equation into a form that does not suffer anymore from catastrophic cancellation.
The original question asked for a solution in 1D, which has a simple solution, but in practice, we need the formula in 3D and then the solution is more complicated. See this notebook for a full derivation.
Example implementation of numerically stable calculation in 3D in Python:
import numpy as np
# numerically stable implementation
#np.vectorize
def msq2(px1, py1, pz1, px2, py2, pz2, m1, m2):
p1_sq = px1 ** 2 + py1 ** 2 + pz1 ** 2
p2_sq = px2 ** 2 + py2 ** 2 + pz2 ** 2
m1_sq = m1 ** 2
m2_sq = m2 ** 2
x1 = m1_sq / p1_sq
x2 = m2_sq / p2_sq
x = x1 + x2 + x1 * x2
a = angle(px1, py1, pz1, px2, py2, pz2)
cos_a = np.cos(a)
if cos_a >= 0:
y1 = (x + np.sin(a) ** 2) / (np.sqrt(x + 1) + cos_a)
else:
y1 = -cos_a + np.sqrt(x + 1)
y2 = 2 * np.sqrt(p1_sq * p2_sq)
return m1_sq + m2_sq + y1 * y2
# numerically stable calculation of angle
def angle(x1, y1, z1, x2, y2, z2):
# cross product
cx = y1 * z2 - y2 * z1
cy = x1 * z2 - x2 * z1
cz = x1 * y2 - x2 * y1
# norm of cross product
c = np.sqrt(cx * cx + cy * cy + cz * cz)
# dot product
d = x1 * x2 + y1 * y2 + z1 * z2
return np.arctan2(c, d)
The numerically stable implementation can never produce a negative result, which is a commonly occurring problem with naive implementations, even in double precision.
Let's compare the numerically stable function with a naive implementation.
# naive implementation
def msq1(px1, py1, pz1, px2, py2, pz2, m1, m2):
p1_sq = px1 ** 2 + py1 ** 2 + pz1 ** 2
p2_sq = px2 ** 2 + py2 ** 2 + pz2 ** 2
m1_sq = m1 ** 2
m2_sq = m2 ** 2
# energies of particles 1 and 2
e1 = np.sqrt(p1_sq + m1_sq)
e2 = np.sqrt(p2_sq + m2_sq)
# dangerous cancelation in third term
return m1_sq + m2_sq + 2 * (e1 * e2 - (px1 * px2 + py1 * py2 + pz1 * pz2))
For the following image, the momenta p1 and p2 are randomly picked from 1 to 1e5, the values m1 and m2 are randomly picked from 1e-5 to 1e5. All implementations get the input values in single precision. The reference in both cases is calculated with mpmath using the naive formula with 100 decimal places.
The naive implementation loses all accuracy for some inputs, while the numerically stable implementation does not.
If you put e.g. m1 = 1e-4, m2 = 1e-4, p1 = 1 and p2 = 1 in the expression, you get about 4e-8 with double precision but 0.0 with single precision calculation. I assume, that your question is about how one can get the 4e-8 as well with single precision calculation.
What you can do is a taylor expansion (around m1 = 0 and m2 = 0) of the expression above.
e ~ e|(m1=0,m2=0) + de/dm1|(m1=0,m2=0) * m1 + de/dm2|(m1=0,m2=0) * m2 + ...
If I calculated correctly, the zeroth and first order terms are 0 and the second order expansion would be
e ~ (p1+p2)/p1 * m1**2 + (p1+p2)/p2 * m2**2
This yields exactly 4e-8 even with single precision calculation. You can of course do more terms in the expansion if you need, until you hit the precision limit of a single float.
Edit
If the mi are not always much smaller than the pi you could further massage the equation to get
The complicated part is now the one in the square brackets. It essentially is sqrt(x+1)-1 for a wide range of x values. If x is very small, we can use the taylor expansion of the square root (e.g. like here). If the x value is larger, the formula works just fine, because the addition and subtraction of 1 are no longer discarding the value of x due to floating point precision. So one threshold for x must be choosen below one switches to the taylor expansion.
I'd like to be able to use the vDSP_biquad function as a one pole filter.
My one pole filter looks like this :
output[i] = onePole->z1 = input[i] * onePole->a0 + onePole->z1 * onePole->b1;
where
b1 = exp(-2.0 * M_PI * (_frequency / sampleRate));
a0 = 1.0 - b1;
This one pole works great, but of course it's not optimized, which is why I'd like to use the Accelerate Framework to speed it up.
Because vDSP_biquad uses the Direct Form II of the biquad implementation, it seems to me I should be able to set the coefficients to use it as a one-pole filter. https://en.wikipedia.org/wiki/Digital_biquad_filter#Direct_form_2
filter->omega = 2 * M_PI * freq / sampleRate;
filter->b1 = exp(-filter->omega);
filter->b0 = 1 - filter->b1;
filter->b2 = 0;
filter->a1 = 0;
filter->a2 = 0;
However, this does not work as a one pole filter. (The implementation of biquad is fine, I use it for many other filter types, it's just these coefficients don't have the desired effect).
What am I doing wrong?
Also open to hearing other ways to optimize a one-pole filter with Accelerate or otherwise.
The formula in the Apple docs is:
y[n] = b0*x[n] + b1*x[n-1] + b2*x[n-2] - a1*y[n-1] - a2*y[n-2]
In your above code, you're using b1 which is two inputs ago. For a one-pole, you'll need to use the previous output, y[n-1].
So I think the coefficients you want are:
a1 = -exp(-2.0 * M_PI * (_frequency / sampleRate))
b0 = 1.0 + a1
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 8 years ago.
Improve this question
Having an issue calculating money after percentages have been added and then subtracting , I know that 5353.29 + 18% = 6316.88 which I need to do in my tsql but I also need to do the reverse and take 18% from 6316.88 to get back to 5353.29 all in tsql, I might have just been looking at this too long but I just cant get the figures to calculate properly, any help please?
newVal = 5353.29(1 + .18)
origVal = newVal/(1 + .18)
6316.88 is 118%, so to get back you need to divide 6316.88 by 118, then multiply by 100.
(6316.88/118)*100=5353.29
To Add n% to X
Result = X * ( 1 + (n / 100.0))
To do the reverse
X = Result / ( 1 + (n / 100.0))
Before I begin the problem, I use P0, P1, P2, and P3 for the four cubic Bezier points, and 't' since it's parametric. Also, I have searched for a similar problem in this site, as well as Google, and couldn't find one. I apologize if this is a common question.
The problem: I am getting a slope of 0 for both dx/dt and dy/dt for cubic Beziers in these two cases
1: t = 0 and P0 == P1
2: t = 1 and P2 == P3
Here's an example to illustrate (1), where t = 0 and P0 == P1.
Find the tangent (i.e dx/dt and dy/dt) of the following cubic Bezier at t = 0:
(100, 100) (100, 100) (150, 150) (200, 100)
To find the tangent, we'd want the first derivative of the cubic Bezier:
Cubic Bezier definition
B(t) = (1-t)^3P0 + 3t(1-t)^2P1 + 3t^2(1-t)P2 + t^3P3
First derivative of a bezier curve (if you'd like to see the steps I used to get here, let me know)
B'(t) = (-3P0 + 9P1 - 9P2 + 3P3)t^2 + (6P0 - 12P1 + 6P2)t + (-3P0 + 3P1)
Plugging in t = 0 to the first derivative equation, we get
B'(0) = -3P0 + 3P1
And finally, recall that P0 = P1 = (100, 100), so dx/dt and dy/dt are:
dx/dt = dy/dt = -3*(100) + 3*(100) = 0
This tells me...there is no tangent at t = 0 for this cubic Bezier. Which makes no sense if you were to graph and look at it.
What I'm doing to get a non-zero slope is to:
Treat the points P1, P2, and P3 like a quadratic Bezier, convert them into the equivalent cubic Bezier, THEN find the first derivative at t = 0.
Is there any way I can avoid doing that? I'm finding it difficult to accept a tangent that has 0 for dx/dt and dy/dt.
Thanks for your help.
The derivative B'(t) at t = 0 is indeed undefined for case 1 (and at t = 1 for case 2).
To see why this is the case, we can run the de Casteljau algorithm "backwards" on your example to double the parameter range of the curve from t = 0 ... 1 to t = -1 ... 1. This results in the following cubic Bezier curve control points:
(300,400) (0,-100) (100,200) (200,100)
If you plot this curve, you'll see your original curve from t = 0.5 ... 1. You'll also see that there is a cusp at t = 0.5 on this extended curve, right at the beginning of your original. This cusp is why your curve is not differentiable at its starting point.
However, the tangent of the curve is not quite the same thing as the derivative. So if all you need is the tangent, you're in luck. (The derivative is tangent to the curve, but so is any other vector perpendicular to the curve's normal.)
It turns out that the tangents at the ends of the curve are generally equivalent to:
P1 - P0 at t = 0
P3 - P2 at t = 1
However, if (and only if) P0 = P1 and/or P2 = P3, then the tangent at the degenerate point (that is, at t = 0 if P0 = P1 and/or t = 1 if P2 = P3) is equivalent to:
P2 - P1
You can verify that this is the case by evaluating B'(t) as t->0.
In fact, if you split the extended curve in two at t = 0.5 and then apply the P2 - P1 equation to each side, you'll see that there are two different tangents at the cusp. The tangent for each half of the curve point in the exact opposite directions. This is another illustration of why the derivative is undefined at this point.
One final note: your trick of treating the points P1, P2, and P3 like a quadratic Bezier will also give you a correct tangent. However, this will not give you the correct derivative.
This question is already correctly answered but I thought you'd like to know the underlying mathematics:
You are looking to find the end-slopes of a cubic bezier. Since the curve (i.e. its x and y values) is parametric in t, you will differentiate x and y w.r.t. t separately. The pair that you arrive at may be conceived of as the instantaneous "velocity" of a point travelling along the curve. So the point's initial velocity is zero (or more precisely a null-vector) in this case, but (most probably) the acceleration (or failing that, at least the rate of change of acceleration) will be non-zero, and hence the point's velocity will become non-zero (a non-null vector) and hence it will will move from those coordinates and trace out the curve.
But the slope as you visually perceive it is not parametric i.e. it does not depend on the time. I.O.W. what you are looking for is dy/dx, and not the pair (dx/dt, dy/dt), and given that dx/dt and dy/dt both are zero at t=0 for your curve, dy/dx = (dy/dt)/(dx/dt) = 0/0 which is indeterminate. To evaluate this, one must apply L'Hopital's rule. You can get the detailed treatment of the rule from the Wikipedia article but basically it means that to evaluate such indeterminate limits, we can differentiate the numerator f and denominator g separately to get f' and g' and then limit(f/g) is equal to limit(f'/g'). Now if p0, p1, p2 and p3 are the points defining your cubic, then:
dy / dt = ypart ( 3 * ( p1 - p0 ) + 6 * t * ( p2 - 2 * p1 + p0 ) + 3 * t ** 2 * ( p3 - 3 * p2 + 3 * p1 - p0 ) )
dx / dt = xpart ( 3 * ( p1 - p0 ) + 6 * t * ( p2 - 2 * p1 + p0 ) + 3 * t ** 2 * ( p3 - 3 * p2 + 3 * p1 - p0 ) )
-> (dy/dt) / (dx/dt) = ypart ( p1 - p0 ) / xpart ( p1 - p0 )
But this becomes indeterminate when p0 == p1. Now by L'Hopital's rule,
limit(t->0) [ (dy/dt) / (dx/dt) ] = limit(t->0) [ (d2y/dt2) / (d2x/dt2) ]
Now:
d2y/dt2 = ypart ( 6 * ( p2 - 2 * p1 + p0 ) + 6 * t * ( p3 - 3 * p2 + 3 * p1 - p0 ) )
d2x/dt2 = xpart ( 6 * ( p2 - 2 * p1 + p0 ) + 6 * t * ( p3 - 3 * p2 + 3 * p1 - p0 ) )
and at t = 0, (d2y/dt2) / (d2x/dt2) = ypart ( p2 - 2 * p1 + p0 ) / xpart ( p2 - 2 * p1 + p0 )
But since p1 == p0, this becomes: ypart ( p2 - p0 ) / xpart ( p2 - p0 ), which is exactly the result what Naaff told you. Note that if even p2 == p0 (a really degenerate curve, especially if cubic, in which case it will be just a straight line!), then even this will be indeterminate, and you can once more differentiate the numerator and denominator to get:
limit(dy/dx) = limit(t->0) [ (d3y/dt3) / (d3x/dt3) ] = ypart ( p3 - p0 ) / xpart ( p3 - p0 )
I hope it was useful for you... (BTW it doesn't seem as if TeX-like notation works here unlike on math.stackexchange, else I would provide math markup.)