I am getting references in a paper on genetic programming, to a "protected division" operation. When I google this, i get mostly papers on genetic programming and various results related to cryptography, but none that explain what it actually is. Does anybody know?
Protected division (often notated with %) checks to see if its second
argument is 0. If so, % typically returns the value 1 (regardless of
the value of the first argument).
http://en.wikipedia.org/wiki/Genetic_programming
In cryptography it doesn't seem to be well-defined, but the top google hit is for protecting against side channel attacks (in that case, via power use - you can guess what numbers are being used in the division by looking at the power consumption of the hardware doing the encryption) http://dl.acm.org/citation.cfm?id=1250996 http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.9.7298&rep=rep1&type=pdf
In GP protected division is a modified division operator which does not signal "division by zero" error when denominator is 0 (zero). It typically returns 1 when denominator is zero.
It divides on threshold function of argument instead of argument.
Thres(x) = epsilon*Theta(x) if fabs(x)<epsilon.
Where Theta() is non-zero variant of theta-function.
Other threshold functions possible. Or sometimes it is just 'epsilon'.
When evolving programs with Genetic Programming (GP), every generated program is tested to get its fitness value.
The protected division is required when the evolved programs are mathematical expressions. In cryptography, the mathematical expression might be used to model a decision-making process.
In the evaluation step, the program may perform a division by zero, which would cause a crash. To avoid this, the protected division is set to return a specific value if the denominator equals zero. I've seen three settings:
If the denominator equals zero, return 1
If the denominator equals zero, return 0
If the denominator equals zero, return the numerator
The setting should be specified somewhere in the paper.
If not, the safest bet is to assume that the protected division returns the numerator.
Given that 1 is a multiplicative neutral and 0 is an additive neutral, they could cause some bias in the programs generated during the evolution but are still commonly used.
Related
While reading the Pony Tutorial on Operations, I noticed that some infix operators have partial and unsafe versions. Coming from C#, Java, Python, and JS/TS, I haven't the faintest clue what those do.
In C# there are checked and unchecked contexts for arithmetic. In a checked block, math that would result in a overflow throws an exception. Are the unsafe operators related to that?
Can someone please explain unsafe and partial operators?
The regular operators, such as add/+, mod/%%, etc. will always return the most sensible result. This leads to certain results, such as division by zero being equal to 0. This is because these functions are mathematically considered non-partial, meaning that for every input, there is a defined output; even if the output may be unusual, like an addition that overflows having its result being smaller than the inputs.
However, there are certain situations where having a clearly defined result for every input is not what the programmer wants. That's where unsafe and partial operators come in.
Since these functions must return defined results (see the division by zero example above), they have to run both extra and expensive instructions for that guarantee. The unsafe version of the operators removes these guarantees, and uses a quicker CPU instruction that may give unexpected results for certain inputs. This is useful when you know that your inputs cannot reach certain conditions (eg. overflow, division by zero), and you want your code to squeeze out some extra performance. From the documented definition of the add_unsafe/+~ and mod_unsafe/%%~ operators in trait Integer¹, for example:
fun add_unsafe(y: A): A =>
"""
Unsafe operation.
If the operation overflows, the result is undefined.
"""
fun mod_unsafe(y: A): A
"""
Calculates the modulo of this number after floored division by `y`.
Unsafe operation.
If y is 0, the result is undefined.
If the operation overflows, the result is undefined.
"""
Alternatively, you may want to detect these conditions that would return mathematically-wrong results in your code at runtime, meaning that the functions do not return a value for every set of inputs, and are therefore partial. These will raise errors that you can handle as usual. Also reading the documentation of the add_partial/+? and mod_partial/%%? operators in trait Integer¹, we find:
fun add_partial(y: A): A ?
"""
Add y to this number.
If the operation overflows this function errors.
"""
fun mod_partial(y: A): A ?
"""
Calculates the modulo of this number and `y` after floored division (`fld`).
The result has the sign of the divisor.
If y is `0` or the operation overflows, this function errors.
"""
¹ This trait is implemented by all integer types in Pony, both signed and unsigned.
I am trying to find out the parameters for the function below:
$$
\log L(\alpha,\beta,v) = v/\beta(e^{-\beta T} -1) + \alpha/\beta \sum_{i=1}^{n}(e^{-\beta(T-t_i)} -1) + \sum_{i=1}^{N}log(v e^{-\beta t_i} + \alpha \sum_{j=1}^{jmax(t_i)} e^{-\beta(t_i - t_j)}).
$$
However, the conventional methods like fmin, fminsearch are not converging properly. Any suggestions on any other methods or open libraries which I can use?
I was trying CVXPY, but they don't support the division by a variable in the expression.
The problem may not be convex (I have not verified this but it could be why CVXPY refused it). We don't have the data so we cannot try things out, but I can give some general advice:
Provide exact gradients (and 2nd derivatives if needed) or use a modeling system with automatic differentiation. Especially first derivatives should be preferably quite precise. With finite differences you may lose half the precision.
Provide a good starting point. May be using an alternative estimation method.
Some solvers can use bounds on the variables to restrict the feasible region where functions will be evaluated. This can be used to restrict the search to interesting areas only and also to protect operations like division and log functions.
According to the documentation, the decimal.Round method uses a round-to-even algorithm which is not common for most applications. So I always end up writing a custom function to do the more natural round-half-up algorithm:
public static decimal RoundHalfUp(this decimal d, int decimals)
{
if (decimals < 0)
{
throw new ArgumentException("The decimals must be non-negative",
"decimals");
}
decimal multiplier = (decimal)Math.Pow(10, decimals);
decimal number = d * multiplier;
if (decimal.Truncate(number) < number)
{
number += 0.5m;
}
return decimal.Round(number) / multiplier;
}
Does anybody know the reason behind this framework design decision?
Is there any built-in implementation of the round-half-up algorithm into the framework? Or maybe some unmanaged Windows API?
It could be misleading for beginners that simply write decimal.Round(2.5m, 0) expecting 3 as a result but getting 2 instead.
The other answers with reasons why the Banker's algorithm (aka round half to even) is a good choice are quite correct. It does not suffer from negative or positive bias as much as the round half away from zero method over most reasonable distributions.
But the question was why .NET use Banker's actual rounding as default - and the answer is that Microsoft has followed the IEEE 754 standard. This is also mentioned in MSDN for Math.Round under Remarks.
Also note that .NET supports the alternative method specified by IEEE by providing the MidpointRounding enumeration. They could of course have provided more alternatives to solving ties, but they choose to just fulfill the IEEE standard.
Probably because it's a better algorithm. Over the course of many roundings performed, you will average out that all .5's end up rounding equally up and down. This gives better estimations of actual results if you are for instance, adding a bunch of rounded numbers. I would say that even though it isn't what some may expect, it's probably the more correct thing to do.
While I cannot answer the question of "Why did Microsoft's designers choose this as the default?", I just want to point out that an extra function is unnecessary.
Math.Round allows you to specify a MidpointRounding:
ToEven - When a number is halfway between two others, it is rounded toward the nearest even number.
AwayFromZero - When a number is halfway between two others, it is rounded toward the nearest number that is away from zero.
Decimals are mostly used for money; banker’s rounding is common when working with money. Or you could say.
It is mostly bankers that need the
decimal type; therefore it does
“banker’s rounding”
Bankers rounding have the advantage that on average you will get the same result if you:
round a set of “invoice lines” before adding them up,
or add them up then round the total
Rounding before adding up saved a lot of work in the days before computers.
(In the UK when we went decimal banks would not deal with half pence, but for many years there was still a half pence coin and shop often had prices ending in half pence – so lots of rounding)
Use another overload of Round function like this:
decimal.Round(2.5m, 0,MidpointRounding.AwayFromZero)
It will output 3. And if you use
decimal.Round(2.5m, 0,MidpointRounding.ToEven)
you will get banker's rounding.
Perhaps this is the wrong sort of question to ask here but I am curious. I know that many languages will simply explode and fail when asked to divide by 0, but are there any programming languages that can intelligently handle this impossible sum - and if so, what do they do? Do they keep processing, treating 350/0 as 350, or stop execution, or what?
The little-known Java programming language gives the special constant Double.POSITIVE_INFINITY or Double.NEGATIVE_INFINITY (depending on the numerator) when you divide by zero in an IEEE floating-point context. Integer division by zero is undefined, and results in an ArithmeticException being thrown, which is quite different from your scenario of "explosion and failure".
The INTERCAL standard library returns #0 on divide by zero
From Wikipedia:
The infinities of the extended real number line can be represented in IEEE floating point datatypes, just like ordinary floating point values like 1, 1.5 etc. They are not error values in any way, though they are often (but not always, as it depends on the rounding) used as replacement values when there is an overflow. Upon a divide by zero exception, a positive or negative infinity is returned as an exact result.
In Java, division by zero in a floating-point context produces the special value Double.POSITIVE_INFINITY or Double.NEGATIVE_INFINITY.
i'd be surprised if any language returns 350 if you do 350/0. Just two examples, but Java throws an Exception that can be caught. C/C++ just crashes (i think it throws a Signal that can probably be caught).
In Delphi, it either throw a compile-time error (if divided by a 0 value const) or a catchable runtime error if it happens at runtime.
It's the same for C and C++.
In PHP you will get a warning:
Warning: Division by zero in
<file.php> on line X
So, in PHP, for something like:
$i = 123 / 0;
$i will be set to nothing. BUT $i is not === NULL and isset($i) returns true and is_string($i) returns false.
Python (at least version 2, I don't have 3) throws a ZeroDivisionError, which can be caught.
num = 42
try:
for divisor in (1,0):
ans = num / divisor
print ans
except ZeroDivisionError:
print "Trying to divide by 0!"
prints out:
42
Trying to divide by 0!
Most SQL implementations raise a "division by zero" error, but MySQL just returns NULL
Floating point numbers as per the IEEE define constants NaN etc. Any continued operation involving thst value will remain unchanged until the end. Integer or whole numbers are different with exceptions being thrown...In java...
In pony division by 0 is 0 but i have yet to find a language where 0/0 is 1
I'm working with polyhedra and trying to choose a language that likes inf.
The total edges for a polyhedron {a,b} where a is edges per polygon and b is edges per corner is
E = 1/(1/a + 1/b - 1/2)
if E is negative it's a negative curvature, but if E is infinity (1/0) it tiles the plane. Examples: {3,6} {4,4}
What is the best method for comparing IEEE floats and doubles for equality? I have heard of several methods, but I wanted to see what the community thought.
The best approach I think is to compare ULPs.
bool is_nan(float f)
{
return (*reinterpret_cast<unsigned __int32*>(&f) & 0x7f800000) == 0x7f800000 && (*reinterpret_cast<unsigned __int32*>(&f) & 0x007fffff) != 0;
}
bool is_finite(float f)
{
return (*reinterpret_cast<unsigned __int32*>(&f) & 0x7f800000) != 0x7f800000;
}
// if this symbol is defined, NaNs are never equal to anything (as is normal in IEEE floating point)
// if this symbol is not defined, NaNs are hugely different from regular numbers, but might be equal to each other
#define UNEQUAL_NANS 1
// if this symbol is defined, infinites are never equal to finite numbers (as they're unimaginably greater)
// if this symbol is not defined, infinities are 1 ULP away from +/- FLT_MAX
#define INFINITE_INFINITIES 1
// test whether two IEEE floats are within a specified number of representable values of each other
// This depends on the fact that IEEE floats are properly ordered when treated as signed magnitude integers
bool equal_float(float lhs, float rhs, unsigned __int32 max_ulp_difference)
{
#ifdef UNEQUAL_NANS
if(is_nan(lhs) || is_nan(rhs))
{
return false;
}
#endif
#ifdef INFINITE_INFINITIES
if((is_finite(lhs) && !is_finite(rhs)) || (!is_finite(lhs) && is_finite(rhs)))
{
return false;
}
#endif
signed __int32 left(*reinterpret_cast<signed __int32*>(&lhs));
// transform signed magnitude ints into 2s complement signed ints
if(left < 0)
{
left = 0x80000000 - left;
}
signed __int32 right(*reinterpret_cast<signed __int32*>(&rhs));
// transform signed magnitude ints into 2s complement signed ints
if(right < 0)
{
right = 0x80000000 - right;
}
if(static_cast<unsigned __int32>(std::abs(left - right)) <= max_ulp_difference)
{
return true;
}
return false;
}
A similar technique can be used for doubles. The trick is to convert the floats so that they're ordered (as if integers) and then just see how different they are.
I have no idea why this damn thing is screwing up my underscores. Edit: Oh, perhaps that is just an artefact of the preview. That's OK then.
The current version I am using is this
bool is_equals(float A, float B,
float maxRelativeError, float maxAbsoluteError)
{
if (fabs(A - B) < maxAbsoluteError)
return true;
float relativeError;
if (fabs(B) > fabs(A))
relativeError = fabs((A - B) / B);
else
relativeError = fabs((A - B) / A);
if (relativeError <= maxRelativeError)
return true;
return false;
}
This seems to take care of most problems by combining relative and absolute error tolerance. Is the ULP approach better? If so, why?
#DrPizza: I am no performance guru but I would expect fixed point operations to be quicker than floating point operations (in most cases).
It rather depends on what you are doing with them. A fixed-point type with the same range as an IEEE float would be many many times slower (and many times larger).
Things suitable for floats:
3D graphics, physics/engineering, simulation, climate simulation....
In numerical software you often want to test whether two floating point numbers are exactly equal. LAPACK is full of examples for such cases. Sure, the most common case is where you want to test whether a floating point number equals "Zero", "One", "Two", "Half". If anyone is interested I can pick some algorithms and go more into detail.
Also in BLAS you often want to check whether a floating point number is exactly Zero or One. For example, the routine dgemv can compute operations of the form
y = beta*y + alpha*A*x
y = beta*y + alpha*A^T*x
y = beta*y + alpha*A^H*x
So if beta equals One you have an "plus assignment" and for beta equals Zero a "simple assignment". So you certainly can cut the computational cost if you give these (common) cases a special treatment.
Sure, you could design the BLAS routines in such a way that you can avoid exact comparisons (e.g. using some flags). However, the LAPACK is full of examples where it is not possible.
P.S.:
There are certainly many cases where you don't want check for "is exactly equal". For many people this even might be the only case they ever have to deal with. All I want to point out is that there are other cases too.
Although LAPACK is written in Fortran the logic is the same if you are using other programming languages for numerical software.
Oh dear lord please don't interpret the float bits as ints unless you're running on a P6 or earlier.
Even if it causes it to copy from vector registers to integer registers via memory, and even if it stalls the pipeline, it's the best way to do it that I've come across, insofar as it provides the most robust comparisons even in the face of floating point errors.
i.e. it is a price worth paying.
This seems to take care of most problems by combining relative and absolute error tolerance. Is the ULP approach better? If so, why?
ULPs are a direct measure of the "distance" between two floating point numbers. This means that they don't require you to conjure up the relative and absolute error values, nor do you have to make sure to get those values "about right". With ULPs, you can express directly how close you want the numbers to be, and the same threshold works just as well for small values as for large ones.
If you have floating point errors you have even more problems than this. Although I guess that is up to personal perspective.
Even if we do the numeric analysis to minimize accumulation of error, we can't eliminate it and we can be left with results that ought to be identical (if we were calculating with reals) but differ (because we cannot calculate with reals).
If you are looking for two floats to be equal, then they should be identically equal in my opinion. If you are facing a floating point rounding problem, perhaps a fixed point representation would suit your problem better.
If you are looking for two floats to be equal, then they should be identically equal in my opinion. If you are facing a floating point rounding problem, perhaps a fixed point representation would suit your problem better.
Perhaps we cannot afford the loss of range or performance that such an approach would inflict.
#DrPizza: I am no performance guru but I would expect fixed point operations to be quicker than floating point operations (in most cases).
#Craig H: Sure. I'm totally okay with it printing that. If a or b store money then they should be represented in fixed point. I'm struggling to think of a real world example where such logic ought to be allied to floats. Things suitable for floats:
weights
ranks
distances
real world values (like from a ADC)
For all these things, either you much then numbers and simply present the results to the user for human interpretation, or you make a comparative statement (even if such a statement is, "this thing is within 0.001 of this other thing"). A comparative statement like mine is only useful in the context of the algorithm: the "within 0.001" part depends on what physical question you're asking. That my 0.02. Or should I say 2/100ths?
It rather depends on what you are
doing with them. A fixed-point type
with the same range as an IEEE float
would be many many times slower (and
many times larger).
Okay, but if I want a infinitesimally small bit-resolution then it's back to my original point: == and != have no meaning in the context of such a problem.
An int lets me express ~10^9 values (regardless of the range) which seems like enough for any situation where I would care about two of them being equal. And if that's not enough, use a 64-bit OS and you've got about 10^19 distinct values.
I can express values a range of 0 to 10^200 (for example) in an int, it is just the bit-resolution that suffers (resolution would be greater than 1, but, again, no application has that sort of range as well as that sort of resolution).
To summarize, I think in all cases one either is representing a continuum of values, in which case != and == are irrelevant, or one is representing a fixed set of values, which can be mapped to an int (or a another fixed-precision type).
An int lets me express ~10^9 values
(regardless of the range) which seems
like enough for any situation where I
would care about two of them being
equal. And if that's not enough, use a
64-bit OS and you've got about 10^19
distinct values.
I have actually hit that limit... I was trying to juggle times in ps and time in clock cycles in a simulation where you easily hit 10^10 cycles. No matter what I did I very quickly overflowed the puny range of 64-bit integers... 10^19 is not as much as you think it is, gimme 128 bits computing now!
Floats allowed me to get a solution to the mathematical issues, as the values overflowed with lots zeros at the low end. So you basically had a decimal point floating aronud in the number with no loss of precision (I could like with the more limited distinct number of values allowed in the mantissa of a float compared to a 64-bit int, but desperately needed th range!).
And then things converted back to integers to compare etc.
Annoying, and in the end I scrapped the entire attempt and just relied on floats and < and > to get the work done. Not perfect, but works for the use case envisioned.
If you are looking for two floats to be equal, then they should be identically equal in my opinion. If you are facing a floating point rounding problem, perhaps a fixed point representation would suit your problem better.
Perhaps I should explain the problem better. In C++, the following code:
#include <iostream>
using namespace std;
int main()
{
float a = 1.0;
float b = 0.0;
for(int i=0;i<10;++i)
{
b+=0.1;
}
if(a != b)
{
cout << "Something is wrong" << endl;
}
return 1;
}
prints the phrase "Something is wrong". Are you saying that it should?
Oh dear lord please don't interpret the float bits as ints unless you're running on a P6 or earlier.
it's the best way to do it that I've come across, insofar as it provides the most robust comparisons even in the face of floating point errors.
If you have floating point errors you have even more problems than this. Although I guess that is up to personal perspective.