I just came across this piece of code:
Dim d As Double
For i = 1 To 10
d = d + 0.1
Next
MsgBox(d)
MsgBox(d = 1)
MsgBox(1 - d)
Can anyone explain me the reason for that? Why d is set to 1?
Floating point types and integer types cannot be compared directly, as their binary representations are different.
The result of adding 0.1 ten times as a floating point type may well be a value that is close to 1, but not exactly.
When comparing floating point values, you need to use a minimum value by which the values can differ and still be considered the same value (this value is normally known as the epsilon). This value depends on the application.
I suggest reading What Every Computer Scientist Should Know About Floating-Point Arithmetic for an in-depth discussion.
As for comaring 1 to 1.0 - these are different types so will not compare to each other.
.1 (1/10th) is a repeating fraction when converted to binary:
.0001100110011001100110011001100110011.....
It would be like trying to show 1/3 as a decimal: you just can't do it accurately.
This is because a double is always only an approximation of the value and not the exact value itself (like a floating point value). When you need an exact decimal value, instead use a Decimal.
Contrast with:
Dim d As Decimal
For i = 1 To 10
d = d + 0.1
Next
MsgBox(1)
MsgBox(d = 1)
MsgBox(1 - d)
Related
Vb.net has a decimal data type.
Unlike normal double or floating points, decimal data type can store values like 0.1
Now say I have a variable like precision.
Say precision is 8
So basically I want to do
Protected Overridable Sub setPairsPricesStep2(decimalPrecission As Long, Optional base As String = "", Optional quote As String = "")
If decimalPrecission = 8 Then
Return
End If
Dim price = 10D ^ (-decimalPrecission)
setPairsPriceStep1(price, base, quote)
End Sub
There is a problem there
the result of Dim price = 10D ^ (-decimalPrecission) is double, not decimal. I can convert it to decimal but then I will lost the precission.
So what is the right way to do it? Should I just use for next but that's hardly elegant.
It's simple
I want a function that given precisions give decimal value.
For example, if precision is 1 I got 0.1. If precision is 5, I got 0.00001
I ended up doing this
For i = 1 To decimalPrecission
price *= 0.1D
Next
But surely there is a better way
Update:
Per comment, I tried
Dim e = 10D ^ -5
Dim e1 = 10D ^ -5L
The type of e and e1 are both double.
I suppose I can do Cdec(e). But then it means I have lost accuracy because normal double cannot store .1 correctly.
I want a function that given precisions give decimal value.
For example, if precision is 1 I got 0.1. If precision is 5, I got 0.00001
Since you are working with the Decimal type, the simplest way to get this result is to use the Decimal constructor that allows you to specify the scale factor.
Public Sub New (lo As Integer, mid As Integer, hi As Integer, isNegative As Boolean, scale As Byte)
From the Remarks section of the above referenced documentation,
The binary representation of a Decimal number consists of a 1-bit sign, a 96-bit integer number, and a scaling factor used to divide the integer number and specify what portion of it is a decimal fraction. The scaling factor is implicitly the number 10 raised to an exponent ranging from 0 to 28.
So you can see that if take the value of one divided by 10 to the first power, the result is 0.1. Likewise, one divided by 10 to the fifth power, the result is 0.00001.
The lo, mid, and hi arguments in the constructor could be obtained by uisng the [Decimal.GetBits Method](Decimal.GetBits Method), but for this simple case, I chose to hard code the values for the value of one stored as a decimal.
To obtain a value of 0.1D:
New Decimal(1, 0, 0, False, 1)
To obtain a value of 0.00001D:
New Decimal(1, 0, 0, False, 5)
Dim stringrepresentation = "1E-" + decimalPrecission.ToString
Dim price = Decimal.Parse(stringrepresentation, System.Globalization.NumberStyles.AllowExponent)
This is what I basically did. Basically I created a string 1E-5, for example, and use decimal.parse to get the decimal
I wonder if there is a better way but I have no idea.
Actually Jimy ways may work too but rounded to a number
Problems like this drive me crazy. Here's the relevant piece of code:
Dim RES As New Size(Math.Floor(Math.Round(mPageSize.Width - mMargins.Left - mMargins.Right - mLabelSize.Width, 4) / (mLabelSize.Width + mSpacing.Width) + 1),
Math.Floor((mPageSize.Height - mMargins.Top - mMargins.Bottom - mLabelSize.Height) / (mLabelSize.Height + mSpacing.Height)) + 1)
Values of the variables (all are of Single type):
mPageSize.Width = 8.5
mMargins.Left = 0.18
mMargins.Right = 0.18
mLabelSize.Width = 4.0
mSpacing.Width = 0.14
For God-knows-what reason, RES evaluates to {Width=1,Height=5} instead of {Width=2,Height=5}. I have evaluated the expressions on the right-side individually and as a whole and they correctly evaluate to {2,5}, but RES would never get correct value. Wonder what am I missing here.
EDIT
I have simplified the problem further. The following code will produce 2.0 if you QuickWatch the RHS, but the variable on the LHS will get 1.0 after you execute this line:
Dim X = Math.Floor(Math.Round(mPageSize.Width - mMargins.Left - mMargins.Right - mLabelSize.Width, 4) / (mLabelSize.Width + mSpacing.Width) + 1)
Time for MS to check it out?
EDIT 2
More info. The following gives correct results:
Dim Temp = mPageSize.Width - mMargins.Left - mMargins.Right - mLabelSize.Width
Dim X = Math.Floor(Temp / CDec(mLabelSize.Width + mSpacing.Width)) + 1
The problem is that the following expression evaluates to a value just below 1:
Math.Round(mPageSize.Width - mMargins.Left - mMargins.Right - mLabelSize.Width, 4) / (mLabelSize.Width + mSpacing.Width)
= 0.99999999985602739 (Double)
But what's the reason for that? The truth is that I don't know exactly. The MSDN does not offer enough information about the implementation of / but here's my guess:
Math.Round returns a Double with value 4.14. The right-hand side of the division is a Single. So you're dividing a Double by a Single. This results in a Double (see MSDN). So far, so good. The MSDN states that all integral data types are widened to Double before the division. Although Single is not an integral data type, this is probably what happens. And here is the problem. The widening does not seem to be performed on the result of the addition, but on its operands.
If you write
Dim sum = (mLabelSize.Width + mSpacing.Width) 'will be 4.14 Single
Math.Round(mPageSize.Width - mMargins.Left - mMargins.Right - mLabelSize.Width, 4) / sum
= 1 (Double)
Here sum is converted to double (resulting in 4.14) and everything is fine. But, if we convert both operands to double, then the conversion of 0.14 introduces some floating point error:
Dim dblLabelSizeWidth As Double = mLabelSize.Width ' will be 4.0
Dim dblSpacing As Double = mSpacing.Width ' will be 0.14000000059604645
The sum is slightly bigger than 4.14, resulting in a quotient slightly smaller than 1.
So the reason is that the conversion to double is not performed on the division's operand, but on the operand's operands, which introduces floating point errors.
You could overcome this problem by adding a small epsilon to the quotient before rounding off. Alternatively you might consider using a more precise data type such as Decimal. But at some point, there will also be floating-point errors with Decimal.
This is due to rounding error: you're taking the floor of a value that is very close to 2, but is less than 2 (while the mathematical value is 2). You should do all your computations with integers, or take rounding errors into account before using operations like floor (not always possible if you want the true value).
EDIT: Since vb.net has a Decimal datatype, you can also use it instead of integers. It may help in some cases like here: the base conversions for 0.18 and 0.14 (not representable exactly in binary) are avoided and the additions and subtractions will be performed exactly here, so that the operands of the division will be computed exactly. Thus, if the result of the division is an integer, you'll get it exactly (instead of possibly a value just below, like what you got with binary). But make sure that your inputs are already in decimal.
I am currently trying to figure out how to multiply two numbers in fixed point representation.
Say my number representation is as follows:
[SIGN][2^0].[2^-1][2^-2]..[2^-14]
In my case, the number 10.01000000000000 = -0.25.
How would I for example do 0.25x0.25 or -0.25x0.25 etc?
Hope you can help!
You should use 2's complement representation instead of a seperate sign bit. It's much easier to do maths on that, no special handling is required. The range is also improved because there's no wasted bit pattern for negative 0. To multiply, just do as normal fixed-point multiplication. The normal Q2.14 format will store value x/214 for the bit pattern of x, therefore if we have A and B then
So you just need to multiply A and B directly then divide the product by 214 to get the result back into the form x/214 like this
AxB = ((int32_t)A*B) >> 14;
A rounding step is needed to get the nearest value. You can find the way to do it in Q number format#Math operations. The simplest way to round to nearest is just add back the bit that was last shifted out (i.e. the first fractional bit) like this
AxB = (int32_t)A*B;
AxB = (AxB >> 14) + ((AxB >> 13) & 1);
You might also want to read these
Fixed-point arithmetic.
Emulated Fixed Point Division/Multiplication
Fixed point math in c#?
With 2 bits you can represent the integer range of [-2, 1]. So using Q2.14 format, -0.25 would be stored as 11.11000000000000. Using 1 sign bit you can only represent -1, 0, 1, and it makes calculations more complex because you need to split the sign bit then combine it back at the end.
Multiply into a larger sized variable, and then right shift by the number of bits of fixed point precision.
Here's a simple example in C:
int a = 0.25 * (1 << 16);
int b = -0.25 * (1 << 16);
int c = (a * b) >> 16;
printf("%.2f * %.2f = %.2f\n", a / 65536.0, b / 65536.0 , c / 65536.0);
You basically multiply everything by a constant to bring the fractional parts up into the integer range, then multiply the two factors, then (optionally) divide by one of the constants to return the product to the standard range for use in future calculations. It's like multiplying prices expressed in fractional dollars by 100 and then working in cents (i.e. $1.95 * 100 cents/dollar = 195 cents).
Be careful not to overflow the range of the variable you are multiplying into. Your constant might need to be smaller to avoid overflow, like using 1 << 8 instead of 1 << 16 in the example above.
Why vb prints out 1??? when d is a double aproximation to 1? shoudnt be 0.99999 or something similar? what if I really need the float value? and how could I print it?
Dim d As Double
For i = 1 To 10
d = d + 0.1
Next
MsgBox(d)
Console.WriteLine(d)
thanks
When using MsgBox or Console.WriteLine, double.ToString() is called in order to convert the double to a string.
By default this uses the G format specifier.
The general ("G") format specifier converts a number to the most compact of either fixed-point or scientific notation, depending on the type of the number and whether a precision specifier is present. The precision specifier defines the maximum number of significant digits that can appear in the result string. If the precision specifier is omitted or zero, the type of the number determines the default precision, as indicated in the following table.
And:
However, if the number is a Decimal and the precision specifier is omitted, fixed-point notation is always used and trailing zeros are preserved.
When converting the infinite 0.9999999.... to a string, since it goes forever, rounding occurs, this results in 1.
A simple test is to run this:
MsgBox((0.9999999999999999999999999).ToString())
There's no text in the documentation about what that means, but it sounds very important to understand in order to not run into trouble. Does someone know what that is all about the "significant digits" of a number?
Although the other answer on this question links to a correct explanation of the concept of significant digits in general, NSNumberFormatter's {uses|minimum|maximum}SignificantDigits properties have nothing to do with precision of calculations.
The significant digits are the group of digits in a number from the first nonzero digit to the last nonzero digit, inclusive, usually unless trailing zeroes are fractional. Restricting output to a specific number of significant digits is useful if a relative (percentage) error is known or desired.
First of all, the minimumSignificantDigits and maximumSignificantDigits have no effect unless usesSignificantDigits is set to YES. If this is the case, their effect is probably most easily explained using examples.
Let's take the numbers a = 123.4567, b = 1.23, and c = 0.00123:
Assuming minimumSignificantDigits = 0, 1 or 2:
If maximumSignificantDigits = 3, then a will be formatted as "123", b as "1.23", and c as "0.00123".
If maximumSignificantDigits = 4, then a will be formatted as "123.5", b as "1.23" and c as "0.00123".
If maximumSignificantDigits = 2, then a will be formatted as "120", b as "1.2" and c as "0.0012".
Assuming minimumSignificantDigits = 4:
If maximumSignificantDigits = 4, then a will be formatted as "123.5", b as "1.230", and c as "0.001230".
Note: The 4 → 5 conversions occur due to the round-to-nearest mode, as the digit following the 4 in a is 5.
See here for a nice tutorial on significant digits. Very simple explanation would be: the number of digits that are used for calculations within your app.