When two variables are declared as integer type and you perform
14/4, you get 4, but when you use integer division, 14\4, you get 3.
I thought when you use integer division it rounds to the closest even number. So 14\4 = 3.5 (4 is the closest even number) should be 4 instead,
right?
In VB.NET, the / operator is defined to return a floating-point result. It converts the variables to double before performing the division.
This is not the case in the integer division \ where the division is performed without the remainder if the quotient is a decimal (decimals are ignored). For example if the quotient is 3.x, then x is ignored
When you cast a floating point number to an integer in VB.NET, the value is rounded to the nearest even number. Apparently rounding a number when converting it to an integer is a behavior that stretches back to the days of the BASIC language.
However, when performing integer division (with the \ operator), the fractional part is simply discarded, no matter what the fractional part is. This is why you get the behavior that you are seeing.
Related
A very weird issue happens when substracting two numbers in vb.net
It returns a wrongful outcome for a very simple math equation :
I am using visual studio 2019
I don't want to use any math.round to fix the value for such simple equation
Any ideas why that issue happens?
Dim diff As Decimal = 100.1 - 100
MsgBox(diff)
it returns 0.0999999999999943
There is a lot more going on in your math than you assume! You have three different primitive types there:
100.1 is a Double
100 is an Integer
diff is a Decimal
If you put Option Strict On at the top of your code, your code will not compile. This will tell you
Option Strict On disallows implicit conversions from 'Double' to 'Decimal'.
So you must either explicitly cast (this is what your Option Strict Off version is doing implicitly)
Dim diff As Decimal = CDec(100.1 - 100)
but since it's the same thing you are already doing implicitly, you will get that same floating point issue. The better thing to do is to start with a Decimal instead of Double in the first place, such as
Dim diff = 100.1D - 100
By using the Decimal literal D at the end of the floating point number it actually does Decimal math, and the result:
What is the difference between:
Dim a As Integer = CInt(2.2)
and
Dim a As Integer = Math.Round(2.2)
?
CInt returns an integer but will round the .5 to nearest even number so:
2 = CInt(2.5)
4 = CInt(3.5)
Are both true, which might not be what you want.
Math.Round can be told to round away from zero. But returns a double, so we still need to cast it
3 = CInt(Math.Round(2.5, MidpointRounding.AwayFromZero))
There is bigger differences in CInt(), Int() and Round()... and others.
Round has parameters of rounding, so it is flexible and user friendly. But it do not change variable type. No "type conversion".
Meanwhile CInt() is a bit cryptic as it rounds too. And it is doing "Type conversion" to integer.
2 = Int(2.555), 3 = CInt(2.555)
2 = Int(2.5), 2 = CInt(2.5)
Some documentation states:
When the fractional part of expression is exactly .5, CInt always rounds it to the nearest even number. For example, .5 rounds to 0, and 1.5 rounds to 2.
But I do not like that "exact 0.5", in real word it is "0.5000001"
So, doing integer math (like calculating bitmaps address Hi and Lo bytes) do not use CInt(). Use old school INT(). Until you get to negative numbers... see the fix() function.
If there is no need to convert type, use floor().
I think all this chaos of number conversion is for some sort of compatibility with some ancient software.
The difference between those two functions is that they do totally different things:
CInt converts to an Integer type
Math.Round rounds the value to the nearest Integer
Math.Round in this instance will get you 2.0, as specified by the MSDN documentation. You are also using the function incorrectly, see the MSDN link above.
Both will raise an Exception if conversion fails, you can use Try..Catch for this.
Side note: You're new to VB.NET, but you might want to try switching to C#. I find that it is a hybrid of VB.NET & C++ and it will be far easier for you to work with than VB.NET.
Executing the following statement results in Access SQL:
CLNG((CCUR(1.225)/1)*100) = 123
The Conversion Goes, Decimal > Currency > Double > Double > Long
If I remove the CCUR conversion function:
CLNG(((1.225)/1)*100) = 122
The Conversion here goes , Decimal > Double > Double > Long
What is the difference between these two?
This extends to being different between Code And Access SQL
In Access SQL
clng((CCUR(1.015)/1)*100)/100 = 1.01 (Wrong Rounding)
In Access VBA
clng((CCUR(1.015)/1)*100)/100 = 1.02 (Appropriate Rounding Here)
Microsoft explain that the CLng function uses Banker's Rounding, here.
When the fractional part is exactly 0.5, CInt and CLng always round it to the nearest even number. For example, 0.5 rounds to 0, and 1.5 rounds to 2. CInt and CLng differ from the Fix and Int functions, which truncate, rather than round, the fractional part of a number. Also, Fix and Int always return a value of the same type as is passed in.
Looking at a similar question and the subsequent answer HERE, it explains that there are changes to the bit calculation behind the scenes, based on how it is calculated, but I'm not sure how the data type effects it.
What am I missing, and why is it calculating this way? How could I reproduce this behavior predictably in SQL Server?
EDIT
After some digging I believe that this is truly the result of a rounding point issue. In SQL server it will round floats to the nearest whole number if it is outside of the 15 digit max of precision. Access seems to hold more somehow, even though a Double is equivalent to a Float(53) in TSQL.
The difference in results is a combination of two different issues: Jet/ACE vs VBA expression evaluation and binary floating point representation of decimal numbers.
The first is that the Jet/ACE expression engine implicitly converts fractional numbers to Decimal while VBA converts them to Double. This can be easily demonstrated (note the Eval() function evaluates an expression using the Jet/ACE db engine):
?Typename(1.015), eval("typename(1.015)")
Double Decimal
The second issue is that of floating point arithmetic. This is somewhat more difficult to demonstrate because VBA always rounds its output, but the issue is more obvious using another language (Python, in this case):
>>> from decimal import Decimal
>>> Decimal(1.015)
Decimal('1.0149999999999999023003738329862244427204132080078125')
The Double type in VBA uses floating-point arithmetic, while the Decimal type uses integer arithmetic (it stores the position of the decimal point behind the scenes).
The upshot to this is that Banker's rounding or traditional rounding is a red herring. The determining factor is whether the binary floating point representation of the number is slightly greater or less than its decimal representation.
To see how this works in your original question see the following VBA:
?Eval("typename((CCUR(1.225)/1))"), Eval("typename(((1.225)/1))")
Double Decimal
?Eval("typename(CCUR(1.225))"), Eval("typename(1.225)")
Currency Decimal
And Python:
>>> Decimal(1.225)
Decimal('1.225000000000000088817841970012523233890533447265625')
I should also point out that your assumption of the conversion to Double in your second example is incorrect. The data type remains Decimal until the final conversion to Long. The difference between the first two functions is that multiplying a Decimal by a Currency type in Jet/ACE results in a Double. This seems like somewhat odd behavior to me, but the code bears it out:
?eval("TypeName(1.225)"), eval("TypeName(1.225)")
Decimal Decimal
?eval("TypeName(CCUR(1.225))"), eval("TypeName((1.225))")
Currency Decimal
?eval("TypeName(CCUR(1.225)/1)"), eval("TypeName((1.225)/1)")
Double Decimal
?eval("TypeName((CCUR(1.225)/1)*100)"), eval("TypeName(((1.225)/1)*100)")
Double Decimal
?eval("TypeName(CLNG((CCUR(1.225)/1)*100))"), eval("TypeName(CLNG(((1.225)/1)*100))")
Long Long
So the conversion in the two cases is actually:
Decimal > Currency > Double > Double > Long (as you correctly assumed); and
Decimal > Decimal > Decimal > Decimal > Long (correcting your initial assumption).
To answer your question in the comment below, Eval() uses the same expression engine as Jet/ACE, so it is functionally equivalent to entering the same formula in an Access query. For further proof, I present the following:
SELECT
TypeName(1.225) as A1,
TypeName(CCUR(1.225)) as A2,
TypeName(CCUR(1.225)/1) as A3,
TypeName((CCUR(1.225)/1)*100) as A4,
TypeName(CLNG((CCUR(1.225)/1)*100)) as A5
SELECT
TypeName(1.225) as B1,
TypeName((1.225)) as B2,
TypeName((1.225)/1) as B3,
TypeName(((1.225)/1)*100) as B4,
TypeName(CLNG(((1.225)/1)*100)) as B5
I am trying to get the integer part of a number after dividing two variables.
ie, get 3 if the value is 3.75
displaycount and itemcount are both integer variables.
Dim cntr As Integer
cntr = Math.Floor(Math.Abs(itemCount / displaycount))
That code produces a blue squiggly in VS2012 with the comment that "runtime errors may occur when converting Double to Integer" BUT Math.Floor is supposed to take a decimal or double and return an integer.
"Math.Floor is supposed to take a decimal or double and return an integer." No, it isn't. It returns a value of the same type as its argument. See the documentation, e.g. Math.Floor Method (Double).
I would have expected VS to suggest a fix of adding CInt() around the RHS of the assignment; did that not appear for you?
If you need an Integer as result, consider using either the CInt, Int or the Fix functions.
CInt rounds to the nearest integer using the bankers's rounding (n.5 rounds towards the closest even number).
Int removes the fractional parts. Negative numbers are truncated towards smaller numbers
Int(-8.4) = -9.
Fix removes the fractional parts. Negative numbers are truncated towards greater numbers
Fix(-8.4) = -8.
See Conversion.Int Method and Type Conversion Functions (Visual Basic).
I am storing a value (represented as a string originally) like this - 12345678901234.12345678912 - in a double variable. After storing, it is represented in an exponential format (with an e). How do i convert this exponential representation to the original(string) representation?
Dim s as string = "1234567891234567.123456789"
Dim d as Double
Double.TryParse(s, d)
Console.WriteLine(d) 'Prints 1.23456789123457E+15
Using Decimal solves the problem but why cant Double do it?
Your string contains 25 significant digits. double simply doesn't retain that amount of information. Even decimal can barely hold that much (28/29 digits). From the docs for System.Double:
By default, a Double value contains 15 decimal digits of precision, although a maximum of 17 digits is maintained internally.
You should read my articles on binary floating point and decimal floating point for more information - they come at the topic from a C# point of view, but you're obviously using the same types from VB.
In your particular case, the exact double value closest to 1234567891234567.123456789 is just 1234567891234567 - you're losing all the information after the decimal point.