When I enter
Debug.Print(Round(0.005, 2))
0
in the Immediate Window in VBA (Excel 2010) I get 0 as a result. I would have expected 0.01.
I cannot imagine VBA calculating wrong so what am I doing wrong?
a bankers rounding vs arithmetic rounding thing
= Round(variable + 0.000001, 0)
Related
I got two different result in VB 6 and VB.NET for a same code, which handles division operation.
ABC = 9.999728
result = IIf(ABC <> 0, 1 / ABC, 10 ^ 10)
In VB6, I got result = 0.1000027
In VB.NET, the result is 0.100002721
However, when I use CSng(Val(CStr(result))) in VB.NET, I get 0.1000027.
1) Why VB.NET produces different precision compared to VB6?
2) Why CSng(Val(CStr(result))) produces same precision as VB6?
3) This may look trivial and simple, but this problem is repeated and propagates to my final result, which is different from the equivalent result in VB6. Can I safely assume that VB.NET result is more precise than VB6 and discard the VB6 result completely?
In VB6:
Dim ABC, result
ABC = 9.999728
result = IIf(ABC <> 0, 1 / ABC, 10 ^ 10)
MsgBox result
Displays:
0.100002720073986
Randomly using CSng() will of course truncate precision, and the obsolete Val() function should be avoided like the plague. If you want a Double then use CDbl() instead.
I using a simple formula in rdlc report. I want to use the forward rounding for example i have value 25.17 and i want to convert into 26 not 25.
But following formula giving me the result = 25.
=ROUND(Sum(Fields!Total.Value, "DataSet1") + First(Fields!Shipping.Value, "DataSet1") - First(Fields!Discount.Value, "DataSet1"),0)
Thanks in advance
In your Expression, use Ceiling function instead of Round.
=Ceiling(Sum(Fields!Total.Value, "DataSet1") + First(Fields!Shipping.Value, "DataSet1") - First(Fields!Discount.Value, "DataSet1"))
Most programming languages have a Round, Floor and Ceiling function. Floor rounds down, ceiling up and round to the nearest.
Further reading for the ceiling function https://msdn.microsoft.com/en-us/library/system.math.ceiling%28v=vs.110%29.aspx
I've read much about rounding in Excel. I found out that VBA's Round() function uses "Bankers rounding" while Application.WorksheetFunction.Round() uses more or less "normal" rounding. But it didn't help me to understand this:
? Round(6.03499,2)
6.03
Why? I want to see 6.04, not 6.03! The trick is that
? Round(Round(6.03499,3),2)
6.04
I thought a bit and developed a subroutine like this:
Option Explicit
Function DoRound(ByVal value As Double, Optional ByVal numdigits As Integer = 0) As Double
Dim i As Integer
Dim res As Double
res = value
For i = 10 To numdigits Step -1
res = Application.Round(res, i)
Next i
DoRound = res
End Function
It works fine.
? DoRound(6.03499,2)
6.04
But it is not cool. Is there any built-in normal rounding in Excel?
If you round 6.03499 to 3 digits it will be 6.035 - which is correct.
If you round 6.03499 to 2 digits it will be 6.03 - which is correct
However - the example where you first round to 3 digits, then to 2 is also correct, by the following statement:
Round(6.03499, 3) gives 6.035
Round(6.035, 2) gives 6.04
If you want Round(6.03499, 2) to give 6.04 you have to use Application.WorksheetFunction.RoundUp
Rounding 6.0349 to two decimals is just not 6.04 hence, no, there is no such function.
Round up will round anything up. Hence, 6.0000000001 will also become 7 if you round to 0 decimals.
I have a problem when i'm trying to calculate in a view a formula whose result is smaller than 1.
e.g. I have the next formula: Arenda*TotalArea/10000 as TotalArenda
If I have Arenda=10 and TotalArea=10 I get TotalArenda=0,00 when normally should be 0.01
Thanks
Make Arenda = 10.0 and TotalArea = 10.0 instead of 10 and 10. This will force SQL not to use integer math and you will get your needed accuracy.
In fact, the only way I can get 0.0 as the result is if the Arenda is 10 (integer) while at least one of TotalArea or 10000 contain a decimal and a trailing 0, and only if I override order of operations by grouping using parentheses such as
select 10.0* (10/10000) as blah
If all are integers you get 0. If all contain decimals you get 0.01. If I remove the parentheses, I get 0.01 if ANY of them are non-integer types.
If precision is highly important I would recommend you cast to decimals and not floats:
select CONVERT(decimal(10,2), Arenda) * CONVERT(decimal(10,2), TotalArea) / 10000.0
You are using colunns, so changing the type may not be feasible. SQL Server does integer division on integers (other databases behave differently). Try one of these:
cast(Arenda as float)*cast(TotalArea as float)/10000
or:
Arenda*TotalArea/10000.0
I am doing a calculation of values using Microsoft Excel vba.
I have a portion of codes like the below:
INT(151.2 * 100)
I notice that the result from this calculation is 15119 but the correct result should be 15220.
It is ok if i remove the INT()
151.2 * 100
The result returned will be 15220.
Can anyone advise why is there such a difference and whether it is correct to simply remove the INT() to achieve the correct result?
Floating-point arithmetic.
a = (151.2 * 100)
Print Int(a)
15119
Print a
15120
Print a = 15120
False
There is no double-precision number to represent the result of 151.2 * 100 exactly as 15120. The closest one is apparently just under 15120. When you Int it, it gets truncated, i.e. rounded down to 15119.
Instead of truncating, you could round:
Print CInt(a)
15120
Print Round(a)
15120
Note that if you have a variable i of type Integer and you just say i = 151.2 * 100 as you suggest, then you are implicitly coercing the result to an integer i.e. implicitly saying i = CInt(151.2 * 100). I think it's better practice to be explicit.