I have trouble comparing 2 double in Excel VBA
suppose that I have the following code
Dim a as double
Dim b as double
a = 0.15
b = 0.01
After a few manipulations on b, b is now equal to 0.6
however the imprecision related to the double data type gives me headache because
if a = b then
//this will never trigger
end if
Do you know how I can remove the trailing imprecision on the double type?
You can't compare floating point values for equality. See this article on "Comparing floating point numbers" for a discussion of how to handle the intrinsic error.
It isn't as simple as comparing to a constant error margin unless you know for sure what the absolute range of the floats is beforehand.
if you are going to do this....
Dim a as double
Dim b as double
a = 0.15
b = 0.01
you need to add the round function in your IF statement like this...
If Round(a,2) = Round(b,2) Then
//code inside block will now trigger.
End If
See also here for additional Microsoft reference.
It is never wise to compare doubles on equality.
Some decimal values map to several floating point representations. So one 0.6 is not always equal to the other 0.6.
If we subtract one from the other, we probably get something like 0.00000000051.
We can now define equality as having a difference smaller that a certain error margin.
Here is a simple function I wrote:
Function dblCheckTheSame(number1 As Double, number2 As Double, Optional Digits As Integer = 12) As Boolean
If (number1 - number2) ^ 2 < (10 ^ -Digits) ^ 2 Then
dblCheckTheSame = True
Else
dblCheckTheSame = False
End If
End Function
Call it with:
MsgBox dblCheckTheSame(1.2345, 1.23456789)
MsgBox dblCheckTheSame(1.2345, 1.23456789, 4)
MsgBox dblCheckTheSame(1.2345678900001, 1.2345678900002)
MsgBox dblCheckTheSame(1.2345678900001, 1.2345678900002, 14)
As has been pointed out, many decimal numbers cannot be represented precisely as traditional floating-point types. Depending on the nature of your problem space, you may be better off using the Decimal VBA type which can represent decimal numbers (base 10) with perfect precision up to a certain decimal point. This is often done for representing money for example where 2-digit decimal precision is often desired.
Dim a as Decimal
Dim b as Decimal
a = 0.15
b = 0.01
Late answer but I'm surprised a solution hasn't been posted that addresses the concerns outlined in the article linked in the (currently) accepted answer, namely that:
Rounding checks equality with absolute tolerance (e.g. 0.0001 units if rounded to 4d.p.) which is rubbish when comparing different values on multiple orders of magnitude (so not just comparing to 0)
Relative tolerance that scales with one of the numbers being compared meanwhile is not mentioned in the current answers, but performs well on non-zero comparisons (however will be bad at comparing to zero as the scaling blows up around then).
To solve this, I've taken inspiration from Python: PEP 485 -- A Function for testing approximate equality to implement the following (in a standard module):
Code
'#NoIndent: Don't want to lose our description annotations
'#Folder("Tests.Utils")
Option Explicit
Option Private Module
'Based on Python's math.isclose https://github.com/python/cpython/blob/17f94e28882e1e2b331ace93f42e8615383dee59/Modules/mathmodule.c#L2962-L3003
'math.isclose -> boolean
' a: double
' b: double
' relTol: double = 1e-09
' maximum difference for being considered "close", relative to the
' magnitude of the input values
' absTol: double = 0.0
' maximum difference for being considered "close", regardless of the
' magnitude of the input values
'Determine whether two floating point numbers are close in value.
'Return True if a is close in value to b, and False otherwise.
'For the values to be considered close, the difference between them
'must be smaller than at least one of the tolerances.
'-inf, inf and NaN behave similarly to the IEEE 754 Standard. That
'is, NaN is not close to anything, even itself. inf and -inf are
'only close to themselves.
'#Description("Determine whether two floating point numbers are close in value, accounting for special values in IEEE 754")
Public Function IsClose(ByVal a As Double, ByVal b As Double, _
Optional ByVal relTol As Double = 0.000000001, _
Optional ByVal absTol As Double = 0 _
) As Boolean
If relTol < 0# Or absTol < 0# Then
Err.Raise 5, Description:="tolerances must be non-negative"
ElseIf a = b Then
'Short circuit exact equality -- needed to catch two infinities of
' the same sign. And perhaps speeds things up a bit sometimes.
IsClose = True
ElseIf IsInfinity(a) Or IsInfinity(b) Then
'This catches the case of two infinities of opposite sign, or
' one infinity and one finite number. Two infinities of opposite
' sign would otherwise have an infinite relative tolerance.
'Two infinities of the same sign are caught by the equality check
' above.
IsClose = False
Else
'Now do the regular computation on finite arguments. Here an
' infinite tolerance will always result in the function returning True,
' since an infinite difference will be <= to the infinite tolerance.
'This is to supress overflow errors as we deal with infinity.
'NaN has already been filtered out in the equality checks earlier.
On Error Resume Next
Dim diff As Double: diff = Abs(b - a)
If diff <= absTol Then
IsClose = True
ElseIf diff <= CDbl(Abs(relTol * b)) Then
IsClose = True
ElseIf diff <= CDbl(Abs(relTol * a)) Then
IsClose = True
End If
On Error GoTo 0
End If
End Function
'#Description "Checks if Number is IEEE754 +/- inf, won't raise an error"
Public IsInfinity(ByVal Number As Double) As Boolean
On Error Resume Next 'in case of NaN
IsInfinity = Abs(Number) = PosInf
On Error GoTo 0
End Function
'#Description "IEEE754 -inf"
Public Property Get NegInf() As Double
On Error Resume Next
NegInf = -1 / 0
On Error GoTo 0
End Property
'#Description "IEEE754 +inf"
Public Property Get PosInf() As Double
On Error Resume Next
PosInf = 1 / 0
On Error GoTo 0
End Property
'#Description "IEEE754 signaling NaN (sNaN)"
Public Property Get NaN() As Double
On Error Resume Next
NaN = 0 / 0
On Error GoTo 0
End Property
'#Description "IEEE754 quiet NaN (qNaN)"
Public Property Get QNaN() As Double
QNaN = -NaN
End Property
Updated to incorporate great feedback from Cristian Buse
Examples
The IsClose function can be used to check for absolute difference:
assert(IsClose(0, 0.0001233, absTol:= 0.001)) 'same to 3 d.p.?
... or relative difference:
assert(IsClose(1234.5, 1234.6, relTol:= 0.0001)) '0.01% relative difference?
... but generally you specify both and if either tolerance is met then the numbers are considered close. It has special handling of +-infinity which are only close to themselves, and NaN which is close to nothing (see the PEP for full justification, or my Code Review post where I'd love feedback on this code :)
The Currency data type may be a good alternative. It handles relatively large numbers with fixed four digit precision.
Work-a-round??
Not sure if this will answer all scenarios, but I ran into a problem comparing rounded double values in VBA. When I compared to numbers that appeared to be identical after rounding, VBA would trigger false in an if-then compare statement.
My fix was to run two conversions, first double to string, then string to double, and then do the compare.
Simulated Example
I did not record the exact numbers that caused the error mentioned in this post, and the amounts in my example do not trigger the problem currently and are intended to represent the type of issue.
Sub Test_Rounded_Numbers()
Dim Num1 As Double
Dim Num2 As Double
Let Num1 = 123.123456789
Let Num2 = 123.123467891
Let Num1 = Round(Num1, 4) '123.1235
Let Num2 = Round(Num2, 4) '123.1235
If Num1 = Num2 Then
MsgBox "Correct Match, " & Num1 & " does equal " & Num2
Else
MsgBox "Inccorrect Match, " & Num1 & " does not equal " & Num2
End If
'Here it would say that "Inccorrect Match, 123.1235 does not equal 123.1235."
End Sub
Sub Fixed_Double_Value_Type_Compare_Issue()
Dim Num1 As Double
Dim Num2 As Double
Let Num1 = 123.123456789
Let Num2 = 123.123467891
Let Num1 = Round(Num1, 4) '123.1235
Let Num2 = Round(Num2, 4) '123.1235
'Add CDbl(CStr(Double_Value))
'By doing this step the numbers
'would trigger if they matched
'100% of the time
If CDbl(CStr(Num1)) = CDbl(CStr(Num2)) Then
MsgBox "Correct Match"
Else
MsgBox "Inccorrect Match"
End If
'Now it says Here it would say that "Correct Match, 123.1235 does equal 123.1235."
End Sub
Depending on your situation and your data, and if you're happy with the level of precision shown by default, you can try comparing the string conversions of the numbers as a very simple coding solution:
if cstr(a) = cstr(b)
This will include as much precision as would be displayed by default, which is generally sufficient to consider the numbers equal.
This would be inefficient for very large data sets, but for me was useful when reconciling imported data which was identical but was not matching after storing the data in VBA Arrays.
Try to use Single values if possible.
Conversion to Double values generates random errors.
Public Sub Test()
Dim D01 As Double
Dim D02 As Double
Dim S01 As Single
Dim S02 As Single
S01 = 45.678 / 12
S02 = 45.678
D01 = S01
D02 = S02
Debug.Print S01 * 12
Debug.Print S02
Debug.Print D01 * 12
Debug.Print D02
End Sub
45,678
45,678
45,67799949646
45,6780014038086
****I don't necessarily want the code to fix this problem, I would just like if someone would be able to explain to me why this is happening or what I could have done better to solve this problem.****
I have a homework assignment where I need to print out the highest and lowest number of a series.I am currently able to print out the highest and lowest numbers but if all of the numbers entered are positive, then it displays my lowest number as 0, and same if all the entered numbers are negative.
So if someone would be able to explain how to solve this, without necessarily giving away the answer, it would be greatly appreciated!
Here is my code:
Module Module1
'This is going to have the user enter a series of numbers,
'Once the user is finished have them enter '-99' to end the series,
'then it is going to return largest and smallest number
Sub Main()
NumSeries()
End Sub
'This is going to get the series from the users
Sub NumSeries()
Dim largeNum As Integer = 0
Dim smallNum As Integer = 0
Dim userNum As Integer = 0
Dim largeTemp As Integer = 0
Dim smallTemp As Integer = 0
Console.WriteLine("Please enter a series of positive and negative numbers")
Console.WriteLine("Then type '-99' to end the series")
Console.WriteLine()
While (userNum <> -99)
Console.Write("Enter num: ")
userNum = Console.ReadLine()
If (userNum > largeTemp) Then
largeTemp = userNum
largeNum = largeTemp
ElseIf (userNum < smallTemp And userNum <> -99) Then
smallTemp = userNum
smallNum = smallTemp
End If
End While
Console.WriteLine("The largest number is " & largeNum)
Console.WriteLine("The smallest number is " & smallNum)
End Sub
End Module
Two points:
You don't need the variables largeTemp and smallTemp.
(The answer to your question) You should initialize largeNum to a very small number, and smallNum to a very large number. For example, if smallNum is set to zero at the beginning of the program, only numbers smaller than zero will replace it. To find an error like this, you should trace through the program either by hand or with a debugger and see what happens at each step. It would be a good idea to do this now so you'll understand the problem. (Alternatively, as Idle-Mind pointed out, you can initialize largeNum and smallNum to the first item in the list.)
Don't use any sentinel values at all. Simply declare a Boolean variable so you know if you the value retrieved is the very first one or not:
Dim FirstEntry As Boolean = True
If it is, then set both largeNum and smallNum to that value. Now just compare each additional entered value with those stored values to see if they are bigger or smaller than the previously known extreme.
A few things you should improve upon:
You can remove the temp variables. They don't serve any other purpose other than unnecessarily consuming memory and CPU time.
You should initialize your min and max variables with highest and lowest possible values respectively. If you think larger than integer values should be allowed, change the type to Long
Dim largeNum As Integer = Integer.MinValue
Dim smallNum As Integer = Integer.MaxValue
Remove the ElseIf. Use two If statements instead. This is so that both variables will be set with the very first input itself.
If (userNum > largeNum) Then largeNum = userNum
If (userNum < smallNum) Then smallNum = userNum
Using MS VS 2013 (VB.net) with SQL Server 2012.
I am querying a database and returning a list of double using a stored procedure. I am then dividing each double by 8760. When the first list returns from the database it has the decimal places. The list looks like this
After the calc has been performed the decimals have been removed. See image below.
As you can see the decimals are removed. As if you take the first one and divide it by 8760 you get 101.27 Anyone know why or how to avoid this?
My code is as follows
Dim hoursInYear As Double = 8760
Dim steamFees As List(Of Double)
Dim steamFee As Double
Dim steamFeePerHour As New List(Of Double)
Dim steamFeeTotal As Double
steamFees = RunDetailsCalculations.getFixedFeesSteam
For Each steamFee In steamFees
steamFeePerHour.Add(steamFee \ hoursInYear)
Next
steamFeeTotal = steamFeePerHour.Sum
You are using the backslash (\) operator, which is for integer division. Integer division always results in an integer (no fractional part). If you want to retain the fractional part after the division, you need to use floating-point division, which is the forward slash (/) operator.
As the MSDN states:
Integer division is carried out using the \ Operator (Visual Basic). Integer division returns the quotient, that is, the integer that represents the number of times the divisor can divide into the dividend without consideration of any remainder. Both the divisor and the dividend must be integral types (SByte, Byte, Short, UShort, Integer, UInteger, Long, and ULong) for this operator. All other types must be converted to an integral type first.
In other words, when you do this:
Dim result As Double = 887146.6 \ 8760
What you are really doing is this:
Dim input1 As Integer = CInt(887146.6) ' 887146
Dim input2 As Integer = 8760
Dim result1 As Integer = input1 \ input2 ' 887146 \ 8760 = 101 (the remainder is dropped)
Dim result2 As Double = CDbl(result1) ' 101.0D
Or, more simply:
Dim result As Double = CDbl(CInt(887146.6) \ 8760)
What is the meaning of the use of the # symbol in Excel VBA?
It is used like this:
a = b /100#
I don't understand the significance of # after the 100?
The type-declaration character for Double is the number sign (#). Also called HASH
Other type declaration characters are:
Integer %
Long &
Currency #
Single !
Double #
String $
Don't understand the significance of # here.
It implies that when the expression is evaluated, the number in front of the
type declaration character is treated as a specific data type instead of as
a Variant.
See this example, which are basically the same.
Sub Sample1()
Dim a#
a = 1.2
Debug.Print a
End Sub
Sub Sample2()
Dim a As Double
a = 1.2
Debug.Print a
End Sub
EDIT
Let me explain it a little more in detail.
Consider this two procedures
Sub Sample1()
Dim a As Double, b As Integer
b = 32767
a = b * 100
Debug.Print a
End Sub
Sub Sample2()
Dim a As Double, b As Integer
b = 32767
a = b * 100#
Debug.Print a
End Sub
Question: One of them will fail. Can you guess which one?
Ans: The 1st procedure Sub Sample1() will fail.
Reason:
In Sample2, when you do b * 100# the result of calculation will be of type Double. Since it is within the limits of Double, so the calculation succeeds and the result is assigned to variable a.
Now in Sample1, when you do b * 100 the result of calculation will be of type Integer, since both the operands are of type integer. But the result of calculation exceeds the limits of Integer storage. As a result it will error out.
Hope it helps :)
Trying to get the user to put 3 numbers in 3 text boxes and get the average.
Private Sub btnAverage_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnAverage.Click
Dim a As Integer = CInt(txtone.Text)
Dim b As Integer = CInt(txtTwo.Text)
Dim c As Integer = CInt(txtThree.Text)
Dim average As Integer
average = (a + b + c) / 3
lstOutput.Text = average
End Sub
Try changing the type of average from Integer to Double
Dim average as Double
Right now you're trying to store the Average in an Integer which can only hold a whole number. Averages tend to be non-whole numbers and need a data type that can represent that. Double is good for most situations. That should fix your problem.
EDIT OP mentioned that lstOutput is a ListBox
This is one of the confusing things with WinForms. Even though every single control has a Text property, not all of them actually do anything. They only apply to elements that directly display a single text block or value. Ex Button, Label, etc ...
A ListBox on the other hand displays a group of items. You want to add a new item to the list.
lstOutput.Items.Add(average.ToString())
The Text property of a list box will get or set the selected item. You haven't added your average to the listbox yet.
Try:
lstOutput.Items.Add(average)
Are you sure that txtOne.text txtTwo.text and txtThree.txt will always be an integer value?
You might need to also change the a,b,c vars to Doubles and check that the user didn't supply non-numeric values.
If the user puts "one" in the txtOne textbox, you'll get an exception kablowee.
(air coding here)
dim a as new double
try
if isnumeric(txtOne.text.tostring.trim) then
a = cdbl(txtOne.text.tostring.trim)
end if
'repeat for b and c ...
catch ex as exception
messagebox.show(ex.message.tostring)
end try
And, I'm not sure if I'm right about this, (maybe someone will enlighten me) but does .NET consider type conversion from string to int differently in these two cases
a = cint(txtOne.text)
and
a = cint(txtOne.text.tostring)
???