I am doing a calculation of values using Microsoft Excel vba.
I have a portion of codes like the below:
INT(151.2 * 100)
I notice that the result from this calculation is 15119 but the correct result should be 15220.
It is ok if i remove the INT()
151.2 * 100
The result returned will be 15220.
Can anyone advise why is there such a difference and whether it is correct to simply remove the INT() to achieve the correct result?
Floating-point arithmetic.
a = (151.2 * 100)
Print Int(a)
15119
Print a
15120
Print a = 15120
False
There is no double-precision number to represent the result of 151.2 * 100 exactly as 15120. The closest one is apparently just under 15120. When you Int it, it gets truncated, i.e. rounded down to 15119.
Instead of truncating, you could round:
Print CInt(a)
15120
Print Round(a)
15120
Note that if you have a variable i of type Integer and you just say i = 151.2 * 100 as you suggest, then you are implicitly coercing the result to an integer i.e. implicitly saying i = CInt(151.2 * 100). I think it's better practice to be explicit.
Related
Problems like this drive me crazy. Here's the relevant piece of code:
Dim RES As New Size(Math.Floor(Math.Round(mPageSize.Width - mMargins.Left - mMargins.Right - mLabelSize.Width, 4) / (mLabelSize.Width + mSpacing.Width) + 1),
Math.Floor((mPageSize.Height - mMargins.Top - mMargins.Bottom - mLabelSize.Height) / (mLabelSize.Height + mSpacing.Height)) + 1)
Values of the variables (all are of Single type):
mPageSize.Width = 8.5
mMargins.Left = 0.18
mMargins.Right = 0.18
mLabelSize.Width = 4.0
mSpacing.Width = 0.14
For God-knows-what reason, RES evaluates to {Width=1,Height=5} instead of {Width=2,Height=5}. I have evaluated the expressions on the right-side individually and as a whole and they correctly evaluate to {2,5}, but RES would never get correct value. Wonder what am I missing here.
EDIT
I have simplified the problem further. The following code will produce 2.0 if you QuickWatch the RHS, but the variable on the LHS will get 1.0 after you execute this line:
Dim X = Math.Floor(Math.Round(mPageSize.Width - mMargins.Left - mMargins.Right - mLabelSize.Width, 4) / (mLabelSize.Width + mSpacing.Width) + 1)
Time for MS to check it out?
EDIT 2
More info. The following gives correct results:
Dim Temp = mPageSize.Width - mMargins.Left - mMargins.Right - mLabelSize.Width
Dim X = Math.Floor(Temp / CDec(mLabelSize.Width + mSpacing.Width)) + 1
The problem is that the following expression evaluates to a value just below 1:
Math.Round(mPageSize.Width - mMargins.Left - mMargins.Right - mLabelSize.Width, 4) / (mLabelSize.Width + mSpacing.Width)
= 0.99999999985602739 (Double)
But what's the reason for that? The truth is that I don't know exactly. The MSDN does not offer enough information about the implementation of / but here's my guess:
Math.Round returns a Double with value 4.14. The right-hand side of the division is a Single. So you're dividing a Double by a Single. This results in a Double (see MSDN). So far, so good. The MSDN states that all integral data types are widened to Double before the division. Although Single is not an integral data type, this is probably what happens. And here is the problem. The widening does not seem to be performed on the result of the addition, but on its operands.
If you write
Dim sum = (mLabelSize.Width + mSpacing.Width) 'will be 4.14 Single
Math.Round(mPageSize.Width - mMargins.Left - mMargins.Right - mLabelSize.Width, 4) / sum
= 1 (Double)
Here sum is converted to double (resulting in 4.14) and everything is fine. But, if we convert both operands to double, then the conversion of 0.14 introduces some floating point error:
Dim dblLabelSizeWidth As Double = mLabelSize.Width ' will be 4.0
Dim dblSpacing As Double = mSpacing.Width ' will be 0.14000000059604645
The sum is slightly bigger than 4.14, resulting in a quotient slightly smaller than 1.
So the reason is that the conversion to double is not performed on the division's operand, but on the operand's operands, which introduces floating point errors.
You could overcome this problem by adding a small epsilon to the quotient before rounding off. Alternatively you might consider using a more precise data type such as Decimal. But at some point, there will also be floating-point errors with Decimal.
This is due to rounding error: you're taking the floor of a value that is very close to 2, but is less than 2 (while the mathematical value is 2). You should do all your computations with integers, or take rounding errors into account before using operations like floor (not always possible if you want the true value).
EDIT: Since vb.net has a Decimal datatype, you can also use it instead of integers. It may help in some cases like here: the base conversions for 0.18 and 0.14 (not representable exactly in binary) are avoided and the additions and subtractions will be performed exactly here, so that the operands of the division will be computed exactly. Thus, if the result of the division is an integer, you'll get it exactly (instead of possibly a value just below, like what you got with binary). But make sure that your inputs are already in decimal.
How do you access the decimal part of a number in MS Access? More specifically I want only the component after the decimal point, but not including the decimal point. This must also work for all whole numbers. I've seen this answered for other SQL engines, but they don't work in Access. I can't be much more specific than this because of the sensitive nature of what I'm actually working on.
For example given the following numbers the input is on the left and the output is on the right. Output can be either text or a number.
Source Correct Incorrect1 Incorrect2
10.0 0 0.0 .0
3.14159 14159 0.14159 .14159
45.65 65 0.65 .65
173.0 0 0.0 .0
143.15 15 0.15 .15
If I was using C# the following code would give me what I want:
private string getDecimalComponent(double input)
{
String strInput = input.ToString();
if (strInput.Contains('.'))
{
return strInput.Split('.')[1];
}
else
{
return "0";
}
}
Subtract the integer portion of the value.
Example:
4.25 - Int(4.25) = 0.25
Or, as a sample SQL expression:
SELECT
[myDecimalNumber],
[myDecimalNumber] - Int([myDecimalNumber]) as [rightOfDecimal]
FROM tableA
Something like:
SELECT 3.14%1 AS mycolumn from mytable
Depends on the circumstance. If this is, for instance, in a textbox, you can use InStr to find the decimal, and then use the Mid() function to get the number after it. If it's part of an arithmetic equation, then I would use the Int() function and subtract one number from the other to get the difference.
If you can elaborate on how it's being used, and in what context, I can edit my answer to give you more specifics.
EDIT: After more info came to light, try this:
Public Function GetParts(Temp1 as Double)
Temp2 = Int(Temp1)
Temp3 = Mid(Temp1, InStr(Temp1, ".") + 1)
MsgBox Temp2
MsgBox Temp3
End Function
A stable solution producing String containing decimal places from Double.
SELECT DecimalPlaces(MyColumn) FROM MyTable
where the above user-defined function contains the following code:
Function DecimalPlaces(ByVal value As Double) As String
Dim intPartLen As String
intPartLen = Len(CStr(CInt(Abs(value))))
If Len(CStr(Abs(value))) > intPartLen Then
DecimalPlaces = Mid(CStr(Abs(value)), intPartLen + 2)
Else
DecimalPlaces = "0"
End If
End Function
It avoids common mistakes, so it is
locale-independent - works with any decimal separator (it only assumes it is a single character)
preserves precision - avoids subtraction, takes the result only from string representation
Note: those Abs() calls are really required (hint: -0.1)
I'm trying to divide one number by another and then immediately ceil() the result. These would normally be variables, but for simplicity let's stick with constants.
If I try any of the following, I get 3 when I want to get 4.
double num = ceil(25/8); // 3
float num = ceil(25/8); // 3
int num = ceil(25/8); // 3
I've read through a few threads on here (tried the nextafter() suggestion from this thread) as well as other sites and I don't understand what's going on. I've checked and my variables are the numbers I expect them to be and I've in fact tried the above, using constants, and am still getting unexpected results.
Thanks in advance for the help. I'm sure it's something simple that I'm missing but I'm at a loss at this point.
This is because you are doing integer arithmetic. The value is 3 before you are calling ceil, because 25 and 8 are both integers. 25/8 is calculated first using integer arithmetic, evaluating to 3.
Try:
double value = ceil(25.0/8);
This will ensure the compiler treats the constant 25.0 as a floating point number.
You can also use an explicit cast to achieve the same result:
double value = ceil(((double)25)/8);
This is because the expressions are evaluated before being passed as an argument to the ceil function. You need to cast one of them to a double first so the result will be a decimal that will be passed to ceil.
double num = ceil((double)25/8);
I am working on an app that needs to utilize a ratio of a given number and multiply that ratio times another number. Problem is that I can't get numbers less that 1 to give me the proper decimal ratio, instead it gives me zero (when it should be .5).
Example:
float number = 1/2; // This gives me zero
double number = 1/2; // This also gives me zero
If you don't specify decimal places you're using integers which means the calculation is performed with integer precision before the result is cast to the type on the LHS. You want to do the the following when using hard coded numbers in your code:
float number = 1.0f / 2.0f;
double number = 1.0 / 2.0;
If you're aiming to use integer variables for an operation, you'll want to cast them to the type that you want for your result.
Try this
float number = 1.0/2.0;
Remember that 1 is an int, so you are essentially taking
(int)1 / (int)2
which returns
(int)0
To cast variables that are ints, do
float number = (float)numerator / (float)denominator;
What is the syntax to round up a decimal leaving two digits after the decimal point?
Example: 2.566666 -> 2.57
If you want regular rounding, you can just use the Math.Round method. If you specifially want to round upwards, you use the Math.Ceiling method:
Dim d As Decimal = 2.566666
Dim r As Decimal = Math.Ceiling(d * 100D) / 100D
Here is how I do it:
Private Function RoundUp(value As Double, decimals As Integer) As Double
Return Math.Ceiling(value * (10 ^ decimals)) / (10 ^ decimals)
End Function
Math.Round is what you're looking for. If you're new to rounding in .NET - you should also look up the difference between AwayFromZero and ToEven rounding. The default of ToEven can sometime take people by surprise.
dim result = Math.Round(2.56666666, 2)
You can use System.Math, specifically Math.Round(), like this:
Math.Round(2.566666, 2)
Math.Round(), as suggested by others, is probably what you want. But the text of your question specifically asked how to "roundup"[sic]. If you always need to round up, regarless of actual value (ie: 2.561111 would still go to 2.57), you can do this:
Math.Ceiling(d * 100)/100D
The basic function for rounding up is Math.Ceiling(d), but the asker specifically wanted to round up after the second decimal place. This would be Math.Ceiling(d * 100) / 100. For example, it may multiply 46.5671 by 100 to get 4656.71, then rounds up to get 4657, then divides by 100 to shift the decimal back 2 places to get 46.57.
I used this way:
Math.Round(d + 0.49D, 2)
Math.Ceiling((14.512555) * 100) / 100
Dot net will give you 14.52. So, you can use above syntax to round the number up for 2 decimal numbers.
I do not understand why people are recommending the incorrect code below:
Dim r As Decimal = Math.Ceiling(d * 100D) / 100D
The correct code to round up should look like this:
Dim r As Double = Math.Ceiling(d)
Math.Ceiling works with data type Double (not Decimal).
The * 100D / 100D is incorrect will break your results for larger numbers.
Math.Ceiling documentation is found here: http://msdn.microsoft.com/en-us/library/zx4t0t48.aspx