Going to be answering my own question here, but I wanted to post this to get some attention in case anyone else ever comes across this.
Within the BusinessObjects Data Services Designer, I have two decimal(36, 10) values, and I'd like to divide one by the other. In order to account for divide-by-zero situations, I have to first check if the denominator is zero, so I end up with an ifthenelse statement like the following:
ifthenelse(Query.Denominator = 0, 0, Query.Numerator / Query.Denominator)
When I execute my job, however, I end up always getting 0 or 1, rather than a decimal value.
And as I said, I already had an answer here, just wanted to provide this for the community.
An ifthenelse statement is interesting, in that it takes its data type from the second parameter. In the code example above, it assumes the correct data type is the data type of the parameter "0", which happens to be an integer.
When the ifthenelse receives a value from "else", it converts it to an integer. So a value of 0.356 becomes 0 and 0.576 becomes 1.
The trick is to cast the "0" to a decimal(36, 10):
ifthenelse(Query.Denominator = 0, cast(0, 'Decimal(36, 10)'), Query.Numerator / Query.Denominator)
Thanks for this, I thought I was going crazy!
Incidentally, for something slightly simpler, this also works:
ifthenelse(Query.Denominator = 0, 0.0, Query.Numerator / Query.Denominator)
Related
The below line of code is returning as a 1 instead of a 2, for reasons I can't comprehend.
console.write(Format((87.20 \ 43.60)))
Surely this should return the result of 2 but I've checked in another environment and it returns a can anyone tell me why?
I have tried putting the code into a second environment but the result was the same I don't understand why it is returning 1 instead of 2, can anyone enlighten me?
Thanks for the help but found the answer.
Decimals are converted to Long before Integer division and due to this are subject to banker's rounding, multiplying both numbers by a 100 before the operation resolves the issue.
Source of information:
Learn Microsoft - VB.Net - \ Operator - Remarks
Thanks,
While I am glad you resolved your own question, I did want to provide an alternative.
When you use the integer division operator, you do not have control as to how the rounding should occur. Whereas if you do a floating-point division operator you can keep the precision and then use one of the built-in Math class methods (documentation) to specify how the number should round.
Take a look at this example:
Dim result = 87.20 / 43.60
Dim roundUp = Math.Ceiling(result)
Dim roundDown = Math.Floor(result)
Dim bankersRounding = Math.Round(result)
Fiddle: https://dotnetfiddle.net/LZEMXV
Because in your example you are using Console.Write (which treats the data as a String) you do not need to cast the value to an Integer data type. However, if you needed the value as an Integer, any one of those variables outside result can be safely converted to an Integer using the Parse method (documentation).
I'm using google bigquery, and a column has values I want to round. If I do, and the rounded value ends in a zero, the zero is not displayed.
I've tried the function FORMAT, which apparently has some .number function, but I have no idea how to use it. Whenever I include any 2 things separated by a comma inside its brackets, it complains that it only takes 1 value.
You would use FORMAT() with the precision specifier. For four decimal places always -- including zeros:
select format('%.4f', 1.23)
If the BQ documentation does not answer your questions, I find that that the function seems to be inspired by the classic C printf()/sprintf() functions.
Unaware if in BigQuery (haven't used it ever) there is a better way I guess this will fix your problem since I just tried it in their console.
Cast your float to a string and then check if your last digit is a 0. In case it's not add it:
SELECT case when RIGHT(cast(0.9 as string), 1) <> '0' then cast(0.9 as string)+'0' else cast(0.9 as string) end as FormattedNumber
Can anyone explain the following results:
?DateAdd("s", 54, 0) = #12:00:54 AM#
True
?DateAdd("s", 55, 0) = #12:00:55 AM#
False
?DateAdd("s", 56, 0) = #12:00:56 AM#
True
UPDATE: Ross Presser's answer provides the what: the difference has to do with the fact that binary fractions cannot always represent decimal fractions. But WHY is the floating point offset different when both expressions evaluate to the same data type?
?TypeName(DateAdd("s", 55, 0))
Date
?TypeName(#12:00:55 AM#)
Date
?VarType(DateAdd("s", 55, 0)) = VarType(#12:00:55 AM#)
True
When I've encountered this sort of floating point artifact in the past, it's usually been because the result was actually two different types, at least at some point during the evaluation. That does not seem to be the case here. I'm still confused.
UPDATE 2: Ross's updated answer provided additional insight into the problem. I've made progress in tracking this down. Each answer seems to raise new questions. It appears that both DateAdd and the date literal are using double precision, but for some reason DateAdd is rounding to 18 decimal places (or perhaps truncating at 19 and not rounding at all):
?CDbl(#12:00:55 AM#) - CDbl(55/86400)
0
?CDbl(DateAdd("s", 55, 0)) - CDbl(55/86400)
-1.0842021724855E-19
?0.000636574074074074 - 0.0006365740740740741
-1.0842021724855E-19
Any ideas why this might be the case?
Date in VBA is expressed as an integer number of days plus a floating point fraction representing the time. Since the time is a float (or perhaps a double), it cannot exactly express every second with perfect precision. 55 seconds is 55/86400, or 0.00063657407 of a day. This is probably not precisely representable in a float.
For more insight, try subtracting the Dateadd value from the literal value, and converting to float.
EDIT: Here's the insight I was talking about:
? cdbl(dateadd("s",55,0)) - cdbl(#12:00:55 AM#)
-1.0842021724855E-19
The parsing algorithm that takes the time literal to a Date structure is apparently doing something different than the dateadd function does, leading to an error in the 19th decimal place. My guess would be that one or the other of these is using Single where it should be using Double. You can call this a bug and report it to Microsoft, I suppose.
EDIT 2: A google search turned up this link where people are talking about the floating point reality beneath VBA's date type. They gave a different example, where the error is in the 17th place instead of the 19th:
? DateAdd("h",2,#8:00#) - #10:00#
-5.55111512312578E-17
And there's also this gentleman who wrote some VBA code to do DateAdd's job more accurately. (The site on that page presents the code in a badly formatted code block that destroys all the newlines, but the code is downloadable.)
I have a report with a field whose value was the expression:
Fields!TotalPrice.Value/Fields!TotalSlots.Value
Although sometimes TotalSlots was blank and thus I was getting a divide by zero runtime error. So I changed the expression to this:
=IIF(Fields!TotalSlots.Value > 0, Fields!TotalPrice.Value/Fields!TotalSlots.Value,"unknown")
but I'm still getting a divide by zero error. How do I work around this zero divisor issue.
Jamie F's answer is correct. As a tip, you can add a function to your report code to make the division a bit easier to implement in multiple cells, e.g.
Public Function Divider (ByVal Dividend As Double, ByVal Divisor As Double)
If IsNothing(Divisor) Or Divisor = 0
Return 0
Else
Return Dividend/Divisor
End If
End Function
You can then call this in a cell like so:
=Code.Divider(Fields!FieldA.Value, Fields!FieldB.Value)
The VB IIF evaluates all arguments, so it will throw an error if any argument throws an error:
Your formula can be written as:
=IIF(Fields!TotalSlots.Value > 0,
Fields!TotalPrice.Value /
IIF(Fields!TotalSlots.Value > 0,
Fields!TotalSlots.Value,
1 ),
"unknown")
Then even when TotalSlots is zero, the formula still won't encounter a division problem.
I don't think your error is on the calculation. First of all, SSRS deals automatically with this situation. See my third column. And the forth shows your expression:
Your problem is probably somewhere else
This only seems to happens when the division is one of the results of an IIF, not if you just write a formula to divide one by the other, e.g.
=IIF(thing=1,10/0,0)
Before it has evaluated thing, it has already tried to calculate both results, causing an error. You can't use IIF in this way to protect from zero, you have to put the IIF on the bottom line of the division, e.g.
=IIF(thing=1, 10/IIF(divisor=0,-99999999999,divisor),0)
This is not satisfactory, since we've introcudes a weird small non zero number as the result, but it may be ok if you just want a non-error.
Technically, the #error is the correct answer.
Function IIF(arg1, arg2, arg3) always calculates all arguments, before returns a result, so your 2nd argument Fields!TotalPrice.Value/Fields!TotalSlots.Value can return #Error.
Try to use IF(arg1, arg2, arg3) function instead IIF.
=IF(Fields!TotalSlots.Value > 0,
Fields!TotalPrice.Value/Fields!TotalSlots.Value,
"unknown")
Was doing some benchmarking with IsNumeric today and compared it to the following function:
Private Function IsNumeric(ByVal str As String) As Boolean
If String.IsNullOrEmpty(Str) Then Return False
Dim c As Char
For i As Integer = 0 To Str.Length - 1
c = Str(i)
If Not Char.IsNumber(c) Then Return False
Next
Return True
End Function
I was pretty surprised with the result.
With a numeric value this one was about 8-10 times faster then regular IsNumeric(), and with an empty or non-numeric value it was 1000-1500 times faster.
What is taking IsNumeric so long? Is there something else going on under the hood that I should consider before replacing it with the function above?
I use IsNumeric in about 50 different places all over my site, mostly for validation of forms and query strings.
Where is your check for locale-specific delimiters and decimal places? Negation? Exponential notation?
You see, your function is only a tiny subset of what numeric strings can be.
1,000,000.00
1,5E59
-123456789
You're missing all of these.
A single char can only contains 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
but a full string may contains any of the following:
1
1234
12.34
-1234
-12.34
0001
so IsNumeric ought to be somewhat slower.
There're also cultural and internationalization issues. i.e. If you are processing a Unicode string, does IsNumeric also process numbers from all other languages as well?
Generally speaking, I would avoid duplicating or replacing language features like this, especially in a language like VB, which is bound to be implementing many use cases for you. At the very least, I would name your method something distinct so that it is not confusing to other developers.
The speed difference is bound to be the fact your code is doing far less work than the VB language feature is.
I would use Integer.TryParse() or Double.TryParse() depending on the data type you are using, since both of these account for regional differences.