I am trying to work out this statement but can't get anywhere.
From what I have found so far, I think I need to use an nullif clause for each divisor, but when I try and do this it still does not work.
Here is the statement with no nullif's that produces a divide by zero error.
(1 - (1 - (x1.hup / (x1.hup / (x1.dp / 100)))) / (1 - (x2.sdp / 100)))
There may be simpler ways to express this. I think this is all you need:
(1 - (1 - (x1.hup / (x1.hup / nullif(x1.dp / 100, 0)))) / nullif(1 - (x2.sdp / 100)), 0))
This is Not the Answer
Create a function to check zero like this
CREATE FUNCTION [dbo].[IsZero] (
#Number FLOAT,
)
RETURNS FLOAT
AS
BEGIN
IF (#Number = 0)
BEGIN
SET #Number = 1
END
RETURN (#Number)
END
Not very elegant but will do the job.
SELECT CASE WHEN (1 - (x2.sdp / 100) = 0 THEN NULL
WHEN (x1.dp / 100) = 0 THEN NULL
WHEN (x1.hup / (x1.dp / 100)) = 0 THEN NULL
WHEN (x1.hup / (x1.hup / (x1.dp / 100))) = 0 THEN NULL
ELSE (1 - (1 - (x1.hup / (x1.hup / (x1.dp / 100)))) / (1 - (x2.sdp / 100)))
END AS field
FROM yourtable
Coalescing to zero isn't the magic bullet you're looking for.
Just try to the simplify your problem; if you have 42 / x and x is null, replacing it with zero will just result in a division error.
I don't know about the formula you are applying in this case, but the main thing you are lacking is validation. In my example above, if x equals to 0 we've got a problem.
Similarly, in 23 / (100 - x), x can't ever be 100, so you must check that beforehand and handle the situation accordingly. No expression can result in 0 if it`s a divisor.
So, try to establish which constraints should be watched before processing; for instance, your statement must not accept 0 for x1.dp or x1.hup, and x2.sdp can be anything but 100 (1 - 100/100 = 0, right?). Should one of these situations happen, you could return an error or something.
Related
To eliminate null values from fields for some reason the use of too many coalesce results in BCD Overflow errors
I eliminated the error to 1 line in the select part.
If any (of the 4) coalesces are replaced by a fixed value it's ok
If any coalesce is removed, the result is also ok
If for any field another is used, the error remains
Replacing the '/' with '-' also gives a result but not the correct value.
All fields are numeric(10,4).
It's Firebird version 2.5.8
select
coalesce(Field1, 0) * coalesce(Field2, 0) * ((100 - coalesce(Field3, 0)) / 100) * ((100 + coalesce(Field4, 0)) / 100)
from Table
A calculated column is expected, the actual result is a BCD overflow
Not quite an answer, just a test.
Firebird 2.1.7, IBExpert, SQL Dialect 3
create table t58096187 (
f1 numeric(10,4),
f2 numeric(10,4),
f3 numeric(10,4),
f4 numeric(10,4)
);
insert into t58096187 values ( 50, 50, 50, 50 );
select
coalesce(F1, 0) * coalesce(F2, 0) * ((100 - coalesce(F3, 0)) / 100) * ((100 + coalesce(F4, 0)) / 100)
from t58096187;
ERROR: Unsuccesful execution ... integer overflow ... cause the most significant bit of the result to carry
•Dialect 3 databases allow numbers (DECIMAL and NUMERIC data types) to be stored as INT64 when the precision is greater than 9
https://firebirdsql.org/manual/gfix-dialect.html
https://firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref25-datatypes-fixedtypes.html
Same error with Numeric(9,4) which should avoid use if int64 internal datatype.
If I declare fields as float though, the select yields result 1875, which is well within Numeric(10,4) datatype.
Maybe some INTERMEDIATE result in reverse-polish goes out of bounds?
This also works if to avoid use of INT64 using
recreate table t58096187 (
f1 numeric(6,2),
f2 numeric(6,2),
f3 numeric(6,2),
f4 numeric(6,2)
)
It seems Firebird 3.0.5 is also affected - https://dbfiddle.uk/?rdbms=firebird_3.0&fiddle=57729b31e0a5019aea68a136638d9f50
There is no error - but no results either!
Numeric-as-Int32 works: https://dbfiddle.uk/?rdbms=firebird_3.0&fiddle=c4a4230e855b0ce4fd2b0c7b3b697cda
Reported as https://www.sql.ru/forum/1317439-a/
Mark's assumption of fractional part accuracy as causing this is probably correct.
recreate table t58096187 (
f1 numeric(10,4),
f2 numeric(10,4),
f3 numeric(10,4),
f4 numeric(10,3)
)
Still works.
https://dbfiddle.uk/?rdbms=firebird_3.0&fiddle=29a6c15d6e1854be230d29aea30307cf
Coalesce seems to have nothing with this my problem, dunno if it is relevant to the topic starter's problem, as it is ambiguous what he meant by "removing coalesce". After I removed coalesce it becomes like that
select
F1 * F2 * F3 * F4
from t58096187
And likely the same error
https://dbfiddle.uk/?rdbms=firebird_3.0&fiddle=1f29aff4102ace57e8fa27d83e59b93f
Apparently the problem is a little more specific: The result of an integer operation caused the most significant bit of the result to carry.
So applying the suggested solution (define the format for the resulting number we want to use) we would do:
select
cast( (cast(coalesce(f1, 0) as numeric(10,4)) * cast(coalesce(f2, 0) as numeric(10,4)) ) as numeric(10,4)) * cast(((100 - coalesce(f3, 0)) / 100) as numeric(10,4)) * cast(((100 + coalesce(f4, 0)) / 100) as numeric(10,4))
-- cast(coalesce(f1, 0) as numeric(10,4)) * cast(coalesce(f2, 0) as numeric(10,4)) * cast(((100 - coalesce(f3, 0)) / 100) as numeric(10,4)) * cast(((100 + coalesce(f4, 0)) / 100) as numeric(10,4)) -- sigue dando el mismo error
-- coalesce(F1, 0) * coalesce(F2, 0) * ((100 - coalesce(F3, 0)) / 100) * ((100 + coalesce(F4, 0)) / 100)
from t58096187;
Thanks to Arioch 'The for the great analysis and example to reproduce the error!
I have an Oracle function. Due to privacy issues, I can't include the whole function, but the relevant line is :-
WHEN in_tariff_length = 36 THEN round( ( ( (in_agreed / 100) * 80) * in_uplift) / 100,2) * 3
..which is rounded to 2 decimal places, and returns 1655.58
When I use
WHEN in_tariff_length = 36 THEN round( ( ( (in_agreed / 100) * 80) * in_uplift) / 100,3) * 3
..which is rounded to 3 decimal places, and returns 1655.568
The result I need is 1655.57.
All values sent to and returned from the function are NUMBER.
You can't round something, multiply by 3 and get 1655.57
551.86 * 3 = 1655.58
551.856 * 3 = 1655.568
1655.57 / 3 = 551.856666667
I suggest multiply first, then round.
Please look closely at the second function
round( ( ( (in_agreed / 100) * 80) * in_uplift) / 100,3)
Round( something, 3) means round to 3 decimal places but not to 2
Trying to make a calculation for student grades in an MCQ. 30 q's, 4 marks for correct and -1 for incorrect.
The code below works when I do not put in the /120 part. For a record with 16 correct answers and 7 wrong answers, it returns a value of 57, which is what I want it to. But when I try to divide it by 120 to get a percentage value, it returns 63.94, can anybody explain how or why this happens?
I can upload more code if needs be.
Without "/ 120"
(Math.Round(((CDbl(objAssessment1.Item("Correct")) * 4) - (CDbl(objAssessment1.Item("Wrong")))), 2))
With "/ 120"
(Math.Round(((CDbl(objAssessment1.Item("Correct")) * 4) - (CDbl(objAssessment1.Item("Wrong"))) / 120), 2))
multiplication/division before addition/substraction
What you got, if we put in the actual values is: 16 * 4 - 7 / 120
What you actually want is: (16 * 4 - 7) / 120
(Math.Round(((CDbl(objAssessment1.Item("Correct")) * 4) - (CDbl(objAssessment1.Item("Wrong"))) / 120), 2))
(Math.Round(((CDbl(16) * 4) - (CDbl(7)) / 120), 2))
(Math.Round(((16 * 4) - (7) / 120), 2))
(Math.Round(((16 * 4) - (7) / 120), 2))
(Math.Round(((64) - (7) / 120), 2))
The 64 and the 7 in parenthesis just evaluate to themselves.
(Math.Round((64 - 7 / 120), 2))
Now remember order of precedence from algebra - multiplication and division first.
(Math.Round((64 - 0.0583333333333333), 2))
Now the subtraction.
(Math.Round(63.94166666666667, 2))
Now the Math.Round
(63.94)
Removing the extra set of parenthesis 63.94 evaluates to itself
63.94
The whole problem was too many parenthesis and you missed on set to perform the subtraction before the division.
jmcilhinney's suggestion in comments would be the best way to straighten this out. This really wasn't a programming problem but an arithmetic problem. :-)
We have a fairly complicated SQL Server 2008 r2 sp2 query with this as one of the lines :-
SUM((t.Quantity * contract.ValueOfOnePoint) * ((
CASE contract.Style
WHEN 3
THEN 1 / (1.0 + ((100.0 - Val) / 100.0 * 90.0 / 365.0))
WHEN 2
THEN 1000 * (6.0 * (1.0 - (POWER((1.0 / (1.0 + ((100.0 - Val) / 200.0))), 20.0))) / ((100.0 - Val) / 200.0) + (100.0 * (POWER((1.0 / (1.0 + ((100.0 - Val) / 200.0))), 20.0))))
END
) - (
CASE contract.Style
WHEN 3
THEN 1.0 / (1.0 + ((100.0 - t.Price) / 100.0 * 90.0 / 365.0))
WHEN 2
THEN 1000 * (6.0 * (1.0 - (POWER((1.0 / (1.0 + ((100.0 - t.Price) / 200.0))), 20.0))) / ((100.00 - t.Price) / 200.00) + (100.0 * (POWER((1.0 / (1.0 + ((100.0 - t.Price) / 200.0))), 20.0))))
END
)
)) AS NativeAmount
I am testing this on a single row which has a style of 3 so only the first line in the case statement should have any affect yet leaving the "WHEN 2" clause in it reduces the accuracy of the formula.
Eg. if I remove both WHEN 2 conditions I get an answer such 123.45678 but with the WHEN 2 line left in I get 123.46. It seems to be rounding for some reason even though the second WHEN should never be in-play.
Any thoughts would be really appreciated - going mad!
Thanks.
James.
You need to combine two things. The return type of the case statement is the same for all the then and else clauses. This is a quote from the documentation:
[The case statement] returns the highest precedence type from the set of types in
result_expressions and the optional else_result_expression. For more
information, see Data Type Precedence (Transact-SQL).
So, the SQL Engine does care about all the clauses in the query (which is the answer to your question).
I don't fully understand what is happening in this case. When you call the power() function, the compiler has to decide on the precision of the numeric value, based on the constants and column types. Based on this SQL Fiddle, it chooses a precision of 38 and a scale of 1. However, simple arithmetic on the values produces a precision of 36 and a scale of 23. I'm not sure why, in the end, this results in rounding the value to two decimal places. Perhaps the logic for assigning types for the into clause doesn't quite match the logic for typing of expressions.
Sorry to ask a dumb question, but this one's got me stumped.
SELECT
81234 / 160000 * 100 AS Try1,
CAST((81234 / 160000 * 100) AS float) AS Try2
The answer is 50.77125 but both values return zero. What's the problem?
Thanks,
Try using a decimal point.
Something like
SELECT
81234 / 160000 * 100 AS Try1,
CAST((81234 / 160000 * 100) AS float) AS Try2,
81234. / 160000. * 100. AS Try3
SQL Fiddle DEMO
From / (Divide) (Transact-SQL)
If an integer dividend is divided by an integer divisor, the result is
an integer that has any fractional part of the result truncated.
you're implementing a division by int
Try this:
SELECT
cast(81234 as float) / cast(160000 as float) * 100
or
SELECT
81234.00 / 160000.00 * 100
the cause of that is that 81234 / 160000 is an integer division so it returns 0, force it to be float like 81234. / 160000 and it will work
Try this
SELECT
81234.00 / 160000 * 100 AS Try1,
CAST((81234.00 / 160000 * 100) AS float) AS Try2
You're dividing a number that is considered an integer. Thus, by adding two decimals it is converted to a decimal number and decimals are displayed.
In your case, the division 81234 / 160000 resulted in a 0, which multiplied by 100 is still 0.
You are doing integer divides. 81234/160000 gives zero. Multiply by 100 and it's still zero.
Try converting to floats before doing the division.
Change your query like below
SELECT
81234.00 / 160000 * 100 AS Try1,
CAST((81234.00 / 160000 * 100) AS float) AS Try2
** 81234.00 / 160000 return 0.5077125 by manual through sql it will think as Integer Division so it will return the value 0.
Actually the value 81234 is integer and if you devide an integer it will return integer only concerting 81234 in to float will return float
SELECT
(CONVERT(FLOAT,81234) / 160000) * 100 AS Try1,
((CONVERT(FLOAT,81234) / 160000 * 100) AS Try2
Thanks
Ashutosh Arya