SQL error: BCD overflow caused by too many coalesce - sql

To eliminate null values from fields for some reason the use of too many coalesce results in BCD Overflow errors
I eliminated the error to 1 line in the select part.
If any (of the 4) coalesces are replaced by a fixed value it's ok
If any coalesce is removed, the result is also ok
If for any field another is used, the error remains
Replacing the '/' with '-' also gives a result but not the correct value.
All fields are numeric(10,4).
It's Firebird version 2.5.8
select
coalesce(Field1, 0) * coalesce(Field2, 0) * ((100 - coalesce(Field3, 0)) / 100) * ((100 + coalesce(Field4, 0)) / 100)
from Table
A calculated column is expected, the actual result is a BCD overflow

Not quite an answer, just a test.
Firebird 2.1.7, IBExpert, SQL Dialect 3
create table t58096187 (
f1 numeric(10,4),
f2 numeric(10,4),
f3 numeric(10,4),
f4 numeric(10,4)
);
insert into t58096187 values ( 50, 50, 50, 50 );
select
coalesce(F1, 0) * coalesce(F2, 0) * ((100 - coalesce(F3, 0)) / 100) * ((100 + coalesce(F4, 0)) / 100)
from t58096187;
ERROR: Unsuccesful execution ... integer overflow ... cause the most significant bit of the result to carry
•Dialect 3 databases allow numbers (DECIMAL and NUMERIC data types) to be stored as INT64 when the precision is greater than 9
https://firebirdsql.org/manual/gfix-dialect.html
https://firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref25-datatypes-fixedtypes.html
Same error with Numeric(9,4) which should avoid use if int64 internal datatype.
If I declare fields as float though, the select yields result 1875, which is well within Numeric(10,4) datatype.
Maybe some INTERMEDIATE result in reverse-polish goes out of bounds?
This also works if to avoid use of INT64 using
recreate table t58096187 (
f1 numeric(6,2),
f2 numeric(6,2),
f3 numeric(6,2),
f4 numeric(6,2)
)
It seems Firebird 3.0.5 is also affected - https://dbfiddle.uk/?rdbms=firebird_3.0&fiddle=57729b31e0a5019aea68a136638d9f50
There is no error - but no results either!
Numeric-as-Int32 works: https://dbfiddle.uk/?rdbms=firebird_3.0&fiddle=c4a4230e855b0ce4fd2b0c7b3b697cda
Reported as https://www.sql.ru/forum/1317439-a/
Mark's assumption of fractional part accuracy as causing this is probably correct.
recreate table t58096187 (
f1 numeric(10,4),
f2 numeric(10,4),
f3 numeric(10,4),
f4 numeric(10,3)
)
Still works.
https://dbfiddle.uk/?rdbms=firebird_3.0&fiddle=29a6c15d6e1854be230d29aea30307cf
Coalesce seems to have nothing with this my problem, dunno if it is relevant to the topic starter's problem, as it is ambiguous what he meant by "removing coalesce". After I removed coalesce it becomes like that
select
F1 * F2 * F3 * F4
from t58096187
And likely the same error
https://dbfiddle.uk/?rdbms=firebird_3.0&fiddle=1f29aff4102ace57e8fa27d83e59b93f

Apparently the problem is a little more specific: The result of an integer operation caused the most significant bit of the result to carry.
So applying the suggested solution (define the format for the resulting number we want to use) we would do:
select
cast( (cast(coalesce(f1, 0) as numeric(10,4)) * cast(coalesce(f2, 0) as numeric(10,4)) ) as numeric(10,4)) * cast(((100 - coalesce(f3, 0)) / 100) as numeric(10,4)) * cast(((100 + coalesce(f4, 0)) / 100) as numeric(10,4))
-- cast(coalesce(f1, 0) as numeric(10,4)) * cast(coalesce(f2, 0) as numeric(10,4)) * cast(((100 - coalesce(f3, 0)) / 100) as numeric(10,4)) * cast(((100 + coalesce(f4, 0)) / 100) as numeric(10,4)) -- sigue dando el mismo error
-- coalesce(F1, 0) * coalesce(F2, 0) * ((100 - coalesce(F3, 0)) / 100) * ((100 + coalesce(F4, 0)) / 100)
from t58096187;
Thanks to Arioch 'The for the great analysis and example to reproduce the error!

Related

SQL: Cannot convert nvarchar to numeric in complex query

I need to get the nearest airport in my database table from the current users position. I found this formula: https://de.scribd.com/presentation/2569355/Geo-Distance-Search-with-MySQL#page=7
So there are a few differences between the formula described in the link above and my current situation: The example was in MySQL, I'm using MS SQL (not a problem, I guess). lat and lon are considered to be database columns with numeric data type, but for some reason the database table was created with two corresponding columns of type varchar.
My problem is: When I want to use an ORDER BY clause, it throws Error converting data type nvarchar to numeric, without it, it works. I did some research on what rubbish was inserted as string and migrated it so that I just have some empty values.
I can't take all because I only need one. But if I do TOP 1 without ORDER BY I don't get any airport rather than the nearest airport. Does anyone know how to fix the query?
Thanks in advance!
SELECT TOP 1
temp.Distance
FROM (
SELECT
(
3956 * 2 * ASIN(
SQRT(
POWER(
SIN((53.6349994 - abs(CAST(latitude_deg AS numeric))) * pi() / 180 / 2), 2) + COS(53.6349994 * pi()/180) * COS(abs(CAST(latitude_deg AS numeric)) * pi()/180) * POWER(SIN((10.0117336 - CAST(longitude_deg AS numeric)) * pi()/180 / 2), 2) ))) AS Distance
FROM Airport_Airports
WHERE
isnumeric(longitude_deg) = 1 AND isnumeric(latitude_deg) = 1 AND
longitude_deg LIKE '%[^0-9.]%' AND latitude_deg LIKE '%[^0-9.]%'
) AS temp
WHERE
temp.Distance < 50000
Order BY
temp.Distance
First, this logic doesn't make sense:
WHERE isnumeric(longitude_deg) = 1 AND
isnumeric(latitude_deg) = 1 AND
longitude_deg LIKE '%[^0-9.]%' AND
latitude_deg LIKE '%[^0-9.]%'
The like is looking for non-numeric characters. I think you intend:
WHERE isnumeric(longitude_deg) = 1 AND
isnumeric(latitude_deg) = 1 AND
longitude_deg NOT LIKE '%[^0-9.]%' AND
latitude_deg NOT LIKE '%[^0-9.]%'
This ensures that the values are numeric.
The solution to your problem -- at least in SQL Server 2012+ -- is to use try_convert() or try_cast():
(3956 * 2 * ASIN(
SQRT(
POWER(
SIN((53.6349994 - abs(try_convert(numeric, latitude_deg))) * pi() / 180 / 2), 2) + COS(53.6349994 * pi()/180) * COS(abs(try_convert(numeric, latitude_deg)) * pi()/180) * POWER(SIN((10.0117336 - try_convert(numeric, longitude_deg)) * pi()/180 / 2), 2) ))) AS Distance
This will prevent any conversion errors.
You shouldn't use just numeric. Use either a floating point representation or something with decimal places, say numeric(20, 10).
The reason this occurs with the order by is because of the SQL optimizer. You clearly have some lat/long values that do not convert correctly to a numeric. SQL Server allows itself to re-arrange operations, so the conversion might take place before the filtering by the where clause. This is part of the overall query optimization.

SQL divide by zero error, nullif not helping

I am trying to work out this statement but can't get anywhere.
From what I have found so far, I think I need to use an nullif clause for each divisor, but when I try and do this it still does not work.
Here is the statement with no nullif's that produces a divide by zero error.
(1 - (1 - (x1.hup / (x1.hup / (x1.dp / 100)))) / (1 - (x2.sdp / 100)))
There may be simpler ways to express this. I think this is all you need:
(1 - (1 - (x1.hup / (x1.hup / nullif(x1.dp / 100, 0)))) / nullif(1 - (x2.sdp / 100)), 0))
This is Not the Answer
Create a function to check zero like this
CREATE FUNCTION [dbo].[IsZero] (
#Number FLOAT,
)
RETURNS FLOAT
AS
BEGIN
IF (#Number = 0)
BEGIN
SET #Number = 1
END
RETURN (#Number)
END
Not very elegant but will do the job.
SELECT CASE WHEN (1 - (x2.sdp / 100) = 0 THEN NULL
WHEN (x1.dp / 100) = 0 THEN NULL
WHEN (x1.hup / (x1.dp / 100)) = 0 THEN NULL
WHEN (x1.hup / (x1.hup / (x1.dp / 100))) = 0 THEN NULL
ELSE (1 - (1 - (x1.hup / (x1.hup / (x1.dp / 100)))) / (1 - (x2.sdp / 100)))
END AS field
FROM yourtable
Coalescing to zero isn't the magic bullet you're looking for.
Just try to the simplify your problem; if you have 42 / x and x is null, replacing it with zero will just result in a division error.
I don't know about the formula you are applying in this case, but the main thing you are lacking is validation. In my example above, if x equals to 0 we've got a problem.
Similarly, in 23 / (100 - x), x can't ever be 100, so you must check that beforehand and handle the situation accordingly. No expression can result in 0 if it`s a divisor.
So, try to establish which constraints should be watched before processing; for instance, your statement must not accept 0 for x1.dp or x1.hup, and x2.sdp can be anything but 100 (1 - 100/100 = 0, right?). Should one of these situations happen, you could return an error or something.

SQLServer CAST and divide 2 decimal numbers, followed by multiplication gives too many decimal places

I'm working in SQL Server 2008. I have an alias column that I'm creating via the following formula:
col1 / col2 * some_number
col1 and col2 are nvarchars in my table, but they're really integers. some_number is an integer. If I blindly do col1 / col2, I will get 0 for most. So, I need to cast them as decimals. I want there to be a maximum of 2 decimal places after the decimal point. Here is what I have currently:
CAST(col1 AS DECIMAL(10,2)) / CAST(col2 AS DECIMAL(10,2)) * 100
However, this returns far more decimal places than just 2. How do I fix my code to return just 2 decimal places?
According to the documents the scale of the result is given by:
s2 = max(6, s1 + p2 + 1)
where p2 represents the precision of the numerator and s2 represents scale of the denominator
So when dividing one DECIMAL(10, 2) by another you can substitute in values:
s2 = max(6, 2 + 10 + 1) = 13
Which is corroborated with a simple example:
SELECT CAST(1 AS DECIMAL(10,2)) / CAST(1 AS DECIMAL(10,2))
= 1.0000000000000 -- 13 decimal points
You need to use another cast on your result to reduce the scale:
SELECT CAST(CAST(col1 AS DECIMAL(10,2)) / CAST(col2 AS DECIMAL(10,2)) * 100 AS DECIMAL(10, 2))
I'd also suggest if the data type of you column is nvarchar, "but they're really integers" that you just bite the bullet and alter the column data type.

SQL Server case statement altering value accuracy

We have a fairly complicated SQL Server 2008 r2 sp2 query with this as one of the lines :-
SUM((t.Quantity * contract.ValueOfOnePoint) * ((
CASE contract.Style
WHEN 3
THEN 1 / (1.0 + ((100.0 - Val) / 100.0 * 90.0 / 365.0))
WHEN 2
THEN 1000 * (6.0 * (1.0 - (POWER((1.0 / (1.0 + ((100.0 - Val) / 200.0))), 20.0))) / ((100.0 - Val) / 200.0) + (100.0 * (POWER((1.0 / (1.0 + ((100.0 - Val) / 200.0))), 20.0))))
END
) - (
CASE contract.Style
WHEN 3
THEN 1.0 / (1.0 + ((100.0 - t.Price) / 100.0 * 90.0 / 365.0))
WHEN 2
THEN 1000 * (6.0 * (1.0 - (POWER((1.0 / (1.0 + ((100.0 - t.Price) / 200.0))), 20.0))) / ((100.00 - t.Price) / 200.00) + (100.0 * (POWER((1.0 / (1.0 + ((100.0 - t.Price) / 200.0))), 20.0))))
END
)
)) AS NativeAmount
I am testing this on a single row which has a style of 3 so only the first line in the case statement should have any affect yet leaving the "WHEN 2" clause in it reduces the accuracy of the formula.
Eg. if I remove both WHEN 2 conditions I get an answer such 123.45678 but with the WHEN 2 line left in I get 123.46. It seems to be rounding for some reason even though the second WHEN should never be in-play.
Any thoughts would be really appreciated - going mad!
Thanks.
James.
You need to combine two things. The return type of the case statement is the same for all the then and else clauses. This is a quote from the documentation:
[The case statement] returns the highest precedence type from the set of types in
result_expressions and the optional else_result_expression. For more
information, see Data Type Precedence (Transact-SQL).
So, the SQL Engine does care about all the clauses in the query (which is the answer to your question).
I don't fully understand what is happening in this case. When you call the power() function, the compiler has to decide on the precision of the numeric value, based on the constants and column types. Based on this SQL Fiddle, it chooses a precision of 38 and a scale of 1. However, simple arithmetic on the values produces a precision of 36 and a scale of 23. I'm not sure why, in the end, this results in rounding the value to two decimal places. Perhaps the logic for assigning types for the into clause doesn't quite match the logic for typing of expressions.

SQL - How do i output string with numbers in sql?

I want to get a number 5000.1 and divide it by 1000 before adding an "F" infront of it.
How do i do this? I tried and failed this:
select "F" + round ( acq.store_size_net / 1000, 0) from acq
I suspect your missing the cast of the number to a text data type
Without knowing the exact dialect of sql you're using im gonna hazard a guess at ms-sql
select 'F' + cast(cast(round ( 5000.1 / 1000, 0)as int) as nvarchar(50))
produces output
F5
This will work in Oracle :
select 'F' || round (acq.store_size_net / 1000, 0) from acq