Inconsistent results when using Round in Oracle - sql

I have an Oracle function. Due to privacy issues, I can't include the whole function, but the relevant line is :-
WHEN in_tariff_length = 36 THEN round( ( ( (in_agreed / 100) * 80) * in_uplift) / 100,2) * 3
..which is rounded to 2 decimal places, and returns 1655.58
When I use
WHEN in_tariff_length = 36 THEN round( ( ( (in_agreed / 100) * 80) * in_uplift) / 100,3) * 3
..which is rounded to 3 decimal places, and returns 1655.568
The result I need is 1655.57.
All values sent to and returned from the function are NUMBER.

You can't round something, multiply by 3 and get 1655.57
551.86 * 3 = 1655.58
551.856 * 3 = 1655.568
1655.57 / 3 = 551.856666667
I suggest multiply first, then round.

Please look closely at the second function
round( ( ( (in_agreed / 100) * 80) * in_uplift) / 100,3)
Round( something, 3) means round to 3 decimal places but not to 2

Related

Round Up Float column to next number with a multiple of 5

Currently I have a column defined as a float(ex 16.98) I need to be able to take the value and find the next highest multiple of 5.
Examples...
Value 16.98 Should Return 17.00
Value 10.46 Should Return 10.50
Value 9.11 Should Return 9.15
Value 8.10 Should Return 8.10
Value 18.65 Should Return 18.65
Notice that if is is a multiple of 5 then it should return itself.
If You are using MySql try this:
SELECT (ceil(cast(val*100 as signed)/5.0)*5.0)/100.0 from tbl;
CAST inside CEIL is very helpful - it eliminates rounding error.
Here is Demo
In SQL SERVER:
SELECT (ceiling(cast(val*100 as int)/5.0)*5.0)/100.0 from tbl;
Where tbl contains values to transform.
SELECT Format((ceiling(cast(unitprice*100 as int)/5.0)*5.0)/100.0, 'g18')
From table
Try this:
SELECT
CEILING(16.98 * 20) / 20,
CEILING(10.46 * 20) / 20,
CEILING(9.11 * 20) / 20,
CEILING(8.10 * 20) / 20,
CEILING(18.65 * 20) / 20
So the query would look like:
SELECT CEILING(yourColumn * 20) / 20 AS Result FROM yourTable;

SQL: Cannot convert nvarchar to numeric in complex query

I need to get the nearest airport in my database table from the current users position. I found this formula: https://de.scribd.com/presentation/2569355/Geo-Distance-Search-with-MySQL#page=7
So there are a few differences between the formula described in the link above and my current situation: The example was in MySQL, I'm using MS SQL (not a problem, I guess). lat and lon are considered to be database columns with numeric data type, but for some reason the database table was created with two corresponding columns of type varchar.
My problem is: When I want to use an ORDER BY clause, it throws Error converting data type nvarchar to numeric, without it, it works. I did some research on what rubbish was inserted as string and migrated it so that I just have some empty values.
I can't take all because I only need one. But if I do TOP 1 without ORDER BY I don't get any airport rather than the nearest airport. Does anyone know how to fix the query?
Thanks in advance!
SELECT TOP 1
temp.Distance
FROM (
SELECT
(
3956 * 2 * ASIN(
SQRT(
POWER(
SIN((53.6349994 - abs(CAST(latitude_deg AS numeric))) * pi() / 180 / 2), 2) + COS(53.6349994 * pi()/180) * COS(abs(CAST(latitude_deg AS numeric)) * pi()/180) * POWER(SIN((10.0117336 - CAST(longitude_deg AS numeric)) * pi()/180 / 2), 2) ))) AS Distance
FROM Airport_Airports
WHERE
isnumeric(longitude_deg) = 1 AND isnumeric(latitude_deg) = 1 AND
longitude_deg LIKE '%[^0-9.]%' AND latitude_deg LIKE '%[^0-9.]%'
) AS temp
WHERE
temp.Distance < 50000
Order BY
temp.Distance
First, this logic doesn't make sense:
WHERE isnumeric(longitude_deg) = 1 AND
isnumeric(latitude_deg) = 1 AND
longitude_deg LIKE '%[^0-9.]%' AND
latitude_deg LIKE '%[^0-9.]%'
The like is looking for non-numeric characters. I think you intend:
WHERE isnumeric(longitude_deg) = 1 AND
isnumeric(latitude_deg) = 1 AND
longitude_deg NOT LIKE '%[^0-9.]%' AND
latitude_deg NOT LIKE '%[^0-9.]%'
This ensures that the values are numeric.
The solution to your problem -- at least in SQL Server 2012+ -- is to use try_convert() or try_cast():
(3956 * 2 * ASIN(
SQRT(
POWER(
SIN((53.6349994 - abs(try_convert(numeric, latitude_deg))) * pi() / 180 / 2), 2) + COS(53.6349994 * pi()/180) * COS(abs(try_convert(numeric, latitude_deg)) * pi()/180) * POWER(SIN((10.0117336 - try_convert(numeric, longitude_deg)) * pi()/180 / 2), 2) ))) AS Distance
This will prevent any conversion errors.
You shouldn't use just numeric. Use either a floating point representation or something with decimal places, say numeric(20, 10).
The reason this occurs with the order by is because of the SQL optimizer. You clearly have some lat/long values that do not convert correctly to a numeric. SQL Server allows itself to re-arrange operations, so the conversion might take place before the filtering by the where clause. This is part of the overall query optimization.

Rounding of the numeric values

I want to round of the values of two columns:
select a.region as "Regions",
a.suminsured,2 as "SumInsured" ,
a.suminsured/b.sum*100 as pct
from (
SELECT region, sum(suminsured) as suminsured
FROM "Exposure_commune" group by region
) a,
(select sum(suminsured) FROM "Exposure_commune") b
I want the suminsured and pct columns to come with 2 decimal places. Can someone tell me what I should do?
You can use directly numeric with two parameters. Second parameter for round decimal.
select sum(column_name::numeric(10,2)) from tablename
Use round() with two parameters, which only works for the data type numeric.
While being at it, your query can be simpler and faster:
SELECT region
, round(sum(suminsured), 2) AS suminsured
, round((sum(suminsured) * 100) / sum(sum(suminsured)) OVER (), 2) AS pct
FROM "Exposure_commune"
GROUP BY 1;
You can use sum() as window function to get the total without additional subquery, which is cheaper. Related:
Postgres window function and group by exception
Multiplying first is typically cheaper and more exact (although that barely matters with numeric).
Data type is not numeric
For data types double precision of real
You can ...
just cast to numeric to use the same function.
multiply by 100, cast to integer and divide by 100.0.
multiply by 100 and use the simple round() and devide by 100.
The simple round() with just one parameter works for floating point types as well.
Demonstrating all three variants:
SELECT region
, round(sum(suminsured), 2) AS suminsured
, (sum(suminsured) * 100)::int / 100.0 AS suminsured2
, round(sum(suminsured) * 100) / 100 AS suminsured3
, round((sum(suminsured) * 100) / sum(sum(suminsured)) OVER (), 2) AS pct
, ((sum(suminsured) * 10000) / sum(sum(suminsured)) OVER ())::int / 100.0 AS pct2
, round((sum(suminsured) * 10000) / sum(sum(suminsured)) OVER ()) / 100 AS pct3
FROM "Exposure_commune"
GROUP BY 1;
SQL Fiddle.

SQL Server case statement altering value accuracy

We have a fairly complicated SQL Server 2008 r2 sp2 query with this as one of the lines :-
SUM((t.Quantity * contract.ValueOfOnePoint) * ((
CASE contract.Style
WHEN 3
THEN 1 / (1.0 + ((100.0 - Val) / 100.0 * 90.0 / 365.0))
WHEN 2
THEN 1000 * (6.0 * (1.0 - (POWER((1.0 / (1.0 + ((100.0 - Val) / 200.0))), 20.0))) / ((100.0 - Val) / 200.0) + (100.0 * (POWER((1.0 / (1.0 + ((100.0 - Val) / 200.0))), 20.0))))
END
) - (
CASE contract.Style
WHEN 3
THEN 1.0 / (1.0 + ((100.0 - t.Price) / 100.0 * 90.0 / 365.0))
WHEN 2
THEN 1000 * (6.0 * (1.0 - (POWER((1.0 / (1.0 + ((100.0 - t.Price) / 200.0))), 20.0))) / ((100.00 - t.Price) / 200.00) + (100.0 * (POWER((1.0 / (1.0 + ((100.0 - t.Price) / 200.0))), 20.0))))
END
)
)) AS NativeAmount
I am testing this on a single row which has a style of 3 so only the first line in the case statement should have any affect yet leaving the "WHEN 2" clause in it reduces the accuracy of the formula.
Eg. if I remove both WHEN 2 conditions I get an answer such 123.45678 but with the WHEN 2 line left in I get 123.46. It seems to be rounding for some reason even though the second WHEN should never be in-play.
Any thoughts would be really appreciated - going mad!
Thanks.
James.
You need to combine two things. The return type of the case statement is the same for all the then and else clauses. This is a quote from the documentation:
[The case statement] returns the highest precedence type from the set of types in
result_expressions and the optional else_result_expression. For more
information, see Data Type Precedence (Transact-SQL).
So, the SQL Engine does care about all the clauses in the query (which is the answer to your question).
I don't fully understand what is happening in this case. When you call the power() function, the compiler has to decide on the precision of the numeric value, based on the constants and column types. Based on this SQL Fiddle, it chooses a precision of 38 and a scale of 1. However, simple arithmetic on the values produces a precision of 36 and a scale of 23. I'm not sure why, in the end, this results in rounding the value to two decimal places. Perhaps the logic for assigning types for the into clause doesn't quite match the logic for typing of expressions.

math expression returns zero

Sorry to ask a dumb question, but this one's got me stumped.
SELECT
81234 / 160000 * 100 AS Try1,
CAST((81234 / 160000 * 100) AS float) AS Try2
The answer is 50.77125 but both values return zero. What's the problem?
Thanks,
Try using a decimal point.
Something like
SELECT
81234 / 160000 * 100 AS Try1,
CAST((81234 / 160000 * 100) AS float) AS Try2,
81234. / 160000. * 100. AS Try3
SQL Fiddle DEMO
From / (Divide) (Transact-SQL)
If an integer dividend is divided by an integer divisor, the result is
an integer that has any fractional part of the result truncated.
you're implementing a division by int
Try this:
SELECT
cast(81234 as float) / cast(160000 as float) * 100
or
SELECT
81234.00 / 160000.00 * 100
the cause of that is that 81234 / 160000 is an integer division so it returns 0, force it to be float like 81234. / 160000 and it will work
Try this
SELECT
81234.00 / 160000 * 100 AS Try1,
CAST((81234.00 / 160000 * 100) AS float) AS Try2
You're dividing a number that is considered an integer. Thus, by adding two decimals it is converted to a decimal number and decimals are displayed.
In your case, the division 81234 / 160000 resulted in a 0, which multiplied by 100 is still 0.
You are doing integer divides. 81234/160000 gives zero. Multiply by 100 and it's still zero.
Try converting to floats before doing the division.
Change your query like below
SELECT
81234.00 / 160000 * 100 AS Try1,
CAST((81234.00 / 160000 * 100) AS float) AS Try2
** 81234.00 / 160000 return 0.5077125 by manual through sql it will think as Integer Division so it will return the value 0.
Actually the value 81234 is integer and if you devide an integer it will return integer only concerting 81234 in to float will return float
SELECT
(CONVERT(FLOAT,81234) / 160000) * 100 AS Try1,
((CONVERT(FLOAT,81234) / 160000 * 100) AS Try2
Thanks
Ashutosh Arya