SQL DECIMAL losing precision when SET with calculation - sql

In SQL Server I declare a decimal variable, then set it from an equation. The decimal variable loses precision. However, if I just select the equation, the precision is intact. How do I set the decimal variable without losing precision?
SQL:
DECLARE #_oDiscount decimal(10,2)
SET #_oDiscount = CAST(9.99 AS decimal(10,2)) * CAST(.5 AS float)
SELECT #_oDiscount AS DecimalVariable, CAST(9.99 AS decimal(10,2)) * CAST(.5 AS float) AS Equation
OUTPUT:
DecimalVariable | Equation
-------------------------------
5.00 | 4.995

Well, YOU'VE defined the decimal to contain only 2 digits after the comma!
Therefore, the result of the calculation being 4.995, it will be rounded to 5.00.
If you would be using DECIMAL(12,4) instead, then there is no loss of precision!
DECLARE #_oDiscount decimal(10,4)
SET #_oDiscount = CAST(9.99 AS decimal(10,4)) * CAST(0.5 AS DECIMAL(10,4))
SELECT #_oDiscount
--> return 4.9950
Also: I would recommend against using FLOAT whenever possible!
And furthermore: there's really no need for all those casts..... just use
SET #_oDiscount = 9.99 * 0.5
and you'll get just the same results.
The DECIMAL(p, s) defines how precise the decimal value will be: p (precision) stands for the total number of digits, while s (scale) stands for the number of digits after the decimal point.

Related

Strange result when dividing two numbers

This code:
DECLARE #remise decimal(10,2)
set #remise = 10 / 100
select #remise
results in 0.00. Why doesn't it result in 0.10, which I have been expecting?
You have two integers - 10 and 100. So when considering the division operator, the system decides to perform integer division. Integer division disregards any remainder and cannot produce a decimal result, so the result is 0.
It doesn't matter that you're subsequently planning to store the result in a decimal(10,2).
One way to solve it would be to use a non-integer as one of the inputs to the division. If you're not using literals, commonly this can be done by multiplying one of the inputs by 1.0:
DECLARE #remise decimal(10,2)
set #remise = (10 * 1.0) / 100
select #remise
Is this SQL Server?
10 and 100 are both integers. The integer division 10 / 100 results in 0. If you put that 0 into a decimal variable, the zero still stays zero.
You want
DECLARE #remise decimal(10,2)
set #remise = 10.0 / 100.0
select #remise

SQL Server: why is Float more accurate than Decimal

This post has the following code:
DECLARE #A DECIMAL(3, 0), #B DECIMAL(18, 0), #F FLOAT
SET #A = 3
SET #B = 3
SET #F = 3
SELECT 1 / #A * 3.0, 1 / #B * 3.0, 1 / #F * 3.0
SELECT 1 / #A * 3 , 1 / #B * 3 , 1 / #F * 3
Using float, the expression evaluates to 1. Using Decimal, the expression evaluates to some collection of 9s after the decimal point. Why does float yield the more accurate answer in this case? I thought that Decimal is more accurate / exact per Difference between numeric, float and decimal in SQL Server and Use Float or Decimal for Accounting Application Dollar Amount?
The decimal values that you have declared are fixed width, and there are no points after the decimal place. This affects the calculations.
SQL Server has a rather complex formula for how to calculate the precision of arithmetical expressions containing decimal numbers. The details are in the documentation. You also need to take into account that numeric constants are in decimal format, rather than numeric.
Also, in the end, you need to convert back to a decimal format with the precision that you want. In that case, you might discover that float and decimal are equivalent.

Sql server rounding issue down to 2 decimal places

I am having a problem getting my "Rounding to 2 decimal places" to behave as I would expect.
Try this trivial example
declare #num numeric (18,2)
declare #vat numeric (18,2)
set #num = 11729.83
set #vat = 1.14
select round(#num/#vat,2)
I am getting an answer of 10289.320000 but I should be getting 10289.33 . The full un rounded number is 10289.324561403508771929824561404 (unless my maths is completely off)
Try this
select cast(round(#num/#vat,3) as decimal(18,2))
Round can either return a value lower than the original, or a value upper than the original. In fact it returns the value closest to the original.
If you want to systematically round a number to its lower or upper value, you could then use FLOOR or CEILING (Thanks #GarethD for refreshing my memory on CEILING...)
select round(floor(100*#num/#vat)/100,2) -> lower value
select round(ceiling(100*#num/#vat)/100,2) -> upper value
Otherwise round will indeed return 10289.32 when the value is strictly lower than 10289.325 (which is the case here)
Try converting to a decimal:
select cast(round(#num / #vat, 2) as numeric(18, 2))
I advocate round() to be explicit about the conversion method.
Try this
declare #num numeric (18,2)
declare #vat numeric (18,2)
set #num = 11729.83
set #vat = 1.14
select round(Convert(Decimal(18,3),(#num/#vat)),2)
I am not sure if I understand correctly. But maybe you've been looking for something like this:
select round(round(round(round(#num / #vat, 5), 4), 3), 2)
You will need to convert it to the appropriate decimal type again:
SELECT CONVERT(DECIMAL(18,2), ROUND(#NUM/#VAT,2))

SQL Sum function of a decimal column returns int instead of decimal

I have a column of decimal type and i need to use the sum function on it like this:
declare #credit decimal = (select
( select ISNULL(SUM(Convert(DECIMAL(13,2), Amount)),0)
from TransactionDetail as t1
where t1.AccountFrom = #fromAccount and t1.AccountTo = #toAccount
) -
( select ISNULL(SUM(Convert(DECIMAL(13,2),Amount)),0)
from TransactionDetail as t1
where t1.AccountFrom = #toAccount and t1.AccountTo = #fromAccount
)
)
select #credit
The output should be a decimal number like :
13.56
However, the result is always int, Any suggestions?
The default scale is 0. If you want the result as a particular format, try explicitly adding precision and scale to the variable:
declare #credit decimal(13, 2) = (select . . .
This behavior is well documented:
The number of decimal digits that will be stored to the right of the
decimal point. This number is substracted from p to determine the
maximum number of digits to the left of the decimal point. The maximum
number of decimal digits that can be stored to the right of the
decimal point. Scale must be a value from 0 through p. cSale can be
specified only if precision is specified. The default scale is 0;

Truncate (not round) decimal places in SQL Server

I'm trying to determine the best way to truncate or drop extra decimal places in SQL without rounding. For example:
declare #value decimal(18,2)
set #value = 123.456
This will automatically round #value to be 123.46, which is good in most cases. However, for this project, I don't need that. Is there a simple way to truncate the decimals I don't need? I know I can use the left() function and convert back to a decimal. Are there any other ways?
ROUND ( 123.456 , 2 , 1 )
When the third parameter != 0 it truncates rather than rounds.
Syntax
ROUND ( numeric_expression , length [ ,function ] )
Arguments
numeric_expression
Is an expression of the exact numeric or approximate numeric data
type category, except for the bit data type.
length
Is the precision to which numeric_expression is to be rounded. length must be an expression of type tinyint, smallint, or int. When length is a positive number, numeric_expression is rounded to the number of decimal positions specified by length. When length is a negative number, numeric_expression is rounded on the left side of the decimal point, as specified by length.
function
Is the type of operation to perform. function must be tinyint, smallint, or int. When function is omitted or has a value of 0 (default), numeric_expression is rounded. When a value other than 0 is specified, numeric_expression is truncated.
select round(123.456, 2, 1)
SELECT Cast(Round(123.456,2,1) as decimal(18,2))
Here's the way I was able to truncate and not round:
select 100.0019-(100.0019%.001)
returns 100.0010
And your example:
select 123.456-(123.456%.001)
returns 123.450
Now if you want to get rid of the ending zero, simply cast it:
select cast((123.456-(123.456%.001)) as decimal (18,2))
returns 123.45
Actually whatever the third parameter is, 0 or 1 or 2, it will not round your value.
CAST(ROUND(10.0055,2,0) AS NUMERIC(10,2))
Do you want the decimal or not?
If not, use
select ceiling(#value),floor(#value)
If you do it with 0 then do a round:
select round(#value,2)
Another truncate with no rounding solution and example.
Convert 71.950005666 to a single decimal place number (71.9)
1) 71.950005666 * 10.0 = 719.50005666
2) Floor(719.50005666) = 719.0
3) 719.0 / 10.0 = 71.9
select Floor(71.950005666 * 10.0) / 10.0
Round has an optional parameter
Select round(123.456, 2, 1) will = 123.45
Select round(123.456, 2, 0) will = 123.46
ROUND(number, decimals, operation)
number => Required. The number to be rounded
decimals => Required. The number of decimal places to round number to
operation => Optional. If 0, it rounds the result to the number of decimal. If another value than 0, it truncates the result to the number of decimals. Default value is 0
SELECT ROUND(235.415, 2, 1)
will give you 235.410
SELECT ROUND(235.415, 0, 1)
will give you 235.000
But now trimming0 you can use cast
SELECT CAST(ROUND(235.415, 0, 1) AS INT)
will give you 235
This will remove the decimal part of any number
SELECT ROUND(#val,0,1)
SELECT CAST(Value as Decimal(10,2)) FROM TABLE_NAME;
Would give you 2 values after the decimal point. (MS SQL SERVER)
Another way is ODBC TRUNCATE function:
DECLARE #value DECIMAL(18,3) =123.456;
SELECT #value AS val, {fn TRUNCATE(#value, 2)} AS result
LiveDemo
Output:
╔═════════╦═════════╗
║ val ║ result ║
╠═════════╬═════════╣
║ 123,456 ║ 123,450 ║
╚═════════╩═════════╝
Remark:
I recommend using built-in ROUND function with 3rd parameter set to 1.
I know this is pretty late but I don't see it as an answer and have been using this trick for years.
Simply subtract .005 from your value and use Round(#num,2).
Your example:
declare #num decimal(9,5) = 123.456
select round(#num-.005,2)
returns 123.45
It will automatically adjust the rounding to the correct value you are looking for.
By the way, are you recreating the program from the movie Office Space?
Try like this:
SELECT cast(round(123.456,2,1) as decimal(18,2))
If you desire to take some number like 89.0904987 and turn it into 89.09 by simply omitting the undesired decimal places, simply use the following:
select cast(yourColumnName as decimal(18,2))
The following screenshot is from W3Schools SQL Data Types section, which describes what decimal(18,2) is doing:
Therefore,
select cast(89.0904987 as decimal(18,2))
gives you: 89.09
Please try to use this code for converting 3 decimal values after a point into 2 decimal places:
declare #val decimal (8, 2)
select #val = 123.456
select #val = #val
select #val
The output is 123.46
I think you want only the decimal value,
in this case you can use the following:
declare #val decimal (8, 3)
SET #val = 123.456
SELECT #val - ROUND(#val,0,1)
I know this question is really old but nobody used sub-strings to round. This as advantage the ability to round really long numbers (limit of your string in SQL server which is usually 8000 characters):
SUBSTRING('123.456', 1, CHARINDEX('.', '123.456') + 2)
I think we can go much easier with simpler example solution found in Hackerrank:
Problem statement: Query the greatest value of the Northern Latitudes
(LAT_N) from STATION that is less than 137.2345. Truncate your answer
to 4 decimal places.
SELECT TRUNCATE(MAX(LAT_N),4)
FROM STATION
WHERE LAT_N < 137.23453;
Solution Above gives you idea how to simply make value limited to 4 decimal points. If you want to lower or upper the numbers after decimal, just change 4 to whatever you want.
Mod(x,1) is the easiest way I think.
select convert(int,#value)