How to get exact value in SQL Server - sql

I am dividing 100000=Amount into 9=Shift into Days=30
But when I need the actual amount so how do I get it?
In SQL:
Declare #R1 as money, #R2 as money, #R3 as money
Set #R1 = 100000 / 9 / 30
Set #R2 = #R1 * 9 * 30
select #R2 As 'ActualAmount'
I need exactly 100,000, but the result is 99900.00 - why?

By default sql will do integer division. Just add *1.0 to make it decimal division
Set #R1=100000*1.0/9/30
SQL DEMO
Use float instead of money
SELECT (100000/9/30)*9*30 result, 'v1' as version
UNION
SELECT (100000*1.0/9/30)*9*30, 'v2'
UNION
SELECT (CAST(100000 as float)/9/30)*9*30, 'v3'
ORDER BY version
OUTPUT

Welcome to the wonderful world of floating point rounding errors. 100000 / 9 = 11111.1111... repeating. Let's simplify the example and pretend we just want 10 / 3; different function, same problem. When 10 / 3 is expressed as a fraction, you could do (10 / 3) * 3, cancel out the threes, and get back to 10. However a computer stores 10 / 3 as 3.33333... repeating, out to as high a precision as it can get. What that precision is, is actually a somewhat complicated question to answer, but suffice it to say, it's not infinity. So what actually gets stored is something like 3.333333....0. When you multiply that by 3, the computer says "ah, you asked me to multiply 3.3333...0 * 3, which is of course 9.99999...0".
Mathematically, your expectation to get 10 back (or in your case, 100,000) is completely reasonable. But because of how a computer stores numbers with repeating decimals, the computer doesn't recognize there's an equivalence between 10 / 3 and 3.33333... (Actually, it sort of does, but the reason it does is probably more to do with luck; more on this later).
As was pointed out in another answer, the reason you're getting 99900 is because it's doing integer division rather than floating point division. So the question then becomes what do you cast 100000 as to get the answer you want?
In your case, float seems to get it right, although it's worth noting that this may not be the case in all circumstances. This probably turns out right because float has 53 decimal places available in its mantissa (significant digits) and with that level of precision, SQL makes an educated guess that you were, in fact, trying to re-form a fraction with infinitely repeating decimal places rather than multiply three numbers very close to, but not quite equal to the original number.
Money on the other hand is usually a bad choice as a data type because it's effectively just a small precision float and leads to problems like this. Decimal also has some problems in this case, as it will actually overshoot the re-formed value.
declare
#Money money = cast(100000 as money) / 2700,
#Float float = cast(100000 as float) / 2700,
#Decimal decimal(38,2) = cast(100000 as decimal(38, 2)) / 2700
select
_Money = #Money,
_MoneyReformed = #Money * 2700,
_Float = #Float,
_FloatReformed = #Float * 2700,
_Decimal = #Decimal,
_DecimalReformed = #Decimal * 2700
If you need a true, universally correct representation of fractions, you'll have to store the numerator and denominator in separate columns so you can manipulate each independently.

Related

SQL Server retain decimal scale and precision for operations on very lage numbers

Following my previous question I am still struggling with an arithmetic overflow error in SQL Server 2017. How can I perform a multiplication/division of a large decimal(38,0) value and retain/regain both the precision and scale?
In my current example I simply want to halve DECLARE #var1 decimal(38,0) = 85070591730234615865699536669866196992; (and truncate the result if #var1 were odd). From this post I have gathered that even if I do not attempt to divide by 2 but multiply by 0.5 instead I would still not obtain a decimal(38,0) result. I could divide by 2,000,000 to make the result fit the resulting decimal(38,6) type, but considering the multiplication rules etc. I can't figure out how to get back to decimal(38,0).
This answer is specific to your specific question -- dividing by 2. If you want a more general solution, please ask a new question.
Dividing by 2 is the same as multiplying by 5 and dividing by 10.
We cannot multiply your value by 5, but we can do the following:
Integer divide by 10.
Multiply by 5.
Add in a little bit extra to account for the missing digit.
But . . . you are thinking . . . how can we divide by 10? Well, you can do that with string operations: just truncate the last digit:
declare #var1 decimal(38,0) = 85070591730234615865699536669866196992;
select (convert(decimal(38,0), left(convert(varchar(38), #var1), 37)) * 5 +
convert(decimal(38, 0), (#var1 % 10) / 2)
) as half_a_big_value
Here is a db<>fiddle.

ROUND function versus data type

My question is conceptual.
In the code snippet below my expectation is to get exactly the same result for all "round" activations.
However, when using a constant as an argument, the result is different than when using a calculated or stored value.
In the documentation I did not find any comment to help me understand the reason for the difference.
In response to the similar question (sql server round function not working well), I was referred to the excellent article https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html, but in the following example, all cases are "float", so I don't think there are reasons for different results.
--
DECLARE #value FLOAT;
SET #value = 4255.0;
CREATE TABLE TESTTABLE (
FLOAT_VALUE FLOAT
)
INSERT INTO TESTTABLE(FLOAT_VALUE) VALUES (#value);
SELECT 0.015 * 4255.0 AS ORIGINAL_VALUE,
ROUND(0.015 * 4255.0, 2) AS ROUND_FROM_CONSTANT,
ROUND(0.015 * CONVERT(FLOAT, 4255.0), 2) AS ROUND_FROM_CAST,
ROUND(0.015 * #value, 2) AS ROUND_FROM_VARIABLE,
ROUND(0.015 * FLOAT_VALUE, 2) AS ROUND_FROM_FIELD
FROM TESTTABLE
The reason you're getting different results is because you have different data types throughout the expressions.
As a literal, 4225.0 is a decimal(5,1) not a float, so only when you explicitly cast the value to a float will it be treated as one. As a float and decimal don't operate the same as well, you get different results.
We'll go through each expression:
0.015 * 4255.0
These are both decimal values, a decimal(3,3) and a decimal(5,1). When dealing with different scales and precisions and multiplication, the following is used to calculate them:
Precision = p1 + p2 + 1
Scale = s1 + s2
Therefore the resulting precision and scale are 3+5+1 and 1+3 = decimal(9,4) This is why you get the result 63.8250.
ROUND(0.015 * 4255.0, 2)
Same as above, but then you round to 2 decimal places; little explanation needed here.
ROUND(0.015 * CONVERT(float, 4255.0), 2)
There are 2 data types here, the decimal(3,3) and the float. Use we have 2 different data types, data type precedence comes into play. As a 0.015 is implicitly cast to a float.
CONVERT(float,0.015) * CONVERT(float, 4255.0) returns 63.825, however, as float is an imprecision value, using ROUND on that ends up returning 63.82; because because the number is actually < 63.825 in the imprecise stored value (perhaps 63824999999999~).
ROUND(0.015 * #value, 2)
Same as 3.
ROUND(0.015 * FLOAT_VALUE, 2)
Same as 3.
References:
Data type precedence (Transact-SQL)
Precision, scale, and Length (Transact-SQL)

SQL Server Real and Float datatype cast conversions do not match

We are undergoing a migration in our tool. Some of the datatypes in our tables have been changed from real to float. To understand the impact I have run the following code:
declare #realVariable as real,
#floatVariable as float
set #realVariable=152304.11999512
set #floatVariable=152304.11999512
select cast(#realVariable AS decimal(15, 4)) as realcol,str(#floatVariable,15,4) as floatcol
The result is different for both real and float variables:
real=152304.1250
float= 152304.1200
How can I make sure that the float result is same as real?
float and real are both approximate-number data types for floating point numeric data.
real is the same as float(24). If you declare float without a suffix, then it means float(53).
So, you are converting from real (which is float(24)) to float(53). You are simply making the numbers more precise.
float(24) has a precision of 7 digits and float(53) has a precision of 15 digits.
See Books Online > float and real (Transact-SQL): https://msdn.microsoft.com/en-GB/library/ms173773.aspx
If we repeat your example and display the variables without conversion, we can see that the real variable is displayed with 7 digits precision, and float variable is displayed with 14 digits precision, which is exactly what you set it to.
DECLARE #realVariable as real,
#floatVariable as float;
-- Set to a number with 14 digits precision
SET #realVariable=152304.11999512;
SET #floatVariable=152304.11999512;
SELECT #realVariable AS '#realVariable', #floatVariable AS '#floatVariable';
The simple answer to your question is that the two numbers are not the same. The one with float(53) is more precise.
Calculating the ABS of the difference of the two numbers and then comparing this to an acceptable threshold is one way to do it. However this would not be satisfactory if the scale can vary widely.
To illustrate why this might not be satisfactory:
r = 0.0001234567
f = 0.00012345678901234
Comparing the difference to a threshold of 0.0005 would not be useful.
So it might be better to use a threshold that is a percentage of the number.
DECLARE #threshold AS float, #delta AS float;
SET #threshold = ABS(#floatVariable / 1000000);
SET #delta = ABS(#floatVariable - CAST(#realVariable AS float));
SELECT #realVariable AS '#realVariable',
#floatVariable AS '#floatVariable',
#threshold AS '#threshold',
#delta AS '#delta',
CASE
WHEN #delta < #threshold THEN N'OK'
ELSE N'Different'
END AS 'Comparison';
Note: This solution might need a bit of adjustment for numbers that are very close to zero.
#RichardCL's description describes the way things work (+1). Now, if you are converting a value stored as a real datatype value into a value stored as a float datatype value, then the entire and precise "original" value will be stored in the resulting value--you will lose no information from your existing set of real values.
Referring to your example, your value of 152304.11999512 cannot be stored precisely as a real value--it will be rounded and stored as 152304.125. Convert this to a float, and you will still have 152304.125.
You need a delta. Compute the absolute value of the difference between the real and float values then make sure that is less than your delta. The delta should be sufficiently small to meet your needs.
For example using your values:
abs(152304.1250 - 152304.1200) <= .0050 so the use .0050 as your delta value.

Strange behavior while rounding in SQL server 2008

At some point I have a numeric(28,10) and I cast it in money (I know its bad but for legacy reason I have to return money) in the same time I also have to set the sign (multiplying by +1/-1).
In a first attempt I had cast the +/-1 to match the numeric type.
For the value 133.3481497944 we encounter a strange behavior (I have simplified the actual code in order to keep only the elements needed to demonstrate the problem):
SELECT CAST(CAST(133.3481497944 AS numeric(28,10))*cast(1 AS numeric(28,10)) AS money)
133.3482
which is not correctly rounded...
Removing the cast solve the problem
SELECT CAST(CAST(133.3481497944 AS numeric(28,10)) * 1 AS money)
133.3481
Did someone know what is happening in SQL? How can a multiplication by 1 and cast(1 AS numeric(28,10)) affect the result of the rounding?
When multiplying numerics, SQL uses the following rules to determine the precision and scale of the output:
p = p1 + p2 + 1
s = s1 + s2
which makes sense - you wouldn't want 1.5 * 2.5 to be truncated to one digit past the decimal. Nor would you want 101 * 201 to be limited to 3 digits of precision, giving you 20300 instead of 20301.
In your case that would result in a precision of 57 and a scale of 20, which isn't possible - the maximum precision and scale is 38.
If the resulting type is too big, decimal digits are sacrificed in order to preserve the integral (most significant) part of the result.
From the SQL Programmability & API Development Team Blog:
In SQL Server 2005 RTM (and previous versions), we decided preserve a minimum scale of 6 in both multiplication and division.
So your answer depands on how big and precise you need the multiplier to be. In order to preserve 10 digits of decimal precision. If the multiplier needs a scale bigger than 9, then decimal digits may be truncated. If you use a smaller precision and scale, you should be fine:
SELECT CAST(CAST(133.3481497944 AS numeric(28,10))*cast(1 AS numeric(9,7)) AS money)
yields 133.3481.
I don't see any ROUNDing here. I only see casting. Don't assume that it will round, when you CAST. Historically, when we cast the environment truncates (SQL server or not) or behaves not as we expect - especially when we're talking about FLOATs.
SELECT
CAST(CAST(133.3481497944 AS numeric(28,10))*cast(1 AS numeric(28,10)) AS money) --Your original,
CAST(1 AS numeric(28,10)) --Just the 1 casted,
CAST(133.3481497944 AS numeric(28,10)) --Your expected calculation,
CAST(133.3481497944 AS numeric(28,10))*cast(1 AS numeric(28,10)) -- The actual calculation
SELECT
CAST(133.3481497944 AS numeric(28,10))*cast(1.5 AS numeric(28,10)),
CAST(133.3481497944 AS numeric(28,10))*1.5,
CAST((133.3481497944*1) AS money),
133.3481497944*1
Returns
133.3482
1.0000000000
133.3481497944
133.348150
200.022225
200.02222469160
133.3481
133.3481497944
So as mentioned above, there really isn't any true rounding, but a loss of precision during the cast. As to exactly why, I don't know. Most likely during the calculation(multiplication) while using the Numeric(28,10) it cuts off some precision.
I added the second lines to show that really you may not need your numeric casting.

Get an accurate percentage calculation

I am trying to divide one number by another in a SQL view. I'm dividing two columns that are each of type int, and the problem is that its giving me a rounded output, rather than an accurate one.
Here is my select:
SELECT numberOne, numberTwo, (numberOne*100/NULLIF(numberTwo, 0)) as PctOutput
This is working, but when it dives 3 by 37 its giving me 8 instead of 8.108.
How would I modify this to give me an accurate answer? Do I need to cast the numbers out of ints? if so - into what?
Try an implicit cast:
SELECT
numberOne,
numberTwo,
((1.0*numberOne)*100/NULLIF(1.0*numberTwo, 0)) as PctOutput
**--Cast the denominator as Float**
SELECT 3 * 100 / NULLIF(CAST(37 AS FLOAT),0) -- 8.10810810810811
**--Multiply by 1.0 in either numerator OR denominator**
SELECT 3 * 100 / NULLIF(37 * 1.0,0) -- 8.108108
SELECT 3.0 * 100 / NULLIF(37,0) -- 8.108108
**--Convert it to decimal**
SELECT CONVERT(DECIMAL(25,13), 3) * 100 /
NULLIF(CONVERT(DECIMAL(25,13), 37),0) -- 8.108108108
You need to cast the numbers from INT into either float or decimal.
If you use literals, they will likely be decimal (NUMERIC), not float (http://stackoverflow.com/questions/1072806/sql-server-calculation-with-numeric-literals)
Note that if you use decimal, you should be aware of the rules of scale and precision in division if you have numbers near the boundaries where you might lose precision:
http://msdn.microsoft.com/en-us/library/ms190476.aspx
In SQL Server the result is always of the same type as the inputs. So yes you do need to convert the inputs.
I generally convert using convert(decimal(9,2),numberOne) but depending you may want to convert(real,numberOne) or use a different level of precision.