I have been calculating different integer percentages with different numbers but each time I get floor rounded number. select 13*100/60 gives me 21 and the actual number is 21.66 which using a round function should give us 22 but it can only give me 21 for all different decimal numbers.
I am using SQL 2017. please help
This is due to the fact that you are dividing ints and not floating-point numbers. Integer division returns an integer.
Try the following instead (noting the .0 on the end of the 60):
SELECT 13 * 100 / 60.0
Making one of the components a floating-point number will automatically output the result as a floating-point number.
Output:
21.666666
Incidentally, if you are working with variables and one of them is a FLOAT, it will automatically produce the output you expect:
DECLARE #A FLOAT
DECLARE #B INT
DECLARE #C INT
SET #A = 13
SET #B = 100
SET #C = 60
SELECT #A * #B / #C
Output:
21.6666666666667
Related
I have a large stored procedure updating a table each night. It grabs a basic amount of information in to a temp table before updating each of the remaining columns through a series of updates.
One of these updates is a division. The division takes two previously updated fields and divides them.
UPDATE C
SET C.RETURN_ON_SALES = C.TW_UNIT_SALES / C.DISTRIBUTION
FROM
#custTop150 C
The column definitions in #custTop150 are as follows:
RETURN_ON_SALES DECIMAL(5,2) NULL
TW_UNIT_SALES INT NULL
[DISTRIBUTION] INT NULL
All of my results for this update are returning as 0.00. To give you an example of some of the divisions it is calculating:
7 / 41 | Returns 0.00 when it should return 0.17073170731
11 / 41 | Returns 0.00 when it should return 0.26829268292
9 / 41 | Returns 0.00 when it should return 0.21941219512
5 / 38 | Returns 0.00 when it should return 0.13157894736
I know that the answer will be pretty simply or right in front of my eyes but at this point in time I am completely lost as to why this is happening.
Thanks
Guess you are working on SQL Server. Then change RETURN_ON_SALES to Decimal(10,10) and CAST the inputs or change its data type to DECIMAL
DECLARE #RETURN_ON_SALES decimal(10,10);
DECLARE #TW_UNIT_SALES INT ;
DECLARE #DISTRIBUTION INT ;
SET #TW_UNIT_SALES=7;
SET #DISTRIBUTION=41;
SET #RETURN_ON_SALES=CAST(#TW_UNIT_SALES AS DECIMAL (10,5))/ CAST (#DISTRIBUTION AS DECIMAL(10,5));
PRINT #RETURN_ON_SALES;
TRY THIS: It seems your columns are defined as INTEGER and in the division it should return in DECIMAL value so for this we can simply change the one of operand value to DECIMAL using CAST and it will return the DECIMAL value as below
DECLARE #first INT = 4, #second INT = 3
SELECT #first/#second -- OUTPUT: 1
SELECT #first/CAST(#second AS DECIMAL(10,5)) -- OUTPUT: 1.33333333333333
Try this:
UPDATE C
SET C.RETURN_ON_SALES = 1.0 * C.TW_UNIT_SALES / C.DISTRIBUTION
FROM
#custTop150 C
Note that the 1.0 will cause a cast to float before assigning to the target variable.
You are not specifying which DBMS so, to make sure it works, you may also try the following (if the previous still does not work):
UPDATE C
SET C.RETURN_ON_SALES = (1.0 * C.TW_UNIT_SALES) / (1.0 * C.DISTRIBUTION)
FROM
#custTop150 C
This post has the following code:
DECLARE #A DECIMAL(3, 0), #B DECIMAL(18, 0), #F FLOAT
SET #A = 3
SET #B = 3
SET #F = 3
SELECT 1 / #A * 3.0, 1 / #B * 3.0, 1 / #F * 3.0
SELECT 1 / #A * 3 , 1 / #B * 3 , 1 / #F * 3
Using float, the expression evaluates to 1. Using Decimal, the expression evaluates to some collection of 9s after the decimal point. Why does float yield the more accurate answer in this case? I thought that Decimal is more accurate / exact per Difference between numeric, float and decimal in SQL Server and Use Float or Decimal for Accounting Application Dollar Amount?
The decimal values that you have declared are fixed width, and there are no points after the decimal place. This affects the calculations.
SQL Server has a rather complex formula for how to calculate the precision of arithmetical expressions containing decimal numbers. The details are in the documentation. You also need to take into account that numeric constants are in decimal format, rather than numeric.
Also, in the end, you need to convert back to a decimal format with the precision that you want. In that case, you might discover that float and decimal are equivalent.
I'm using this query to round off the numbers and this round off the next value. Now I need to round off to the before value means if the value is 45.67 then the value should be 45. I tried these two queries and still I need to tweak the values.
Method1:
parsename('$' + convert(varchar,convert(money,round(sum(Column1 * Column2),0)),1),2)
Method2:
parsename('$' + convert(varchar,convert(money,floor(Column1 * Column2),0),1),2)
Really appreciate any suggestions.
The CEILING function returns the smallest integer greater than or equal to the specified numeric expression. The FLOOR function returns the largest integer less than or equal to the specified numeric expression. For example, in considering a numeric expression of 12.9273, CEILING returns 13 and FLOOR returns 12. The return value of both FLOOR and CEILING has the same data type as the input numeric expression.
SELECT CEILING(12.9273);
Here is the result set.
13
SELECT FLOOR(12.9273);
Here is the result set.
12
http://technet.microsoft.com/en-us/library/ms190927%28v=sql.105%29.aspx
To round down you can use FLOOR()
E.G.
DECLARE #number numeric(5,2)
SET #number = 45.67
SELECT FLOOR(#number)
You'd get the result 45
With your example, it looks like it's already working?
declare #number1 numeric(5,2)
declare #number2 numeric(5,2)
set #number1 = 1.23
set #number2 = 21.69
select parsename('$' + convert(varchar,convert(money,floor(#number1 * #number2),0),1),2)
select #number1 * #number2
Results
$26
26.6787
DECLARE #a int;
DECLARE #b int;
SET #a = 9;
SET #b = 2;
SELECT CEILING (#a/#b);
It is returning as 4 instead of 5. Why?
Edit: I would like to get next smallest integer if the quotient is not whole number.
Try:
SELECT CEILING (#a/CAST(#b AS float))
And consider NULLIF(#b,0) too, to avoid a Division By Zero error.
After dividing 9 by 2 a decimal fraction is Truncated to its integer part - 4, not Rounded to 5. Try:
SELECT 9/2
Resilt is 4. Then CEILING(4) = 4
To get next integer declare variables as data types that can handle decimal part: NUMERIC,FLOAT, REAL.
SQL Server does integer division. So 9/2 = 4 in SQL Server.
Taking the ceiling of an integer is the same integer.
I have the following set of calculations in excel that I want to be able to use in a stored procedure.
Excel
CellA: 45/448.2 = 0.100401606425703
CellB: 1-CellA = 0.899598393574297
CellC: 1-CellB = 0.100401606425703
CellD: CellC * 448.2 = 45.000000000000000
In SQL I am doing the following:
declare #a decimal(18,15) = 45/448.2
declare #b decimal(18,15) = 1-#a
declare #c decimal(18,15) = 1-#b
declare #d decimal(18,15) = #c * 448.2
I have also tried running the calculation in one line
declare #e decimal(18,15) = (1-(1-(45/448.2)))*448.2
when I return the values SQL gives me the following:
#a: 0.100401000000000
#b: 0.899599000000000
#c: 0.100401000000000
#d: 44.999728200000000
#e: 44.999728200000000
I've tried adjusting the precision of the decimals in SQL but I nothing makes a difference, it only returns the first 6 digits of the decimal.
Does Excel do any optimization when running the formula?
Any ideas?
Even just your first line is enough to show the problem:
declare #a decimal(18,15) = 45/448.2
print #a
gives
---------------------------------------
0.100401000000000
This is because of data types. When you say
448.2
it is (per the documentation) interpreted as a constant of type decimal, and also per the documentation,
In Transact-SQL statements, a constant with a decimal point is
automatically converted into a numeric data value, using the minimum
precision and scale necessary. For example, the constant 12.345 is
converted into a numeric value with a precision of 5 and a scale of 3.
So 448.2 is decimal(4,3). 45 is integer, which when combined with a decimal is treated as having precision of 10 and scale 0 . When we divide, the rules say
Operation Result precision Result scale
e1 / e2 p1 - s1 + s2 + max(6, s1 + p2 + 1) max(6, s1 + p2 + 1)
which in this case gives a result precision of 10 - 3 + 0 + max(6, 0 + 3 + 1) and scale of max(6, 0 + 3 + 1), which comes out to 13 and 6.
That result scale of 6 is why the result only has those six decimal places.
The way to fix it is to get your operands into an appropriate type before acting on them; for example, here are two ways:
Force a number to be treated as floating-point:
declare #a decimal(18,15) = 45/448.2e0
select #a
---------------------------------------
0.100401606425703
Explicitly supply a decimal scale:
declare #a decimal(18,15) = 45/cast(448.2 as decimal(18,10))
select #a
---------------------------------------
0.100401606425703