So I'm dividing an integer by a decimal, and storing the result in a decimal column. However, it always drops the fractional component(the part after the decimal point). If I multiply the result by 10 or 100 I get a more accurate result, but dividing again drops the fractional part again.
The two fields I've inserted into were a precision 5, scale 0 decimal and a precision 5, scale 3 decimal.
I've also tried casting the integer into a decimal and that doesn't make a difference, neither does multiplying by 1.0.
I'm out of ideas or tricks to try.
Thanks, Buzkie
Turns out I was casting incorrectly. After doing using the correct format
CAST(int AS DECIMAL(5,3))
it worked. I had left off the precision and scale before.
Related
I have a double type column in impala
while I am trying to cut it upto some decimal places
I got this error
ERROR: AnalysisException: No matching function with signature: truncate(DOUBLE, TINYINT).
e.g select truncate(cast(0.4893617021276596 as double),7);
any workaround will be welcome
You can use round():
select round(col, 6)
If you actually want a truncate, then subtract 0.0000005:
select round(col - 0.0000005, 6)
Using the DECIMAL type, it is possible to represent numbers with greater precision than the FLOAT or DOUBLE types can represent.
The maximum allowed precision and scale of the DECIMAL type are both 38.
Precision is the total number of digits, regardless of the location of the decimal point.
Scale is the number of digits after the decimal place.
To represent the number 8.54 without a loss of precision, you would need a
DECIMAL type with precision of at least 3, and scale of at least 2.
Example:
Note that the DECIMAL(17,16) type means there is a total of 17 digits, with 16 of them after the decimal point.
DECIMAL(17,16) 3.1415926535897932
You could ALTER your table with DECIMAL type as follow:
ALTER TABLE my_table CHANGE field field DECIMAL(precision, scale);
or as suggest #Gordon Linoff, you could use round() function.
I would like to remove zeros after two decimal places in DB2. I have more than 1000 rows for this column
For example
3.6900 needs to be converted to 3.69
I used cast in the query after my research and it gave me the correct result but I would like to understand what is DECIMAL(12,2) and how does this work ? Is there any better way to eliminate zeros?
SELECT CAST(CG.RATE AS DECIMAL(12,2)) AS test from fd.OFFERS CG
Please let me know.
what is DECIMAL(12,2) and how does this work?
The DECIMAL data type represents numbers with a specified decimal precision. You can read a description of the numeric data types:
A DECIMAL number is a packed decimal number with an implicit decimal point. The position of the decimal point is determined by the precision and the scale of the number. The scale, which is the number of digits in the fractional part of the number, cannot be negative or greater than the precision. The maximum precision is 31 digits.
I have a problem with round in SQL Server 2014: when I round a number to 2 decimal places sometimes the rounded number is different if I cast to float before or not.
For example, if I execute:
select round(cast(3.945 as float),2)
select round(3.945,2)
I have:
3.94
3.950
But if I execute:
select round(cast(3.935 as float),2)
select round(3.935,2)
I have:
3.94
3.940
It seems incorrect, rounding 3.935 and 3.945 casting to float before, I obtain the same value. Is this a bug?
The problem here is that float is a binary floating point type, where the representation is an approximation of the value. Floats do not losslessly convert to or from base 10, because there is no power of 10 that is also a power of 2. So when this is converted it is done in a way that leaves a roundoff error that pushes the value just before the rounding threshold.
Oddly I cannot reproduce the same behaviour on PostgreSQL and I am not entirely sure why (it may be that on PostgreSQL, round takes a numeric value and this forces a conversion back).
Never use floats where absolute accuracy is required. This occurs not only in databases, but in almost every programming language as well.
As #ChrisTravers says in his answer the issue with rounding a float is that you're not getting exact arithmetic. i.e. That explains why round(3.945,2) rounds up to 3.95 whilst round(3.945E0,2) effectively rounds down to 3.94.
If you're wondering why you see more than 2 decimal places in some cases, that's because of the type you're dealing with. i.e. 3.94 is a float, so doesn't have a specified number of decimal places; whilst 3.950 is the result of rounding a decimal(4,3); which even though we've rounded to 2 decimal places doesn't affect the precision of the type (i.e. it's still decimal(4,3); not converted to decimal(4,2) or decimal(3,2)).
If the purpose of this rounding is for display purposes, you're best of using the str function. i.e.
select str(3.945,4,2) --decimal
select str(3.945E0,4,2) --float
In the above the 4 is the length of the string (i.e. includes the decimal point as a character), and the 2 is the number of decimal places to show.
NB: In this scenario you're chaning the data type to varchar(4).
The below code allows you to see what type you get after performing an operation:
declare #result sql_variant = str(3.945E0,4,2)
select sql_variant_property(#result, 'BaseType') [BaseType]
,sql_variant_property(#result, 'MaxLength') [MaxLength]
,sql_variant_property(#result, 'Precision') [Precision]
,sql_variant_property(#result, 'Scale') [Scale]
I have a table with a smallint column that contains percentages as whole numbers (i.e., 50, 75, 85, etc.)
When I divide this column by 100, as in
SELECT MY_COLUMN/100 AS PCT_AS_FRACTION
FROM MY_TABLE
the result is rounded to the nearest whole number.
For example, for a row that contains the number "50", I get zero as my result.
I can duplicate this with a simple statement:
SELECT 50 / 100 AS TEST_VALUE
Why is that, and how can I get more precision in my result?
When you do integer division (integer divided by integer) you always get an integer answer. 50/100 = .50, which is 0 in integer-speak.
Have you tried dividing MY_COLUMN by 100.0?
Cast whole numbers.
SELECT (cast(50 AS float)/100)
You're doing integer division.
50/100 is 0 with a remainder of 50.
You need to use floating point division.
NB - Be careful in that the remainder of integer division is not rounded. Rather, it is dropped. This is equivalent to calling the FLOOR sql function.
This is common, and is defined as such because when multiplying two integers, a fraction will never occur. Therefore a fraction-handling methodology is never assumed when multiplying integers, but the same cannot be said for integer division.
This can often have an impact when doing dateTime arithmetic in SQL.
When you are using /(Divide) operator it
Returns the data type of the argument with the higher precedence.
and
If an integer dividend is divided by an integer divisor, the result is
an integer that has any fractional part of the result truncated.
So, you need to cast at least one of the operands to appropriate type: decimal and numeric or float or real.
You're dividing 2 integers which results in another integer.
It should possible to cast that way, too
SELECT (50/100)::numeric;
For a SQL int that is being converted to a float, how do I set the precision of the floating point number?
This is the selection I would like to truncate to two or 3 decimal places:
AVG(Cast(e.employee_level as Float))avg_level,
Thanks!
In TSQL, you can specify two different sizes for float, 24 or 53. This will set the precision to 7 or 15 digits respectively.
If all you want to do is truncate to a set number of decimal places, you can use ROUND, ie:
ROUND(AVG(CAST(e.employee_level as float)), 3)
As a general rule, you can't specify the number of digits after the decimal point for a floating-point number. Floating point data types store the closest floating-point approximation to any given value. The closest floating-point approximation is unlikely to have the number of digits you want. Although you might be able to suppress every digit after the third one, that will only change the appearance of the value, not the value itself.
Integers are a different story. An integer--stored, converted, or cast to a floating-point data type--will be stored exactly over a large range. Floating-point data types don't have to store any fractional units for integers.
I'd suggest, though that the best practice for you is to
avoid casting integers to floating-point if you don't need fractional units, or
cast integers to decimal or numeric if you do need fractional units, or
handle display issues entirely in application code.
I have had the same issue when calculating a percentage and needing a resulting string value.
Example: 68 is what % of 379
Result is a float = 17.9419525065900
You can cast/convert to Numeric with the Precision = 2 and get 17.94
If you need the value as a string you can then cast it as a VarChar if needed.
You can use Round() as well but in this case it only makes 17.9419525065900 = 17.9400000000000.
You can also use Ceiling() and Floor() to get the next highest or lowest integer.
Ceiling(17.9419525065900) = 18
Floor(17.9419525065900) = 17
Using these combinations you should be able to achieve a result in any format you need.