I have a column with datatype float in Teradata. I want to find the Maximum precision and scale for that column.
Note: My column's scale part has more than 10 digits in most of the places.
Sample Data
123.12321323002
13123213.13200003
33232.213123001
The output I need is
Precsion 19 (scale + length of 13123213) and scale is 11 (length of 12321323002)
or
8 (length of 13123213), 11 (length of 12321323002).
I tried to find them buy converting the column as varchar and splitting them based on the '.' and make the integer and fractional part as 2 columns and then finding the max length of 2 columns. But when I'm select the data, Teradata rounds off the scale part. So after that, if I convert them as char, I'm getting lesser value for scale part.
For example:
org data: 1234.12312000123101
data when I select from Teradata: 1234.12312000123
This is a bit long for a comment.
Teradata uses the IEEE format for real/float values. This gives 15-17 digits of precision. Alas, you need 19 digits, so the values you want are not being stored in the database. You cannot recover them.
What you can do is fix the database, so it uses numeric/decimal/number. This supports what you want: NUMERIC(19, 11). You would then need to reload the data so it is correctly stored in the database.
When you need high precision without predefined scale simply switch to the NUMBER datatype, which is a mixture of DECIMAL and FLOAT.
Exact numeric, at least 38 digits precision, no predefined scale, range of 1E-130 .. 1E125.
Float on steroids :-)
Related
I have a double type column in impala
while I am trying to cut it upto some decimal places
I got this error
ERROR: AnalysisException: No matching function with signature: truncate(DOUBLE, TINYINT).
e.g select truncate(cast(0.4893617021276596 as double),7);
any workaround will be welcome
You can use round():
select round(col, 6)
If you actually want a truncate, then subtract 0.0000005:
select round(col - 0.0000005, 6)
Using the DECIMAL type, it is possible to represent numbers with greater precision than the FLOAT or DOUBLE types can represent.
The maximum allowed precision and scale of the DECIMAL type are both 38.
Precision is the total number of digits, regardless of the location of the decimal point.
Scale is the number of digits after the decimal place.
To represent the number 8.54 without a loss of precision, you would need a
DECIMAL type with precision of at least 3, and scale of at least 2.
Example:
Note that the DECIMAL(17,16) type means there is a total of 17 digits, with 16 of them after the decimal point.
DECIMAL(17,16) 3.1415926535897932
You could ALTER your table with DECIMAL type as follow:
ALTER TABLE my_table CHANGE field field DECIMAL(precision, scale);
or as suggest #Gordon Linoff, you could use round() function.
I would like to remove zeros after two decimal places in DB2. I have more than 1000 rows for this column
For example
3.6900 needs to be converted to 3.69
I used cast in the query after my research and it gave me the correct result but I would like to understand what is DECIMAL(12,2) and how does this work ? Is there any better way to eliminate zeros?
SELECT CAST(CG.RATE AS DECIMAL(12,2)) AS test from fd.OFFERS CG
Please let me know.
what is DECIMAL(12,2) and how does this work?
The DECIMAL data type represents numbers with a specified decimal precision. You can read a description of the numeric data types:
A DECIMAL number is a packed decimal number with an implicit decimal point. The position of the decimal point is determined by the precision and the scale of the number. The scale, which is the number of digits in the fractional part of the number, cannot be negative or greater than the precision. The maximum precision is 31 digits.
I have create a table named:
sub
,with field:
addn numeric (24,6) NULL
I have used the values in this txt file to insert.
https://drive.google.com/file/d/1z2ixHDOvHiM5bqDSI3fCbkuXa0Syfjrn/view
Question:
Why is it the if I query this:
select SUM(addn) from sub
Result:
131546008007.610000
and if I paste the result of this in Excel:
select * from sub
the sum is:
131546008007.57
Note:
there are 4 (-0.01) in the query. I don't know if this is trigger and how to solve this
This is topic about Precision, Scale and Length. Numeric datatype is stored/managed in different way than float datatype, for instance.
Try this query, and you will have the same result than in Excel:
select sum(cast(addn as float)) from sub
131546008007.57
Here you have some links where they are explaining that the float datatype is an approximate number, and the decimal is more accurate than the float datatype. So you can see with this than Excel is using approximate numbers.
Precision, Scale, and Length
Here they are explaining than in financial applications you should NOT use floating-point datatypes, so it is good you're using numeric in your DB, and therefore you can rely on your SQL DB in this example.
And here they state that:
Excel was designed in accordance to the IEEE Standard for Binary Floating-Point Arithmetic (IEEE 754). The standard defines how floating-point numbers are stored and calculated. The IEEE 754 standard is widely used because it allows-floating point numbers to be stored in a reasonable amount of space and calculations can occur relatively quickly.
.
Excel store 15 significant digits of precision.
For a SQL int that is being converted to a float, how do I set the precision of the floating point number?
This is the selection I would like to truncate to two or 3 decimal places:
AVG(Cast(e.employee_level as Float))avg_level,
Thanks!
In TSQL, you can specify two different sizes for float, 24 or 53. This will set the precision to 7 or 15 digits respectively.
If all you want to do is truncate to a set number of decimal places, you can use ROUND, ie:
ROUND(AVG(CAST(e.employee_level as float)), 3)
As a general rule, you can't specify the number of digits after the decimal point for a floating-point number. Floating point data types store the closest floating-point approximation to any given value. The closest floating-point approximation is unlikely to have the number of digits you want. Although you might be able to suppress every digit after the third one, that will only change the appearance of the value, not the value itself.
Integers are a different story. An integer--stored, converted, or cast to a floating-point data type--will be stored exactly over a large range. Floating-point data types don't have to store any fractional units for integers.
I'd suggest, though that the best practice for you is to
avoid casting integers to floating-point if you don't need fractional units, or
cast integers to decimal or numeric if you do need fractional units, or
handle display issues entirely in application code.
I have had the same issue when calculating a percentage and needing a resulting string value.
Example: 68 is what % of 379
Result is a float = 17.9419525065900
You can cast/convert to Numeric with the Precision = 2 and get 17.94
If you need the value as a string you can then cast it as a VarChar if needed.
You can use Round() as well but in this case it only makes 17.9419525065900 = 17.9400000000000.
You can also use Ceiling() and Floor() to get the next highest or lowest integer.
Ceiling(17.9419525065900) = 18
Floor(17.9419525065900) = 17
Using these combinations you should be able to achieve a result in any format you need.
Let's say I have the following 2 queries:
select sum(cast(2666 as float)) * cast(.3 as float)
select sum(cast(2666 as real)) * cast(.3 as real)
The 1st query returns: 799.8
The 2nd query returns: 799.800031781197
Why does the 2nd query not return the same thing as the 1st?
Binary floating point types (like real and float) cannot exactly represent decimal numbers. In particular it is not possible to exactly store 0.3 as a binary floating point number. Instead a number very close to 0.3 is stored. This is called a representation error.
The size of the error is different for real and float because they have different precision.
If you want to store decimal numbers more accurately, consider using decimal or numeric. But note that even though these types can accurately store decimal values up to a certain number of digits, calculations can still produce numbers that cannot be represented exactly. For example the result of 0.1 / 0.3 can not be stored exactly in a decimal even though both 0.1 and 0.3 can. In this case the result will be rounded to the nearest value that can be stored in the type (e.g. 0.333333333 depending on the precision).