Considering the taxable amount (rateable), the number of days and the rate, I must calculate the interest applied (with the compound rate formula) but I have a problem with floating-point and rounding applied by DB2 V12 (Z/OS).
(Note: The same statement on DB2 LUW does not give me problems)
The value of the interest that I expect is 17,84€ instead I get 17,86€
I'm using this statement:
SELECT CAST(CAST(RATEABLE AS DECFLOAT)
* ( 1 - ( POWER ( ( 1 + CAST(RATE AS DECFLOAT) / 100 ),
( -1 * CAST(NUMBER_DAYS AS DECFLOAT) / CAST(DIVISOR AS DECFLOAT) )
)
)
) AS DECIMAL(18, 2)
) AS PAYMENT_INTEREST
FROM (
--- I simulate accessing my DB2 table.
SELECT CAST(92247.38 AS DECIMAL(18, 2)) AS RATEABLE,
CAST(0.249000 AS DECIMAL(12, 6)) AS RATE,
INTEGER(28) AS NUMBER_DAYS,
INTEGER(360) AS DIVISOR
FROM SYSIBM.SYSDUMMY1
) AS TEMP
If I defined the "RATE" field as DEC (12,3) the calculation is correct but obviously I would not be able to manage more rates with more decimals.
Now, what am I missing here?
Thanks.
Don’t use float datatypes as they will give you this issue. Just define your fields as decimals with enough decimal points to hold the data that you need them to hold. You can always round them in your queries if you need show them with reduced precision
Related
I have a table in PostgreSQL, that have a Float column. In my select I use AVG() on that column, so often it gives a number with many decimals. Is there any way to retrict the number of decimals to a maximum of 3, meaning there can be less but not more than 3.
This is the Query:
SELECT team, AVG(score) FROM team_score_table GROUP BY team
You can use round():
select round(val::numeric, 3)
You can also convert to a numeric, but you need a precision appropriate for your values:
select val::numeric(20, 3)
I actually prefer the explicit cast() because it sets the data type of the column to a numeric with an explicit scale -- so downstream apps are aware of the number of decimal places intended in the result.
round() returns a numeric value but it is a "generic" numeric, with no specified scale and precision.
You can see the difference in this example.
You can use a several functions to do that:
SELECT round(42.43666, 2) -- 42.44
SELECT trunc(42.43666, 2) -- 42.43
or cast:
SELECT cast(42.43666 as numeric(20, 2)) -- 42.44
according to your example should be:
SELECT team, round(AVG(score)::numeric, 2) FROM team_score_table GROUP BY team
SELECT team, trunc(AVG(score)::numeric, 2) FROM team_score_table GROUP BY team
SELECT team, cast(AVG(score) as numeric(20,2)) FROM team_score_table GROUP BY team
I am currently running a query that runs a sum function and also divides this number. Currently I get values like 0.0904246741698848, and 1.6419814808335567. I want these decimals to be trimmed to 2 spaces past the decimal point. Their schema is a float. Here is my code. Thanks for the help.
#standardSQL
SELECT
Serial,
MAX(createdAt) AS Latest_Use,
SUM(ConnectionTime/3600) as Total_Hours,
COUNT(DISTINCT DeviceID) AS Devices_Connected
FROM `dataworks-356fa.FirebaseArchive.Firebase_ConnectionInfo`
WHERE PeripheralType = 1 or PeripheralType = 2 or PeripheralType = 12
GROUP BY Serial
ORDER BY Latest_Use DESC
#standardSQL
WITH `data` AS (
SELECT 0.0904246741698848 AS val UNION ALL
SELECT 1.6419814808335567
)
SELECT val, ROUND(val, 2) AS rounded_val
FROM `data`
for example, assuming your want apply this to your Total_Hours column :
#standardSQL
SELECT
Serial,
MAX(createdAt) AS Latest_Use,
ROUND(SUM(ConnectionTime/3600),2) AS Total_Hours,
COUNT(DISTINCT DeviceID) AS Devices_Connected
FROM `dataworks-356fa.FirebaseArchive.Firebase_ConnectionInfo`
WHERE PeripheralType = 1 OR PeripheralType = 2 OR PeripheralType = 12
GROUP BY Serial
ORDER BY Latest_Use DESC
I found that rounding was problematic if my data had a whole number such as 2.00 and I needed all of my data to reflect 2 decimal places as these were for prices that end up getting displayed. Big Query was returning 2.0 no matter what I specified to round to using ROUND.
Assuming you're working with data that never surpasses 2 decimal places, and it is stored as a STRING, this code will work (if it's more decimal places, add another 0 to the addition for each space).
FORMAT("%.*f",2,CAST(GROSS_SALES_AMT AS FLOAT64) + .0001)
This will take a float in BigQuery and format it with two decimal points.
CAST(SUM(ConnectionTime/3600) AS STRING FORMAT '999,999.99')
Note: Add a a currency symbol (e.g., $) for currency ($999,999.99).
Example:
You can always use the round() function.
If you are looking for precision after decimal (as using round will round-off the values) you can use substr(str(value),precision) which will give exact output after decimal.
I'm trying to understand the repercussions of having a column with real data type in a database(Sql Server) table, in my scenario. I have a column in my database with real datatype. The data is static, its always one of the values in range 0.0, 0.1, 0.2.. to 1.0.
Use Case:
I have to Sum up the values in the column and use the sum value in some arithmetic calculations which include financial data.
Concern?:
When I sum up values in the column it gives me result with more decimal places.
Test:
I want sum up the column values and use it in multiplication and division calculations.
I want to repeat #1 for same set of values in a column with decimal and float data types.
Procedure: I have created three different tables with a single column and same set of values but different data types, one with decimal, float and real. And perform the arithmetic calculations on each of them.
CREATE TABLE #tReal(d real);
INSERT INTO #tReal
SELECT 0.1 UNION ALL
SELECT 1 UNION ALL
SELECT 1 UNION ALL
SELECT 0.9 UNION ALL
SELECT 1 UNION ALL
SELECT 0.9 UNION ALL
SELECT 1;
select SUM(d) from #tReal
Expected Result : 5.9
Actual Result : 5.8999999538064
And, If I perform round operation on the sum the result is as expected
declare #sumofd real
select #sumofd = SUM(d) from #tReal
select round(#sumofd , 1)
Result: 5.9
Also, if I update datatype from real to float
CREATE TABLE #tfloat(d float);
INSERT INTO #tfloat
SELECT 0.1 UNION ALL
SELECT 1 UNION ALL
SELECT 1 UNION ALL
SELECT 0.9 UNION ALL
SELECT 1 UNION ALL
SELECT 0.9 UNION ALL
SELECT 1;
select SUM(d) from #tfloat
Expected Result : 5.9
Actual Result : 5.9
And, This is the same case if I update datatype from real to decimal
CREATE TABLE #tDecimal(d DECIMAL(3,2));
INSERT INTO #tDecimal
SELECT 0.1 UNION ALL
SELECT 1 UNION ALL
SELECT 1 UNION ALL
SELECT 0.9 UNION ALL
SELECT 1 UNION ALL
SELECT 0.9 UNION ALL
SELECT 1;
select SUM(d) from #tDecimal
Expected Result : 5.94
Actual Result : 5.94
And, If I perform some basic arithmetic operations on the sum, without rounding, like
declare #sumofdReal real
declare #sumofdFloat float
declare #sumofdDecimal decimal(3,2)
select #sumofdReal = SUM(d) from #tReal
select #sumofdFloat = SUM(d) from #tfloat
select #sumofdDecimal = SUM(d) from #tDecimal
Multiplication:
select #sumofdReal * 2
Result: 11.8
select #sumofdFloat * 2
Result: 11.8
select #sumofdDecimal * 2
Result: 11.88
Division:
select #sumofdReal / 2
Result: 2.95
select #sumofdFloat / 2
Result: 2.95
select #sumofdDecimal / 2
Result: 2.97000
Drop Tables:
drop table #tReal
drop table #tfloat
drop table #tDecimal
Following are my questions
In my scenario, having a datatype as real will have any repercussions?
If #1 is yes, what kind of repercussions? Do I have to change datatype to float or decimal? Why?
If #1 is No, Why? Please explain?
Is there a point to which float and real types can produce exact result and anything beyond it rounding errors will show up?
real in SQL Server is a synonym for float(24), which takes 4 bytes and up to 7 digits of precision.
float by itself is the same as float(53) which is also the same as double which takes up 8 bytes and has up to 15 digits of precision.
In my scenario, having a datatype as real will have any repercussions?
Possibly. decimal, float, and real have different characteristics and are not fully interchangeable. If you want to maintain the exact decimal representation, use the decimal datatype with the appropriate scale and precision. If your numbers represent imprecise data (such as temperature, height, time, or other "natural" measurements that cannot be measured without some imprecision) and want faster mathematical operations then use float (or real if you don;t need more than 7 digits of precision.
When I sum up values in the column it gives me result with more decimal places.
When you add floating point numbers, SQL determines how big the result can be and will use the appropriate data type. When you add two reals, the result could have more than 7 digits of precision, so it may use float as the resulting data type.
Do I have to change datatype to float or decimal? Why?
Use float if you may have more than 7 digits of precision, don't need absolute precision from a decimal standpoint, and want to use a floating-point type for faster calculations.
If you want a fixed scale and precision and want to minimize the imprecision involved with floating point numbers use decimal.
Working with floating-point values always has repercussions, because they are not represented exactly and some rounding errors will always occur. You have to be most careful when comparing values with constants, e.g. if (a == 0.0) as this won't always work if 'a' is the result of mathematical operations. Float and Decimal just give you different ranges and precisions.
If you are using these values in financial calculations, then you should always (pretty much) use decimal types, otherwise you will run into issues with floating point rounding at some point. People get very wound up when you start losing pennies.
You may want to read some more on floating point arithmetic here:
http://floating-point-gui.de/
Decimal types will not be subject to these and should be exact.
For an example, you will eventually run into the situation where you try and round a number to 2 DP, e.g. 0.695, which you would expect to round up to 0.70.
DECLARE #x REAL = 0.695
SELECT ROUND(#x, 2)
-- outputs 0.69
DECLARE #y DECIMAL(6,4) = 0.695
SELECT ROUND(#y, 2)
-- outputs 0.7000
This is down to the fact that you cannot accurately represent 0.695 as a floating point number and it is in fact marginally less than 0.695 as saved.
I have a problem with sql(maths).
I have a total payable given to vendor which is 33.333, and I need to divide the amount with two users. So I have select
select (16.666 * 2) from dual which gives me 33.332 that is .1 less than the total amount I have to give.
If I have this sql
select (16.667 * 2) from dual, then it gives me 33.334 which .1 greater than 33.333.
How can I divide the total amount which I could equally distribute?
Thanks
I'm not sure from where are you executing your query, but it works here (SQLDeveloper, 10g):
SELECT (33.333 / 2) FROM dual;
16,6665
SELECT (16.6665 * 2) FROM dual;
33,333
Do it the other way around:
select 33.333/2
You are most likely working with the wrong column type. You should be using DECIMAL instead of e.g. FLOAT.
Here is a good summary: http://lists.mysql.com/mysql/189592
Depending on the SQL standard you are using the type can be MONEY, DECIMAL or NUMBER.
I have a series of calculation times in a DB2 SQL DB that are stored as float with a default value of 0.0.
The table being updated is as follows:
CREATE TABLE MY_CALC_DATA_TABLE
(
CALCDATE TIMESTAMP,
INDIV_CALC_DURATION_IN_S FLOAT WITH DEFAULT 0.0,
CALC_TIME_PERCENTAGE FLOAT WITH DEFAULT 0.0
)
Using a sproc. I am calculating the sum as follows:
CREATE OR REPLACE PROCEDURE MY_SCHEMA.MY_SPROC (IN P_DATE TIMESTAMP)
LANGUAGE SQL
NO EXTERNAL ACTION
BEGIN
DECLARE V_TOTAL_CALC_TIME_IN_S FLOAT DEFAULT 0.0;
-- other stuff setting up and joining data
-- Calculate the total time taken to perform the
-- individual calculations
SET V_TOTAL_CALC_TIME_IN_S =
(
SELECT
SUM(C.INDIV_CALC_DURATION_IN_S)
FROM
MY_SCHEMA.MY_CALC_DATA_TABLE C
WHERE
C.CALCDATE = P_DATE
)
-- Now calculate each individual calculation's percentage
-- of the toal time.
UPDATE
MY_SCHEMA.MY_CALC_DATA_TABLE C
SET
C.CALC_TIME_PERCENTAGE =
(C.INDIV_CALC_DURATION_IN_S / V_TOTAL_CALC_TIME_IN_S) * 100
WHERE
C.CALCDATE = P_DATE;
END#
Trouble is, when I do a sum of all the CALC_TIME_PERCENTAGE values for the specified CALC_DATE it is always less than 100% with the sum being values like 80% or 70% for different CALC_DATES.
We are talking between 35k and 55k calculations here with the maximum individual calculation's percentage of the total, as calculated above, being 11% and lots of calculations in the 0.00000N% range.
To calculate the total percentage I am using the simple query:
SELECT
SUM(C.CALC_TIME_PERCENTAGE)
FROM
MY_SCHEMA.MY_CALC_DATA_TABLE C
WHERE
C.CALCDATE = P_DATE;
Any suggestions?
Update: Rearranging the calc. as suggested fixed the problem. Thanks. BTW In DB2 FLOAT and DOUBLE are the same type. And now to read that suggested paper on floats.
If the field C.INDIV_CALC_DURATION_IN_S were Integer, I would assume it's a rounding error. Reading again, that is not the problem as the datatype is FLOAT.
You can still try using this. I wouldn't be surprised if this yielded (slighly) different results than the previous method:
SET
C.CALC_TIME_PERCENTAGE =
(C.INDIV_CALC_DURATION_IN_S * 100.0 / V_TOTAL_CALC_TIME_IN_S)
But you mention that there a lot of rows in a calculation for a certain date, so it may be a rounding error due to that. Try with DOUBLE datatype in both fields (or at least the CALC_TIME_PERCENTAGE field) and see if the difference from 100% gets smaller.
I'm not sure if DB2 has DECIMAL(x,y) datatype. It may be more appropriate in this case.
Another problem is how you find the sum of CALC_TIME_PERCENTAGE. I suppose you (and everyone else) would use the:
SELECT
P_DATE, SUM(CALC_TIME_PERCENTAGE)
FROM
MY_SCHEMA.MY_CALC_DATA_TABLE C
GROUP BY P_DATE
This way, you have no way to determine in what order the summation will be done. It may not be even possible to determine that but you can try:
SELECT
P_DATE, SUM(CALC_TIME_PERCENTAGE)
FROM
( SELECT
P_DATE, CALC_TIME_PERCENTAGE
FROM
MY_SCHEMA.MY_CALC_DATA_TABLE C
ORDER BY P_DATE
, CALC_TIME_PERCENTAGE ASC
) AS tmp
GROUP BY P_DATE
The optimizer may disregard the interior ORDER BY but it's worth a shot.
Another possibility for this big difference is that rows are deleted from the table between the UPDATE and the SHOW percent SUM operations.
You can test if that happens by running the calculations (without UPDATE) and summing up:
SELECT
P_DATE
, SUM( INDIV_CALC_DURATION_IN_S * 100.0 / T.TOTAL )
AS PERCENT_SUM
FROM
MY_SCHEMA.MY_CALC_DATA_TABLE C
, ( SELECT SUM(INDIV_CALC_DURATION_IN_S) AS TOTAL
FROM MY_SCHEMA.MY_CALC_DATA_TABLE
) AS TMP
GROUP BY P_DATE
Might be a rounding problem. Try C.INDIV_CALC_DURATION_IN_S * 100 / V_TOTAL_CALC_TIME_IN_S instead.
If C.INDIV_CALC_DURATION_IN_S is very small but you have a large number of rows (and thus V_TOTAL_CALC_TIME_IN_S becomes large in comparison) then
(C.INDIV_CALC_DURATION_IN_S / V_TOTAL_CALC_TIME_IN_S) * 100
is likely to lose precision, especially if you're using FLOATs.
If this is the case, then changing the calculation (as mentioned elsewhere) to
(C.INDIV_CALC_DURATION_IN_S * 100) / V_TOTAL_CALC_TIME_IN_S
should increase the total, although it may not get you all the way to 100%
If that's the case and a lot of the measurements are small fractions of a second, I'd consider looking beyond this procedure: could the times be recorded in, say, milli- or micro-seconds? Either would give you some headroom for additional significant digits.