The code below calculates the conversion rate of a dataset. The code relevant to this question is in line 13. When I calculate the conversion rate, I divide the total number of purchases made on the website by the total number of users (people who browse) on the website. The output I get is 0.495 but I don't understand why I need the '1.0 *' at the start of line 13 for this to work? I don't know the purpose of this part of the code, but without it the code doesn't work.
Your code is not MySQL code. MySQL does not support square braces around identifiers. One database that does is SQL Server. And it does integer division. So, in SQL Server, 1/2 is 0 rather than 0.5.
The * 1.0 simply converts the integer to something with a decimal point.
Assuming userid is not NULL, this is more easily expressed as:
avg( is_purchase * 1.0 )
forpas and Gordon Linoff are correct that SQL Server performs integer division by default. In future, and for more complex calculations, you can use CTEs or subqueries employing CAST to represent values as floating point values prior to division. E.g., at lines 6-7 you would have:
CAST(p.userid IS NOT NULL AS float) AS is_purchase
Related
I have an Ingres table with following columns
from_date ingresdate
to_date ingresdate
model_amt money
The dates can reflect a period of any number of days, and the model_amt is always a weekly figure. I need to work out the the total model_amt for the period
To do this I need to know how many days are covered by the period, and then divide model_amt by 7, and multiply it by the number of days
however, I am getting incorrect results using the code below
select model_amt, date_part('day',b.to_date - b.from_date),
model_amt / 7 * int4( (date_part('day',b.to_date - b.from_date)) )
from table
For example, where model_amt = 88.82 and the period is for 2 weeks, I get the following output
+-------------------------------------------------------+
¦model_amt ¦col2 ¦col3 ¦
+--------------------+-------------+--------------------¦
¦ #88.82¦ 14¦ #177.66¦
+-------------------------------------------------------+
But 88.82 / 7 * 14 = 177.64, not 177.66?
Any ideas what is going on? The same issue happens regardless of whether I include the int4 function around the date_part.
* Update 15:28 *
The solution was to add a float8 function around the model_amt
float8(model_amt)/ 7 * interval('days', to_date - from_date)
Thanks for the responses.
In computers, floating point numbers are notoriously inaccurate. You can multiply do all kinds of basic mathematics calculations on floating point numbers and they'll be off by a few decimals.
Some information can be found here; but its very googleable :). http://effbot.org/pyfaq/why-are-floating-point-calculations-so-inaccurate.htm
Generally to avoid inaccuracies, you need to use a language specific feature (e.g. BigDecimal in Java) to "perfectly" store the decimals. Alternatively, you can represent decimals as separate integers (e.g. main number is one integer and the decimal is another integer) and combine them later.
So, I suspect this is just ingres showing the normal floating point inaccuracies and that there are known workarounds for it in that database.
Update
Here's a support article from Actian specifically about ingres floating point issues which seems useful: https://communities.actian.com/s/article/Floating-Point-Numbers-Causes-of-Imprecision.
I've got a simple calculation (910 / 28 = 3.5) and I'm trying to perform this in a SQL query:
SELECT CONVERT(DECIMAL(5,2), (910 / 28),2) AS average
But the answer is coming out at 32.00, I'm obviously missing something simple could someone spare a moment to point out my error please?
Thanks,
C
Use this:
SELECT CONVERT(DECIMAL(5,2), (910.0 / 28)) AS average
By taking the quotient as 910.0 / 28 SQL Server will retain decimal precision. Then, make your cast to a decimal with two places. By the way, as far as I know CONVERT typically takes just two parameters when converting a number to decimal.
we can use this query for dynamic value from table:
SELECT CONVERT(DECIMAL(5,2), (cast (910 as decimal))/ 28) AS average
It will give the desire output
Unsure if this applies to your database, but in Trino SQL (a sort of database middleware layer), I find adding a decimal point followed by two zeros to any of two operands in this query (e.g., select 910.00/23 AS average or select 910/23.00 AS average) returns a non-integer value (39.57, instead of 39).
Adding 3 zeros after the decimal (select 910.000/23 AS average) returns a 3-decimal place result (39.565), and so on.
Try this query...
SELECT CAST(910 AS decimal(5,2)) / CAST(28 AS decimal(5,2)) as average
Try use this
select cast(round(910/28.0,2) as numeric(36,2)) AS average
I have the following sql query that I am running:
select sum(cast(mkt_value as decimal(20,7)))/1204438043.37 from table1
and I get the following result which is correct but I need at least 10 decimal places not 6:
0.009347
If I run the following query:
select sum(mkt_value) from table1
I get 11258490.2400.
If I divide 11258490.24 by 1204438043.37 in excel I get 0.009347504674 which is the answer I'm looking for.
Please help me correct my SQL!!
Your cast is breaking this. It doesn't have enough space to give you more than six decimal places. What you're saying is literally "give me the result of this division with at most six decimal places", and then you're suprised the result only has six decimal places :)
The solution is either to omit the cast (if the data type is money, it's fine) or increase the scale of the decimal, eg. decimal(20, 11). The second parameter of the decimal type says the maximal amount of decimal places (-1) in the number. Also, consider only casting the result of the sum instead of all the items.
Note that most operations in MS SQL return a value of the same data type. So:
select 3 / 4; -- Returns 0
select cast(1000 as smallint) * cast(1000 as smallint);
-- Error - 1 000 000 is too big for smallint
The same thing happens in your sum and also in the division that happens right after it :)
You need to apply casting after calculating final value
select CAST (sum(mkt_value)/1204438043.37 as decimal(15,10)) from table1.
This means result would be having maximum 15 digits and mandatory 10 decimal places.
I would suggest the following query,Since your final decimal places are getting truncated.Hence a cast would be required to your final result set.
select
cast(cast (sum(mkt_value) as decimal(20,7))/cast (1204438043.37 as decimal(20,7)) as decimal(20,12)) from table1
Thank you..Let me know if this works
I am trying to do a select count(*) from table from a table and I am not able to do it because of this error. I am not aware of the number of rows in the table.
I am not doing any other aggregation in my query apart from this.
I guess it has something to do with the count value which is too large to be stored in INTEGER.
What is the alternative?
When your session runs in Teradata mode the result of a COUNT is INTEGER as you already noticed (in ANSI mode it will be a DECIMAL with at least 15 digits).
The workaround is simple, cast it to a bigint:
SELECT CAST(COUNT(*) AS BIGINT)...
This is the #2 Google hit for Teradata 2616, so I want to add something. If you're getting 2616 "Numeric overflow occured" from a SUM in Teradata, the solution is to CAST, then SUM. The CAST has to be inside the SUM:
SELECT
SUM(CAST(WHATEVER_QTY AS DECIMAL(38,0)))
FROM
TER_DATABASE.WHATEVER_TABLE ;
In my case, DECIMAL(38,0) worked, but BIGINT was 2616. You are welcome to experiment. Here's the link at info.teradata:
http://www.info.teradata.com/HTMLPubs/DB_TTU_14_00/index.html#page/SQL_Reference/B035_1145_111A/Data_Type_Conversions.098.297.html
I know when you work with Money it's better (if not imperative) to use Decimal data type, especially when you work with Large Amount of Money :). But I want to store price of my products as less memory demanding float numbers because they don't really need such a precision. Now when i want to calculate the whole Income of the products sold, it could become a very large number and it must have great precision too. I want to know what would be the result if I do this summation by SUM keyword in a SQL query. I guess it will be stored in a Double variable and this surely lose some precision. How can I force it to do calculation using Decimal numbers? Perhaps someone who knows about the internals of SQL engines could answer my question. It's good to mention that I use Access Database Engine, but any general answer would be appreciated too. This might be an example of the query I would use:
SELECT SUM(Price * Qty) FROM Invoices
or
SELECT SUM(Amount) FROM Invoices
Amount and Price are stored as float(Single) data type and Qty as int32.
If you want to do the calculation as a double, then cast one of the values to that type:
SELECT SUM(cast(Price as double) * Qty)
FROM Invoices;
SELECT SUM(cast(Amount as double))
FROM Invoices;
real double precision
Note that naming is not consistent among databases. For instance "binary_float" is 5 bytes (based on IEEE 4-byte float) and "binary_double" is 9 bytes (based on IEEE 8-bytes double). But, "float" is 8-bytes in SQL Server, but 4-byte in MySQL. SQL Server and Postgres use "real" for the 4-byte version. MySQL and Postgres use "double" for the 8-byte version.
EDIT:
After writing this, I saw the reference to Access in the question (this should really be a tag). In Access, you would use cdbl() instead of cast():
SELECT SUM(cdbl(Price) * Qty)
FROM Invoices;
SELECT SUM(cdbl(Amount))
FROM Invoices;
If the choice is between a float and a 4-byte (unsigned) int (both requiring the same amount of storage in memory) there are pros and cons:
The float cannot accurately handle cents assuming that a price has
the format $$$$$.cc (1/100ths are not precisely representable in the
floating-point - single as well as double - format), so this will
introduce rounding errors which are usually unacceptable in
money-related applications.
The int - assuming that you express the price in cents - will allow
precise values in the range -2^31 to 2^31-2^0 (about 2 * 10^9) cents for signed values
and 0 to 2^32-2^0 (about 4 * 10^9) for unsigned. The downside is that it may feel
"unnatural" to use cents instead of dollars and cents but this is
mostly a problem inside the developers mind: the actual "problems" -
if you wish to call them that - arise when printing the values in
dollars and cents which require a slightly more complex formatting
but this is a very small price in relation to how the rest of the
application can be simplified.
Later, when summing or performing other calculations - the integer cent and quantity values are first converted to double precision floating-point. The double precision format allows expressing integer values (assuming integer cents) precisiely in the range -(2^53-2^0) to 2^53-2^0 which probably (you need to check) covers your needs. Keep in mind, though, that you will still have integer cents in the double which need to be converted to dollars and cents.
EDIT_______________________
"6-7 decimal digits of precision" is most easily explained by the range of integers representable in the single-precision format. Since the SP format significand is 24 bits long (1 implicit + 23 explicit) this allows integers in the range 2^0 to 2^24-2^0 or 1 to 16777215. 16777215 is more than six (999999) but less than seven (9999999) decimal digits, hence "6-7 decimal digits." The double-precision format features a 53 bit significand (1 + 52) which results in an integer range of 2^0 to 2^53-2^0.
The real SP precision is "24 sequential binary digits of precision."
If you can make do with cents in 50 unit increments your range in SP will be 2^-1 to 2^23-2^-1 or 0.5 to 8388607.5
If you can make do with cents in 25 unit increments your range in SP will be 2^-2 to 2^22-2^-2 or 0.25 to 4194303.75.
Actually, as #Phylogenesis said in the first comment, when I think about, we don't sell enough items to overflow the precision on a double value, just like items are not expensive enough to overflow the precision on a float value.As I guessed, I tested and found that if you run simple SELECT SUM(Amount) FROM Invoices query, the result will be a double value. But following what suggested by #Gordon Linoff, the safest approach for obsessive-compulsive people is to use a cast to Decimal or Currency(Access). So the query in Access syntax will be:
SELECT SUM(CCur(Price) * Qty)
FROM Invoices;
SELECT SUM(CCur(Amount))
FROM Invoices;
which CCur function converts Single(c# float) values to Currency(c# decimal). Its good to know that conversion to Double is not necessary, because the engine does it itself. So the easier approach which is also safe is to just run the simple query.