I have this formula in SQL server:
((FP.Month1/30) * (30 - DAY(GETDATE()))) + FP.month2 + ((FP.month3/30) * (Day(GETDATE())))
I have been testing out results and they are coming out slightly wrong. So I broke it down to the minimal amount, this:
(FP.Month1/30)
and when I run that, it comes out wrong but the reason this comes out wrong is it is ignoring the decimals. Month1 is 140 so if you divide that by 30, it is supposed to come out as 4.666667 but it is coming out as 4 instead. So that will of course result in the total formula resulting in a wrong result. How do I make sure SQL server uses decimals also.
FYI, I don't know if this effects it but the final result needs to be rounded to a whole number.
SQL server does integer division. So instead of 30, use 30.0:
(FP.Month1 / 30.0)
As a note, 30.4 is a better estimate of the number of days in a month.
Related
The code below calculates the conversion rate of a dataset. The code relevant to this question is in line 13. When I calculate the conversion rate, I divide the total number of purchases made on the website by the total number of users (people who browse) on the website. The output I get is 0.495 but I don't understand why I need the '1.0 *' at the start of line 13 for this to work? I don't know the purpose of this part of the code, but without it the code doesn't work.
Your code is not MySQL code. MySQL does not support square braces around identifiers. One database that does is SQL Server. And it does integer division. So, in SQL Server, 1/2 is 0 rather than 0.5.
The * 1.0 simply converts the integer to something with a decimal point.
Assuming userid is not NULL, this is more easily expressed as:
avg( is_purchase * 1.0 )
forpas and Gordon Linoff are correct that SQL Server performs integer division by default. In future, and for more complex calculations, you can use CTEs or subqueries employing CAST to represent values as floating point values prior to division. E.g., at lines 6-7 you would have:
CAST(p.userid IS NOT NULL AS float) AS is_purchase
SQL keeps rounding the results of my calculations and I do not seem to find an article on how to fix this.
I am trying to do calculations within SQL. This to BI self service easier.
SELECT
COUNT(CONTRACTID) / (SELECT COUNT(RENTALOBJECTID) FROM PMCCONTRACTOBJECT) AS result
FROM PMCCONTRACT
The result I am getting is 3 while I expect to get 3.2973.
Wow sorry, i added the wrong code.
Should make more sense now
since you are getting 3, you possibly allowed an integer division when you didn't mean to - always cast to float or suitable Decimal before dividing - or use suitable data types in your calculation
see the following example as an illustration of what happens
select 10/3, cast(10 as float) / cast(3 as float)
Well i gave little information i was able to find the awnser
SELECT cast (count(CONTRACTID) as decimal) / (SELECT count(RENTALOBJECTID)
FROM PMCCONTRACTOBJECT) AS result FROM PMCCONTRACT
I have an Ingres table with following columns
from_date ingresdate
to_date ingresdate
model_amt money
The dates can reflect a period of any number of days, and the model_amt is always a weekly figure. I need to work out the the total model_amt for the period
To do this I need to know how many days are covered by the period, and then divide model_amt by 7, and multiply it by the number of days
however, I am getting incorrect results using the code below
select model_amt, date_part('day',b.to_date - b.from_date),
model_amt / 7 * int4( (date_part('day',b.to_date - b.from_date)) )
from table
For example, where model_amt = 88.82 and the period is for 2 weeks, I get the following output
+-------------------------------------------------------+
¦model_amt ¦col2 ¦col3 ¦
+--------------------+-------------+--------------------¦
¦ #88.82¦ 14¦ #177.66¦
+-------------------------------------------------------+
But 88.82 / 7 * 14 = 177.64, not 177.66?
Any ideas what is going on? The same issue happens regardless of whether I include the int4 function around the date_part.
* Update 15:28 *
The solution was to add a float8 function around the model_amt
float8(model_amt)/ 7 * interval('days', to_date - from_date)
Thanks for the responses.
In computers, floating point numbers are notoriously inaccurate. You can multiply do all kinds of basic mathematics calculations on floating point numbers and they'll be off by a few decimals.
Some information can be found here; but its very googleable :). http://effbot.org/pyfaq/why-are-floating-point-calculations-so-inaccurate.htm
Generally to avoid inaccuracies, you need to use a language specific feature (e.g. BigDecimal in Java) to "perfectly" store the decimals. Alternatively, you can represent decimals as separate integers (e.g. main number is one integer and the decimal is another integer) and combine them later.
So, I suspect this is just ingres showing the normal floating point inaccuracies and that there are known workarounds for it in that database.
Update
Here's a support article from Actian specifically about ingres floating point issues which seems useful: https://communities.actian.com/s/article/Floating-Point-Numbers-Causes-of-Imprecision.
I've got a simple calculation (910 / 28 = 3.5) and I'm trying to perform this in a SQL query:
SELECT CONVERT(DECIMAL(5,2), (910 / 28),2) AS average
But the answer is coming out at 32.00, I'm obviously missing something simple could someone spare a moment to point out my error please?
Thanks,
C
Use this:
SELECT CONVERT(DECIMAL(5,2), (910.0 / 28)) AS average
By taking the quotient as 910.0 / 28 SQL Server will retain decimal precision. Then, make your cast to a decimal with two places. By the way, as far as I know CONVERT typically takes just two parameters when converting a number to decimal.
we can use this query for dynamic value from table:
SELECT CONVERT(DECIMAL(5,2), (cast (910 as decimal))/ 28) AS average
It will give the desire output
Unsure if this applies to your database, but in Trino SQL (a sort of database middleware layer), I find adding a decimal point followed by two zeros to any of two operands in this query (e.g., select 910.00/23 AS average or select 910/23.00 AS average) returns a non-integer value (39.57, instead of 39).
Adding 3 zeros after the decimal (select 910.000/23 AS average) returns a 3-decimal place result (39.565), and so on.
Try this query...
SELECT CAST(910 AS decimal(5,2)) / CAST(28 AS decimal(5,2)) as average
Try use this
select cast(round(910/28.0,2) as numeric(36,2)) AS average
This question already has answers here:
SQL Server Strange Ceiling() behavior
(2 answers)
Closed 8 years ago.
I've found a really weird behavior in SQL Server 2012, the CEILING of 100 gives me 101 and sometimes 100.
I need to get the ceiling of a number considering 2 decimals, that means convert a 0.254 to 0.26
So I tried to run
SELECT CEILING(field * 100.0) / 100.0
FROM Table
This should work, and it does, at least for most of the data.
Any idea on how to solve it?
What you are seeing here is floating point errors. When you store a number in a floating point column, it isn't exact, so the number 1 may actually be 1.0000000000000000000000001. So multiplying it by 100 gives you a number a tiny bit greater than 100, hence CEILING rounds it up to 101.
The solution is to ROUND the number first which will remove the floating point errors. Note I have used 5 as the number of decimal places, you will need to decide on your own value of precision.
SELECT CEILING(ROUND(field,5)*100.0)/100.0 FROM Table