With this code I keep on getting 3.500000. How do I make it 3.5 with one decimal point? I can get it higher but not lower.
(Select AVG(CONVERT(DECIMAL(12,2),Cats)) from #Catsnumber)
This will work
Select CONVERT(DECIMAL(12,2),AVG(Cats))from #Catsnumber
Use either ROUND or CAST to a precise numeric data type with the precision you want
Related
I have a issue with round, trunc function from BigQuery standard query .
I expected at 3953.7, 3053.67, 3053.667. f1_, f2_ result is different. It is a bug??
I expected at 3.195, 3.195, 3.1955, 3.1965, 3.1945.
f1_, f3_ result is different. Is it my fault?
The ROUND() is used to round a numeric field to the nearest number of decimals specified.
There is a limitation of floating point values.
They can only represent binary values, but cannot precisely represent decimal digits after the decimal point (see here).
In case of SELECT ROUND(3053.665,2) you receive: 3053.66, you can overcome it by using: ROUND(value + 0.005, 2), which allows you to receive 3053.67.
Anyway, if you want to take care about precise decimal results, you should use the NUMERIC type. The following query gives results that you expect:
SELECT ROUND(3953.65,1), ROUND(numeric '3053.665',2), ROUND(numeric '3053.6665',3)
TRUNC(), the following query gives results that you expect:
SELECT TRUNC(3.1955,3), TRUNC(numeric'3.195',3), TRUNC(3.1955,4), TRUNC(numeric '3.1965',4), TRUNC(3.1945,4)
BigQuery parses fractional numbers as floating point by default for better performance, while other databases parses fractional numbers as NUMERIC by default. This means the other databases would interpret TRUNC(3.03,2) in the same way BigQuery interprets TRUNC(numeric '3.03',2).
I hope it will helps you.
This is due to the fact that, in BigQuery, digits are stored as floating point values by default.
You can fin more information about how these work in Wikipedia, but the main idea is that some numbers are not stored as they are but as the closest approximation its representation allows. If instead of 3.03 it is internally represented as 3.0299999999..., when you trunc it the result will be 3.02.
Same thing happens with round function, if 3053.665 is internally stored as 3053.6649999999..., the result of rounding it will be 3053.66.
If you specify it to be stored as NUMERIC, it then works as "expected":
select trunc(numeric '3.195', 3)
gives as result 3.195
You can find more information about Numeric Types in the official BigQuery Documentation.
I'm trying to save decimal value in DB but getting this error.
The value I'm trying to save 0.00038711029163348665 or 1.00038711029163348665 or 2.00038711029163348665, and column in DB is Decimal(18,18) not sure what I'm doing wrong
The precision DECIMAL(18,18) means 18 total units of precision, all 18 of which come after the decimal point. You need at least one unit of precision to the left of the decimal point to handle the whole number components of the input, so you should be using DECIMAL(19,18), or something similar.
See the demo to verify that DECIMAL(19,18) resolves your problem, at least for the data you showed us.
Decimal(18,18) has 18 digits to the right of the decimal place and none to the left, so 1.00038711029163348665 doesn't fit.
Use a larger type. Why not just use decimal(38, 18)?
I have data for pounds and pence stored within concatenated strings (unfortunately no way around this) but can not guarantee 2 decimal places.
E.g. I may get a value of 119.109, so this must translated to 2 decimal places with truncation, i.e. 119.10, NOT 119.11.
For this reason I am avoiding "CAST as Decimal" because I do not want to round. Instead I am using ROUND(amount, 2, 1) to force truncation at 2 decimal places.
This works for the most part but sometimes exhibits strange behaviour. For example, 119.10 outputs as 119.09. This can be replicated as:
ROUND(CAST('119.10' AS varchar),2,1)
My target field is Decimal(19,4) (but the 3rd and 4th decimal places will always be 0, it is a finance system so always pounds and pence...).
I assume the problem is something to do with ROUNDing a varchar....but I don't know anyway around this without having to CAST and therefore introduce rounding that way?
What is happening here?
Any ideas greatly appreciated.
This is due to the way floating point numbers work, and the fact that your string number is implicitly converted to a floating point number before being rounded. In your test case:
ROUND(CAST('119.10' AS varchar),2,1)
You are implicitly converting 119.10 to float so that it can be passed to the ROUND function, 119.10 exactly cannot be stored as a float, proven by running the following:
SELECT CAST(CONVERT(FLOAT, '119.10') AS DECIMAL(30, 20))
Which returns:
119.09999999999999000000
Therefore, when you round this with truncate you get 119.09.
For what it is worth, you should always specify a length when converting to, or declaring a varchar
I need to do some very precise reporting in SQL Server Reporting Services. I'm actually attempting to show 13 decimal places. The odd part is even when I format the field C13, Reporting Services seems to round after an arbitrary number of total digits rather than anything to do with the format string.
For example if I have:
1000.01234567890123
What I end up with is:
1000.0123456789000
If on the other hand I have:
10.01234567890123
What I end up with is:
10.01234567890100
So it appears that I only end up with 15 actual digits from my source number. Has anyone seen this before, or know how to resolve it?
It sounds to me like you are using the Float datatype. Instead, I would suggest you use the decimal data type instead. You'll probably want to use something like Decimal(20, 14). You'll still need to be very careful about the math you perform because SQL Server will modify the resulting data type when you perform math on decimals.
Actually, I found the problem was Microsoft's Double.ToString() method. I had to use a "G13" as a formatting string to get all of the decimal places. Go figure.
I am working on a legacy ASP application. I am attempting to insert a value (40.33) into a field in SQL Server 2000 that happens to be a float type. Every place I can see (via some logging) in the application is sending 40.33 to the Stored Procedure. When I run SQL Profiler against the database while the call is happening, the value that I see in the trace is 4.033000183105469e+001
Where is all the extra garbage coming from (the 183105469)?
Why is it that when I pass in 40, or 40.25 there is nothing extra?
Is this just one of the weird side effects of using float? When I am writing something I normally use money or decimal or something else, so not that familiar with the float datatype.
Yes, this is a weird, although well-known, side effect of using FLOAT.
In Microsoft SQL Server, you should use exact numeric datatypes such as NUMERIC, DECIMAL, MONEY or SMALLMONEY if you need exact numerics with scale.
Do not use FLOAT.
I think this is probably just a precision issue - the 0.33 part of the number can't be represented exactly in binary - this is probably the closest that you can get to.
The problem is that floats are not 100% accurate. If you need your numbers to be exact (especially when dealing with monetary values)... you should use a Decimal type.