I have a Stored Procedure that does a lots of calculation, stores the results in several temporary table.
Finally calculating the sum and rounding to two decimal and stores in a temporary table and selects that.
All the intermediate and final temporary table has datatype float for the column of concern.
original Scenario:
Declare #Intermediate table
{
--several other columns
Labor float
--several other columns
};
---Lots of calculation ---xx-----
Declare #Final table
{
--several other columns
LaborTotal float
--several other columns
};
INSERT INTO #Final SELECT ROUND(ISNULL((SELECT SUM([Labor]) FROM #Intermediate ),0),2) AS LaborTotal;
SELECT * FROM #Final;
Result: 7585.22 --> when rounded //Here is the error Expecting 7585.23
7585.225 --> when not rounded
TestCases :
DECLARE #test float = 7585.225;
SELECT ROUND(#test,2) AS Result; --> results 7585.23
SELECT ROUND(7585.225,2) AS Result --> results 7585.23
Inserted individual values to a temporary table, and then calculated the sum
DECLARE #TmpTable table
(
MaterialAmount float
,LaborAmount float
);
INSERT INTO #TmpTable VALUES (12.10,1218.75);
INSERT INTO #TmpTable VALUES (12.10,1090.125);
INSERT INTO #TmpTable VALUES (12.10,900);
INSERT INTO #TmpTable VALUES (12.10,1632.6);
INSERT INTO #TmpTable VALUES (12.10,1625);
INSERT INTO #TmpTable VALUES (12.10,1118.75);
SELECT ROUND(ISNULL((SELECT SUM(MaterialAmount) FROM #TmpTable), 0),2) AS MatSum,
ISNULL((SELECT SUM(LaborAmount) FROM #TmpTable), 0) AS LabSumUnrounded, --> 7585.225
ROUND(ISNULL((SELECT SUM(LaborAmount) FROM #TmpTable), 0),2) AS LabSum; --> 7585.23
SELECT ROUND(SUM(MaterialAmount),2),
ROUND(SUM(LaborAmount),2) ---> 7585.23
FROM #TmpTable;
Any idea/suggestion why i am getting 0.01 difference in my original scenario, while getting exact values in all my testcases ?
Thanks in advance.
This is because you are using float database type.
Float should not be used to represent values that require precision, since they are stored as approximations, different manipulations can give you different results.
In sql server you can use decimal and numeric data types for numerical precision:
http://msdn.microsoft.com/en-us/library/ms187746.aspx
Try this way:
SELECT ROUND(#test,2) AS Result; --> results 7585.23
SELECT ROUND(convert(float,7585.225),2) AS Result --> results 7585.23
When you store the value as float it stores the approximate value but not exact value, when you want to store exact value use Decimal, money or small money data type. Here in your example when I converted the numeric value to float, it stored the approximate value of number.
http://msdn.microsoft.com/en-us/library/ms187912%28v=sql.105%29.aspx
Probably you are getting some sort of precission error prior to the display of the results. Try something like this:
SELECT ROUND(ROUND(ISNULL((SELECT SUM([Labor]) FROM #Intermediate ),0),3) ,2)
The inner instance of ROUND to 3 gets rid of those pesky decimals, leaving just 7585.225, the outer ROUND actualy does what it is supossed to do.
Related
I want to save my adjusted query results into a table. For example I have codes 350.8, 351.94 and I have T-SQL code to remove the decimal points resulting in the adjusted results 350,351 etc. I want to save the results into a table, not Excel. Is this possible?
I see you can create a new table but with the same columns, not the new adjusted results. The below doesn't work as SQL Server doesn't recognise adjusted1, 2 and 3.
CREATE TABLE DiagAdj
(
encounter_id NUMERIC,
di_1 INT,
di_2 INT,
di_3 INT,
adjusted1 INT,
adjusted2 INT,
adjusted3 INT,
);
INSERT INTO DiagAdj (encounter_id, adjusted1, adjusted2, adjusted3)
SELECT encounter_id, adjusted1, adjusted2, adjusted3
FROM dbo.Encounters
Decimal places removed. I want to save down adjusted3 results into a table
SELECT
encounter_id, di_3, -- now, try to cast to int the remainder, ending right before the decimal
adjusted3 = TRY_CONVERT(int,LEFT(di_3, COALESCE(NULLIF(CHARINDEX('.', di_3) - 1, -1), 255)))
FROM
dbo.Encounters;
Why don't you just cast each decimal column to integer:
INSERT INTO DiagAdj (encounter_id, adjusted1, adjusted2, adjusted3)
SELECT
encounter_id,
CAST(diag1 AS DECIMAL(10,0)),
CAST(diag2 AS DECIMAL(10,0)),
CAST(diag3 AS DECIMAL(10,0))
FROM dbo.Encounters;
I've come across an issue (that I've partially solved) but can't seem to find a reason behind the failing in the first place.
I have a field in a table which holds a combination of alpha and numerical values. The field is a char(20) data type (which is wrong, but unchangeable) and holds either a NULL value, 'Unknown' or the "numbers" 0, 50, 100. The char field pads the values with trailing white space. This is a known and we can't do a thing about it.
To remove the Unknown values, we have a series of coalesce statements in place, and these two return the error message as per the title.
,coalesce(DHMCC.[HESA Module Total Proportion Taught], 'Missing')
,cast(isnull(DHMCC.[HESA Module Total Proportion Taught] ,'Missing') as varchar(10))
The query I have is why am I getting this error when I'm not converting a data type of varchar to float (or am I?)
Does anyone have an idea as to where to look next to try to fix this error?
The STR() function accepts a float datatype as the first argument, therefore SQL Server is implicitly converting whatever you pass to this function, which in your case is the CHAR(20) column. Since unknown can't be converted to a float, you get the error.
If you run the following with the actual execution plan enabled:
DECLARE #T TABLE (Col CHAR(20));
INSERT #T VALUES (NULL);
SELECT Result = ISNULL(STR(Col, 25, 0), 'Missing')
FROM #T
Then checkthe execution plan XML you will see the implicit conversion:
<ScalarOperator ScalarString="isnull(str(CONVERT_IMPLICIT(float(53),[Col],0),(25),(0)),'Missing')">
The simplest solution is probably to use a case expression and not bother with any conversion at all (only if you know you will only have the 5 values you listed:
DECLARE #T TABLE (Col CHAR(20));
INSERT #T VALUES (NULL), ('0'), ('50'), ('100');--, ('Unknown');
SELECT Result = CASE WHEN Col IS NULL OR Col = 'Unknown' THEN 'Missing' ELSE Col END
FROM #T;
Result
---------
Missing
0
50
100
Missing
If you really want the STR() function, you can make the conversion explicit, but use TRY_CONVERT() so that anything that is not a float simply returns NULL:
DECLARE #T TABLE (Col CHAR(20));
INSERT #T VALUES (NULL), ('0'), ('50'), ('100');--, ('Unknown');
SELECT Result = ISNULL(STR(TRY_CONVERT(FLOAT, Col), 25, 0), 'Missing')
FROM #T
Result
------------
Missing
0
50
100
Missing
Although, since you the numbers you have stated are integers, I would be inclined to convert them to integers rather than floats:
DECLARE #T TABLE (Col CHAR(20));
INSERT #T VALUES (NULL), ('0'), ('50'), ('100'), ('Unknown');
SELECT Result = ISNULL(CONVERT(VARCHAR(10), TRY_CONVERT(INT, Col)), 'Missing')
FROM #T;
Result
---------
Missing
0
50
100
Missing
Thanks to #GarethD
I've only just come across TRY_CONVERT and this seems like the better option, so thanks him for that pointer, also trying with TRY_CAST as well.
The data really should be held in a varchar field, it's referential and not for calculation, and this seems to work equally as well,
-- Declare #varText as varchar(16) = '10 '
-- Declare #varText as char(16) = 'Unknown'
-- Declare #varText as char(16) = ''
SELECT
ISNULL(NULLIF(TRY_CAST(LTRIM(RTRIM(#varText)) as varchar(16)), ''), 'Missing') AS HESA
I've created this test scenario which works ok.
I have a conundrum....
There is a table with one NVARCHAR(50) Float column that has many rows with many numbers of various decimal lengths:
'3304.063'
'3304.0625'
'39.53'
'39.2'
I need to write a query to find only numbers with decimal places >= 4
First the query I wrote was:
SELECT
Column
FROM Tablename
WHERE Column LIKE '%.[0-9][0-9]%'
The above code finds all numbers with decimal places >= 2:
'3304.063'
'3304.0625'
'39.53'
Perfect! Now, I just need to increase the [0-9] by 2...
SELECT
Column
FROM Tablename
WHERE Column LIKE '%.[0-9][0-9][0-9][0-9]%'
this returned nothing! What?
Does anyone have an explanation as to what went wrong as well and/or a possible solution? I'm kind of stumped and my hunch is that it is some sort of 'LIKE' limitation..
Any help would be appreciated!
Thanks.
After your edit, you stated you are using FLOAT which is an approximate value stored as 4 or 8 bytes, or 7 or 15 digits of precision. The documents explicitly state that not all values in the data type range can be represented exactly. It also states you can use the STR() function when converting it which you'll need to get your formatting right. Here is how:
declare #table table (columnName float)
insert into #table
values
('3304.063'),
('3304.0625'),
('39.53'),
('39.2')
--see the conversion
select * , str(columnName,20,4)
from #table
--now use it in a where clause.
--Return all values where the last digit isn't 0 from STR() the conversion
select *
from #table
where right(str(columnName,20,4),1) != 0
OLD ANSWER
Your LIKE statement would do it, and here is another way just to show they both work.
declare #table table (columnName varchar(64))
insert into #table
values
('3304.063'),
('3304.0625'),
('39.53'),
('39.2')
select *
from #table
where len(right(columnName,len(columnName) - charindex('.',columnName))) >= 4
select *
from #table
where columnName like '%.[0-9][0-9][0-9][0-9]%'
One thing that could be causing this is a space in the number somewhere... since you said the column type was VARCHAR this is a possibility, and could be avoided by storing the value as DECIMAL
declare #table table (columnName varchar(64))
insert into #table
values
('3304.063'),
('3304. 0625'), --notice the space here
('39.53'),
('39.2')
--this would return nothing
select *
from #table
where columnName like '%.[0-9][0-9][0-9][0-9]%'
How to find out if this is the case?
select *
from #table
where columnName like '% %'
Or, anything but numbers and decimals:
select *
from #table
where columnName like '%[^.0-9]%'
The following is working fine for me:
declare #tab table (val varchar(50))
insert into #tab
select '3304.063'
union select '3304.0625'
union select '39.53'
union select '39.2'
select * from #tab
where val like '%.[0-9][0-9][0-9][0-9]%'
Assuming your table only has numerical data, you can cast them to decimal and then compare:
SELECT COLUMN
FROM tablename
WHERE CAST(COLUMN AS DECIMAL(19,4)) <> CAST(COLUMN AS DECIMAL(19,3))
You'd want to test the performance of this against using the character data type solutions that others have already suggested.
You can use REVERSE:
declare #vals table ([Val] nvarchar(50))
insert into #vals values ('3304.063'), ('3304.0625'), ('39.53'), ('39.2')
select [Val]
from #Vals
where charindex('.',reverse([Val]))>4
On applying abs on divisible data, whole number quotient is converted to decimal.
create table #data ( ConditionValue money, ConditionRateNbr real)
insert into #data values(9665.77,37.61)
select abs(conditionvalue/conditionrateNbr) Using_Abs ,*
from #data
--Using_Abs ConditionValue ConditionRateNbr
--256.999969482422 9665.77 37.61
Why this happens and how to solve?
It has to do with the "real" data type being an "approximate-number", see https://msdn.microsoft.com/en-us/library/ms173773.aspx. My knowledge of SQL is not deep enough to give a detailed explanation but I do have a solution, use the decimal data type i/o real:
create table #data ( ConditionValue money, ConditionRateNbr decimal(38,20))
insert into #data values(9665.77,37.61)
select abs(conditionvalue/conditionrateNbr) Using_Abs ,*
from #data
IMO it's usually smarter to use decimal than real to avoid this kind of issue.
Assuming sql server, but if you run the following script:
CREATE TABLE #data
(
ConditionValue MONEY
, ConditionRateNbr REAL
)
INSERT INTO #data
VALUES ( 9665.77, 37.61 )
SELECT *
INTO data
FROM (
SELECT ABS(ConditionValue / ConditionRateNbr) Using_Abs
, conditionValue / ConditionRateNbr test
, *
FROM #data
)a
SELECT * FROM INFORMATION_SCHEMA.COLUMNS AS C WHERE C.TABLE_NAME = 'data'
DROP TABLE #data
DROP TABLE data
You'll see that the result of the division is a real. The ISO synonym for real is float(24). The ABS(numeric_expression) function Returns the same type as numeric_expression. Apparently, the ABS function returns a default float, which is float(53). float(53) contains 15 digits, so you get 15 digits returned.
So the returned datatype is not, as you might at first think, a decimal, but instead another float, defaulted to its max size (so to speak).
If you define your fields as decimals, see here what you are likely to get.
What I have seen using money is more reliable than decimal.
Here is another example.
Why on earth decimal considers 3/1.5 as 1.5, I do not know !!!
declare #real real = 1.5, #realup money=3
select #realUp numerator, #real denominator, #realup /cast (#real as money) money, #realup /cast(#real as decimal) decimal
--numerator denominator money decimal
--3.00 1.5 2.00 1.50000000000000000000000
I have two table. First table(Table1) use to get the records and second table(Table2) used to insert first table record into it. But I am little bit confused after getting result.
In table 1 and table 2 column "Amount" have same data type i.e nvarchar(max)
Table1
Id Amount
1 Null
2 -89437.43
2 -533.43
3 22403.88
If I run this query
Insert into Table2(Amount)
Select Amount from Table1
Then get result like this, I don't know why values are automatically rounded off
Table2
Id Amount
1 Null
2 -89437.4
2 -533.43
3 22403.9
SQL Server will round float values when converting back and to from string types.
And then you have the fun bits of empty string being 0, as well other strange effects
SELECT CAST(CAST('' AS float) AS nvarchar(MAX))
SELECT CAST(CAST('0.E0' AS float) AS nvarchar(MAX))
Use decimal.
If you need to store "blank" (how does this differ from NULL?) use a separate bit column to allow that extra value
Here is good explanation about your question.
Eigher you explicitly give float or decimal or numeric(xx,x) (x means numeric value)
Then it will convert as the data, other wise it round off the last value.
Insert into Table2(Amount)
Select cast(Amount as numeric(18,2) --or , cast (Amount as float)
from Table1
Check this link:-
TSQL Round up decimal number
In my case I was doing the conversion to the correct data type but had decimal(18,0) for the column in the table. So make sure the decimal places are represented properly for the column decimal(18,2).
Perhaps it's your query tool that's truncating to 8 characters.
Check the actual fields lengths to see if the problem is really in the database:
SELECT LEN(Amount)
FROM Table2
WHERE Amount LIKE '%-89437.%'
Unreproducible. Running this script on SQL Server 2012:
DECLARE #T1 TABLE ([Amount] nvarchar(max) NULL);
DECLARE #T2 TABLE ([Amount] nvarchar(max) NULL);
INSERT INTO #T1 ([Amount])
VALUES (NULL),('-89437.43'),('-533.43'),('22403.88');
Insert into #T2(Amount)
Select Amount from #T1;
SELECT * FROM #T2;
Produces this result:
Amount
NULL
-89437.43
-533.43
22403.88
The problem you describe does not exist.
This will show you the problem:
DECLARE #T1 TABLE ([Amount123456789] money NULL);
DECLARE #T2 TABLE ([Amount123456789] nvarchar(max) NULL);
INSERT INTO #T1 ([Amount123456789])
VALUES (NULL),('-89437.43123'),('-533.43456'),('22403.88789'),(22403.88789);
Insert into #T2(Amount123456789)
Select Amount123456789 from #T1;
SELECT * FROM #T1;
SELECT * FROM #T2;