Error converting data type nvarchar to numeric - SQL Server - sql

I am trying to take an average of a column in my database. The column is AMOUNT and it is stored as NVARCHAR(300),null.
When I try to convert it to a numeric value I get the following error:
Msg 8114, Level 16, State 5, Line 1
Error converting datatype NVARCHAR to NUMBER
Here is what I have right now.
SELECT AVG(CAST(Reimbursement AS DECIMAL(18,2)) AS Amount
FROM Database
WHERE ISNUMERIC(Reimbursement) = 1
AND Reimbursement IS NOT NULL

You would think that your code would work. However, SQL Server does not guarantee that the WHERE clause filters the database before the conversion for the SELECT takes place. In my opinion this is a bug. In Microsoft's opinion, this is an optimization feature.
Hence, your WHERE is not guaranteed to work. Even using a CTE doesn't fix the problem.
The best solution is TRY_CONVERT() available in SQL Server 2012+:
SELECT AVG(TRY_CONVERT(DECIMAL(18,2), Reimbursement)) AS Amount
FROM Database
WHERE ISNUMERIC(Reimbursement) = 1 AND Reimbursement IS NOT NULL;
In earlier versions, you can use CASE. The CASE does guarantee the sequential ordering of the clauses, so:
SELECT AVG(CASE WHEN ISNUMERIC(Reimbursement) = 1 AND Reimbursement IS NOT NULL
THEN CONVERT(DECIMAL(18,2), Reimbursement))
END)
FROM Database;
Because AVG() ignores NULL values, the WHERE is not necessary, but you can include it if you like.
Finally, you could simplify your code by using a computed column:
alter database add Reimbursement_Value as
(CASE WHEN ISNUMERIC(Reimbursement) = 1 AND Reimbursement IS NOT NULL
THEN CONVERT(DECIMAL(18,2), Reimbursement))
END);
Then you could write the code as:
select avg(Reimbursement_Value)
from database
where Reimbursement_Value is not null;

Quote from MSDN...
ISNUMERIC returns 1 for some characters that are not numbers, such as plus (+), minus (-), and valid currency symbols such as the dollar sign ($). For a complete list of currency symbols, see money and smallmoney
select isnumeric('+')---1
select isnumeric('$')---1
so try to add to avoid non numeric numbers messing with your ouput..
WHERE Reimbursement NOT LIKE '%[^0-9]%'
If you are on SQLServer 2012,you could try using TRY_Convert which outputs null for conversion failures..
SELECT AVG(try_convert( DECIMAL(18,2),Reimbursement))
from
table

I am guessing that since it is Nvarchar you are going to find some values in there with a '$','.', or a (,). I would run a query likt this:
SELECT Amount
FROM database
WHERE Amount LIKE '%$%' OR
Amount LIKE '%.%' OR
Amount LIKE '%,%'
See what you get and my guess you will get some rows returned and then update those rows and try it again.
Currently your query would pull all numbers that are not all numeric which is a reason why it is failing too. Instead try running this:
SELECT AVG(CAST(Reimbursement AS DECIMAL(18,2)) AS Amount
FROM Database
--Changed ISNUMERIC() = to 0 for true so it will only pull numeric numbers.
WHERE ISNUMERIC(Reimbursement) = 0 and Reimbursement IS NOT NULL

Related

SQL Code Error converting data type varchar to float

The following code encounters an error when executed in Microsoft Server Management Studion:
USE [DST]
GO
Select
CAST([Balance] as float)
FROM [RAW_XXX]
WHERE ISNUMERIC(Balance) = 1
Msg 8114, Level 16, State 5, Line 2
Error converting data type varchar to float.
I thought that the ISNUMERIC would exclude anything that can not be cast or converted.
It is a massive database in SQLServer 2012 so I am unsure how to find the data that is causing the error.
Use TRY_CONVERT to flush out the offending records:
SELECT *
FROM [RAW_XXX]
WHERE TRY_CONVERT(FLOAT, Balance) IS NULL;
The issue with your current logic is that something like $123.45 would be true according to ISNUMERIC, but would fail when trying to cast as floating point.
By the way, if you wanted a more bare bones way of finding records not castable to float you could just rely on LIKE:
SELECT *
FROM [RAW_XXX]
WHERE Balance NOT LIKE '%[^0-9.]%' AND Balance NOT LIKE '%.%.%';
The first LIKE condition ensures that Balance consists only of numbers and decimal points, and the second condition ensures that at most one decimal point appears. Checkout the demo below to see this working.
Demo

Numeric comparison of a varchar column to a decimal column

I have the following tables
productinfo:
ID|productname|productarea|productcost|productid
sales:
ID|salesid|productid|
salesdata:
ID|productid|productname|salestotal
Where I am having trouble is: salesdata.salestotal is a varchar column and may have nulls
Comparing the salesdata.saletotal to the productinfo.productcost columns.
I can do a
cast(salesdata.saletotal as float(7)) > X
and it works. I would like to do is
cast(salesdata.saletotal as float(7)) > productinfo.productcost
where
sales.productid = salesdata.productid and
productinfo.productid = salesdata.productid
However when I do that I get an error:
Error when converting type varchar to float
I found this post which was similar but am unable to get any other columns with it. I can not change the current db structure.
You can add a IsNumeric() to your where clause to check that salesdata.saletotal can be parsed to float
SELECT
cast(salesdata.saletotal as float(7)) > productinfo.productcost
FROM Sales,Salesdata
WHERE
sales.productid = salesdata.productid and
productinfo.productid = salesdata.productid and
IsNumeric(salesdata.saletotal) =1
Or you can use Not Like (best than IsNumeric)
salesdata.saletotal NOT LIKE '%[^0-9]%'
if you are using sql server version 2012 and above, you can use try_cast() or try_parse()
try_cast(salesdata.saletotal as float)>productinfo.productcost
or
try_parse(salesdata.saletotal as float)>productinfo.productcost
Prior to 2012, I would use patindex('%[^0-9.-]%',salesdata.saletotal)=0 to determine if something is numeric, because isnumeric() is somewhat broken.
e.g. rextester: http://rextester.com/UZE48454
case when patindex('%[^0-9.-]%',salesdata.saletotal)>0
then null
when isnull(cast(salesdata.saletotal as float(7)),0.0)
> isnull(cast(productinfo.productcost as float(7)),0)
then 1
else 0
end

Search Through All Between Values SQL

I have data following data structure..
_ID _BEGIN _END
7003 99210 99217
7003 10225 10324
7003 111111
I want to look through every _BEGIN and _END and return all rows where the input value is between the range of values including the values themselves (i.e. if 10324 is the input, row 2 would be returned)
I have tried this filter but it does not work..
where #theInput between a._BEGIN and a._END
--THIS WORKS
where convert(char(7),'10400') >= convert(char(7),a._BEGIN)
--BUT ADDING THIS BREAKS AND RETURNS NOTHING
AND convert(char(7),'10400') < convert(char(7),a._END)
Less than < and greater than > operators work on xCHAR data types without any syntactical error, but it may go semantically wrong. Look at examples:
1 - SELECT 'ab' BETWEEN 'aa' AND 'ac' # returns TRUE
2 - SELECT '2' BETWEEN '1' AND '10' # returns FALSE
Character 2 as being stored in a xCHAR type has greater value than 1xxxxx
So you should CAST types here. [Exampled on MySQL - For standard compatibility change UNSIGNED to INTEGER]
WHERE CAST(#theInput as UNSIGNED)
BETWEEN CAST(a._BEGIN as UNSIGNED) AND CAST(a._END as UNSIGNED)
You'd better change the types of columns to avoid ambiguity for later use.
This would be the obvious answer...
SELECT *
FROM <YOUR_TABLE_NAME> a
WHERE #theInput between a._BEGIN and a._END
If the data is string (assuming here as we don't know what DB) You could add this.
Declare #searchArg VARCHAR(30) = CAST(#theInput as VARCHAR(30));
SELECT *
FROM <YOUR_TABLE_NAME> a
WHERE #searchArg between a._BEGIN and a._END
If you care about performance and you've got a lot of data and indexes you won't want to include function calls on the column values.. you could in-line this conversion but this assures that your predicates are Sargable.
SELECT * FROM myTable
WHERE
(CAST(#theInput AS char) >= a._BEGIN AND #theInput < a.END);
I also saw several of the same type of questions:
SQL "between" not inclusive
MySQL "between" clause not inclusive?
When I do queries like this, I usually try one side with the greater/less than on either side and work from there. Maybe that can help. I'm very slow, but I do lots of trial and error.
Or, use Tony's convert.
I supposed you can convert them to anything appropriate for your program, numeric or text.
Also, see here, http://technet.microsoft.com/en-us/library/aa226054%28v=sql.80%29.aspx.
I am not convinced you cannot do your CAST in the SELECT.
Nick, here is a MySQL version from SO, MySQL "between" clause not inclusive?

Return Character from Numeric Field using Cast & Coalesce

Using SQL Server 2008 R2
Relatively basic SQL user, apologies if this is a simple question but need a little help to tidy up the output of a fairly complex script I have been given.
I have a number of columns being returned for which where there is a NULL, I want to replace all NULL's with a standard set of characters, currently "---". Using ISNULL works for most columns. However, for some columns we are looking at 2 tables to find a value so have after doing some research on here, I have modified a line I am having trouble with as follows:
Previous
isnull (ff.ff_sales,aa.ff_sales) as 'Total Revenue'
Latest
cast(coalesce(ff.ff_sales,aa.ff_sales,'') as FLOAT) as 'Total Revenue'
The initial line returned 'NULL' if both ff.ff_sales & aa.ff_sales were empty, now with the latest line using cast & coalesce I get '0'. However, I am trying to achieve a situation where I get '---' as per all other fields where a NULL exists. I don't want it to return '0' for a Sales field as this is misleading. I have tried using VARCHAR instead of FLOAT but am unsure if this is the right thing to do at this stage?
1st column is using ISNULL, 2nd column current output with cast & coalesce, 3rd column is what I want to get to:
Total Revenue Total Revenue Total Revenue
67755 67755 67755
6.123 6.123 6.123
494.75 494.75 494.75
0 0 0
1139909 1139909 1139909
12346.45 12346.45 12346.45
129.866 129.866 129.866
NULL 0 ---
NULL 0 ---
554 554 554
Thanks for your help!
You can use case to decide what should be output in this scenario, like so:
select
case
when isnull (ff.ff_sales,aa.ff_sales) is null then '---'
else cast(isnull (ff.ff_sales,aa.ff_sales) as varchar)
end as 'Total Revenue'
This will force the output to be returned as varchar though, because you cannot cast '---' as a numeric type.
Alternatively, you could just let the value be NULL in DB, and in your presentation layer, replace a DBNull or equivalent value with '---'. This will let you keep total revenue as a numeric field at the DB level.

How does one filter based on whether a field can be converted to a numeric?

I've got a report that has been in use quite a while - in fact, the company's invoice system rests in a large part upon this report (Disclaimer: I didn't write it). The filtering is based upon whether a field of type VarChar(50) falls between two numeric values passed in by the user.
The problem is that the field the data is being filtered on now not only has simple non-numeric values such as '/A', 'TEST' and a slew of other non-numeric data, but also has numeric values that seem to be defying any type of numeric conversion I can think of.
The following (simplified) test query demonstrates the failure:
Declare #StartSummary Int,
#EndSummary Int
Select #StartSummary = 166285,
#EndSummary = 166289
Select SummaryInvoice
From Invoice
Where IsNull(SummaryInvoice, '') <> ''
And IsNumeric(SummaryInvoice) = 1
And Convert(int, SummaryInvoice) Between #StartSummary And #EndSummary
I've also attempted conversions using bigint, real and float and all give me similar errors:
Msg 8115, Level 16, State 2, Line 7
Arithmetic overflow error converting
expression to data type int.
I've tried other larger numeric datatypes such as BigInt with the same error. I've also tried using sub-queries to sidestep the conversion issue by only extracting fields that have numeric data and then converting those in the wrapper query, but then I get other errors which are all variations on a theme indicating that the value stored in the SummaryInvoice field can't be converted to the relevant data type.
Short of extracting only those records with numeric SummaryInvoice fields to a temporary table and then querying against the temporary table, is there any one-step solution that would solve this problem?
Edit: Here's the field data that I suspect is causing the problem:
SummaryInvoice
11111111111111111111111111
IsNumeric states that this field is numeric - which it is. But attempting to convert it to BigInt causes an arithmetic overflow. Any ideas? It doesn't appear to be an isolated incident, there seems to have been a number of records populated with data that causes this issue.
It seems that you are gonna have problems with the ISNUMERIC function, since it returns 1 if can be cast to any number type (including ., ,, e0, etc). If you have numbers longer than 2^63-1, you can use DECIMAL or NUMERIC. I'm not sure if you can use PATINDEX to perform an regex look on SummaryInvoice, but if you can, then you should try this:
SELECT SummaryInvoice
FROM Invoice
WHERE ISNULL(SummaryInvoice, '') <> ''
AND CASE WHEN PATINDEX('%[^0-9]%',SummaryInvoice) > 0 THEN CONVERT(DECIMAL(30,0), SummaryInvoice) ELSE -1 END
BETWEEN #StartSummary And #EndSummary
You can't guarantee what order the WHERE clause filters will be applied.
One ugly option to decouple inner and outer.
SELECT
*
FROM
(
Select TOP 2000000000
SummaryInvoice
From Invoice
Where IsNull(SummaryInvoice, '') <> ''
And IsNumeric(SummaryInvoice) = 1
ORDER BY SummaryInvoice
) foo
WHERE
Convert(int, SummaryInvoice) Between #StartSummary And #EndSummary
Another using CASE
Select SummaryInvoice
From Invoice
Where IsNull(SummaryInvoice, '') <> ''
And
CASE WHEN IsNumeric(SummaryInvoice) = 1 THEN Convert(int, SummaryInvoice) ELSE -1 END
Between #StartSummary And #EndSummary
YMMV
Edit: after question update
use decimal(38,0) not int
Change ISNUMERIC(SummaryInvoice) to ISNUMERIC(SummaryInvoice + '0e0')
AND with IsNumeric(SummaryInvoice) = 1, will not short circuit in SQL Server.
But may be you can use
AND (CASE IsNumeric(SummaryInvoice) = 1 THEN Convert(int, SummaryInvoice) ELSE 0 END)
Between #StartSummary And #EndSummary
Your first issue is to fix your database structure so bad data cannot get into the field. You are putting a band-aid on a wound that needs stitches and wondering why it doesn't heal.
Database refactoring is not fun, but it needs to be done when there is a data integrity problem. I assume you aren't really invoicing someone for 11,111,111,111,111,111,111,111,111 or 'test'. So don't allow those values to ever get entered (if you can't change the structure to the correct data type, consider a trigger to prevent bad data from going in) and delete the ones you do have that are bad.