I've got a report that has been in use quite a while - in fact, the company's invoice system rests in a large part upon this report (Disclaimer: I didn't write it). The filtering is based upon whether a field of type VarChar(50) falls between two numeric values passed in by the user.
The problem is that the field the data is being filtered on now not only has simple non-numeric values such as '/A', 'TEST' and a slew of other non-numeric data, but also has numeric values that seem to be defying any type of numeric conversion I can think of.
The following (simplified) test query demonstrates the failure:
Declare #StartSummary Int,
#EndSummary Int
Select #StartSummary = 166285,
#EndSummary = 166289
Select SummaryInvoice
From Invoice
Where IsNull(SummaryInvoice, '') <> ''
And IsNumeric(SummaryInvoice) = 1
And Convert(int, SummaryInvoice) Between #StartSummary And #EndSummary
I've also attempted conversions using bigint, real and float and all give me similar errors:
Msg 8115, Level 16, State 2, Line 7
Arithmetic overflow error converting
expression to data type int.
I've tried other larger numeric datatypes such as BigInt with the same error. I've also tried using sub-queries to sidestep the conversion issue by only extracting fields that have numeric data and then converting those in the wrapper query, but then I get other errors which are all variations on a theme indicating that the value stored in the SummaryInvoice field can't be converted to the relevant data type.
Short of extracting only those records with numeric SummaryInvoice fields to a temporary table and then querying against the temporary table, is there any one-step solution that would solve this problem?
Edit: Here's the field data that I suspect is causing the problem:
SummaryInvoice
11111111111111111111111111
IsNumeric states that this field is numeric - which it is. But attempting to convert it to BigInt causes an arithmetic overflow. Any ideas? It doesn't appear to be an isolated incident, there seems to have been a number of records populated with data that causes this issue.
It seems that you are gonna have problems with the ISNUMERIC function, since it returns 1 if can be cast to any number type (including ., ,, e0, etc). If you have numbers longer than 2^63-1, you can use DECIMAL or NUMERIC. I'm not sure if you can use PATINDEX to perform an regex look on SummaryInvoice, but if you can, then you should try this:
SELECT SummaryInvoice
FROM Invoice
WHERE ISNULL(SummaryInvoice, '') <> ''
AND CASE WHEN PATINDEX('%[^0-9]%',SummaryInvoice) > 0 THEN CONVERT(DECIMAL(30,0), SummaryInvoice) ELSE -1 END
BETWEEN #StartSummary And #EndSummary
You can't guarantee what order the WHERE clause filters will be applied.
One ugly option to decouple inner and outer.
SELECT
*
FROM
(
Select TOP 2000000000
SummaryInvoice
From Invoice
Where IsNull(SummaryInvoice, '') <> ''
And IsNumeric(SummaryInvoice) = 1
ORDER BY SummaryInvoice
) foo
WHERE
Convert(int, SummaryInvoice) Between #StartSummary And #EndSummary
Another using CASE
Select SummaryInvoice
From Invoice
Where IsNull(SummaryInvoice, '') <> ''
And
CASE WHEN IsNumeric(SummaryInvoice) = 1 THEN Convert(int, SummaryInvoice) ELSE -1 END
Between #StartSummary And #EndSummary
YMMV
Edit: after question update
use decimal(38,0) not int
Change ISNUMERIC(SummaryInvoice) to ISNUMERIC(SummaryInvoice + '0e0')
AND with IsNumeric(SummaryInvoice) = 1, will not short circuit in SQL Server.
But may be you can use
AND (CASE IsNumeric(SummaryInvoice) = 1 THEN Convert(int, SummaryInvoice) ELSE 0 END)
Between #StartSummary And #EndSummary
Your first issue is to fix your database structure so bad data cannot get into the field. You are putting a band-aid on a wound that needs stitches and wondering why it doesn't heal.
Database refactoring is not fun, but it needs to be done when there is a data integrity problem. I assume you aren't really invoicing someone for 11,111,111,111,111,111,111,111,111 or 'test'. So don't allow those values to ever get entered (if you can't change the structure to the correct data type, consider a trigger to prevent bad data from going in) and delete the ones you do have that are bad.
Related
Apologies if this has been asked before - I've spent a couple of hours searching but not found anything that's helped.
It's quite simple really - I've been asked to create a query which includes a field that when it was set up (not by me) was created as a VARCHAR instead of an INT.
I need to do some calculations on this field, however some users have been entering text into it, so the calculations fail as it can't convert the data to an INT.
Is there anything I can add to a CASE statement to handle where there's text?
I was thinking something like the below, but don't know what the actual code is:
CASE
WHEN [Field1] IS TEXT THEN 1 ;
ELSE [Field2] as [Chose name]
END
Edit: Note that this is in MS SQL Server.
Thanks.
In SQL Server, you can use try_convert() and isnull() for this:
isnull(try_convert(int, field), 1)
try_convert() attempts you cast field to an int. When that fails, null is returned; you can trap that with isnull() and turn the result to 1 instead.
Note that this only works as long as field is not null (otherwise, you would get 1 as a result).
In SQL Server
Declare #Salary varchar(100);
Set #Salary = 50000;
Select Case when ISNUMERIC(#Salary) = 1 then 1
else 0 end as [Check]
May be this will be Helpful.
I am comparing data from two different databases (one MariaDB and one SQL Server) within my Node project, and am then doing inserts and updates as necessary depending on the comparison results.
I have a question about this code that I use to iterate through results in Node, going one at a time and passing in values to check against (note - I am more familiar with Node and JS than with SQL, hence this question):
SELECT TOP 1
CASE
WHEN RM00101.CUSTCLAS LIKE ('%CUSR%')
THEN CAST(REPLACE(LEFT(LR301.DOCNUMBR, CHARINDEX('-', LR301.DOCNUMBR)), '-', '') AS INT)
ELSE 0
END AS Id,
CASE
WHEN LR301.RMDTYPAL = 7 THEN LR301.ORTRXAMT * -1
WHEN LR301.RMDTYPAL = 9 THEN LR301.ORTRXAMT * -1
ELSE LR301.ORTRXAMT
END DocumentAmount,
GETDATE() VerifyDate
FROM
CRDB..RM20101
INNER JOIN
CRDB..RM00101 ON LR301.CUSTNMBR = RM00101.CUSTNMBR
WHERE
CONVERT(BIGINT, (REPLACE(LEFT(LR301.DOCNUMBR, CHARINDEX('-', LR301.DOCNUMBR)), '-', ''))) = 589091
Currently, the above works for me for finding records that match. However, if I enter a value that doesn't yet exist - in this line below, like so:
WHERE CONVERT(BIGINT, (REPLACE(LEFT( LR301.DOCNUMBR, CHARINDEX('-', LR301.DOCNUMBR)), '-', ''))) = 789091
I get this error:
Error converting data type varchar to bigint.
I assume the issue is that, if the value isn't found, it can't cast it to an INTEGER, and so it errors out. Sound right?
What I ideally want is for the query to execute successfully, but just return 0 results when a match is not found. In JavaScript I might doing something like an OR clause to handle this:
const array = returnResults || [];
But I'm not sure how to handle this with SQL.
By the way, the value in SQL Server that's being matched is of type char(21), and the values look like this: 00000516542-000. The value in MariaDB is of type INT.
So two questions:
Will this error out when I enter a value that doesn't currently match?
If so, how can I handle this so as to just return 0 rows when a match isn't found?
By the way, as an added note, someone suggested using TRY_CONVERT, but while this works in SQL Server, it doesn't work when I use it with the NODE mssql package.
I think the issue is happening because the varchar value is not always made of numbers. You can make the comparison in varchar format itself to avoid this issue:
WHERE (REPLACE(LEFT( LR301.DOCNUMBR, CHARINDEX('-', LR301.DOCNUMBR)), '-', '')) = '789091'
Hope this helps.
Edit: based on the format in the comment, this should do the trick;
WHERE REPLACE(LTRIM(REPLACE(REPLACE(LEFT( LR301.DOCNUMBR, CHARINDEX('-', LR301.DOCNUMBR)),'0',' '),'-','')),' ','0') = '789091'
I am trying to take an average of a column in my database. The column is AMOUNT and it is stored as NVARCHAR(300),null.
When I try to convert it to a numeric value I get the following error:
Msg 8114, Level 16, State 5, Line 1
Error converting datatype NVARCHAR to NUMBER
Here is what I have right now.
SELECT AVG(CAST(Reimbursement AS DECIMAL(18,2)) AS Amount
FROM Database
WHERE ISNUMERIC(Reimbursement) = 1
AND Reimbursement IS NOT NULL
You would think that your code would work. However, SQL Server does not guarantee that the WHERE clause filters the database before the conversion for the SELECT takes place. In my opinion this is a bug. In Microsoft's opinion, this is an optimization feature.
Hence, your WHERE is not guaranteed to work. Even using a CTE doesn't fix the problem.
The best solution is TRY_CONVERT() available in SQL Server 2012+:
SELECT AVG(TRY_CONVERT(DECIMAL(18,2), Reimbursement)) AS Amount
FROM Database
WHERE ISNUMERIC(Reimbursement) = 1 AND Reimbursement IS NOT NULL;
In earlier versions, you can use CASE. The CASE does guarantee the sequential ordering of the clauses, so:
SELECT AVG(CASE WHEN ISNUMERIC(Reimbursement) = 1 AND Reimbursement IS NOT NULL
THEN CONVERT(DECIMAL(18,2), Reimbursement))
END)
FROM Database;
Because AVG() ignores NULL values, the WHERE is not necessary, but you can include it if you like.
Finally, you could simplify your code by using a computed column:
alter database add Reimbursement_Value as
(CASE WHEN ISNUMERIC(Reimbursement) = 1 AND Reimbursement IS NOT NULL
THEN CONVERT(DECIMAL(18,2), Reimbursement))
END);
Then you could write the code as:
select avg(Reimbursement_Value)
from database
where Reimbursement_Value is not null;
Quote from MSDN...
ISNUMERIC returns 1 for some characters that are not numbers, such as plus (+), minus (-), and valid currency symbols such as the dollar sign ($). For a complete list of currency symbols, see money and smallmoney
select isnumeric('+')---1
select isnumeric('$')---1
so try to add to avoid non numeric numbers messing with your ouput..
WHERE Reimbursement NOT LIKE '%[^0-9]%'
If you are on SQLServer 2012,you could try using TRY_Convert which outputs null for conversion failures..
SELECT AVG(try_convert( DECIMAL(18,2),Reimbursement))
from
table
I am guessing that since it is Nvarchar you are going to find some values in there with a '$','.', or a (,). I would run a query likt this:
SELECT Amount
FROM database
WHERE Amount LIKE '%$%' OR
Amount LIKE '%.%' OR
Amount LIKE '%,%'
See what you get and my guess you will get some rows returned and then update those rows and try it again.
Currently your query would pull all numbers that are not all numeric which is a reason why it is failing too. Instead try running this:
SELECT AVG(CAST(Reimbursement AS DECIMAL(18,2)) AS Amount
FROM Database
--Changed ISNUMERIC() = to 0 for true so it will only pull numeric numbers.
WHERE ISNUMERIC(Reimbursement) = 0 and Reimbursement IS NOT NULL
I have this query
SELECT
CASE WHEN dbo.CFE_PPHY.P77 IS NOT NULL OR dbo.CFE_PPHY.P77 <>''
THEN MONTH(dbo.CFE_PPHY.P77)
WHEN dbo.CFE_PPHY.P70 IS NOT NULL OR dbo.CFE_PPHY.P70 <>''
THEN MONTH(dbo.CFE_SERVICE_EVTS.C10_2)
ELSE COALESCE(CONVERT(VARCHAR,dbo.CFE_PPHY.P77)+
CONVERT(VARCHAR,dbo.CFE_SERVICE_EVTS.C10_2),'toto') END
AS CFELiasse_DateEffetEIRL_MM_N
FROM CFE_PPHY LEFT JOIN CFE_SERVICE_EVTS ON CFE_PPHY.colA = CFE_SERVICE_EVTS.colB
The ELSE part is giving me headaches.
The columns CFE_PPHY.P77 and CFE_SERVICE_EVTS.C10_2 have date time format. I'm turning them into varchar. Yet when I'm running the query, I have the following error
Msg 245, Level 16, State 1, Line 1 Conversion failed when converting the varchar value 'toto' to data type int.
Obviously, I cannot turn toto to an integer. Fair enough. However, from my point of view, I've converted the datetime format to a varchar format, so it should do the work.
Where am I wrong?
Thanks
You have to convert all of your case expressions to varchar. SQL is deciding to case the field as int so 'toto' is invalid. If all expressions are converted to varchar this error should be solved.
http://blog.sqlauthority.com/2010/10/08/sql-server-simple-explanation-of-data-type-precedence/
Have a closer look at your case expression: in the first and second conditional branches you're returning MONTH(... which is obviously integer.
But in third branch you're returning varchar thus SQL server tries to convert it to int according to data type of previous branches and failing to do it.
Try like this,
SELECT CASE
WHEN dbo.CFE_PPHY.P77 IS NOT NULL
OR dbo.CFE_PPHY.P77 <> ''
THEN convert(VARCHAR, MONTH(dbo.CFE_PPHY.P77))
WHEN dbo.CFE_PPHY.P70 IS NOT NULL
OR dbo.CFE_PPHY.P70 <> ''
THEN convert(VARCHAR, MONTH(dbo.CFE_SERVICE_EVTS.C10_2))
ELSE COALESCE(CONVERT(VARCHAR, dbo.CFE_PPHY.P77) + CONVERT(VARCHAR, dbo.CFE_SERVICE_EVTS.C10_2), 'toto')
END AS CFELiasse_DateEffetEIRL_MM_N
FROM CFE_PPHY
LEFT JOIN CFE_SERVICE_EVTS ON CFE_PPHY.colA = CFE_SERVICE_EVTS.colB
First, when converting to a string, always include a length (in SQL Server). The default length varies by context and may not be correct.
Second, the comparison of date/time values to '' is not necessary. This is not really valid value for a date/time -- although it does get converted to a 0 which is 1900-01-01. The NULL comparison should be sufficient. Otherwise, be explicit.
Third, string concatenation will return NULL if any of the arguments are NULL.
Fourth, table aliases make a query easier to write and to read.
As far as I can tell, your case is a bit over complicated. In the ELSE, we know that dbo.CFE_PPHY.P77 is NULL, because of the first condition. So, how about:
SELECT (CASE WHEN p.P77 IS NOT NULL
THEN CAST(MONTH(p.P77) as VARCHAR(255))
WHEN p.P70 IS NOT NULL
THEN CAST(MONTH(e.C10_2) as VARCHAR(255))
ELSE 'toto'
END) AS CFELiasse_DateEffetEIRL_MM_N
FROM CFE_PPHY p LEFT JOIN
CFE_SERVICE_EVTS e
ON p.colA = e.colB;
I'm selecting a value out of a table that can either be an integer or a nvarchar. It's stored as nvarchar. I want to conditionally call a function that will convert this value if it is an integer (that is, if it can be converted into an integer), otherwise I want to select the nvarchar with no conversion.
This is hitting a SQL Server 2005 database.
select case
when T.Value (is integer) then SomeConversionFunction(T.Value)
else T.Value
end as SomeAlias
from SomeTable T
Note that it is the "(is integer)" part that I'm having trouble with. Thanks in advance.
UPDATE
Check the comment on Ian's answer. It explains the why and the what a little better. Thanks to everyone for their thoughts.
select case
when ISNUMERIC(T.Value) then T.Value
else SomeConversionFunction(T.Value)
end as SomeAlias
Also, have you considered using the sql_variant data type?
The result set can only have one type associated with it for each column, you will get an error if the first row converts to an integer and there are strings that follow:
Msg 245, Level 16, State 1, Line 1
Conversion failed when converting the nvarchar value 'word' to data type int.
try this to see:
create table testing
(
strangevalue nvarchar(10)
)
insert into testing values (1)
insert into testing values ('word')
select * from testing
select
case
when ISNUMERIC(strangevalue)=1 THEN CONVERT(int,strangevalue)
ELSE strangevalue
END
FROM testing
best bet is to return two columns:
select
case
when ISNUMERIC(strangevalue)=1 THEN CONVERT(int,strangevalue)
ELSE NULL
END AS StrangvalueINT
,case
when ISNUMERIC(strangevalue)=1 THEN NULL
ELSE strangevalue
END AS StrangvalueString
FROM testing
or your application can test for numeric and do your special processing.
You can't have a column that is sometimes an integer and sometimes a string. Return the string and check it using int.TryParse() in the client code.
ISNUMERIC. However, this accepts +, - and decimals so more work is needed.
However, you can't have the columns as both datatypes in one go: you'll need 2 columns.
I'd suggest that you deal with this in your client or use an ISNUMERIC replacement
IsNumeric will get you part of the way there. You can then add some further code to check whether it is an integer
for example:
select top 10
case
when isnumeric(mycolumn) = 1 then
case
when convert(int, mycolumn) = mycolumn then
'integer'
else
'number but not an integer'
end
else
'not a number'
end
from mytable
To clarify some other answers, your SQL statement can't return different data types in one column (it looks like the other answers are saying you can't store different data types in one column - yours are all strign represenations).
Therefore, if you use ISNUMERIC or another function, the value will be cast as a string in the table that is returned anyway if there are other strigns being selected.
If you are selecting only one value then it could return a string or a number, however your front end code will need to be able to return the different data types.
Just to add to some of the other comments about not being able to return different data types in the same column... Database columns should know what datatype they are holding. If they don't then that should be a BIG red flag that you have a design problem somewhere, which almost guarantees future headaches (like this one).