Table name: products,
column names: rice_price, sugar_price
I would like to get the average of both columns separately. For example;
SELECT
AVG(rice_price) avg_rice,
AVG(sugar_price) avg_sugar
FROM
products
If I run this query on SQL server, I get the message below
Msg 8117, Level 16, State 1, Line 4
Operand data type nvarchar is invalid for avg operator.
What could be the solution?
If most of them look like numbers, you could use this, which will exclude the ones that don't convert nicely, by handling them as null:
SELECT
AVG(try_convert(numeric(18,4),rice_price)) avg_rice,
AVG(try_convert(numeric(18,4),sugar_price)) avg_sugar
FROM
products
But you should be changing your datatypes as has been pointed out in the comments. This kind of query will help you discover the ones that aren't good.
SELECT *
FROM products
WHERE rice_price IS NOT NULL
AND try_convert(numeric(18,4),rice_price) IS NULL
SELECT *
FROM products
WHERE sugar_price IS NOT NULL
AND try_convert(numeric(18,4),sugar_price) IS NULL
The IS_NUMERIC function can work for this too, but I find I have switched to using TRY_CONVERT in this situation, because it feels more flexible - I can use whatever data type I need.
Related
Initial situation
I have a relatively large table (ca. 0.7 Mio records) where an nvarchar field "MediaID" contains largely media IDs in proper hexadecimal notation (as they should).
Within my "sequential" query (each query depends on the output of the query before, this is all in pure T-SQL) I have to convert these hexadecimal values into decimal bigint values in order to do further calculations and filtering on these calculated values for the subsequent queries.
--> So far, no problem. The "sequential" query works fine.
Problem
Unfortunately, some of these Media IDs do contain non-hex characters - most probably because there was some typing errors by the people which have added them or through import errors from the previous business system.
Because of these non-hex chars, the whole query fails (of course) because the conversion hits an error.
For my current purpose, such rows must be skipped/ignored as they are clearly wrong and cannot be used (there are no medias / data carriers in use with the current business system which can have non-hex character IDs).
Manual editing of the data is not an option as there are too many errors and it is not clear with what the data must be replaced.
Challenge
To create a query which only returns records which have valid hex values within the media ID field.
(Unfortunately, my SQL skills are not enough to create the above query. Your help is highly appreciated.)
The relevant section of the larger query looks like this (xxxx is where your help comes in :-))
select
pureMediaID
, mediaID
, CUSTOMERID
,CONTRACT_CUSTOMERID
from
(
select concat('0x', Replace(Ltrim(Replace(mediaID, '0', ' ')), ' ', '0')) AS pureMediaID
--, CUSTOMERID
, *
from M_T_CONTRACT_CUSTOMERS
where mediaID is not null
and mediaID like '0%'
and xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
) as inner1
EDIT: As per request I have added here some good and some bad data:
Good:
4335463357
4335459809
1426427996
4335463509
4335515039
4335465134
4427370396
4335415661
4427369036
4335419089
004BB03433
004e7cf9c6
00BD23133
00EE13D8C1
00CCB5522C
00C46522C
00dbbe3433
Bad:
4564589+
AB6B8BFC.8
7B498DFCnm
DB218DFChb
d<tgfh8CFC
CB9E8AFCzj
B458DFCjhl
rytzju8DFC
BFCtdsjshj
DB9888FCgf
9BC08CFCyx
EB198DFCzj
4B628CFChj
7B2B8DFCgg
After I did upgrade the compatibility level of the SQL instance to SQL2016 (it was below 2012 before) I could use try_convert with same syntax as the original convert function as donPablo has pointed out. With that the query could run fully through and every MediaID which is not a correct hex value gets nicely converted into a null value - really, really nice.
Exactly what I needed.
Unfortunately, the solution of ALICE... didn't work out for me as this was also (strangely) returning records which had the "+" character within them.
Edit: The added comment of Alice... where you create a calculated field like this:
CASE WHEN "KEY" LIKE '%[^0-9A-F]%' THEN 0 ELSE 1 end as xyz
and then filter in the next query like this:
where xyz = 1
works also with SQL Instances with compatibility level < SQL 2012.
Great addition for people which still have to work with older SQL instances.
An option (although not ideal in terms of performance) is to check the characters in the MediaID through a case statement and regular expression
Hexadecimals cannot contain characters other than A-F and numbers between 0 and 9
CASE WHEN MediaID LIKE '%[0-9A-F]%' THEN 1 ELSE 0 END
I would recommend writing a function that can be used to evaluate MediaID first and checks if it is hexadecimal and then running the query for conversion
The following code encounters an error when executed in Microsoft Server Management Studion:
USE [DST]
GO
Select
CAST([Balance] as float)
FROM [RAW_XXX]
WHERE ISNUMERIC(Balance) = 1
Msg 8114, Level 16, State 5, Line 2
Error converting data type varchar to float.
I thought that the ISNUMERIC would exclude anything that can not be cast or converted.
It is a massive database in SQLServer 2012 so I am unsure how to find the data that is causing the error.
Use TRY_CONVERT to flush out the offending records:
SELECT *
FROM [RAW_XXX]
WHERE TRY_CONVERT(FLOAT, Balance) IS NULL;
The issue with your current logic is that something like $123.45 would be true according to ISNUMERIC, but would fail when trying to cast as floating point.
By the way, if you wanted a more bare bones way of finding records not castable to float you could just rely on LIKE:
SELECT *
FROM [RAW_XXX]
WHERE Balance NOT LIKE '%[^0-9.]%' AND Balance NOT LIKE '%.%.%';
The first LIKE condition ensures that Balance consists only of numbers and decimal points, and the second condition ensures that at most one decimal point appears. Checkout the demo below to see this working.
Demo
One of the client input'd large JSON PAYLOAD. 1000+ JSON PAYLOAD's ReceiptID field contains "NULL"/"some other word" instead of valid Blank/AlphaNumeric/Numeric.
Right now using, the following COALESCE & ISNULL based two query to narrow down for smaller subset. But what is the best quick approach to filter out these to new dirtyRowTable which would help to ask client to replay the same.
Using the following two query to find exact row that have bad data.
--Below SQL using ISNULL, returns all 1000 rows
Select top 1000 EventStoreId,
isnull(JSON_VALUE(payload,'$.ReceiptId'),0) ReceiptId
from dbo.EventStore order by 1 desc
--another SQL below using COALESCE, returns only 512 rows and error'd since 513th have value 'NULL'.
(Error: Msg 245, Level 16, State 1, Line 16
Conversion failed when converting the nvarchar value 'NULL' to data type int.)
Select top 1000 EventStoreId,
COALESCE(JSON_VALUE(payload,'$.ReceiptId'),0) ReceiptId
from dbo.EventStore order by 1 desc
COALESCE() -- although standard -- has the downfall that it evaluates the first argument twice. So when the first argument is non-trivial, ISNULL() is the better approach in SQL Server.
That said, the type of the ISNULL() expression is the type of the first expression. So, it is returning a string. The better way to write the code avoids implicit type conversions:
ISNULL(JSON_VALUE(payload, '$.ReceiptId'), N'0') ReceiptId
I am trying to take an average of a column in my database. The column is AMOUNT and it is stored as NVARCHAR(300),null.
When I try to convert it to a numeric value I get the following error:
Msg 8114, Level 16, State 5, Line 1
Error converting datatype NVARCHAR to NUMBER
Here is what I have right now.
SELECT AVG(CAST(Reimbursement AS DECIMAL(18,2)) AS Amount
FROM Database
WHERE ISNUMERIC(Reimbursement) = 1
AND Reimbursement IS NOT NULL
You would think that your code would work. However, SQL Server does not guarantee that the WHERE clause filters the database before the conversion for the SELECT takes place. In my opinion this is a bug. In Microsoft's opinion, this is an optimization feature.
Hence, your WHERE is not guaranteed to work. Even using a CTE doesn't fix the problem.
The best solution is TRY_CONVERT() available in SQL Server 2012+:
SELECT AVG(TRY_CONVERT(DECIMAL(18,2), Reimbursement)) AS Amount
FROM Database
WHERE ISNUMERIC(Reimbursement) = 1 AND Reimbursement IS NOT NULL;
In earlier versions, you can use CASE. The CASE does guarantee the sequential ordering of the clauses, so:
SELECT AVG(CASE WHEN ISNUMERIC(Reimbursement) = 1 AND Reimbursement IS NOT NULL
THEN CONVERT(DECIMAL(18,2), Reimbursement))
END)
FROM Database;
Because AVG() ignores NULL values, the WHERE is not necessary, but you can include it if you like.
Finally, you could simplify your code by using a computed column:
alter database add Reimbursement_Value as
(CASE WHEN ISNUMERIC(Reimbursement) = 1 AND Reimbursement IS NOT NULL
THEN CONVERT(DECIMAL(18,2), Reimbursement))
END);
Then you could write the code as:
select avg(Reimbursement_Value)
from database
where Reimbursement_Value is not null;
Quote from MSDN...
ISNUMERIC returns 1 for some characters that are not numbers, such as plus (+), minus (-), and valid currency symbols such as the dollar sign ($). For a complete list of currency symbols, see money and smallmoney
select isnumeric('+')---1
select isnumeric('$')---1
so try to add to avoid non numeric numbers messing with your ouput..
WHERE Reimbursement NOT LIKE '%[^0-9]%'
If you are on SQLServer 2012,you could try using TRY_Convert which outputs null for conversion failures..
SELECT AVG(try_convert( DECIMAL(18,2),Reimbursement))
from
table
I am guessing that since it is Nvarchar you are going to find some values in there with a '$','.', or a (,). I would run a query likt this:
SELECT Amount
FROM database
WHERE Amount LIKE '%$%' OR
Amount LIKE '%.%' OR
Amount LIKE '%,%'
See what you get and my guess you will get some rows returned and then update those rows and try it again.
Currently your query would pull all numbers that are not all numeric which is a reason why it is failing too. Instead try running this:
SELECT AVG(CAST(Reimbursement AS DECIMAL(18,2)) AS Amount
FROM Database
--Changed ISNUMERIC() = to 0 for true so it will only pull numeric numbers.
WHERE ISNUMERIC(Reimbursement) = 0 and Reimbursement IS NOT NULL
I've 2 columns which I want to use a condition on. But I get this error message and my query is correct I'll come to that soon.
Msg 8114, Level 16, State 5, Line 1
Error converting data type varchar to float.
So this is the problem, I have a temp-table in which ID-number looks like this 9001011234 we can call it A, in the other one that I want to check with it looks like this 900101-1234 and this one for B this is the swedish format for Id-numbers.
So in my condition I want to check this to get the right amount and the correct result.
where A = B
The rest of the query is fine, when I remove this condition it gives me a result. It's just this one bit that is incorrect.
You have a VARCHAR format that can't be trivially transformed to a number. I'd use REPLACE(b,'-','') = a to fix the format, and let SQL Server take care of the rest.
Say:
where A = CAST(REPLACE(B, '-', '') AS float)
You are trying to compare values that are not the same datatype. eg Where 'one' = 1
You will need to convert one of the values to the same datatype as the other.