Converting varchar values to decimal while handling NULLs and EMPTY strings as 0 - sql

I have a column of varchar datatype populated with a mix of values such as 1.2, 5.33 while also having NULLs and EMPTY strings.
I want to convert all the values to decimal while treating NULLs and EMPTY strings as 0.
I can change the NULLs or EMPTY strings using the CONVERT function like below. Like this I replace the NULLs and EMPTY strings with a varhcar 0.
CASE WHEN Column1 = '' OR Column1= NULL THEN '0' ELSE Column1 END AS 'NewColumn1'
However what I want to do is to be able to then convert this data (output of NewColumn1) into decimal but when I place the CASE statement into a CONVERT or a CAST function I will have errors.
I also tried the following.
CONVERT(DECIMAL(10,4), ISNULL(Column1, '0')) however it fails since here I am not handling the EMPTY strings.
Any ideas how can I solve this problem.

Simple way:
SELECT CONVERT(DECIMAL(10, 4), ISNULL(NULLIF(Column1, ''), '0'))
Your CASE statement doesn't work because you're cheking if Column1 = NULL. You sholud check if it IS NULL.
CASE WHEN Column1 = '' OR Column1 IS NULL THEN '0' ELSE Column1 END AS 'NewColumn1'

Try this,
SELECT CAST(ISNULL(NULLIF(Column1, ''),0) AS DECIMAL(12,2)) FROM table

Related

Why do query results change when using coalesce on a char field in a where clause?

I'm seeing some unexpected behavior when querying with a where clause on a coalesced char field in an Oracle database.
It seems that the results of
CASE WHEN COALESCE(char_field, 'some_val') = 'someOtherVal'
are different than the results of
CASE WHEN char_field = 'someOtherVal'
The specific comparisons in which I've noticed this weird output are 'between', 'in', and 'equals'. These are the weird outputs I'm seeing:
Between seems to be non-inclusive on the upper end
In and equals return false for every comparison
Here's some sql to replicate the weirdness:
CREATE TABLE delete_me( some_char CHAR(8) );
INSERT ALL
INTO delete_me (some_char) VALUES ('1')
INTO delete_me (some_char) VALUES ('2')
INTO delete_me (some_char) VALUES ('4')
INTO delete_me (some_char) VALUES ('5')
INTO delete_me (some_char) VALUES ('abc1')
INTO delete_me (some_char) VALUES (null)
SELECT 1 FROM DUAL;
SELECT some_char,
COALESCE(some_char, 'wasNull') AS coalesce_some_char,
CASE
WHEN (some_char BETWEEN '1' AND '5')
THEN 'true'
ELSE 'false'
END AS between_1_5,
CASE
WHEN (COALESCE(some_char, 'wasNull') BETWEEN '1' AND '5')
THEN 'true'
ELSE 'false'
END AS coalesce_between_1_5,
CASE
WHEN (some_char IN ('1', '5'))
THEN 'true'
ELSE 'false'
END AS in_1_5,
CASE
WHEN (COALESCE(some_char, 'wasNull') IN ('1', '5'))
THEN 'true'
ELSE 'false'
END AS coalesce_in_1_5,
CASE
WHEN (some_char = 'abc1')
THEN 'true'
ELSE 'false'
END AS equals_abc1,
CASE
WHEN (COALESCE(some_char, 'wasNull') = 'abc1')
THEN 'true'
ELSE 'false'
END AS coalesce_equals_abc1
FROM delete_me;
I would have expected the output of the comparisons on coalesced fields to match those on non-coalesced fields for all operators except IS NULL.
Does anyone have any idea why these results wouldn't match up?
Your problem is in the data type of some_char. When a column of type CHAR is compared to a string, Oracle blank pads the string to the length of the column (see the docs). In the tests you are doing, the values match in length ('1' vs '1') or are completely different ('1' vs 'abc1') so everything works fine. However when you use COALESCE on a CHAR field, the output of COALESCE is the fully blank padded column value returned as a VARCHAR (see the docs for NVL) and so the comparison string is not blank padded, and you are then comparing '1 ' vs '1', which fails. There are a couple of ways to work around this. You can TRIM the output of COALESCE i.e.
TRIM(COALESCE(some_char, 'wasNull'))
or change the data type of some_char to VARCHAR(8) instead.
I've made a demo of all of this on dbfiddle

SQL Server CASE statement with multiple THEN clauses

I have seen several similar questions but none cover what I need. I need to put another THEN statement after the first one. My column contains int's. When it returns NULL I need it to display a blank space, but when I try the below code, I just get '0'.
CASE
WHEN Column1 IS NULL
THEN ''
ELSE Column1
END
If I try to put a sting after THEN then it tells me that it cannot convert it from int. I need to convert it to varchar and then change its output to a blank space afterwards, such as:
e.g.
CASE
WHEN Column1 IS NULL
THEN CONVERT(varchar(10), Column1)
THEN ''
ELSE Column1
END
Is there a way of doing this?
Thanks
Rob
A case expression returns a single value -- with a given type. If you want a string result, then you need to be sure that all paths in the case return strings:
CASE WHEN Column1 IS NULL
THEN ''
ELSE CAST(Column1 AS VARCHAR(255))
END
This is more simply written using COALESCE():
COALESCE(CAST(Column1 as VARCHAR(255)), '')
You cannot display an integer as a "blank" (other than using a NULL value).

How many ways can you generate an error converting varchar to numeric that won't be caught by ISNUMERIC()?

I am in the process of loading a bunch of tables into SQL Server and converting them from varchar to specific data types (int, date, etc.). One frustration is how many different ways there are to break the conversion from string to numeric (int, decimal, etc) and that there is not an easy diagnostic tool to find the offending rows (besides ISNUMERIC() which doesn't work all the time).
Here is my list of ways to break the conversion that won't get caught by ISNUMERIC().
The string contains scientific notation (ie 3.55E-10)
The string contains a blank ('')
The string contains a non-alphanumeric symbol ('$', '-', ',')
Here's what I'm currently using to compensate:
SELECT
CASE
WHEN [MyColumn] IN ('','-') THEN NULL -- deals with blanks
WHEN [MyColumn] LIKE '%E%' THEN CONVERT(DECIMAL(20, 4), CONVERT(FLOAT(53), [MyColumn])) -- deals with scientific notation
ELSE CAST(REPLACE(REPLACE([MyColumn] , '$', ''), '-', '') AS DECIMAL(20, 4))
END [MyColumn] -- deals with special characters
FROM
MyTable
Does anyone else have others? Or good ways to diagnose?
Don't use ISNUMERIC(). If you are on 2012+ then you could use TRY_CAST or TRY_CONVERT.
If you are on older versions, you could use some syntax like this:
SELECT *
FROM #TableA
WHERE ColA NOT LIKE '%[^0-9]%'
You can try to use LIKE '%[0-9]%' instead of ISNUMERIC()
SELECT col, CASE WHEN col NOT LIKE '%[^0-9]%' and col<>''
THEN 1
ELSE 0
END
FROM T
You can use NOT LIKE to exclude anything that isn't a digit... and REPLACE for commas and periods. Naturally, you can add other nested REPLACE functions for values you want to accept.
declare #var varchar(64) = '55,5646'
SELECT
CASE
WHEN replace(replace(#var,'.',''),',','') NOT LIKE '%[^0-9]%'
THEN 1
ELSE 0
END
This allows you to accept decimals for your decimal / numeric / float conversions.

SQL: How to make a replace on the field ''

I have a very but tricky question for you guys. So, listen I have a field with spaces and numbers in one of my table columns. The key part is transform the content in a decimal field. The drawback is basically that for some rows I could get something like:
' 1584.00 '
' 156546'
'545.00 '
' '
So, to clean up my column, I have done a LTRIM and RTRIM so spaces gone. So now for a couple of records where the record were just spaces the new content is ''. Finally I need to convert this result to a decimal.
Issue: The thing is that for field that contend just the spaces the new result is '' and I'm not able to apply a REPLACE on this because it's a blank and the code below doesn't work:
SELECT REPLACE('','','0')
-- Final current verison
SELECT CAST(COALESCE(REPLACE(REPLACE([Gross_Weight],' ','0'),',',''),'0') AS DECIMAL(13,3))
How could I figure it out?
thanks so much
SELECT COALESCE(NULLIF(MyColumn, ''), 0)
This has the side-effect that you will also turn NULL values into 0, which you might not want. If that's a problem then a simple CASE statement should do the trick:
SELECT CASE WHEN MyColumn = '' THEN 0 ELSE CAST(MyColumn AS DECIMAL(10, 4)) END
Obviously you'll also have to incorporate any other manipulations that you're already doing.
No need for replace, just concatenate a zero to your column, like
SELECT RTRIM('0' + LTRIM(column))
I presume your data is in a table.
Lets call this table 'DATA' and the column 'VALUE'
Then you might use the below query
UPDATE DATA SET VALUE = 0 where VALUE = ''
To select the value do the below
select case ltrim(rtrim([Gross_Weight])) when ''
THEN 0
ELSE ltrim(rtrim([Gross_Weight])) END
Let me know if i get the requirement wrong.

Removing Leading Zeros Part 2

SQL Server 2012
I have 3 columns in my table that will be using a function. '[usr].[Udf_OverPunch]'. and substring.
Here is my code:
[usr].[Udf_OverPunch](SUBSTRING(col001, 184, 11)) as REPORTED_GAP_DISCOUNT_PREVIOUS_AMOUNT
This function works appropriately for what I need it to do. It is basically converting symbols or letters to a designated number based on a data dictionary.
The problem I am having is that there are leading zeros. I just asked a questions about leading zeroes but it won't allow me to do it with the function columns because of the symbols cannot be converted to int.
This is what I am using to get rid of leading zeros (but leave one zero) in my code for the other columns:
cast(cast(SUBSTRING(col001, 217, 6) as int) as varchar(25)) as PREVIOUS_REPORTING_PERIOD
This works well at turning a value of '000000' to just one '0' or a value of '000060' to '60' but will not work with the function because of the symbol or letter (when trying to convert to int).
As I mentioned, I have 3 columns which produce values that look something like this when the function is not being used:
'0000019753{'
'0000019748G'
'0000019763H'
My goal here is to use the function while also removing the leading zeros (unless they are all zeros then keep one zero).
This is what I attempted that isn't working because the value contains a character that isn't an integer:
[usr].[Udf_OverPunch]cast(cast(SUBSTRING(col001, 184, 6) as int) as varchar(25)) as REPORTED_GAP_DISCOUNT_PREVIOUS_AMOUNT,
Please let me know if you have any ideas or need more information. :)
select case when col like '%[^0]%' then substring(col,patindex('%[^0]%',col),len(col)) when col like '%0%' then '0' else col end
from tab
or
select case when col like '%[^0]%' then right(col,len(ltrim(replace(col,'0',' ')))) when col like '%0%' then '0' else col end
from tab
I am handling such replacement with T-SQL CLR function that allows replacement using regular expressions. So, the solution will be like this:
[dbo].[fn_Utils_RegexReplace] ([value], '^0{1,}(?=.)', '')
You need to create such function because there are no regular expression support in T-SQL (build-in).
How to create regex replace function in T-SQL?
For example:
try this,
declare #i varchar(50)='0000019753}'--'0000019753'
select case when ISNUMERIC(#i)=1 then
cast(cast(#i as bigint) as varchar(50)) else #i end
or
[usr].[Udf_OverPunch]( case when ISNUMERIC(col001)=1 then
cast(cast(col001 as bigint) as varchar(50)) else col001 end)