Apologies for the multiple basic questions - I am very new to SQL and still trying to work things out.
I would like to insert records from my staging table to another table in my database, both removing the double quotes in the source file with a 'replace' function and converting the data from nvarchar (staging table) to datetime2. I can't quite work out how to do this: if I loop the 'case when' within 'replace', as below, then SQL doesn't recognise my data and nulls it out:
CASE WHEN ISDATE (REPLACE([Column1], '"', '')) = 1
THEN CONVERT(datetime2, Column1, 103)
ELSE null END
However if I loop my 'replace' within my 'case when', as below, SQL gives me an error message saying that it is unable to convert nvarchar into datetime2:
LTRIM(REPLACE([Column1], '"', '')
,CASE WHEN ISDATE(Column1) = 1 THEN CONVERT(datetime2, Column1, 103)
ELSE null END
What order / syntax do I need to be using to achieve this? An example of the data field would be:
"16/10/2017"
It uploads to my staging table as nvarchar
"16/10/2017"
and I would like to move it into my table2 as datetime2:
16/10/2017
Instead of isdate(), use try_convert():
TRY_CONVERT(datetime2, LTRIM(REPLACE([Column1], '"', ''), 103)
I think your confusion is that you need to do the string manipulation before the conversion. To do this, the string manipulation needs to be an argument to the conversion.
You are doing it right. The problem is, convert needs value without " ", and hence your convert was failing.
Just try this :
select
CASE WHEN ISDATE (REPLACE([Column1], '"', '')) = 1
THEN CONVERT(datetime2, (REPLACE([Column1], '"', '')), 103)
ELSE null END
from #tbl
more details : cast and convert doc
Related
I have a hive table which I am querying thru Impala and I have a very basic issue. The table has time duration as a string and sometimes due to error the value is not populated and it's empty.
When I use a query like the select cast(duration as decimal) from my table then if it encounters null in the sample, it fails with an error like UDF ERROR. STRING TO DECIMAL PARSE FAILED.
So my question is how to handle null here. I want Impala to ignore those rows which has null or simply treat as 0 rather than coming to a grinding halt.
I tried running with the -c option at impala-shell to ignore the error but it did not help.
When I try to use coalesce to return 0 instead of null it gives different error.
when I try ZEROIFNULL it gives another error.
Nothing seems to be working.
This seems a very basic issue so I assume there should be some good way to handle nulls. Unfortunately can not find anything even if I google for hours. so please help. :)
After lots of research I realized all the standard way to handle null was not working as the data which I was "thinking" to be null was not actually null but empty string. Seems like Impala treats null and empty string differently. Found another question on this board another post and that helped to handle empty string like below:
trim(duration) not like ''
using above I was able to exclude those rows which has empty string. thanks for reading.
This should work:
select
case col
when null then 0
when '' then 0
else cast(col as decimal(15, 2))
end as col
from table;
I prefer to use regex and nullif for these situations. Regex to pull out the decimal value and nullif to account for the empty string situation.
select cast(nullif(regexp_extract(null, '[\\+-]?\\d+\\.?\\d*', 0), '') as decimal(2,1)) as shouldbenull;
select cast(nullif(regexp_extract('', '[\\+-]?\\d+\\.?\\d*', 0), '') as decimal(2,1)) as shouldbenull;
select cast(nullif(regexp_extract('dirtydata', '[\\+-]?\\d+\\.?\\d*', 0), '') as decimal(2,1)) as shouldbenull;
select cast(nullif(regexp_extract('1.4', '[\\+-]?\\d+\\.?\\d*', 0), '') as decimal) as shouldroundto1;
select cast(nullif(regexp_extract('1.4', '[\\+-]?\\d+\\.?\\d*', 0), '') as decimal(2,1)) as shouldbe1_4;
If you need your final answer to be zero, then just wrap it in a zeroifnull function.
select zeroifnull(cast(nullif(regexp_extract('', '[\\+-]?\\d+\\.?\\d*', 0), '') as decimal(2,1))) as shouldbezero;
select zeroifnull(cast(nullif(regexp_extract('1.4', '[\\+-]?\\d+\\.?\\d*', 0), '') as decimal(2,1))) as shouldbe1_4;
I am trying to concat the columns here but when I encounter a column with empty / blank string, the concat failed.
I need to do some formatting for each column with different data type, so I am not using the CONCAT function. Using the conventional way like
SELECT CONVERT(varchar, [Priority]) + '~' + CONVERT(varchar,[AP_type]) + '~' + [AP_Name] + '~'
FROM table
Any suggestions on how I can concat empty string ?
Results I am looking :
0~0~~~~In~In
Thanks.
Couple of things.
always best to specify the length when converting to varchar. For example varchar(50)
concat() will handle nulls as empty string and there is no need to convert. Oddly enough, char(0) creates the odd behavior.
Example
Declare #YourTable Table ([priority] varchar(50),[ap_type] varchar(50),[ap_name] varchar(50),[ap_par] varchar(50),[infoText] varchar(50),[TxtCame] varchar(50),[TxtWent] varchar(50))
Insert Into #YourTable Values
(0,0,'','','','In','In')
,(0,0,'','',null,'In','In') -- Has Null
,(0,0,'','',char(0),'In','In') -- Has Char(0) ... Truncates without NullIf()
Select NewString = concat(priority,'~',ap_type,'~',ap_name,'~',ap_par,'~',NullIf(infoText,char(0)),'~',TxtCame,'~',TxtWent)
from #YourTable
Returns
NewString
0~0~~~~In~In
0~0~~~~In~In
0~0~~~~In~In -- NullIf() was required to fix
I'm trying to do a SELECT for concatenating 2 columns in MSSQL 2008R2.
COALESCE(SeriePolita, '') + COALESCE(CAST(NumarPolita AS numeric), '') as NrSeriePolita
Both SeriePolita and NumarPolita are NVARCHAR. The difference between them is that SeriePolita contains text and NumarPolita only number.
I need to convert NumarPolita from 0008838630 to 8838630.
As you ca see in the code line, i used CAST() for NumarPolita but i get this message:
Conversion failed when converting the nvarchar value 'AAA' to data type int.
Is there another way to covert a NVARCHAR to INT/NUMERIC?
Thank you!
You need to cast it back to NVARCHAR to concatenate it, since concatenation of nvarchar with numeric is not possible:
COALESCE(SeriePolita, '') + COALESCE(CAST(CAST(NumarPolita AS numeric) AS NVARCHAR), '') as NrSeriePolita
You can use a CASE expression and ISNUMERIC:
SELECT
CASE
WHEN ISNUMERIC(NumarPolita)
THEN COALESCE(SeriePolita, '') + COALESCE(CAST(NumarPolita AS numeric), '')
ELSE ''
END AS NrSeriePolita
FROM ...
Alternatively, if you only need the records where this conversion is possible, you can use ISNUMERIC in the WHERE clause.
How to convert Varchar to Int in sql server 2008.
i have following code when i tried to run it wont allowed me to convert Varchar to Int.
Select Cast([Column1] as INT)
Column1 is of Varchar(21) NOT NULL type and i wanted to convert it into Int.
actually i am trying to insert Column1 into another table having Field as INT.
can someone please help me to convert this ?
Spaces will not be a problem for cast, however characters like TAB, CR or LF will appear as spaces, will not be trimmed by LTRIM or RTRIM, and will be a problem.
For example try the following:
declare #v1 varchar(21) = '66',
#v2 varchar(21) = ' 66 ',
#v3 varchar(21) = '66' + char(13) + char(10),
#v4 varchar(21) = char(9) + '66'
select cast(#v1 as int) -- ok
select cast(#v2 as int) -- ok
select cast(#v3 as int) -- error
select cast(#v4 as int) -- error
Check your input for these characters and if you find them, use REPLACE to clean up your data.
Per your comment, you can use REPLACE as part of your cast:
select cast(replace(replace(#v3, char(13), ''), char(10), '') as int)
If this is something that will be happening often, it would be better to clean up the data and modify the way the table is populated to remove the CR and LF before it is entered.
you can use convert function :
Select convert(int,[Column1])
That is how you would do it, is it throwing an error? Are you sure the value you are trying to convert is convertible? For obvious reasons you cannot convert abc123 to an int.
UPDATE
Based on your comments I would remove any spaces that are in the values you are trying to convert.
That is the correct way to convert it to an INT as long as you don't have any alpha characters or NULL values.
If you have any NULL values, use
ISNULL(column1, 0)
Try the following code. In most case, it is caused by the comma issue.
cast(replace([FIELD NAME],',','') as float)
Try with below command, and it will ask all values to INT
select case when isnumeric(YourColumn + '.0e0') = 1
then cast(YourColumn as int)
else NULL
end /* case */
from YourTable
There are two type of convert method in SQL.
CAST and CONVERT have similar functionality. CONVERT is specific to SQL Server, and allows for a greater breadth of flexibility when converting between date and time values, fractional numbers, and monetary signifiers. CAST is the more ANSI-standard of the two functions.
Using Convert
Select convert(int,[Column1])
Using Cast
Select cast([Column1] as int)
This question already has answers here:
How to concatenate text from multiple rows into a single text string in SQL Server
(47 answers)
Closed 7 years ago.
I have a sql function that includes this code:
DECLARE #CodeNameString varchar(100)
SELECT CodeName FROM AccountCodes ORDER BY Sort
I need to concatenate all results from the select query into CodeNameString.
Obviously a FOREACH loop in C# code would do this, but how do I do it in SQL?
If you're on SQL Server 2005 or up, you can use this FOR XML PATH & STUFF trick:
DECLARE #CodeNameString varchar(100)
SELECT
#CodeNameString = STUFF( (SELECT ',' + CodeName
FROM dbo.AccountCodes
ORDER BY Sort
FOR XML PATH('')),
1, 1, '')
The FOR XML PATH('') basically concatenates your strings together into one, long XML result (something like ,code1,code2,code3 etc.) and the STUFF puts a "nothing" character at the first character, e.g. wipes out the "superfluous" first comma, to give you the result you're probably looking for.
UPDATE: OK - I understand the comments - if your text in the database table already contains characters like <, > or &, then my current solution will in fact encode those into <, >, and &.
If you have a problem with that XML encoding - then yes, you must look at the solution proposed by #KM which works for those characters, too. One word of warning from me: this approach is a lot more resource and processing intensive - just so you know.
DECLARE #CodeNameString varchar(max)
SET #CodeNameString=''
SELECT #CodeNameString=#CodeNameString+CodeName FROM AccountCodes ORDER BY Sort
SELECT #CodeNameString
#AlexanderMP's answer is correct, but you can also consider handling nulls with coalesce:
declare #CodeNameString nvarchar(max)
set #CodeNameString = null
SELECT #CodeNameString = Coalesce(#CodeNameString + ', ', '') + cast(CodeName as varchar) from AccountCodes
select #CodeNameString
For SQL Server 2005 and above use Coalesce for nulls and I am using Cast or Convert if there are numeric values -
declare #CodeNameString nvarchar(max)
select #CodeNameString = COALESCE(#CodeNameString + ',', '') + Cast(CodeName as varchar) from AccountCodes ORDER BY Sort
select #CodeNameString
from msdn Do not use a variable in a SELECT statement to concatenate values (that is, to compute aggregate values). Unexpected query results may occur. This is because all expressions in the SELECT list (including assignments) are not guaranteed to be executed exactly once for each output row
The above seems to say that concatenation as done above is not valid as the assignment might be done more times than there are rows returned by the select
Here is another real life example that works fine at least with 2008 release (and later).
This is the original query which uses simple max() to get at least one of the values:
SELECT option_name, Field_M3_name, max(Option_value) AS "Option value", max(Sorting) AS "Sorted"
FROM Value_list group by Option_name, Field_M3_name
ORDER BY option_name, Field_M3_name
Improved version, where the main improvement is that we show all values comma separated:
SELECT from1.keys, from1.option_name, from1.Field_M3_name,
Stuff((SELECT DISTINCT ', ' + [Option_value] FROM Value_list from2
WHERE COALESCE(from2.Option_name,'') + '|' + COALESCE(from2.Field_M3_name,'') = from1.keys FOR XML PATH(''),TYPE)
.value('text()[1]','nvarchar(max)'),1,2,N'') AS "Option values",
Stuff((SELECT DISTINCT ', ' + CAST([Sorting] AS VARCHAR) FROM Value_list from2
WHERE COALESCE(from2.Option_name,'') + '|' + COALESCE(from2.Field_M3_name,'') = from1.keys FOR XML PATH(''),TYPE)
.value('text()[1]','nvarchar(max)'),1,2,N'') AS "Sorting"
FROM ((SELECT DISTINCT COALESCE(Option_name,'') + '|' + COALESCE(Field_M3_name,'') AS keys, Option_name, Field_M3_name FROM Value_list)
-- WHERE
) from1
ORDER BY keys
Note that we have solved all possible NULL case issues that I can think of and also we fixed an error that we got for numeric values (field Sorting).