This is TERADATA (not SQL Server, not Oracle )
I have a column of phone numbers:
(312)9879878
(298)989-9878
430-394-2934
394s9048ds987
..........
I need to clean this column into
3129879878
2989899878
4303942934
3949048987
..........
So that only numbers should stay. All other letters, special characters, hyphens ... should be removed. How can I do this?
Which release of TD is running at your site?
If it's 14 or you got the oTranslate UDF installed you can simply do an old trick nesting Translate:
oTranslate(phonenum, oTranslate(phonenum, '0123456789', ''), '')
Answer :
DECLARE #Input varchar(1000)
SET #Input = '01 vishal 98-)6543'
DECLARE #pos INT
SET #Pos = PATINDEX('%[^0-9]%',#Input)
WHILE #Pos > 0
BEGIN
SET #Input = STUFF(#Input,#pos,1,'')
SET #Pos = PATINDEX('%[^0-9]%',#Input)
END
SELECT #Input
Thank You,
Vishal Patel
I have this function to pull numerics (0-9) from a string:
CREATE FUNCTION NumbersOnly(#STR VARCHAR(2000))
RETURNS VARCHAR(1000)
AS
BEGIN
DECLARE #N INT
DECLARE #NN VARCHAR(1000)
SET #N = 0
SET #NN = ''
WHILE #N <= LEN(#STR)
BEGIN
IF SUBSTRING(#STR,#N,1) >= '0'
AND SUBSTRING(#STR,#N,1) <= '9'
BEGIN
SET #NN = #NN + SUBSTRING(#STR,#N,1)
END
SET #N = #N + 1
END
RETURN #NN
END
Related
we are having to ID some data coming from a bad import and any help would be appreciated.
For instance a string like below and identifier char for the charindex.
SET #InputString = 'The quick brown fox jumped "over" or "under" the log'
SET #IdentifierChar = '"'
The issue we are having is that we can run our test against a hard coded string like the one above and get the result of 'over'. we have tried to put it in a while loop and then we get 'over','or','under'. The Expected Result for us would be only returning 'over', 'under' and not the or.
Our first go at a test was something like below just to see try and split:
DECLARE #InputString Nvarchar(MAX)
DECLARE #IdentifierChar NCHAR(1)
SET #InputString = 'The quick brown fox jumped "over" or "under" the log'
SET #IdentifierChar = '"'
declare #FirstID int
declare #SecondID int
declare #Length int
declare #TargetString Nvarchar(MAX)
Set #FirstID = CHARINDEX(#IdentifierChar,#InputString,1)
Set #SecondID = CHARINDEX(#IdentifierChar,#InputString,#FirstID+1)
Set #Length = #SecondID-#FirstID
Set #TargetString = SUBSTRING(#InputString,#FirstID+1,#Length-1)
Like I said then we literally just threw it in a hard coded loop and set the value of the substring to the last position of the identifier of the specialcharacter just to test and see how the charindex was splitting out the strings between the quotes and we did not think about it getting the 'or' as well.
so here is the dirty loop:
Set #COUNT = 0
Set #Length = 0
WHILE(#COUNT)<3
BEGIN
Set #FirstID = CHARINDEX(#IdentifierChar,#InputString,#Length)
Set #SecondID = CHARINDEX(#IdentifierChar,#InputString,#FirstID+1)
Set #Length = #SecondID-#FirstID
Set #TargetString = SUBSTRING(#InputString,#FirstID+1,#Length-1)
SET #COUNT = #COUNT+1
Set #Length =#SecondID
END
There's probably a better way to parse this out, but here's my minimal modification to your code to make it work, with comments where I changed things:
DECLARE #InputString Nvarchar(MAX)
DECLARE #IdentifierChar NCHAR(1)
SET #InputString = 'The quick brown fox jumped "over" or "under" the log'
SET #IdentifierChar = '"'
declare #FirstID int
declare #SecondID int
declare #Length int
declare #TargetString Nvarchar(MAX)
declare #COUNT int -- added this missing from your code above
Set #COUNT = 0
Set #Length = 0
WHILE(#COUNT)<2 -- only need 2 here now
BEGIN
Set #FirstID = CHARINDEX(#IdentifierChar,#InputString,#Length)
Set #SecondID = CHARINDEX(#IdentifierChar,#InputString,#FirstID+1)
Set #Length = #SecondID-#FirstID
Set #TargetString = SUBSTRING(#InputString,#FirstID+1,#Length-1)
SET #COUNT = #COUNT+1
Set #Length =#SecondID+1 -- added one
print #TargetString -- so we can see what it finds
END
Your main issue was updating #Length at the bottom of your loop -- when you thought you were pointing PAST the double quote after "over", you were actually pointing right at it and finding it a second time as an open-quote before " or ".
Here's a User Defined Function that I wrote and keep in my toolbox. This is a slightly off-label use, but it should work well for you.
If we consider this string to be five substrings, delimited by the four double-quote characters, then we can split on those just take substrings 2 and 4. (Getting a third or fourth quoted value would be as easy as getting substrings 6 or 8) Trying to get an element that doesn't exist will just return a NULL.
After executing the CREATE statement below, which will create the dbo.SPLIT_LIST function, you can call it like this:
declare #InputString varchar(255)
SET #InputString = 'The quick brown fox jumped "over" or "under" the log'
select dbo.SPLIT_LIST(#InputString, '"', 2, ''),
dbo.SPLIT_LIST(#InputString, '"', 4, '')
And you'll get your two output values. What's nice about a function like this is you can just throw it in your select statements and operate over many records at a time, instead of per each.
CREATE function dbo.SPLIT_LIST(
#string nvarchar(max),
#delimiter nvarchar(50),
#i int,
#text_qualifier nvarchar(1)
)
returns nvarchar(max)
/*
returns a selected element from a delimited list
select dbo.SPLIT_LIST_w_Qualifier('"twenty,one","twenty,two","twenty,three"', ',', 2,'"')
returns: 'twenty,two'
Note: can ignore embedded text qualifiers
*/
as
BEGIN
declare #value nvarchar(max),
#d_length int,
#next_delimiter nvarchar(51),
#q_length int, --length of the text qualifier
#trim int,
#using_qualifier int
set #d_length = len(#delimiter)
set #q_length = len(#text_qualifier)
set #string = ltrim(rtrim(#string))
--works by chopping off the leading value from the string each round
while #i > 0 and #string is not null and len(#string) > 0
begin
--if the remaining #string starts with the text qualifier,
--then the currently parsed value should end with the text qualifier+delimiter
if left(#string,1) = #text_qualifier
begin
set #using_qualifier = 1
--chop off leading qualifier
set #string = ltrim(right(#string,(len(#string)-len(#text_qualifier))))
end
else
begin
set #using_qualifier = 0
end
if (#using_qualifier = 0) -- If we are NOT using a text qualifier for this element
begin
if (charindex(#delimiter, #string) > 0) --If there is a remaining delimiter
begin
set #value = ltrim(rtrim(left(#string, charindex(#delimiter, #string)-1)))
set #string = ltrim(rtrim(right(#string, len(#string)-charindex(#delimiter, #string) - #d_length + 1)))
end
else --no remaining delimiters
begin
set #value = #string
set #string = null
end
end
else -- If we ARE using a text qualifier for this element
begin
if (charindex((#text_qualifier+#delimiter), #string) > 0) --If there is a remaining qualifier+delimiter
begin
set #value = ltrim(rtrim(left(#string, charindex((#text_qualifier+#delimiter), #string)-1)))
set #string = ltrim(rtrim(right(#string, len(#string)-charindex((#text_qualifier+#delimiter), #string) - #d_length - #q_length + 1)))
end
else --no remaining qualifier+delimiters
begin
--Does the remaining string END with the text qualifier?
if (charindex(REVERSE(#text_qualifier), REVERSE(#string)) = 1)
begin
set #value = ltrim(rtrim(left(#string, len(#string)-#q_length)))
set #string = null
end
else if (charindex((#text_qualifier), #string) > 0) --Is there a remaining qualifier at all?
begin
set #value = ltrim(rtrim(left(#string, charindex((#text_qualifier), #string)-1)))
set #string = null
end
else --no final closing qualifier
begin
set #value = #string
set #string = null
end
end
end
set #i = #i - 1
--print #value
end
if #i = 0 return #value --should exit here
return NULL --a parse too far exists here
END
Depending on the size of your data set and the complexity of your overall query, you could use a recursive CTE:
;with inputStr as (select 'The quick brown fox jumped "over" or "under" or "around" the log' as s)
,cte as (
select right(s,len(s) - charindex('"',s) + 1) as s --get the first occurence of "
from inputStr
union ALL
select right(right(s,len(s)-1),len(s) - charindex('"',s) + 1) as s --get the second occurence of " in the above string
from cte
where charindex('"',s) > 0 --where " exists
)
select left(s,charindex('"',right(s,len(s)-1))+1) as quoted --select from the first " to the second "
from cte
where (len(s) - len(replace(s,'"',''))) % 2 <> 1 --even number of "
and left(s,charindex('"',right(s,len(s)-1))+1) like '"%"'
Just wanted to update. I continued to toy around with the code and got something to work in case anyone wants to use similar logic in the future. This did what we discussed above.
DECLARE #TargetString NVARCHAR(MAX)
DECLARE #stringLen int
DECLARE #splitTbl TABLE(siteId NVARCHAR(MAX))
DECLARE #idChar NCHAR(1)
SET #TargetString = 'The quick brown fox jumped "over" or "under" the "log"'
SET #stringLen = CHARINDEX(' ', #TargetString)
SET #idChar = '"'
WHILE CHARINDEX(' ', #TargetString) > 0
BEGIN
SET #stringLen = CHARINDEX(' ', #TargetString);
INSERT INTO #splitTbl
SELECT SUBSTRING(#TargetString,1,#stringLen - 1);
SET #TargetString = SUBSTRING(#TargetString, #stringLen + 1,
LEN(#TargetString));
END
DECLARE #buildResults NVARCHAR(MAX)
INSERT INTO #splitTbl
SELECT #TargetString
DECLARE #buildLike NVARCHAR(MAX)
SET #buildLike = '%'+#idChar+'%'
SELECT #buildResults = COALESCE(#buildResults + ', ', '')
+SUBSTRING(siteId, 2, lEN(siteId) - 2)
FROM #splitTbl
WHERE siteId LIKE #buildLike
I want to toggle case of entire string.
I am able to do for characters, not for string.
DECLARE #Char AS VARCHAR(1)
SET #Char='a'
IF ASCII(#Char)>=97 AND ASCII(#Char) <=122
PRINT UPPER(#Char)
IF ASCII(#Char)>=65 AND ASCII(#Char) <=90
PRINT LOWER(#Char)
How, I can change case for entire string?
For Ex. "AbCdE", I want to change it to "aBcDe".
You can do it by creating functions:
First make function for one character:
CREATE FUNCTION ToggleChar
(
#Char VARCHAR(1)
)
RETURNS VARCHAR(1)
AS
BEGIN
RETURN CHAR(ASCII(UPPER(#Char))+ASCII(LOWER(#Char))-ASCII(#Char))
END
Then, create function for string:
CREATE FUNCTION ToggleCase
(
#Str VARCHAR(MAX)
)
RETURNS VARCHAR(MAX)
AS
BEGIN
DECLARE #ResultStr VARCHAR(MAX)
SET #ResultStr=''
WHILE ( #Str<>'')
BEGIN
SET #ResultStr=#ResultStr + [dbo].[ToggleChar](#Str)
SET #Str= SUBSTRING(#Str,2,LEN(#Str))
END
RETURN #ResultStr
END
Now, use this function to toggle string.
SELECT dbo.ToggleCase('AbCdE') AS ToggleString
Try this:
DECLARE #Name VARCHAR(10) = 'SaMplE'
DECLARE #Count INT = 1
WHILE #Count <= LEN(#Name)
BEGIN
SET #Name = STUFF(#Name, #Count, 1,
CASE
WHEN ASCII(SUBSTRING(#Name,#Count,1)) BETWEEN 97 AND 122 THEN
UPPER(SUBSTRING(#Name,#Count,1))
WHEN ASCII(SUBSTRING(#Name,#Count,1)) BETWEEN 65 AND 90 THEN
LOWER(SUBSTRING(#Name,#Count,1))
END)
SET #Count = #Count + 1
END
SELECT #Name
I have an issue converting the following hex string to ascii in tsql:
'd520088000000000000000004000000200000000000003770001c7cc5353482d322e302d4f70656e5353485f352e3570312044656269616e2d347562756e7475340d0a0000034c0614403d6544c243e7535320dc7ab5c3c8de0000007e6469666669652d68656c6c6d616e2d67726f75702d65786368616e67652d7368613235362c6469666669652d68656c6c6d616e2d67726f75702d65786368616e67652d736861312c6469666669652d68656c6c6d616e2d67726f757031342d736861312c6469666669652d68656c6c6d616e2d67726f7570312d73686131000000497373682d7273612d636572742d763030406f70656e7373682e636f6d2c7373682d6473732d636572742d763030406f70656e7373682e636f6d2c7373682d7273612c7373682d6473730000009d6165733132382d6374722c6165733139322d6374722c6165733235362d6374722c617263666f75723235362c617263666f75723132382c6165733132382d6362632c336465732d6362632c626c6f77666973682d6362632c636173743132382d6362632c6165733139322d6362632c6165733235362d6362632c617263666f75722c72696a6e6461656c2d636263406c797361746f722e6c69752e73650000009d6165733132382d6374722c6165733139322d6374722c6165733235362d6374722c617263666f75723235362c617263666f75723132382c6165733132382d6362632c336465732d6362632c626c6f77666973682d6362632c636173743132382d6362632c6165733139322d6362632c6165733235362d6362632c617263666f75722c72696a6e6461656c2d636263406c797361746f722e6c69752e736500000069686d61632d6d64352c686d61632d736861312c756d61632d3634406f70656e7373682e636f6d2c686d61632d726970656d643136302c686d61632d726970656d64313630406f70656e7373682e636f6d2c686d61632d736861312d39362c686d61632d6d64352d393600000069686d61632d6d64352c686d61632d736861312c756d61632d3634406f70656e7373682e636f6d2c686d61632d726970656d643136302c686d61632d726970656d64313630406f70656e7373682e636f6d2c686d61632d736861312d39362c686d61632d6d64352d39360000001a6e6f6e652c7a6c6962406f70656e7373682e636f6d2c7a6c69620000001a6e6f6e652c7a6c6962406f70656e7373682e636f6d2c7a6c69620000000000000000000000000000000000000000'
It should convert to
'? ??????????#??????????w????SSH-2.0-OpenSSH_5.5p1
Debian-4ubuntu4?????L??#=eD?C?SS
?z???????~diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1???Issh-rsa-cert-v00#openssh.com,ssh-dss-cert-v00#openssh.com,ssh-rsa,ssh-dss????aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc#lysator.liu.se????aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc#lysator.liu.se???ihmac-md5,hmac-sha1,umac-64#openssh.com,hmac-ripemd160,hmac-ripemd160#openssh.com,hmac-sha1-96,hmac-md5-96???ihmac-md5,hmac-sha1,umac-64#openssh.com,hmac-ripemd160,hmac-ripemd160#openssh.com,hmac-sha1-96,hmac-md5-96????none,zlib#openssh.com,zlib????none,zlib#openssh.com,zlib????????????????????'
Any ideas? I tried using cast and convert but no success.
Any help is much appreciated.
I am using the below function no outside code - tables are populated by an external application
*
declare #hexstring VARCHAR(8000))
RETURNS VARCHAR(8000)
AS
begin
declare #char1 char(1), #char2 char(1), #strlen int, #currpos int, #result varchar(8000)
set #strlen=len(#hexstring)
set #currpos=1
set #result=''
while #currpos<#strlen
begin
set #char1=substring(#hexstring,#currpos,1)
set #char2=substring(#hexstring,#currpos+1,1)
if (#char1 between '0' and '9' or #char1 between 'A' and 'F')
and (#char2 between '0' and '9' or #char2 between 'A' and 'F')
set #result=#result+
char((ascii(#char1)-case when #char1 between '0' and '9' then 48 else 55 end)*16+
ascii(#char2)-case when #char2 between '0' and '9' then 48 else 55 end)
set #currpos = #currpos+2
end
return #result
end
GO
*
Here is a SQL function that will convert a hex string:
BEGIN
DECLARE #hexstring AS VARCHAR(8000)
SET #hexstring = '4368726973204Ce4747461'
DECLARE #strlen AS INT;
SET #strlen = Len(#hexstring)
DECLARE #currpos AS INT
SET #currpos = 1
DECLARE #hexpos AS VARCHAR(16)
SET #hexpos = '0123456789abcdef'
DECLARE #result AS VARCHAR(8000)
SET #result = ''
DECLARE #ch AS INT
WHILE #currpos < #strlen
BEGIN
SET #ch = CONVERT( INT, 16 * (CHARINDEX( SUBSTRING( #hexstring, #currpos, 1), #hexpos, 1) - 1)
+ (CHARINDEX(SUBSTRING(#hexstring, #currpos+1, 1), #hexpos, 1) - 1))
SET #result = #result + CASE WHEN #ch >= 32 AND #ch < 128 THEN CHAR(#ch) ELSE '?' END
SET #currpos = #currpos + 2
END
SELECT #result
END
I added a short hex string to show the function works.
Your hex string didn't convert so well. It may be because of the nulls in it, so I added the #char >= 32 part to strip out control codes (only convert printable ASCII characters, otherwise insert a ?) and you get a string that looks like the one you are after.
The conversion can be done with a single statement. Since you were using a function I provided my answer as such. Note that this solution requires that the input #hexstring is not prefixed with 0x.
CREATE FUNCTION [dbo].[HexToAscii]
(
#hexstring VARCHAR(MAX)
)
RETURNS VARCHAR(MAX)
AS
BEGIN
RETURN CONVERT(VARCHAR(MAX), CONVERT(VARBINARY(MAX), #hexstring, 2))
END
So I'm trying to create my first recursive udf (using MS SQL) to strip anything that's not letters and numbers from a string.
This was inspired by this post (Replace with wildcard, in SQL)
CREATE FUNCTION uf_RemoveNonAlphaNumericChar(
#p_CharIndex int,
#p_Value Varchar(max) )
RETURNS varchar(max)
AS
BEGIN
SET #p_CharIndex = PATINDEX('%[^0-9,a-z]%', #p_Value)
SET #p_Value = STUFF(#p_Value,#p_CharIndex , 1, SPace(0) )
IF #p_CharIndex > 0
BEGIN
EXEC #p_Value = uf_RemoveNonAlphaNumericChar #p_CharIndex = #p_CharIndex,
#p_Value = #p_Value
END
RETURN #p_Value
END
This is one step in a bigger problem where I'm trying to split a string that could be XXX###YYYY into three parts when some of the parts may be missing.
And I'm trying to do it without a while loop (that solution already exists but runs slow).
if Patindex had a start position (in MS SQL), I would already be done. Of course, it would also not be as much fun. Or as cuss-filled...
I found yours problem. You removing symbol if even you dont find it ;)
Look at updated answer:
CREATEFUNCTION uf_RemoveNonAlphaNumericChar(
#p_CharIndex int,
#p_Value Varchar(max) )
RETURNS varchar(max)
AS
BEGIN
SET #p_CharIndex = PATINDEX('%[^0-9,a-z]%', #p_Value)
IF #p_CharIndex > 0
BEGIN
SET #p_Value = STUFF(#p_Value,#p_CharIndex , 1, SPace(0) )
EXEC #p_Value = uf_RemoveNonAlphaNumericChar #p_CharIndex = #p_CharIndex,
#p_Value = #p_Value
END
RETURN #p_Value
END
Does it have to be recursion?
CREATE FUNCTION [dbo].[uf_RemoveNonAlphaNumericChar]
(
#val varchar(max)
)
RETURNS varchar(1000)
AS
BEGIN
DECLARE #s VARCHAR(max), #i INT
SET #s = #val
SET #i = PATINDEX('%[^a-z0-9]%', #s)
WHILE #i > 0
BEGIN
SET #s = REPLACE(#s, SUBSTRING(#s, #i, 1), '')
SELECT #i = PATINDEX('%[^a-z0-9]%', #s)
END
RETURN #s
END
Input
Column
ab2e
mnop
a2t1y
output
Id Col1 Col2 Col3 Col4 Col5 Col6
1 a b e
2 m n o p
3 a t y
The numbers indicates the number of spaces
Since in the first input, there is 2 after b, so the letter e will appear after 2 spaces from b.
In the second input since there is no space, the the letters will appear after each other
Thanks
If you've already got a way of distributing a 'normal' string's contents between the columns and only need a solution for expanding strings like ab2e into strings like ab[space][space]e, then here's a possible solution:
DECLARE #InputString varchar(100), #pos int, #result varchar(100);
SET #InputString = 'a2t1y';
SET #result = #InputString;
SET #pos = PATINDEX('%[0-9]%', #result);
WHILE #pos <> 0 BEGIN
SET #result = STUFF(#result, #pos, 1, SPACE(SUBSTRING(#result, #pos, 1)));
SET #pos = PATINDEX('%[0-9]%', #result);
END
SELECT #result;
The output:
---------------------
a t y
It would probably be a nice idea to implement it as a function:
CREATE FUNCTION ExpandString (#String varchar(100))
RETURNS varchar(100)
AS BEGIN
DECLARE #pos int, #result varchar(100);
SET #result = #String;
SET #pos = PATINDEX('%[0-9]%', #result);
WHILE #pos <> 0 BEGIN
SET #result = STUFF(#result, #pos, 1, SPACE(SUBSTRING(#result, #pos, 1)));
SET #pos = PATINDEX('%[0-9]%', #result);
END
RETURN #result;
END
so you could call it on a column like this:
SELECT …, dbo.ExpandString(t.SomeColumn), …
It should be noted, though, that this solution only supports single-digit 'macros', i.e. a12b would be converted to a[1 space][2 spaces]b with this function, which is not necessarily what you'd expect. So, if you need it to recognise integers as sequences of numeric characters between non-numerics, here's an alternative solution:
CREATE FUNCTION ExpandString (#String varchar(100))
RETURNS varchar(100)
AS BEGIN
DECLARE #pos int, #lastpos int, #len int, #isnum bit,
#sub varchar(100), #result varchar(100);
SET #result = '';
SET #pos = 1;
SET #len = LEN(#String);
SET #isnum = ISNUMERIC(SUBSTRING(#String, #pos, 1));
WHILE #pos <= #len BEGIN
SET #lastpos = #pos;
WHILE #pos <= #len AND ISNUMERIC(SUBSTRING(#String, #pos, 1)) = #isnum
SET #pos = #pos + 1;
SET #sub = SUBSTRING(#String, #lastpos, #pos - #lastpos);
SET #result = #result + CASE #isnum WHEN 1 THEN SPACE(#sub) ELSE #sub END;
SET #isnum = #isnum ^ 1;
END;
RETURN #result;
END
Both versions recognise numbers both at the beginning and at the end of the input string.