I need to replace a list of characters in a string with some mapped characters.
I have a table 'dbo.CharacterMappings' with 2 columns: 'CharacterToFilter' and 'ReplacementCharacter'.
Say that there are 3 records in this table:
Filter Replacement
$ s
# a
0 o
How would I replace all of the filter characters in a string based on these mappings?
i.e. 'Hell0 c#t$' needs to become 'Hello cats'.
I cant really think of any way of doing this without resorting to a table variable and then looping through it. I.e. have a table variable with a 'count' column then use a loop to select 1 row at a time based on this column. Then I can use the REPLACE function to update the characters one at a time.
Edit: I should note that I always want to strip out these characters (I don't need to worry about $5 -> s5 for example).
declare #s varchar(50)= 'Hell0 c#t$'
select #s = REPLACE(#s, CharacterToFilter, ReplacementCharacter)
from CharacterMappings
select #s
You could create a function:
CREATE FUNCTION [dbo].[ReplaceAll]
(
#text varchar(8000)
)
RETURNS VARCHAR(8000)
AS
BEGIN
SELECT #text =
REPLACE(#text,cm.Filter, cm.Replacement)
FROM CharacterMappings cm;
RETURN #text
END
Then this
select dbo.[ReplaceAll]('Hell0 c#t$');
returns Hello cats
Related
In SQL HANA, I need to find how many times a given word is repeated in a string column whose values are delimited by "," and output it as a separate column.
Example, the string column contains:
ZN,ZN,ZS,ZQ
Expected result for "ZN":
2
You might find it acceptable to search only the string ZN by ignoring the fact that there's a comma.
You may count the number of occurrences of any substring by using the string function OCCURRENCES_REGEXPR:
SELECT OCCURRENCES_REGEXPR('(ZN)' IN STRINGCOLUMN) "occurrences_zn" FROM TABLE;
If you really want to clearly specify that ZN is to be searched as an entire word between commas or at the edges, then you may find a better regular expression (the question is then more about regular expressions and not SQL HANA, and you may find existing answers in Stack Overflow).
I can't remember where I found the trick, but in SQL Server, the following works like a charm:
DECLARE #myStringToSearch nvarchar(250) = 'ZN,ZN,ZS,ZQ'
DECLARE #searchValue nvarchar(5) = 'ZN'
SELECT (LEN(#myStringToSearch) - LEN(REPLACE(#myStringToSearch, #searchValue, ''))) / LEN(#searchValue)
The last line compares the length of the original string with the length of the same string, but this time replacing your search value (ZN) with a blank string. In our case, this would result in 4, because ZN is 2 characters, and it was removed twice. However, we're not interested in how many characters were removed, but in how many times the value was encountered, so we divide that result by the length of your search string (2).
Output of the query:
2
You could easily implement this as a DEFAULT constraint in your table, provided your search string is the same across every row.
I wrote one anonymous block in sql , which can be converted to HANA Table function and can be used to achieve expected result.
DO
BEGIN
DECLARE FULL_STRING VARCHAR(100);
DECLARE TRIM_STRING VARCHAR(100);
DECLARE VAL_STRING VARCHAR(100);
FULL_STRING ='ZN,ZN,ZS,ZQ';
FULL_STRING=CONCAT(FULL_STRING,',');
--SELECT :FULL_STRING FROM DUMMY;
VAL_STRING=SUBSTRING(:FULL_STRING,1,LOCATE(:FULL_STRING,',',1)-1);
VAR_TABLE=SELECT :VAL_STRING STRINGVAL FROM DUMMY;
TRIM_STRING=SUBSTRING(:FULL_STRING,LOCATE(:FULL_STRING,',',1)+1 ,LENGTH(:FULL_STRING));
--SELECT * FROM :VAR_TABLE;
--SELECT :TRIM_STRING FROM DUMMY;
WHILE :TRIM_STRING IS NOT NULL AND LENGTH(:TRIM_STRING)>0
DO
VAL_STRING=SUBSTRING(:TRIM_STRING,1,LOCATE(:TRIM_STRING,',',1)-1);
--SELECT :VAL_STRING FROM DUMMY;
VAR_TABLE=SELECT STRINGVAL FROM :VAR_TABLE
UNION ALL
SELECT :VAL_STRING FROM DUMMY;
TRIM_STRING=SUBSTRING(:TRIM_STRING,LOCATE(:TRIM_STRING,',',1)+1 ,LENGTH(:TRIM_STRING));
--i=i+1;
--SELECT :TRIM_STRING FROM DUMMY;
END WHILE ;
SELECT STRINGVAL,COUNT(STRINGVAL) FROM :VAR_TABLE GROUP BY STRINGVAL;
--SELECT :TRIM_STRING FROM DUMMY;
I have a table with the following columns:
dbo.SomeInfo
- Id
- Name
- InfoCode
Now I need to update the above table's InfoCode as
Update dbo.SomeInfo
Set InfoCode= REPLACE(Replace(RTRIM(LOWER(Name)),' ','-'),':','')
This replaces all spaces with - & lowercase the name
When I do check the InfoCode, I see there are Names with some special characters like
Cathe Friedrich''s Low Impact
coffeyfit-cardio-box-&-burn
Jillian Michaels: Cardio
Then I am manually writing the update sql against this as
Update dbo.SomeInfo
SET InfoCode= 'cathe-friedrichs-low-impact'
where Name ='Cathe Friedrich''s Low Impact '
Now, this solution is not realistic for me. I checked the following links related to Regex & others around it.
UPDATE and REPLACE part of a string
https://www.codeproject.com/Questions/456246/replace-special-characters-in-sql
But none of them is hitting the requirement.
What I need is if there is any character other [a-z0-9] replace it - & also there should not be continuous -- in InfoCode
The above Update sql has set some values of InfoCode as the-dancer's-workout®----starter-package
Some Names have value as
Sleek Technique™
The Dancer's-workout®
How can I write Update sql that could handle all such special characters?
Using NGrams8K you could split the string into characters and then rather than replacing every non-acceptable character, retain only certain ones:
SELECT (SELECT '' + CASE WHEN N.token COLLATE Latin1_General_BIN LIKE '[A-z0-9]'THEN token ELSE '-' END
FROM dbo.NGrams8k(V.S,1) N
ORDER BY position
FOR XML PATH(''))
FROM (VALUES('Sleek Technique™'),('The Dancer''s-workout®'))V(S);
I use COLLATE here as on my default collation in my instance the '™' is ignored, therefore I use a binary collation. You may want to use COLLATE to switch the string back to its original collation outside of the subquery.
This approach is fully inlinable:
First we need a mock-up table with some test data:
DECLARe #SomeInfo TABLE (Id INT IDENTITY, InfoCode VARCHAR(100));
INSERT INTO #SomeInfo (InfoCode) VALUES
('Cathe Friedrich''s Low Impact')
,('coffeyfit-cardio-box-&-burn')
,('Jillian Michaels: Cardio')
,('Sleek Technique™')
,('The Dancer''s-workout®');
--This is the query
WITH cte AS
(
SELECT 1 AS position
,si.Id
,LOWER(si.InfoCode) AS SourceText
,SUBSTRING(LOWER(si.InfoCode),1,1) AS OneChar
FROM #SomeInfo si
UNION ALL
SELECT cte.position +1
,cte.Id
,cte.SourceText
,SUBSTRING(LOWER(cte.SourceText),cte.position+1,1) AS OneChar
FROM cte
WHERE position < DATALENGTH(SourceText)
)
,Cleaned AS
(
SELECT cte.Id
,(
SELECT CASE WHEN ASCII(cte2.OneChar) BETWEEN 65 AND 90 --A-Z
OR ASCII(cte2.OneChar) BETWEEN 97 AND 122--a-z
OR ASCII(cte2.OneChar) BETWEEN 48 AND 57 --0-9
--You can easily add more ranges
THEN cte2.OneChar ELSE '-'
--You can easily nest another CASE to deal with special characters like the single quote in your examples...
END
FROM cte AS cte2
WHERE cte2.Id=cte.Id
ORDER BY cte2.position
FOR XML PATH('')
) AS normalised
FROM cte
GROUP BY cte.Id
)
,NoDoubleHyphens AS
(
SELECT REPLACE(REPLACE(REPLACE(normalised,'-','<>'),'><',''),'<>','-') AS normalised2
FROM Cleaned
)
SELECT CASE WHEN RIGHT(normalised2,1)='-' THEN SUBSTRING(normalised2,1,LEN(normalised2)-1) ELSE normalised2 END AS FinalResult
FROM NoDoubleHyphens;
The first CTE will recursively (well, rather iteratively) travers down the string, character by character and a return a very slim set with one row per character.
The second CTE will then GROUP the Ids. This allows for a correlated sub-query, where the actual check is performed using ASCII-ranges. FOR XML PATH('') is used to re-concatenate the string. With SQL-Server 2017+ I'd suggest to use STRING_AGG() instead.
The third CTE will use a well known trick to get rid of multiple occurances of a character. Take any two characters which will never occur in your string, I use < and >. A string like a--b---c will come back as a<><>b<><><>c. After replacing >< with nothing we get a<>b<>c. Well, that's it...
The final SELECT will cut away a trailing hyphen. If needed you can add similar logic to get rid of a leading hyphen. With v2017+ There was TRIM('-') to make this easier...
The result
cathe-friedrich-s-low-impact
coffeyfit-cardio-box-burn
jillian-michaels-cardio
sleek-technique
the-dancer-s-workout
You can create a User-Defined-Function for something like that.
Then use the UDF in the update.
CREATE FUNCTION [dbo].LowerDashString (#str varchar(255))
RETURNS varchar(255)
AS
BEGIN
DECLARE #result varchar(255);
DECLARE #chr varchar(1);
DECLARE #pos int;
SET #result = '';
SET #pos = 1;
-- lowercase the input and remove the single-quotes
SET #str = REPLACE(LOWER(#str),'''','');
-- loop through the characters
-- while replacing anything that's not a letter to a dash
WHILE #pos <= LEN(#str)
BEGIN
SET #chr = SUBSTRING(#str, #pos, 1)
IF #chr LIKE '[a-z]' SET #result += #chr;
ELSE SET #result += '-';
SET #pos += 1;
END;
-- SET #result = TRIM('-' FROM #result); -- SqlServer 2017 and beyond
-- multiple dashes to one dash
WHILE #result LIKE '%--%' SET #result = REPLACE(#result,'--','-');
RETURN #result;
END;
GO
Example snippet using the function:
-- using a table variable for demonstration purposes
declare #SomeInfo table (Id int primary key identity(1,1) not null, InfoCode varchar(100) not null);
-- sample data
insert into #SomeInfo (InfoCode) values
('Cathe Friedrich''s Low Impact'),
('coffeyfit-cardio-box-&-burn'),
('Jillian Michaels: Cardio'),
('Sleek Technique™'),
('The Dancer''s-workout®');
update #SomeInfo
set InfoCode = dbo.LowerDashString(InfoCode)
where (InfoCode LIKE '%[^A-Z-]%' OR InfoCode != LOWER(InfoCode));
select *
from #SomeInfo;
Result:
Id InfoCode
-- -----------------------------
1 cathe-friedrichs-low-impact
2 coffeyfit-cardio-box-burn
3 jillian-michaels-cardio
4 sleek-technique-
5 the-dancers-workout-
I have a string like
ADN120_XK7760069988881LJ
in one of my columns. I have to extract the number with 13 digits length. For example, in the above case, I want to extract 7760069988881 in SQL Server.
using patindex() with substring() (using a variable for the pattern and replicate() to simplify repeating [0-9] 13 times):
create table t (val varchar(128));
insert into t values ('ADN120_XK7760069988881LJ');
declare #pattern varchar(128) = '%'+replicate('[0-9]',13)+'%';
select substring(val,patindex(#pattern,val),13)
from t;
rextester demo: http://rextester.com/MOEVG64754
returns 7760069988881
Creating TEMP table with your query
SELECT 'ADN120_XK7760069988881LJ' CODE INTO #TEMP
Solution using regular expression
SELECT SUBSTRING(CODE,PATINDEX('%[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]%',CODE),13)
FROM #TEMP
Couldn't reduce the number of times [0-9] used
Hope this helps
I am trying to use wildcard expression to fetch data related to a sequence of numbers. Can I know how to use a series of numbers inside wildcard expression LIKE [0-10].
here is my query:
select grade from table where grade LIKE [1-12]?
output: is 1 and 2
I referred to t-SQL book and they talk about LIKE N[1-12]. What's the difference between LIKE [1-12] and N[1-12]?
I can use between 1 and 12 to fetch my data. But I am just curious how to use a wildcard for series of numbers with LIKE operator?
In SQL Server, like has three wildcards. Underscore '_' represents any single character. % represents zero or more characters. And square brackets.
The expression between the square brackets represents one single character. So,
x like '[abc]'
matches "a", "b", or "c" -- and nothing else. The following matches any digit:
x like '[0123456789]'
This, however, starts to get cumbersome to type out. So, SQL Server offers the shorthand:
x like '[0-9]'
This just means any character from the range starting with 0 and ending at 9.
You could match any hex character with:
x like '[0-9ABCDEF]'
So, additional characters are allowed in the range.
When you write
x like '[1-12]'
You are saying x like the range of characters from 1 to 1, plus the character 2. This is more easily written as:
x like '[12]'
In any case, you shouldn't store numeric values as strings, and you shouldn't use like on numbers. It is much better to write:
grade between 1 and 12
Or something like that.
But if you already have a column with a sequence of numbers and don't know the size, what I've done was this function:
CREATE FUNCTION Keep_Only_Int (#X VARCHAR(MAX)) RETURNS BIGINT AS BEGIN
IF #X IS NULL RETURN NULL
DECLARE #T AS INT = LEN(#X), #I AS INT = 0, #J AS CHAR(1), #RET AS VARCHAR(50) = ''
WHILE #I < #T BEGIN
SET #I += 1
SET #J = SUBSTRING(#X, #I, 1)
IF ASCII(#J) BETWEEN 48 AND 57 --Numbers, is needed because ¹, ² and ³ are going to return true in the link
SET #RET += #J
END
IF LEN(#RET) > 19 RETURN NULL --Bigger then bigint
RETURN NULLIF(#RET, '')
END
An example of usage:
create table #a (content varchar(100))
insert #a values ('My number is 123, whatever')
insert #a values ('My number is 1234, whatever')
insert #a values ('My number is ¹²³4, whatever') --> Special numbers
insert #a values ('My number is one, whatever') --> No number
insert #a values ('My number is 1234567890123456789, whatever')
insert #a values ('My number is 12345678901234567890, whatever')--> This is too big!
select *
, dbo.Keep_Only_Int(content)
from #a
The function already convert the field to BIGINT, so you can use an between statement
select *
from #a
where dbo.Keep_Only_Int(content) between 1 and 2000
It is not focused on a great performance, if you are using a table too big I'd recomend creating a specific code for that
I have in my application a user defined function which takes a comma separated list as an argument. It splits the items and plugs them in to a table variable and returns the result.
This function works well, except that when the items in the comma separated list exceed 1000, it ignores the remainder. That is to say, if I plug in 1239, the first 1000 rows will be returned and the remaining 239 are entirely ignored. There are no errors when this occurs.
I can't help but feel that this is due to some sort of limitation that I should know about, but I can't seem to find any information about it. Is it a limitation on the amount of rows that can be stored in a table variable? Or am I missing something in the actual code itself? Can anyone assist? Going squirrely-eyed over here.
ALTER FUNCTION [dbo].[ufnConvertArrayToIntTable] (#IntArray VARCHAR(8000))
RETURNS #retIntTable TABLE
(
ID int
)
AS
BEGIN
DECLARE #Delimiter char(1)
SET #Delimiter = ','
DECLARE #Item varchar(8)
IF CHARINDEX(#Delimiter,#IntArray,0) <> 0
BEGIN
WHILE CHARINDEX(#Delimiter,#IntArray,0) <> 0
BEGIN
SELECT
#Item = RTRIM(LTRIM(SUBSTRING(#IntArray,1,CHARINDEX(#Delimiter,#IntArray,0)-1))),
#IntArray = RTRIM(LTRIM(SUBSTRING(#IntArray,CHARINDEX(#Delimiter,#IntArray,0)+1,LEN(#IntArray))))
IF LEN(#Item) > 0
INSERT INTO #retIntTable SELECT #Item
END
IF LEN(#IntArray) > 0
INSERT INTO #retIntTable SELECT #IntArray
END
ELSE
BEGIN
IF LEN(#IntArray) > 0
INSERT INTO #retIntTable SELECT #IntArray
END
RETURN
END;
You define your input variable as varchar(8000) and your #Item variable is varchar(8). Are your items typically 8 characters each? Is the string you send in w/ over 1000 items more than 8000 characters? Try changing your input to varchar(max) instead.
Are all of your comma seperated values 8 chars long? If so, then the input parameter will only be able to hold 888 (8000 / 9(including the comma) of them..
It's because your input parameter is limited to 8000 characters.
You might try calling the function using substring... Maybe:
WHERE
[myField] IN(Select ID from [dbo].[ufnConvertArrayToIntTable](substring(#inputarray, 1, 4000))
OR
[myField] IN(Select ID from [dbo].[ufnConvertArrayToIntTable](substring(#inputarray, 4001, 8000))
...