LIKE operator for sequence of Numbers - sql

I am trying to use wildcard expression to fetch data related to a sequence of numbers. Can I know how to use a series of numbers inside wildcard expression LIKE [0-10].
here is my query:
select grade from table where grade LIKE [1-12]?
output: is 1 and 2
I referred to t-SQL book and they talk about LIKE N[1-12]. What's the difference between LIKE [1-12] and N[1-12]?
I can use between 1 and 12 to fetch my data. But I am just curious how to use a wildcard for series of numbers with LIKE operator?

In SQL Server, like has three wildcards. Underscore '_' represents any single character. % represents zero or more characters. And square brackets.
The expression between the square brackets represents one single character. So,
x like '[abc]'
matches "a", "b", or "c" -- and nothing else. The following matches any digit:
x like '[0123456789]'
This, however, starts to get cumbersome to type out. So, SQL Server offers the shorthand:
x like '[0-9]'
This just means any character from the range starting with 0 and ending at 9.
You could match any hex character with:
x like '[0-9ABCDEF]'
So, additional characters are allowed in the range.
When you write
x like '[1-12]'
You are saying x like the range of characters from 1 to 1, plus the character 2. This is more easily written as:
x like '[12]'
In any case, you shouldn't store numeric values as strings, and you shouldn't use like on numbers. It is much better to write:
grade between 1 and 12
Or something like that.

But if you already have a column with a sequence of numbers and don't know the size, what I've done was this function:
CREATE FUNCTION Keep_Only_Int (#X VARCHAR(MAX)) RETURNS BIGINT AS BEGIN
IF #X IS NULL RETURN NULL
DECLARE #T AS INT = LEN(#X), #I AS INT = 0, #J AS CHAR(1), #RET AS VARCHAR(50) = ''
WHILE #I < #T BEGIN
SET #I += 1
SET #J = SUBSTRING(#X, #I, 1)
IF ASCII(#J) BETWEEN 48 AND 57 --Numbers, is needed because ¹, ² and ³ are going to return true in the link
SET #RET += #J
END
IF LEN(#RET) > 19 RETURN NULL --Bigger then bigint
RETURN NULLIF(#RET, '')
END
An example of usage:
create table #a (content varchar(100))
insert #a values ('My number is 123, whatever')
insert #a values ('My number is 1234, whatever')
insert #a values ('My number is ¹²³4, whatever') --> Special numbers
insert #a values ('My number is one, whatever') --> No number
insert #a values ('My number is 1234567890123456789, whatever')
insert #a values ('My number is 12345678901234567890, whatever')--> This is too big!
select *
, dbo.Keep_Only_Int(content)
from #a
The function already convert the field to BIGINT, so you can use an between statement
select *
from #a
where dbo.Keep_Only_Int(content) between 1 and 2000
It is not focused on a great performance, if you are using a table too big I'd recomend creating a specific code for that

Related

SQL Return only specific characters in SELECT from VARCHAR column

I am stuck on an issue in SQL Server. I have a VARCHAR column called Name in my table:
I am trying to get the column to only return valid characters when doing a select on it. For example, I am only accepting any letters [A-Z], numbers [0-9] or a question mark [?] but list can change so need to be flexible. The reason why I am only accepting certain characters is due to our supplier specification which I send data to. It will break their system if I send then an invalid character.
SELECT Name FROM #table
For the purpose of asking the question, I have included a small example below where I insert into a table variable. My question is aimed towards the select part as I am trying to work on data already inserted.
DECLARE #table AS TABLE
(
ID INT ,
Name VARCHAR(500) ,
Age INT
)
INSERT INTO #table
VALUES (1, 'Hello ## World! Test8.?##', 23),
(2, 'Need specific characters only Test8.? ]]', 22)
-- Only accept [A-Z][0-9][?]
SELECT Name FROM #table
Please note, the scenario above is a small example and the data is just dummy data I just added to make it easier to ask the question. The data already exist. I have no control over it. I only have access to it and need to tidy it up via doing a select.
Expected results with only returning valid characters:
For first row it will return "Hello World Test8?" and for second row it will return "Need specific chatacters only Test8?".
What I have tried so far is doing a replace on the select to get the result:
-- Only accept [A-Z][0-9][?]
SELECT REPLACE(REPLACE(REPLACE(REPLACE(Name, '#', ''), '!', ''), ']', ''), '.', '') FROM #table
However, this only works if I knew which characters are invalid. As mentioned earlier in question, I only know the opposite which are valid characters. A valid character is a letter [A-Z] or number [0-9] or a question mark. This means I have a massive list of invalid characters I need to add if I went towards a replace solution.
Any idea how I can achieve this within the select statement?
I am on SQL Server Version 2012.
There is no built-in functionality for this, though this was implemented by people before:
https://raresql.com/2013/03/11/sql-server-function-to-parse-alphanumeric-characters-from-string/
Using this (all copyrights to the author) would be:
CREATE FUNCTION dbo.[UDF_Extract_Alphanumeric_From_String]
(
#String VARCHAR(MAX) -- Variable for string
)
RETURNS VARCHAR(MAX)
BEGIN
DECLARE #RETURN_STRING VARCHAR(MAX)
 
; WITH N1 (n) AS (SELECT 1 UNION ALL SELECT 1),
N2(n) AS (SELECT 1 FROM N1 AS X, N1 AS Y),
N3(n) AS (SELECT 1 FROM N2 AS X, N2 AS Y),
N4(n) AS (SELECT ROW_NUMBER() OVER(ORDER BY X.n)
FROM N3 AS X, N3 AS Y)
 
SELECT #RETURN_STRING=ISNULL(#RETURN_STRING,'')+ SUBSTRING(#String,Nums.n,1)
FROM N4 Nums
WHERE Nums.n <=LEN(#String) AND PATINDEX('%[0-9A-Za-z ]%',SUBSTRING(#String,Nums.n,1)) > 0
 
RETURN #RETURN_STRING
END
 
GO
SELECT dbo.[UDF_Extract_Alphanumeric_From_String] ('Hello ## World! Test8.?##') as [Result]
--OUTPUT
Result
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Hello World Test8
(1 row affected)
Completion time: 2022-12-20T22:47:24.8872397+01:00
Here's a different approach with a UDF...
CREATE FUNCTION LeaveValidChars
(
#p1 varchar(100)
)
RETURNS varchar(100)
AS
BEGIN
DECLARE #Result varchar(100)='', #p INT = 0, #c CHAR(1);
WHILE #p < LEN(#p1)
BEGIN
SET #c=substring(#p1, #p, 1)
IF CHARINDEX(#c,'ABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890?')>0
SET #result=#result+#c;
SET #p=#p+1
END
RETURN #Result;
END
GO

How to replace all special characters in string

I have a table with the following columns:
dbo.SomeInfo
- Id
- Name
- InfoCode
Now I need to update the above table's InfoCode as
Update dbo.SomeInfo
Set InfoCode= REPLACE(Replace(RTRIM(LOWER(Name)),' ','-'),':','')
This replaces all spaces with - & lowercase the name
When I do check the InfoCode, I see there are Names with some special characters like
Cathe Friedrich''s Low Impact
coffeyfit-cardio-box-&-burn
Jillian Michaels: Cardio
Then I am manually writing the update sql against this as
Update dbo.SomeInfo
SET InfoCode= 'cathe-friedrichs-low-impact'
where Name ='Cathe Friedrich''s Low Impact '
Now, this solution is not realistic for me. I checked the following links related to Regex & others around it.
UPDATE and REPLACE part of a string
https://www.codeproject.com/Questions/456246/replace-special-characters-in-sql
But none of them is hitting the requirement.
What I need is if there is any character other [a-z0-9] replace it - & also there should not be continuous -- in InfoCode
The above Update sql has set some values of InfoCode as the-dancer's-workout®----starter-package
Some Names have value as
Sleek Technique™
The Dancer's-workout®
How can I write Update sql that could handle all such special characters?
Using NGrams8K you could split the string into characters and then rather than replacing every non-acceptable character, retain only certain ones:
SELECT (SELECT '' + CASE WHEN N.token COLLATE Latin1_General_BIN LIKE '[A-z0-9]'THEN token ELSE '-' END
FROM dbo.NGrams8k(V.S,1) N
ORDER BY position
FOR XML PATH(''))
FROM (VALUES('Sleek Technique™'),('The Dancer''s-workout®'))V(S);
I use COLLATE here as on my default collation in my instance the '™' is ignored, therefore I use a binary collation. You may want to use COLLATE to switch the string back to its original collation outside of the subquery.
This approach is fully inlinable:
First we need a mock-up table with some test data:
DECLARe #SomeInfo TABLE (Id INT IDENTITY, InfoCode VARCHAR(100));
INSERT INTO #SomeInfo (InfoCode) VALUES
('Cathe Friedrich''s Low Impact')
,('coffeyfit-cardio-box-&-burn')
,('Jillian Michaels: Cardio')
,('Sleek Technique™')
,('The Dancer''s-workout®');
--This is the query
WITH cte AS
(
SELECT 1 AS position
,si.Id
,LOWER(si.InfoCode) AS SourceText
,SUBSTRING(LOWER(si.InfoCode),1,1) AS OneChar
FROM #SomeInfo si
UNION ALL
SELECT cte.position +1
,cte.Id
,cte.SourceText
,SUBSTRING(LOWER(cte.SourceText),cte.position+1,1) AS OneChar
FROM cte
WHERE position < DATALENGTH(SourceText)
)
,Cleaned AS
(
SELECT cte.Id
,(
SELECT CASE WHEN ASCII(cte2.OneChar) BETWEEN 65 AND 90 --A-Z
OR ASCII(cte2.OneChar) BETWEEN 97 AND 122--a-z
OR ASCII(cte2.OneChar) BETWEEN 48 AND 57 --0-9
--You can easily add more ranges
THEN cte2.OneChar ELSE '-'
--You can easily nest another CASE to deal with special characters like the single quote in your examples...
END
FROM cte AS cte2
WHERE cte2.Id=cte.Id
ORDER BY cte2.position
FOR XML PATH('')
) AS normalised
FROM cte
GROUP BY cte.Id
)
,NoDoubleHyphens AS
(
SELECT REPLACE(REPLACE(REPLACE(normalised,'-','<>'),'><',''),'<>','-') AS normalised2
FROM Cleaned
)
SELECT CASE WHEN RIGHT(normalised2,1)='-' THEN SUBSTRING(normalised2,1,LEN(normalised2)-1) ELSE normalised2 END AS FinalResult
FROM NoDoubleHyphens;
The first CTE will recursively (well, rather iteratively) travers down the string, character by character and a return a very slim set with one row per character.
The second CTE will then GROUP the Ids. This allows for a correlated sub-query, where the actual check is performed using ASCII-ranges. FOR XML PATH('') is used to re-concatenate the string. With SQL-Server 2017+ I'd suggest to use STRING_AGG() instead.
The third CTE will use a well known trick to get rid of multiple occurances of a character. Take any two characters which will never occur in your string, I use < and >. A string like a--b---c will come back as a<><>b<><><>c. After replacing >< with nothing we get a<>b<>c. Well, that's it...
The final SELECT will cut away a trailing hyphen. If needed you can add similar logic to get rid of a leading hyphen. With v2017+ There was TRIM('-') to make this easier...
The result
cathe-friedrich-s-low-impact
coffeyfit-cardio-box-burn
jillian-michaels-cardio
sleek-technique
the-dancer-s-workout
You can create a User-Defined-Function for something like that.
Then use the UDF in the update.
CREATE FUNCTION [dbo].LowerDashString (#str varchar(255))
RETURNS varchar(255)
AS
BEGIN
DECLARE #result varchar(255);
DECLARE #chr varchar(1);
DECLARE #pos int;
SET #result = '';
SET #pos = 1;
-- lowercase the input and remove the single-quotes
SET #str = REPLACE(LOWER(#str),'''','');
-- loop through the characters
-- while replacing anything that's not a letter to a dash
WHILE #pos <= LEN(#str)
BEGIN
SET #chr = SUBSTRING(#str, #pos, 1)
IF #chr LIKE '[a-z]' SET #result += #chr;
ELSE SET #result += '-';
SET #pos += 1;
END;
-- SET #result = TRIM('-' FROM #result); -- SqlServer 2017 and beyond
-- multiple dashes to one dash
WHILE #result LIKE '%--%' SET #result = REPLACE(#result,'--','-');
RETURN #result;
END;
GO
Example snippet using the function:
-- using a table variable for demonstration purposes
declare #SomeInfo table (Id int primary key identity(1,1) not null, InfoCode varchar(100) not null);
-- sample data
insert into #SomeInfo (InfoCode) values
('Cathe Friedrich''s Low Impact'),
('coffeyfit-cardio-box-&-burn'),
('Jillian Michaels: Cardio'),
('Sleek Technique™'),
('The Dancer''s-workout®');
update #SomeInfo
set InfoCode = dbo.LowerDashString(InfoCode)
where (InfoCode LIKE '%[^A-Z-]%' OR InfoCode != LOWER(InfoCode));
select *
from #SomeInfo;
Result:
Id InfoCode
-- -----------------------------
1 cathe-friedrichs-low-impact
2 coffeyfit-cardio-box-burn
3 jillian-michaels-cardio
4 sleek-technique-
5 the-dancers-workout-

Replacing characters in a string based on rows in a table sql

I need to replace a list of characters in a string with some mapped characters.
I have a table 'dbo.CharacterMappings' with 2 columns: 'CharacterToFilter' and 'ReplacementCharacter'.
Say that there are 3 records in this table:
Filter Replacement
$ s
# a
0 o
How would I replace all of the filter characters in a string based on these mappings?
i.e. 'Hell0 c#t$' needs to become 'Hello cats'.
I cant really think of any way of doing this without resorting to a table variable and then looping through it. I.e. have a table variable with a 'count' column then use a loop to select 1 row at a time based on this column. Then I can use the REPLACE function to update the characters one at a time.
Edit: I should note that I always want to strip out these characters (I don't need to worry about $5 -> s5 for example).
declare #s varchar(50)= 'Hell0 c#t$'
select #s = REPLACE(#s, CharacterToFilter, ReplacementCharacter)
from CharacterMappings
select #s
You could create a function:
CREATE FUNCTION [dbo].[ReplaceAll]
(
#text varchar(8000)
)
RETURNS VARCHAR(8000)
AS
BEGIN
SELECT #text =
REPLACE(#text,cm.Filter, cm.Replacement)
FROM CharacterMappings cm;
RETURN #text
END
Then this
select dbo.[ReplaceAll]('Hell0 c#t$');
returns Hello cats

SQL How to find if all values from one field exist in another field in any order

I am trying to match data from an external source to an in house source. For example one table would have a field with a value of "black blue" and another table would have a field with a value of "blue black". I am trying to figure out how to check if all individual words in the first table are contained in a record the 2nd table in any order. It's not always two words that need to be compared it could be 3 or 4 as well. I know I could use a cursor and build dynamic sql substituting the space with the AND keywod and using the contains function but I'm hoping not to have to do that.
Any help would be much appreciated.
Try doing something like this: Split the data from the first table on the space into a temporary table variable. Then use CHARINDEX to determine if each word is contained in the second table's record. Then just do this for each word in the first record and if the count is the same as the successful checks then you know every word from the first record is used in the second.
Edit: Use a Split function such as:
CREATE FUNCTION dbo.Split (#sep char(1), #s varchar(512))
RETURNS table
AS
RETURN (
WITH Pieces(pn, start, stop) AS (
SELECT 1, 1, CHARINDEX(#sep, #s)
UNION ALL
SELECT pn + 1, stop + 1, CHARINDEX(#sep, #s, stop + 1)
FROM Pieces
WHERE stop > 0
)
SELECT pn,
SUBSTRING(#s, start, CASE WHEN stop > 0 THEN stop-start ELSE 512 END) AS s
FROM Pieces
)
Here's another method you could try, you could sample some simple attributes of your strings such as, length, number of spaces, etc.; then you could use a cross-join to create all of the possible string match combinations.
Then within your where-clause you can sort by matches, the final piece of which in this example is a check using the patindex() function to see if the sampled piece of the first string is in the second string.
-- begin sample table variable set up
declare #s table(
id int identity(1,1)
,string varchar(255)
,numSpace int
,numWord int
,lenString int
,firstPatt varchar(255)
);
declare #t table(
id int identity(1,1)
,string varchar(255)
,numSpace int
,numWord int
,lenString int
);
insert into #t(string)
values ('my name');
insert into #t(string)
values ('your name');
insert into #t(string)
values ('run and jump');
insert into #t(string)
values ('hello my name is');
insert into #s(string)
values ('name my');
insert into #s(string)
values ('name your');
insert into #s(string)
values ('jump and run');
insert into #s(string)
values ('my name is hello');
update #s
set numSpace = len(string)-len(replace(string,' ',''));
update #s
set numWord = len(string)-len(replace(string,' ',''))+1;
update #s
set lenString = len(string);
update #s
set firstPatt = rtrim(substring(string,1,charindex(' ',string,0)));
update #t
set numSpace = len(string)-len(replace(string,' ',''));
update #t
set numWord = len(string)-len(replace(string,' ',''))+1;
update #t
set lenString = len(string);
-- end sample table variable set up
-- select all combinations of strings using a cross join
-- and sort the entries in your where clause
-- the pattern index checks to see if the sampled string
-- from the first table variable is in the second table variable
select *
from
#s s cross join #t t
where
s.numSpace = t.numspace
and s.numWord = t.numWord
and s.lenString = t.lenString
and patindex('%'+s.firstPatt+'%',t.string)>0;

Table variable row limitation?

I have in my application a user defined function which takes a comma separated list as an argument. It splits the items and plugs them in to a table variable and returns the result.
This function works well, except that when the items in the comma separated list exceed 1000, it ignores the remainder. That is to say, if I plug in 1239, the first 1000 rows will be returned and the remaining 239 are entirely ignored. There are no errors when this occurs.
I can't help but feel that this is due to some sort of limitation that I should know about, but I can't seem to find any information about it. Is it a limitation on the amount of rows that can be stored in a table variable? Or am I missing something in the actual code itself? Can anyone assist? Going squirrely-eyed over here.
ALTER FUNCTION [dbo].[ufnConvertArrayToIntTable] (#IntArray VARCHAR(8000))
RETURNS #retIntTable TABLE
(
ID int
)
AS
BEGIN
DECLARE #Delimiter char(1)
SET #Delimiter = ','
DECLARE #Item varchar(8)
IF CHARINDEX(#Delimiter,#IntArray,0) <> 0
BEGIN
WHILE CHARINDEX(#Delimiter,#IntArray,0) <> 0
BEGIN
SELECT
#Item = RTRIM(LTRIM(SUBSTRING(#IntArray,1,CHARINDEX(#Delimiter,#IntArray,0)-1))),
#IntArray = RTRIM(LTRIM(SUBSTRING(#IntArray,CHARINDEX(#Delimiter,#IntArray,0)+1,LEN(#IntArray))))
IF LEN(#Item) > 0
INSERT INTO #retIntTable SELECT #Item
END
IF LEN(#IntArray) > 0
INSERT INTO #retIntTable SELECT #IntArray
END
ELSE
BEGIN
IF LEN(#IntArray) > 0
INSERT INTO #retIntTable SELECT #IntArray
END
RETURN
END;
You define your input variable as varchar(8000) and your #Item variable is varchar(8). Are your items typically 8 characters each? Is the string you send in w/ over 1000 items more than 8000 characters? Try changing your input to varchar(max) instead.
Are all of your comma seperated values 8 chars long? If so, then the input parameter will only be able to hold 888 (8000 / 9(including the comma) of them..
It's because your input parameter is limited to 8000 characters.
You might try calling the function using substring... Maybe:
WHERE
[myField] IN(Select ID from [dbo].[ufnConvertArrayToIntTable](substring(#inputarray, 1, 4000))
OR
[myField] IN(Select ID from [dbo].[ufnConvertArrayToIntTable](substring(#inputarray, 4001, 8000))
...