SQL Server 2012: Remove text from end of string - sql

I'm new to SQL so please forgive me if I use incorrect terminology and my question sounds confused.
I've been tasked with writing a stored procedure which will be sent 3 variables as strings (varchar I think). I need to take two of the variables and remove text from the end of the variable and only from the end.
The strings/text I need to remove from the end of the variables are
co
corp
corporation
company
lp
llc
ltd
limited
For example this string
Global Widgets LLC
would become
Global Widgets
However it should only apply once so
Global Widgets Corporation LLC
Should become
Global Widgets Corporation
I then need to use the altered variables to do a SQL query.
This is to be used as a backup for an integration piece we have which makes a callout to another system. The other system takes the same variables and uses Regex to remove the strings from the end of variables.
I've tried different combinations of PATINDEX, SUBSTRING, REPLACE, STUFF but cannot seem to come up with something that will do the job.
===============================================================
Edit: I want to thank everyone for the answers provided so far, but I left out some information that I didn't think was important but judging by the answers seems like it would affect the processing.
My proc will start something like
ALTER PROC [dbo].[USP_MyDatabaseTable] #variableToBeAltered nvarchar(50)
AS
I will then need to remove all , and . characters. I've already figured out how to do this. I will then need to do the processing on #variableToBeAltered (technically there will be two variables) to remove the strings I listed previously. I must then remove all spaces from #variableToBeAltered. (Again I figured that part out). Then finally I will use #variableToBeAltered in my SQL query something like
SELECT [field1] AS myField
,[field2] AS myOtherField
FROM [MyData].[dbo].[MyDatabaseTable]
WHERE [field1] = (#variableToBeAltered);
I hope this information is more useful.

I'd keep all of your suffixes in a table to make this a little easier. You can then perform code like this either within a query or against a variable.
DECLARE #company_name VARCHAR(50) = 'Global Widgets Corporation LLC'
DECLARE #Suffixes TABLE (suffix VARCHAR(20))
INSERT INTO #Suffixes (suffix) VALUES ('LLC'), ('CO'), ('CORP'), ('CORPORATION'), ('COMPANY'), ('LP'), ('LTD'), ('LIMITED')
SELECT #company_name = SUBSTRING(#company_name, 1, LEN(#company_name) - LEN(suffix))
FROM #Suffixes
WHERE #company_name LIKE '%' + suffix
SELECT #company_name
The keys here are that you are only matching with strings that end in the suffix and it uses SUBSTRING rather than REPLACE to avoid accidentally removing copies of any of the suffixes from the middle of the string.
The #Suffixes table is a table variable here, but it makes more sense for you to just create it and fill it as a permanent table.
The query will just find the one row (if any) that matches its suffix with the end of your string. If a match is found then the variable will be set to a substring with the length of the suffix removed from the end. There will usually be a trailing space, but for a VARCHAR that will just get dropped off.
There are still a couple of potential issues to be aware of though...
First, if you have a company name like "Watco" then the "co" would be a false positive here. I'm not sure what can be done about that other than maybe making your suffixes include a leading space.
Second, if one suffix ends with one of your other suffixes then the ordering that they get applied could be a problem. You could get around this by only applying the row with the greatest length for suffix, but it gets a little more complicated, so I've left that out for now.

Building on the answer given by Tom H, but applying across the entire table:
set nocount on;
declare #suffixes table(tag nvarchar(20));
insert into #suffixes values('co');
insert into #suffixes values('corp');
insert into #suffixes values('corporation');
insert into #suffixes values('company');
insert into #suffixes values('lp');
insert into #suffixes values('llc');
insert into #suffixes values('ltd');
insert into #suffixes values('limited');
declare #companynames table(entry nvarchar(100),processed bit default 0);
insert into #companynames values('somecompany llc',0);
insert into #companynames values('business2 co',0);
insert into #companynames values('business3',0);
insert into #companynames values('business4 lpx',0);
while exists(select * from #companynames where processed = 0)
begin
declare #currentcompanyname nvarchar(100) = (select top 1 entry from #companynames where processed = 0);
update #companynames set processed = 1 where entry = #currentcompanyname;
update #companynames
set entry = SUBSTRING(entry, 1, LEN(entry) - LEN(tag))
from #suffixes
where entry like '%' + tag
end
select * from #companynames

You can use a query like below:
-- Assuming that you can maintain all patterns in a table or a temp table
CREATE TABLE tbl(pattern varchar(100))
INSERT INTO tbl values
('co'),('llc'),('beta')
--#a stores the string you need to manipulate, #lw & #b are variables to aid
DECLARE #a nvarchar(100), #b nvarchar(100), #lw varchar(100)
SET #a='alpha beta gamma'
SET #b=''
-- #t is a flag
DECLARE #t int
SET #t=0
-- Below is a loop
WHILE(#t=0 OR LEN(#a)=0 )
BEGIN
-- Store the current last word in the #lw variable
SET #lw=reverse(substring(reverse(#a),1, charindex(' ', reverse(#a)) -1))
-- check if the word is in pattern dictionary. If yes, then Voila!
SELECT #t=1 FROM tbl WHERE #lw like pattern
-- remove the last word from #a
SET #a=LEFT(#a,LEN(#a)-LEN(#lw))
IF (#t<>1)
BEGIN
-- all words which were not pattern are joined back onto this stack
SET #b=CONCAT(#lw,#b)
END
END
-- get back the remaining word
SET #a=CONCAT(#a,#b)
SELECT #a
drop table tbl
Do note that this method overcomes Tom's problem of
if you have a company name like "Watco" then the "co" would be a false positive here. I'm not sure what can be done about that other than maybe making your suffixes include a leading space.

use the replace function in SQL 2012,
declare #var1 nvarchar(20) = 'ACME LLC'
declare #var2 nvarchar(20) = 'LLC'
SELECT CASE
WHEN ((PATINDEX('%'+#var2+'%',#var1) <= (LEN(#var1)-LEN(#var2)))
Or (SUBSTRING(#var1,PATINDEX('%'+#var2+'%',#var1)-1,1) <> SPACE(1)))
THEN #var1
ELSE
REPLACE(#var1,#var2,'')
END
Here is another way to overcome the 'Runco Co' situation.
declare #var1 nvarchar(20) = REVERSE('Runco Co')
declare #var2 nvarchar(20) = REVERSE('Co')
Select REVERSE(
CASE WHEN(CHARINDEX(' ',#var1) > LEN(#var2)) THEN
SUBSTRING(#var1,PATINDEX('%'+#var2+'%',#var1)+LEN(#var2),LEN(#var1)-LEN(#var2))
ELSE
#var1
END
)

Related

How to replace all special characters in string

I have a table with the following columns:
dbo.SomeInfo
- Id
- Name
- InfoCode
Now I need to update the above table's InfoCode as
Update dbo.SomeInfo
Set InfoCode= REPLACE(Replace(RTRIM(LOWER(Name)),' ','-'),':','')
This replaces all spaces with - & lowercase the name
When I do check the InfoCode, I see there are Names with some special characters like
Cathe Friedrich''s Low Impact
coffeyfit-cardio-box-&-burn
Jillian Michaels: Cardio
Then I am manually writing the update sql against this as
Update dbo.SomeInfo
SET InfoCode= 'cathe-friedrichs-low-impact'
where Name ='Cathe Friedrich''s Low Impact '
Now, this solution is not realistic for me. I checked the following links related to Regex & others around it.
UPDATE and REPLACE part of a string
https://www.codeproject.com/Questions/456246/replace-special-characters-in-sql
But none of them is hitting the requirement.
What I need is if there is any character other [a-z0-9] replace it - & also there should not be continuous -- in InfoCode
The above Update sql has set some values of InfoCode as the-dancer's-workout®----starter-package
Some Names have value as
Sleek Technique™
The Dancer's-workout®
How can I write Update sql that could handle all such special characters?
Using NGrams8K you could split the string into characters and then rather than replacing every non-acceptable character, retain only certain ones:
SELECT (SELECT '' + CASE WHEN N.token COLLATE Latin1_General_BIN LIKE '[A-z0-9]'THEN token ELSE '-' END
FROM dbo.NGrams8k(V.S,1) N
ORDER BY position
FOR XML PATH(''))
FROM (VALUES('Sleek Technique™'),('The Dancer''s-workout®'))V(S);
I use COLLATE here as on my default collation in my instance the '™' is ignored, therefore I use a binary collation. You may want to use COLLATE to switch the string back to its original collation outside of the subquery.
This approach is fully inlinable:
First we need a mock-up table with some test data:
DECLARe #SomeInfo TABLE (Id INT IDENTITY, InfoCode VARCHAR(100));
INSERT INTO #SomeInfo (InfoCode) VALUES
('Cathe Friedrich''s Low Impact')
,('coffeyfit-cardio-box-&-burn')
,('Jillian Michaels: Cardio')
,('Sleek Technique™')
,('The Dancer''s-workout®');
--This is the query
WITH cte AS
(
SELECT 1 AS position
,si.Id
,LOWER(si.InfoCode) AS SourceText
,SUBSTRING(LOWER(si.InfoCode),1,1) AS OneChar
FROM #SomeInfo si
UNION ALL
SELECT cte.position +1
,cte.Id
,cte.SourceText
,SUBSTRING(LOWER(cte.SourceText),cte.position+1,1) AS OneChar
FROM cte
WHERE position < DATALENGTH(SourceText)
)
,Cleaned AS
(
SELECT cte.Id
,(
SELECT CASE WHEN ASCII(cte2.OneChar) BETWEEN 65 AND 90 --A-Z
OR ASCII(cte2.OneChar) BETWEEN 97 AND 122--a-z
OR ASCII(cte2.OneChar) BETWEEN 48 AND 57 --0-9
--You can easily add more ranges
THEN cte2.OneChar ELSE '-'
--You can easily nest another CASE to deal with special characters like the single quote in your examples...
END
FROM cte AS cte2
WHERE cte2.Id=cte.Id
ORDER BY cte2.position
FOR XML PATH('')
) AS normalised
FROM cte
GROUP BY cte.Id
)
,NoDoubleHyphens AS
(
SELECT REPLACE(REPLACE(REPLACE(normalised,'-','<>'),'><',''),'<>','-') AS normalised2
FROM Cleaned
)
SELECT CASE WHEN RIGHT(normalised2,1)='-' THEN SUBSTRING(normalised2,1,LEN(normalised2)-1) ELSE normalised2 END AS FinalResult
FROM NoDoubleHyphens;
The first CTE will recursively (well, rather iteratively) travers down the string, character by character and a return a very slim set with one row per character.
The second CTE will then GROUP the Ids. This allows for a correlated sub-query, where the actual check is performed using ASCII-ranges. FOR XML PATH('') is used to re-concatenate the string. With SQL-Server 2017+ I'd suggest to use STRING_AGG() instead.
The third CTE will use a well known trick to get rid of multiple occurances of a character. Take any two characters which will never occur in your string, I use < and >. A string like a--b---c will come back as a<><>b<><><>c. After replacing >< with nothing we get a<>b<>c. Well, that's it...
The final SELECT will cut away a trailing hyphen. If needed you can add similar logic to get rid of a leading hyphen. With v2017+ There was TRIM('-') to make this easier...
The result
cathe-friedrich-s-low-impact
coffeyfit-cardio-box-burn
jillian-michaels-cardio
sleek-technique
the-dancer-s-workout
You can create a User-Defined-Function for something like that.
Then use the UDF in the update.
CREATE FUNCTION [dbo].LowerDashString (#str varchar(255))
RETURNS varchar(255)
AS
BEGIN
DECLARE #result varchar(255);
DECLARE #chr varchar(1);
DECLARE #pos int;
SET #result = '';
SET #pos = 1;
-- lowercase the input and remove the single-quotes
SET #str = REPLACE(LOWER(#str),'''','');
-- loop through the characters
-- while replacing anything that's not a letter to a dash
WHILE #pos <= LEN(#str)
BEGIN
SET #chr = SUBSTRING(#str, #pos, 1)
IF #chr LIKE '[a-z]' SET #result += #chr;
ELSE SET #result += '-';
SET #pos += 1;
END;
-- SET #result = TRIM('-' FROM #result); -- SqlServer 2017 and beyond
-- multiple dashes to one dash
WHILE #result LIKE '%--%' SET #result = REPLACE(#result,'--','-');
RETURN #result;
END;
GO
Example snippet using the function:
-- using a table variable for demonstration purposes
declare #SomeInfo table (Id int primary key identity(1,1) not null, InfoCode varchar(100) not null);
-- sample data
insert into #SomeInfo (InfoCode) values
('Cathe Friedrich''s Low Impact'),
('coffeyfit-cardio-box-&-burn'),
('Jillian Michaels: Cardio'),
('Sleek Technique™'),
('The Dancer''s-workout®');
update #SomeInfo
set InfoCode = dbo.LowerDashString(InfoCode)
where (InfoCode LIKE '%[^A-Z-]%' OR InfoCode != LOWER(InfoCode));
select *
from #SomeInfo;
Result:
Id InfoCode
-- -----------------------------
1 cathe-friedrichs-low-impact
2 coffeyfit-cardio-box-burn
3 jillian-michaels-cardio
4 sleek-technique-
5 the-dancers-workout-

split string from a string by sql

I have this type of data in a column in my table,,,
QCIM1J77477, 4650125572, QCCR1J77891, 4650312729, QCCR1J74974 --- remove without comma
or
QCIM1E107498,QCIM1E109835,4650092399/ QCCR1E91190, -- remove 4650092399
I want only that string which starts from QC,remove apart from QC,
so please tell me how can I achive it?
Beneath a piece of t-sql script that creates a temporary table #t with temporary variables. Here the temporary table exists untill you break your session, temporary variables exist during the execution of the script. I have a drop table statement at the bottom. Figure out yourself what you want with the table data and whether you want the data put in somewhere else, for example in a not-temporary table :).
I assume you want all the pieces of the string that contain 'QC' as seperate values. If you want your data back as it was originally, that is multiple strings per one column, then you could also do a group by trick. Then you do need a unique identifier of some sort, like name, id, guid of each row or identity.
create table #t ([QCs] nvarchar(100))
declare #str nvarchar(500)
set #str = 'QCIM1E107498,QCIM1E109835,4650092399/ QCCR1E91190'
--replace the above temporary variable with the column you are selecting
declare #sql nvarchar(4000)
select #sql = 'insert into #t select '''+ replace(#str,',',''' union all select ''') + ''''
print #sql
exec ( #sql )
select
QCs
,PATINDEX('%QC%',QCs) as StartPosition
,SUBSTRING(QCs,PATINDEX('%QC%',QCs),12) as QCsNew
from #t where QCs like '%QC%'
drop table #t
With PATINDEX you find the position where in the string 'QC' starts, and with SUBSTRING you tell the dbms to give back (here) 12 characters starting from the found StartPosition.
Beneath what the result looks like. QCsNew is your desired result.
QCs StartPosition QCsNew
QCIM1E107498 1 QCIM1E107498
QCIM1E109835 1 QCIM1E109835
4650092399/ QCCR1E91190 13 QCCR1E91190

SQL Server 2008 query to find rows containing non-alphanumeric characters in a column

I was actually asked this myself a few weeks ago, whereas I know exactly how to do this with a SP or UDF but I was wondering if there was a quick and easy way of doing this without these methods. I'm assuming that there is and I just can't find it.
A point I need to make is that although we know what characters are allowed (a-z, A-Z, 0-9) we don't want to specify what is not allowed (##!$ etc...). Also, we want to pull the rows which have the illegal characters so that it can be listed to the user to fix (as we have no control over the input process we can't do anything at that point).
I have looked through SO and Google previously, but was unable to find anything that did what I wanted. I have seen many examples which can tell you if it contains alphanumeric characters, or doesn't, but something that is able to pull out an apostrophe in a sentence I have not found in query form.
Please note also that values can be null or '' (empty) in this varchar column.
Won't this do it?
SELECT * FROM TABLE
WHERE COLUMN_NAME LIKE '%[^a-zA-Z0-9]%'
Setup
use tempdb
create table mytable ( mycol varchar(40) NULL)
insert into mytable VALUES ('abcd')
insert into mytable VALUES ('ABCD')
insert into mytable VALUES ('1234')
insert into mytable VALUES ('efg%^&hji')
insert into mytable VALUES (NULL)
insert into mytable VALUES ('')
insert into mytable VALUES ('apostrophe '' in a sentence')
SELECT * FROM mytable
WHERE mycol LIKE '%[^a-zA-Z0-9]%'
drop table mytable
Results
mycol
----------------------------------------
efg%^&hji
apostrophe ' in a sentence
Sql server has very limited Regex support. You can use PATINDEX with something like this
PATINDEX('%[a-zA-Z0-9]%',Col)
Have a look at PATINDEX (Transact-SQL)
and Pattern Matching in Search Conditions
I found this page with quite a neat solution. What makes it great is that you get an indication of what the character is and where it is. Then it gives a super simple way to fix it (which can be combined and built into a piece of driver code to scale up it's application).
DECLARE #tablename VARCHAR(1000) ='Schema.Table'
DECLARE #columnname VARCHAR(100)='ColumnName'
DECLARE #counter INT = 0
DECLARE #sql VARCHAR(MAX)
WHILE #counter <=255
BEGIN
SET #sql=
'SELECT TOP 10 '+#columnname+','+CAST(#counter AS VARCHAR(3))+' as CharacterSet, CHARINDEX(CHAR('+CAST(#counter AS VARCHAR(3))+'),'+#columnname+') as LocationOfChar
FROM '+#tablename+'
WHERE CHARINDEX(CHAR('+CAST(#counter AS VARCHAR(3))+'),'+#columnname+') <> 0'
PRINT (#sql)
EXEC (#sql)
SET #counter = #counter + 1
END
and then...
UPDATE Schema.Table
SET ColumnName= REPLACE(Columnname,CHAR(13),'')
Credit to Ayman El-Ghazali.
SELECT * FROM TABLE_NAME WHERE COL_NAME LIKE '%[^0-9a-zA-Z $#$.$-$''''$,]%'
This works best for me when I'm trying to find any special characters in a string

SQL Delete Where Not In

I have a relation mapping table like this:
attributeid bigint
productid bigint
To clean relations that are not used any more, I want to delete all recors where productid = x and attributeid not in (#includedIds), like the following example:
#attributetypeid bigint,
#productid bigint,
#includedids varchar(MAX)
DELETE FROM reltable
WHERE productid = #productid AND
attributetypeid = #attributetypeid AND
attributeid NOT IN (#includedids);
When running the SQL with the includedids param containing more than 1 id - like this: 25,26 - I get a SqlException saying:
Error converting data type varchar to bigint.
And that is of course due to the , in that varchar(max) param...
How should I construct my delete statement to make it work?
SET QUOTED_IDENTIFIER ON
GO
CREATE FUNCTION [dbo].[ListToTable] (
/*
FUNCTION ListToTable
Usage: select entry from listtotable('abc,def,ghi') order by entry desc
PURPOSE: Takes a comma-delimited list as a parameter and returns the values of that list into a table variable.
*/
#mylist varchar(8000)
)
RETURNS #ListTable TABLE (
seqid int not null,
entry varchar(255) not null)
AS
BEGIN
DECLARE
#this varchar(255),
#rest varchar(8000),
#pos int,
#seqid int
SET #this = ' '
SET #seqid = 1
SET #rest = #mylist
SET #pos = PATINDEX('%,%', #rest)
WHILE (#pos > 0)
BEGIN
set #this=substring(#rest,1,#pos-1)
set #rest=substring(#rest,#pos+1,len(#rest)-#pos)
INSERT INTO #ListTable (seqid,entry) VALUES (#seqid,#this)
SET #pos= PATINDEX('%,%', #rest)
SET #seqid=#seqid+1
END
set #this=#rest
INSERT INTO #ListTable (seqid,entry) VALUES (#seqid,#this)
RETURN
END
Run that script in your SQL Server database to create the function ListToTable. Now, you can rewrite your query like so:
#attributetypeid bigint,
#productid bigint,
#includedids varchar(MAX)
DELETE FROM reltable
WHERE productid = #productid AND
attributetypeid = #attributetypeid AND
attributeid NOT IN (SELECT entry FROM ListToTable(#includedids));
Where #includedids is a comma delimited list that you provide. I use this function all the time when working with lists. Keep in mind this function does not necessarily sanitize your inputs, it just looks for character data in a comma delimited list and puts each element into a record. Hope this helps.
Joel Spolsky answered a very similar question here: Parameterize an SQL IN clause
You could try something similar, making sure to cast your attributetypeid as a varchar.
You can't pass a list as an parameter (AFAIK).
Can you rewrite the sql to use a subquery, something like this:
delete from reltable
WHERE productid = #productid AND
attributetypeid = #attributetypeid AND
attributeid NOT IN (select id from ... where ... );
?
That comma delimited list can be sent to a user defined function which will return it as a simple table. That table can then be queried by your NOT IN.
If you need the fn I can provide.. It's been about 5 yrs since I used sql much and I'll have to dust off that section of my brain..
Erland has the definitive guide for dealing with lists to table in SQL 2005, SQL 2008 gives you table based params.
On a side note I would avoid a NOT IN pattern for large lists, cause it does not scale, instead look at using left joins.

Filtering With Multi-Select Boxes With SQL Server

I need to filter result sets from sql server based on selections from a multi-select list box. I've been through the idea of doing an instring to determine if the row value exists in the selected filter values, but that's prone to partial matches (e.g. Car matches Carpet).
I also went through splitting the string into a table and joining/matching based on that, but I have reservations about how that is going to perform.
Seeing as this is a seemingly common task, I'm looking to the Stack Overflow community for some feedback and maybe a couple suggestions on the most commonly utilized approach to solving this problem.
I solved this one by writing a table-valued function (we're using 2005) which takes a delimited string and returns a table. You can then join to that or use WHERE EXISTS or WHERE x IN. We haven't done full stress testing yet, but with limited use and reasonably small sets of items I think that performance should be ok.
Below is one of the functions as a starting point for you. I also have one written to specifically accept a delimited list of INTs for ID values in lookup tables, etc.
Another possibility is to use LIKE with the delimiters to make sure that partial matches are ignore, but you can't use indexes with that, so performance will be poor for any large table. For example:
SELECT
my_column
FROM
My_Table
WHERE
#my_string LIKE '%|' + my_column + '|%'
.
/*
Name: GetTableFromStringList
Description: Returns a table of values extracted from a delimited list
Parameters:
#StringList - A delimited list of strings
#Delimiter - The delimiter used in the delimited list
History:
Date Name Comments
---------- ------------- ----------------------------------------------------
2008-12-03 T. Hummel Initial Creation
*/
CREATE FUNCTION dbo.GetTableFromStringList
(
#StringList VARCHAR(1000),
#Delimiter CHAR(1) = ','
)
RETURNS #Results TABLE
(
String VARCHAR(1000) NOT NULL
)
AS
BEGIN
DECLARE
#string VARCHAR(1000),
#position SMALLINT
SET #StringList = LTRIM(RTRIM(#StringList)) + #Delimiter
SET #position = CHARINDEX(#Delimiter, #StringList)
WHILE (#position > 0)
BEGIN
SET #string = LTRIM(RTRIM(LEFT(#StringList, #position - 1)))
IF (#string <> '')
BEGIN
INSERT INTO #Results (String) VALUES (#string)
END
SET #StringList = RIGHT(#StringList, LEN(#StringList) - #position)
SET #position = CHARINDEX(#Delimiter, #StringList, 1)
END
RETURN
END
I've been through the idea of doing an
instring to determine if the row value
exists in the selected filter values,
but that's prone to partial matches
(e.g. Car matches Carpet)
It sounds to me like you aren't including a unique ID, or possibly the primary key as part of values in your list box. Ideally each option will have a unique identifier that matches a column in the table you are searching on. If your listbox was like below then you would be able to filter for specifically for cars because you would get the unique value 3.
<option value="3">Car</option>
<option value="4">Carpret</option>
Then you just build a where clause that will allow you to find the values you need.
Updated, to answer comment.
How would I do the related join
considering that the user can select
and arbitrary number of options from
the list box? SELECT * FROM tblTable
JOIN tblOptions ON tblTable.FK = ? The
problem here is that I need to join on
multiple values.
I answered a similar question here.
One method would be to build a temporary table and add each selected option as a row to the temporary table. Then you would simply do a join to your temporary table.
If you want to simply create your sql dynamically you can do something like this.
SELECT * FROM tblTable WHERE option IN (selected_option_1, selected_option_2, selected_option_n)
I've found that a CLR table-valued function which takes your delimited string and calls Split on the string (returning the array as the IEnumerable) is more performant than anything written in T-SQL (it starts to break down when you have around one million items in the delimited list, but that's much further out than the T-SQL solution).
And then, you could join on the table or check with EXISTS.