I was actually asked this myself a few weeks ago, whereas I know exactly how to do this with a SP or UDF but I was wondering if there was a quick and easy way of doing this without these methods. I'm assuming that there is and I just can't find it.
A point I need to make is that although we know what characters are allowed (a-z, A-Z, 0-9) we don't want to specify what is not allowed (##!$ etc...). Also, we want to pull the rows which have the illegal characters so that it can be listed to the user to fix (as we have no control over the input process we can't do anything at that point).
I have looked through SO and Google previously, but was unable to find anything that did what I wanted. I have seen many examples which can tell you if it contains alphanumeric characters, or doesn't, but something that is able to pull out an apostrophe in a sentence I have not found in query form.
Please note also that values can be null or '' (empty) in this varchar column.
Won't this do it?
SELECT * FROM TABLE
WHERE COLUMN_NAME LIKE '%[^a-zA-Z0-9]%'
Setup
use tempdb
create table mytable ( mycol varchar(40) NULL)
insert into mytable VALUES ('abcd')
insert into mytable VALUES ('ABCD')
insert into mytable VALUES ('1234')
insert into mytable VALUES ('efg%^&hji')
insert into mytable VALUES (NULL)
insert into mytable VALUES ('')
insert into mytable VALUES ('apostrophe '' in a sentence')
SELECT * FROM mytable
WHERE mycol LIKE '%[^a-zA-Z0-9]%'
drop table mytable
Results
mycol
----------------------------------------
efg%^&hji
apostrophe ' in a sentence
Sql server has very limited Regex support. You can use PATINDEX with something like this
PATINDEX('%[a-zA-Z0-9]%',Col)
Have a look at PATINDEX (Transact-SQL)
and Pattern Matching in Search Conditions
I found this page with quite a neat solution. What makes it great is that you get an indication of what the character is and where it is. Then it gives a super simple way to fix it (which can be combined and built into a piece of driver code to scale up it's application).
DECLARE #tablename VARCHAR(1000) ='Schema.Table'
DECLARE #columnname VARCHAR(100)='ColumnName'
DECLARE #counter INT = 0
DECLARE #sql VARCHAR(MAX)
WHILE #counter <=255
BEGIN
SET #sql=
'SELECT TOP 10 '+#columnname+','+CAST(#counter AS VARCHAR(3))+' as CharacterSet, CHARINDEX(CHAR('+CAST(#counter AS VARCHAR(3))+'),'+#columnname+') as LocationOfChar
FROM '+#tablename+'
WHERE CHARINDEX(CHAR('+CAST(#counter AS VARCHAR(3))+'),'+#columnname+') <> 0'
PRINT (#sql)
EXEC (#sql)
SET #counter = #counter + 1
END
and then...
UPDATE Schema.Table
SET ColumnName= REPLACE(Columnname,CHAR(13),'')
Credit to Ayman El-Ghazali.
SELECT * FROM TABLE_NAME WHERE COL_NAME LIKE '%[^0-9a-zA-Z $#$.$-$''''$,]%'
This works best for me when I'm trying to find any special characters in a string
Related
On MSSQL when I run my query I get the following output when run as Results To Text:
Description Another Column
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- -----------
This is a Description 43289
But interestingly there aren't any blanks at all at the end of the string, and the length is also the correct one (without any blanks)
I want to have an output like this:
Description Another Column
-------------- --------------
My description 543893
I've tried rtrim, ltrim, replace, but none is working. The datatype is nvarchar(max).
As I say, if you really need to do this, you have to do two passes through the data:
create table #t (description nvarchar(max),otherColumn int)
insert into #t(description,otherColumn) values
(N'Short Desc.',21),
(N'Loooooonnnnnngggggg Description',99)
declare #maxLength int
declare #sql nvarchar(max)
select #maxLength = MAX(LEN(description)) from #t
if #maxLength <= 4000
begin
set #sql = N'SELECT CONVERT(nvarchar(' + CONVERT(nvarchar(4),#maxLength) +
N'),description) as description'
end
else
begin
set #sql = N'SELECT description'
end
set #sql = #sql + N',otherColumn from #t'
exec sp_executesql #sql
Result:
description otherColumn
------------------------------- -----------
Short Desc. 21
Loooooonnnnnngggggg Description 99
Even if you use RTRIM, etc, when the system is compiling the query, it doesn't know the nature of the data contained in individual rows1. And so it can only assign the data type nvarchar(max) as the type of the resulting column.
In turn, when SSMS starts to receive the result set, it's informed that the column is an nvarchar(max). At the point at which it's printing the column headers, it doesn't know the nature of the lengths of the results it will receive, so it just has to adopt a sensible default display width that can cope with any data it receives.
I'd normally leave such format considerations to an actual presentation layer (e.g. report builder or application), rather than fiddling around and trying to force SQL/SMS to do this sort of formatting.
1E.g. one of the rows might have the description containing the complete works of Shakespeare, in the original Klingon.
Try this
select Description,len(rtrim(Description)) as count from table
Edit
The actual problem is you set the result mode to text. Change it to grid mode. You will get the correct result
I know this is old, but for anyone who might need it in the future (including myself with how bad my memory is).
I ran into this same problem. Here was the solution I came up with:
SELECT CASE
WHEN LEN([Description]) >= 20
THEN CAST((LEFT([Description], 17) + '...') AS VARCHAR(20))
ELSE CAST([Description] AS VARCHAR(20))
END AS [Description]
,[Another Column]
FROM [dbo].[YourTableName];
I hope you find this helpful, even though it is a bit of a long-winded way to do it.
I'm new to SQL so please forgive me if I use incorrect terminology and my question sounds confused.
I've been tasked with writing a stored procedure which will be sent 3 variables as strings (varchar I think). I need to take two of the variables and remove text from the end of the variable and only from the end.
The strings/text I need to remove from the end of the variables are
co
corp
corporation
company
lp
llc
ltd
limited
For example this string
Global Widgets LLC
would become
Global Widgets
However it should only apply once so
Global Widgets Corporation LLC
Should become
Global Widgets Corporation
I then need to use the altered variables to do a SQL query.
This is to be used as a backup for an integration piece we have which makes a callout to another system. The other system takes the same variables and uses Regex to remove the strings from the end of variables.
I've tried different combinations of PATINDEX, SUBSTRING, REPLACE, STUFF but cannot seem to come up with something that will do the job.
===============================================================
Edit: I want to thank everyone for the answers provided so far, but I left out some information that I didn't think was important but judging by the answers seems like it would affect the processing.
My proc will start something like
ALTER PROC [dbo].[USP_MyDatabaseTable] #variableToBeAltered nvarchar(50)
AS
I will then need to remove all , and . characters. I've already figured out how to do this. I will then need to do the processing on #variableToBeAltered (technically there will be two variables) to remove the strings I listed previously. I must then remove all spaces from #variableToBeAltered. (Again I figured that part out). Then finally I will use #variableToBeAltered in my SQL query something like
SELECT [field1] AS myField
,[field2] AS myOtherField
FROM [MyData].[dbo].[MyDatabaseTable]
WHERE [field1] = (#variableToBeAltered);
I hope this information is more useful.
I'd keep all of your suffixes in a table to make this a little easier. You can then perform code like this either within a query or against a variable.
DECLARE #company_name VARCHAR(50) = 'Global Widgets Corporation LLC'
DECLARE #Suffixes TABLE (suffix VARCHAR(20))
INSERT INTO #Suffixes (suffix) VALUES ('LLC'), ('CO'), ('CORP'), ('CORPORATION'), ('COMPANY'), ('LP'), ('LTD'), ('LIMITED')
SELECT #company_name = SUBSTRING(#company_name, 1, LEN(#company_name) - LEN(suffix))
FROM #Suffixes
WHERE #company_name LIKE '%' + suffix
SELECT #company_name
The keys here are that you are only matching with strings that end in the suffix and it uses SUBSTRING rather than REPLACE to avoid accidentally removing copies of any of the suffixes from the middle of the string.
The #Suffixes table is a table variable here, but it makes more sense for you to just create it and fill it as a permanent table.
The query will just find the one row (if any) that matches its suffix with the end of your string. If a match is found then the variable will be set to a substring with the length of the suffix removed from the end. There will usually be a trailing space, but for a VARCHAR that will just get dropped off.
There are still a couple of potential issues to be aware of though...
First, if you have a company name like "Watco" then the "co" would be a false positive here. I'm not sure what can be done about that other than maybe making your suffixes include a leading space.
Second, if one suffix ends with one of your other suffixes then the ordering that they get applied could be a problem. You could get around this by only applying the row with the greatest length for suffix, but it gets a little more complicated, so I've left that out for now.
Building on the answer given by Tom H, but applying across the entire table:
set nocount on;
declare #suffixes table(tag nvarchar(20));
insert into #suffixes values('co');
insert into #suffixes values('corp');
insert into #suffixes values('corporation');
insert into #suffixes values('company');
insert into #suffixes values('lp');
insert into #suffixes values('llc');
insert into #suffixes values('ltd');
insert into #suffixes values('limited');
declare #companynames table(entry nvarchar(100),processed bit default 0);
insert into #companynames values('somecompany llc',0);
insert into #companynames values('business2 co',0);
insert into #companynames values('business3',0);
insert into #companynames values('business4 lpx',0);
while exists(select * from #companynames where processed = 0)
begin
declare #currentcompanyname nvarchar(100) = (select top 1 entry from #companynames where processed = 0);
update #companynames set processed = 1 where entry = #currentcompanyname;
update #companynames
set entry = SUBSTRING(entry, 1, LEN(entry) - LEN(tag))
from #suffixes
where entry like '%' + tag
end
select * from #companynames
You can use a query like below:
-- Assuming that you can maintain all patterns in a table or a temp table
CREATE TABLE tbl(pattern varchar(100))
INSERT INTO tbl values
('co'),('llc'),('beta')
--#a stores the string you need to manipulate, #lw & #b are variables to aid
DECLARE #a nvarchar(100), #b nvarchar(100), #lw varchar(100)
SET #a='alpha beta gamma'
SET #b=''
-- #t is a flag
DECLARE #t int
SET #t=0
-- Below is a loop
WHILE(#t=0 OR LEN(#a)=0 )
BEGIN
-- Store the current last word in the #lw variable
SET #lw=reverse(substring(reverse(#a),1, charindex(' ', reverse(#a)) -1))
-- check if the word is in pattern dictionary. If yes, then Voila!
SELECT #t=1 FROM tbl WHERE #lw like pattern
-- remove the last word from #a
SET #a=LEFT(#a,LEN(#a)-LEN(#lw))
IF (#t<>1)
BEGIN
-- all words which were not pattern are joined back onto this stack
SET #b=CONCAT(#lw,#b)
END
END
-- get back the remaining word
SET #a=CONCAT(#a,#b)
SELECT #a
drop table tbl
Do note that this method overcomes Tom's problem of
if you have a company name like "Watco" then the "co" would be a false positive here. I'm not sure what can be done about that other than maybe making your suffixes include a leading space.
use the replace function in SQL 2012,
declare #var1 nvarchar(20) = 'ACME LLC'
declare #var2 nvarchar(20) = 'LLC'
SELECT CASE
WHEN ((PATINDEX('%'+#var2+'%',#var1) <= (LEN(#var1)-LEN(#var2)))
Or (SUBSTRING(#var1,PATINDEX('%'+#var2+'%',#var1)-1,1) <> SPACE(1)))
THEN #var1
ELSE
REPLACE(#var1,#var2,'')
END
Here is another way to overcome the 'Runco Co' situation.
declare #var1 nvarchar(20) = REVERSE('Runco Co')
declare #var2 nvarchar(20) = REVERSE('Co')
Select REVERSE(
CASE WHEN(CHARINDEX(' ',#var1) > LEN(#var2)) THEN
SUBSTRING(#var1,PATINDEX('%'+#var2+'%',#var1)+LEN(#var2),LEN(#var1)-LEN(#var2))
ELSE
#var1
END
)
I have this type of data in a column in my table,,,
QCIM1J77477, 4650125572, QCCR1J77891, 4650312729, QCCR1J74974 --- remove without comma
or
QCIM1E107498,QCIM1E109835,4650092399/ QCCR1E91190, -- remove 4650092399
I want only that string which starts from QC,remove apart from QC,
so please tell me how can I achive it?
Beneath a piece of t-sql script that creates a temporary table #t with temporary variables. Here the temporary table exists untill you break your session, temporary variables exist during the execution of the script. I have a drop table statement at the bottom. Figure out yourself what you want with the table data and whether you want the data put in somewhere else, for example in a not-temporary table :).
I assume you want all the pieces of the string that contain 'QC' as seperate values. If you want your data back as it was originally, that is multiple strings per one column, then you could also do a group by trick. Then you do need a unique identifier of some sort, like name, id, guid of each row or identity.
create table #t ([QCs] nvarchar(100))
declare #str nvarchar(500)
set #str = 'QCIM1E107498,QCIM1E109835,4650092399/ QCCR1E91190'
--replace the above temporary variable with the column you are selecting
declare #sql nvarchar(4000)
select #sql = 'insert into #t select '''+ replace(#str,',',''' union all select ''') + ''''
print #sql
exec ( #sql )
select
QCs
,PATINDEX('%QC%',QCs) as StartPosition
,SUBSTRING(QCs,PATINDEX('%QC%',QCs),12) as QCsNew
from #t where QCs like '%QC%'
drop table #t
With PATINDEX you find the position where in the string 'QC' starts, and with SUBSTRING you tell the dbms to give back (here) 12 characters starting from the found StartPosition.
Beneath what the result looks like. QCsNew is your desired result.
QCs StartPosition QCsNew
QCIM1E107498 1 QCIM1E107498
QCIM1E109835 1 QCIM1E109835
4650092399/ QCCR1E91190 13 QCCR1E91190
The answer escapes me...maybe because it is not possible...
Example that works...
SELECT * FROM TABLEA WHERE FIELD1 IN ('aaa','bbb','ccc')
Example that does not work...
Attempt to leverage variable so that I can define the values once in a string of statements
DECLARE #ListValues VARCHAR(50)
SET #ListValues = '''aaa'',''bbb'',''ccc'''
SELECT * FROM TABLEA WHERE FIELD1 IN (#ListValues)
This is is obviously only a small part of the equation and for other reasons...
I cannot leverage a table for the values and change this to a true sub-query
The closest question I could find was this one... but does not cover my requirements obviously...
Storing single quotes in varchar variable SQL Server 2008
Thanks in advance.
You can do this using dynamic SQL:
DECLARE #ListValues VARCHAR(MAX)
,#SQL VARCHAR(MAX)
SELECT #ListValues = '''aaa'',''bbb'',''ccc'''
,#SQL = 'SELECT * FROM TABLEA WHERE FIELD1 IN ('+#ListValues+')'
EXEC (#SQL)
It doesn't work because the IN operator expects a list of items - here strings.
What you're supplying with your #ListValues variable however is a single string - not a list of strings.
What you could do is use a table variable and store your values in it:
DECLARE #ListOfValues TABLE (ItemName VARCHAR(50))
INSERT INTO #ListOfValues(ItemName)
VALUES('aaa'), ('bbb'), ('ccc')
SELECT *
FROM TABLEA
WHERE FIELD1 IN (SELECT ItemName FROM #ListOfValues)
Build your whole SQL query dynamically (say it's stored in a string variable #sql),
and then execute it with EXEC (#sql). Better yet, use the sp_executesql SP
because this approach is more secure (less prone to SQL injection) than EXEC.
See: sp_executesql
The IN operator in SQLServer expect a list of values, your variable is a single string, the query parsed will be different
SELECT * FROM TABLEA WHERE FIELD1 IN ('aaa','bbb','ccc')
SELECT * FROM TABLEA WHERE FIELD1 IN ("'aaa','bbb','ccc'")
Attention: the double quotes are there only for readability, to get the string with single quote in it.
if you know a programming language the first one is like searching in an array, the second is a string.
To store a list in your variable it need to a table
DECLARE #varTable TABLE (field1 varchar())
So that you can use it in your IN
SELECT * FROM TABLEA WHERE FIELD1 IN (SELECT field1 FROM #varTable)
To add values to the table variable use an INSERT statament like usual for tables.
I have a row of strings that are in the following format:
'Order was assigned to lastname,firsname'
I need to cut this string down into just the last and first name but it is always a different name for each record.
The 'Order was assigned to' part is always the same.......
Thanks
I am using SQL Server. It is multiple records with different names in each record.
In your specific case you can use something like:
SELECT SUBSTRING(str, 23) FROM table
However, this is not very scalable, should the format of your strings ever change.
If you are using an Oracle database, you would want to use SUBSTR instead.
Edit:
For databases where the third parameter is not optional, you could use SUBSTRING(str, 23, LEN(str))
Somebody would have to test to see if this is better or worse than subtraction, as in Martin Smith's solution but gives you the same result in the end.
In addition to the SUBSTRING methods, you could also use a REPLACE function. I don't know which would have better performance over millions of rows, although I suspect that it would be the SUBSTRING - especially if you were working with CHAR instead of VARCHAR.
SELECT REPLACE(my_column, 'Order was assigned to ', '')
For SQL Server
WITH testData AS
(
SELECT 'Order was assigned to lastname,firsname' as Col1 UNION ALL
SELECT 'Order was assigned to Bloggs, Jo' as Col1
)
SELECT SUBSTRING(Col1,23,LEN(Col1)-22) AS Name
from testData
Returns
Name
---------------------------------------
lastname,firsname
Bloggs, Jo
on MS SQL Server:
declare #str varchar(100) = 'Order was assigned to lastname,firsname'
declare #strLen1 int = DATALENGTH('Order was assigned to ')
declare #strLen2 int = len(#str)
select #strlen1, #strLen2, substring(#str,#strLen1,#strLen2),
RIGHT(#str, #strlen2-#strlen1)
I would require that a colon or some other delimiter be between the message and the name.
Then you could just search for the index of that character and know that anything after it was the data you need...
Example with format changing over time:
CREATE TABLE #Temp (OrderInfo NVARCHAR(MAX))
INSERT INTO #Temp VALUES ('Order was assigned to :Smith,Mary')
INSERT INTO #Temp VALUES ('Order was assigned to :Holmes,Larry')
INSERT INTO #Temp VALUES ('New Format over time :LootAt,Me')
SELECT SUBSTRING(OrderInfo, CHARINDEX(':',OrderInfo)+1, LEN(OrderInfo))
FROM #Temp
DROP TABLE #Temp