I want to know a flexible way to extract the string between two '-'. The issue is that '-' may or may not exist in the source string. I have the following code which works fine when '-' exists twice in the source string, marking the start and end of the extracted string. But it throws an error "Invalid length parameter passed to the LEFT or SUBSTRING function" when there is only one '-' or none at all or if the string is blank. Can someone please help? Thanks
declare #string varchar(100) = 'BLAH90-ExtractThis-WOW'
SELECT SUBSTRING(#string,CHARINDEX('-',#string)+1, CHARINDEX('-',#string,CHARINDEX('-',#string)+1) -CHARINDEX('-',#string)-1) as My_String
Desired Output: ExtractThis
If there is one dash only e.g. 'BLAH90-ExtractThisWOW' then the output should be everything after the first dash i.e. ExtractThisWOW. If there are no dashes then the string will have a blank space instead e.g. 'BLAH90 ExtractThisWOW' and should return everything after the blank space i.e. ExtractThisWOW.
You can try something like this.
When there is no dash, it starts at the space if there is one or take the whole string if not.
Then I look if there is only one dash or 2
declare #string varchar(100) = 'BLAH90-ExtractThis-WOW'
declare #dash_pos integer = CHARINDEX('-',#string)
SELECT CASE
WHEN #dash_pos = 0 THEN
RIGHT(#string,LEN(#string)-CHARINDEX(' ',#string))
ELSE (
CASE
WHEN #dash_pos = LEN(#string)-CHARINDEX('-',REVERSE(#string))+1
THEN RIGHT(#string,LEN(#string)-#dash_pos)
ELSE SUBSTRING(#string,#dash_pos+1, CHARINDEX('-',#string,#dash_pos+1) -
#dash_pos -1)
END
)
END as My_String
Try this. If there are two dashes, it'll take what is inside. If there is only one or none, it'll keep the original string.
declare #string varchar(100) = 'BLAH-90ExtractThisWOW'
declare #dash_index1 int = case when #string like '%-%' then CHARINDEX('-', #string) else -1 end
declare #dash_index2 int = case when #string like '%-%'then len(#string) - CHARINDEX('-', reverse(#string)) + 1 else -1 end
SELECT case
when #dash_index1 <> #dash_index2 then SUBSTRING(#string,CHARINDEX('-',#string)+1, CHARINDEX('-',#string,CHARINDEX('-',#string)+1) -CHARINDEX('-',#string)-1)
else #string end
as My_String
Take your existing code:
declare #string varchar(100) = 'BLAH90-ExtractThis-WOW'
SELECT SUBSTRING(#string,CHARINDEX('-',#string)+1, CHARINDEX('-',#string,CHARINDEX('-',#string)+1) -CHARINDEX('-',#string)-1) as My_String
insert one line, like so:
declare #string varchar(100) = 'BLAH90-ExtractThis-WOW'
SET #string = #string + '--'
SELECT SUBSTRING(#string,CHARINDEX('-',#string)+1, CHARINDEX('-',#string,CHARINDEX('-',#string)+1) -CHARINDEX('-',#string)-1) as My_String
and you're done. (If NULL, you will get NULL returned. Also, this will return all data based on the FIRST dash found in the string, regardless of however many dashes are in the string.)
I have a requirement to run a function over certain fields to identify and redact any numbers which are 5 digits or longer, ensuring all but the last 4 digits are replaced with *
For example: "Some text with 12345 and 1234 and 12345678" would become "Some text with *2345 and 1234 and ****5678"
I've used PATINDEX to identify the the starting character of the pattern:
PATINDEX('%[0-9][0-9][0-9][0-9][0-9]%', TEST_TEXT)
I can recursively call that to get the starting character of all the occurrences, but I'm struggling with the actual redaction.
Does anyone have any pointers on how this can be done? I know to use REPLACE to insert the *s where they need to be, it's just the identification of what I should actually be replacing I'm struggling with.
Could do it on a program, but I need it to be T-SQL (can be a function if needed).
Any tips greatly appreciated!
You can do this using the built in functions of SQL Server. All of which used in this example are present in SQL Server 2008 and higher.
DECLARE #String VARCHAR(500) = 'Example Input: 1234567890, 1234, 12345, 123456, 1234567, 123asd456'
DECLARE #StartPos INT = 1, #EndPos INT = 1;
DECLARE #Input VARCHAR(500) = ISNULL(#String, '') + ' '; --Sets input field and adds a control character at the end to make the loop easier.
DECLARE #OutputString VARCHAR(500) = ''; --Initalize an empty string to avoid string null errors
WHILE (#StartPOS <> 0)
BEGIN
SET #StartPOS = PATINDEX('%[0-9][0-9][0-9][0-9][0-9]%', #Input);
IF #StartPOS <> 0
BEGIN
SET #OutputString += SUBSTRING(#Input, 1, #StartPOS - 1); --Seperate all contents before the first occurance of our filter
SET #Input = SUBSTRING(#Input, #StartPOS, 500); --Cut the entire string to the end. Last value must be greater than the original string length to simply cut it all.
SET #EndPos = (PATINDEX('%[0-9][0-9][0-9][0-9][^0-9]%', #Input)); --First occurance of 4 numbers with a not number behind it.
SET #Input = STUFF(#Input, 1, (#EndPos - 1), REPLICATE('*', (#EndPos - 1))); --#EndPos - 1 gives us the amount of chars we want to replace.
END
END
SET #OutputString += #Input; --Append the last element
SET #OutputString = LEFT(#OutputString, LEN(#OutputString))
SELECT #OutputString;
Which outputs the following:
Example Input: ******7890, 1234, *2345, **3456, ***4567, 123asd456
This entire code could also be made as a function since it only requires an input text.
A dirty solution with recursive CTE
DECLARE
#tags nvarchar(max) = N'Some text with 12345 and 1234 and 12345678',
#c nchar(1) = N' ';
;
WITH Process (s, i)
as
(
SELECT #tags, PATINDEX('%[0-9][0-9][0-9][0-9][0-9]%', #tags)
UNION ALL
SELECT value, PATINDEX('%[0-9][0-9][0-9][0-9][0-9]%', value)
FROM
(SELECT SUBSTRING(s,0,i)+'*'+SUBSTRING(s,i+4,len(s)) value
FROM Process
WHERE i >0) calc
-- we surround the value and the string with leading/trailing ,
-- so that cloth isn't a false positive for clothing
)
SELECT * FROM Process
WHERE i=0
I think a better solution it's to add clr function in Ms SQL Server to manage regexp.
sql-clr/RegEx
Here is an option using the DelimitedSplit8K_LEAD which can be found here. https://www.sqlservercentral.com/articles/reaping-the-benefits-of-the-window-functions-in-t-sql-2 This is an extension of Jeff Moden's splitter that is even a little bit faster than the original. The big advantage this splitter has over most of the others is that it returns the ordinal position of each element. One caveat to this is that I am using a space to split on based on your sample data. If you had numbers crammed in the middle of other characters this will ignore them. That may be good or bad depending on you specific requirements.
declare #Something varchar(100) = 'Some text with 12345 and 1234 and 12345678';
with MyCTE as
(
select x.ItemNumber
, Result = isnull(case when TRY_CONVERT(bigint, x.Item) is not null then isnull(replicate('*', len(convert(varchar(20), TRY_CONVERT(bigint, x.Item))) - 4), '') + right(convert(varchar(20), TRY_CONVERT(bigint, x.Item)), 4) end, x.Item)
from dbo.DelimitedSplit8K_LEAD(#Something, ' ') x
)
select Output = stuff((select ' ' + Result
from MyCTE
order by ItemNumber
FOR XML PATH('')), 1, 1, '')
This produces: Some text with *2345 and 1234 and ****5678
I have a string:
DECLARE #UserComment AS VARCHAR(1000) = 'bjones marked inspection on system UP for site COL01545 as Refused to COD won''t pay upfront :Routeid: 12 :Inspectionid: 55274'
Is there a way for me to extract everything from the string after 'Inspectionid: ' leaving me just the InspectionID to save into a variable?
Your example doesn't quite work correctly. You defined your variable as varchar(100) but there are more characters in your string than that.
This should work based on your sample data.
DECLARE #UserComment AS VARCHAR(1000) = 'bjones marked inspection on system UP for site COL01545 as Refused to COD won''t pay upfront :Routeid: 12 :Inspectionid: 55274'
select right(#UserComment, case when charindex('Inspectionid: ', #UserComment, 0) > 0 then len(#UserComment) - charindex('Inspectionid: ', #UserComment, 0) - 13 else len(#UserComment) end)
I would do this as:
select stuff(#UserComment, 1, charindex(':Inspectionid: ', #UserComment) + 14, '')
This works even if the string is not found -- although it will return the whole string. To get an empty string in this case:
select stuff(#UserComment, 1, charindex(':Inspectionid: ', #UserComment + ':Inspectionid: ') + 14, '')
Firstly, let me say that your #UserComment variable is not long enough to contain the text you're putting into it. Increase the size of that first.
The SQL below will extract the value:
DECLARE #UserComment AS VARCHAR(1000); SET #UserComment = 'bjones marked inspection on system UP for site COL01545 as Refused to COD won''t pay upfront :Routeid: 12 :Inspectionid: 55274'
DECLARE #pos int
DECLARE #InspectionId int
DECLARE #IdToFind varchar(100)
SET #IdToFind = 'Inspectionid: '
SET #pos = CHARINDEX(#IdToFind, #UserComment)
IF #pos > 0
BEGIN
SET #InspectionId = CAST(SUBSTRING(#UserComment, #pos+LEN(#IdToFind)+1, (LEN(#UserComment) - #pos) + 1) AS INT)
PRINT #InspectionId
END
You could make the above code into a SQL function if necessary.
If the Inspection ID is always 5 digits then the last argument for the Substring function (length) can be 5, i.e.
SELECT SUBSTRING(#UserComment,PATINDEX('%Inspectionid:%',#UserComment)+14,5)
If the Inspection ID varies (but is always at the end - which your question slightly implies), then the last argument can be derived by subtracting the position of 'InspectionID:' from the overall length of the string. Like this:
SELECT SUBSTRING(#UserComment,PATINDEX('%Inspectionid:%',#UserComment)+14,LEN(#usercomment)-(PATINDEX('%Inspectionid:%',#UserComment)+13))
I need a SQL query to get the value between two known strings (the returned value should start and end with these two strings).
An example.
"All I knew was that the dog had been very bad and required harsh punishment immediately regardless of what anyone else thought."
In this case the known strings are "the dog" and "immediately". So my query should return "the dog had been very bad and required harsh punishment immediately"
I've come up with this so far but to no avail:
SELECT SUBSTRING(#Text, CHARINDEX('the dog', #Text), CHARINDEX('immediately', #Text))
#Text being the variable containing the main string.
Can someone please help me with where I'm going wrong?
The problem is that the second part of your substring argument is including the first index.
You need to subtract the first index from your second index to make this work.
SELECT SUBSTRING(#Text, CHARINDEX('the dog', #Text)
, CHARINDEX('immediately',#text) - CHARINDEX('the dog', #Text) + Len('immediately'))
I think what Evan meant was this:
SELECT SUBSTRING(#Text, CHARINDEX(#First, #Text) + LEN(#First),
CHARINDEX(#Second, #Text) - CHARINDEX(#First, #Text) - LEN(#First))
An example is this: You have a string and the character $
String :
aaaaa$bbbbb$ccccc
Code:
SELECT SUBSTRING('aaaaa$bbbbb$ccccc',CHARINDEX('$','aaaaa$bbbbb$ccccc')+1, CHARINDEX('$','aaaaa$bbbbb$ccccc',CHARINDEX('$','aaaaa$bbbbb$ccccc')+1) -CHARINDEX('$','aaaaa$bbbbb$ccccc')-1) as My_String
Output:
bbbbb
You need to adjust for the LENGTH in the SUBSTRING. You were pointing it to the END of the 'ending string'.
Try something like this:
declare #TEXT varchar(200)
declare #ST varchar(200)
declare #EN varchar(200)
set #ST = 'the dog'
set #EN = 'immediately'
set #TEXT = 'All I knew was that the dog had been very bad and required harsh punishment immediately regardless of what anyone else thought.'
SELECT SUBSTRING(#Text, CHARINDEX(#ST, #Text), (CHARINDEX(#EN, #Text)+LEN(#EN))-CHARINDEX(#ST, #Text))
Of course, you may need to adjust it a bit.
I had a similar need to parse out a set of parameters stored within an IIS logs' csUriQuery field, which looked like this: id=3598308&user=AD\user¶meter=1&listing=No needed in this format.
I ended up creating a User-defined function to accomplish a string between, with the following assumptions:
If the starting occurrence is not found, a NULL is returned, and
If the ending occurrence is not found, the rest of the string is returned
Here's the code:
CREATE FUNCTION dbo.str_between(#col varchar(max), #start varchar(50), #end varchar(50))
RETURNS varchar(max)
WITH EXECUTE AS CALLER
AS
BEGIN
RETURN substring(#col, charindex(#start, #col) + len(#start),
isnull(nullif(charindex(#end, stuff(#col, 1, charindex(#start, #col)-1, '')),0),
len(stuff(#col, 1, charindex(#start, #col)-1, ''))+1) - len(#start)-1);
END;
GO
For the above question, the usage is as follows:
DECLARE #a VARCHAR(MAX) = 'All I knew was that the dog had been very bad and required harsh punishment immediately regardless of what anyone else thought.'
SELECT dbo.str_between(#a, 'the dog', 'immediately')
-- Yields' had been very bad and required harsh punishment '
Try this and replace '[' & ']' with your string
SELECT SUBSTRING(#TEXT,CHARINDEX('[',#TEXT)+1,(CHARINDEX(']',#TEXT)-CHARINDEX('[',#TEXT))-1)
I have a feeling you might need SQL Server's PATINDEX() function. Check this out:
Usage on Patindex() function
So maybe:
SELECT SUBSTRING(#TEXT, PATINDEX('%the dog%', #TEXT), PATINDEX('%immediately%',#TEXT))
SELECT
SUBSTRING( '123#yahoo.com', charindex('#','123#yahoo.com',1) + 1, charindex('.','123#yahoo.com',1) - charindex('#','123#yahoo.com',1) - 1 )
DECLARE #Text VARCHAR(MAX), #First VARCHAR(MAX), #Second VARCHAR(MAX)
SET #Text = 'All I knew was that the dog had been very bad and required harsh punishment immediately regardless of what anyone else thought.'
SET #First = 'the dog'
SET #Second = 'immediately'
SELECT SUBSTRING(#Text, CHARINDEX(#First, #Text),
CHARINDEX(#Second, #Text) - CHARINDEX(#First, #Text) + LEN(#Second))
You're getting the starting position of 'punishment immediately', but passing that in as the length parameter for your substring.
You would need to substract the starting position of 'the dog' from the charindex of 'punishment immediately', and then add the length of the 'punishment immediately' string to your third parameter. This would then give you the correct text.
Here's some rough, hacky code to illustrate the process:
DECLARE #text VARCHAR(MAX)
SET #text = 'All I knew was that the dog had been very bad and required harsh punishment immediately regardless of what anyone else thought.'
DECLARE #start INT
SELECT #start = CHARINDEX('the dog',#text)
DECLARE #endLen INT
SELECT #endLen = LEN('immediately')
DECLARE #end INT
SELECT #end = CHARINDEX('immediately',#text)
SET #end = #end - #start + #endLen
SELECT #end
SELECT SUBSTRING(#text,#start,#end)
Result: the dog had been very bad and required harsh punishment immediately
Among the many options is to create a simple function.
Can keep your code cleaner.
Gives the ability to handle errors if the start or end marker/string is not present.
This function also allows for trimming leading or trailing whitespace as an option.
SELECT dbo.GetStringBetweenMarkers('123456789', '234', '78', 0, 1)
Yields:
56
--Code to create the function
USE [xxxx_YourDB_xxxx]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE FUNCTION [dbo].[GetStringBetweenMarkers] (#FullString varchar(max), #StartMarker varchar(500), #EndMarker varchar(500), #TrimLRWhiteSpace bit, #ReportErrorInResult bit)
RETURNS varchar(max)
AS
BEGIN
--Purpose is to simply return the string between 2 string markers. ew 2022-11-06
--Will perform a LTRIM and RTRIM if #TrimLRWhiteSpace = 1
--Will report errors of either marker not being found in the RETURNed string if #ReportErrorInResult = 1.
-- When #ReportErrorInResult = 0, if the start marker isn't found, will return everything from the start of the #FullString to the left of the end marker.
-- When #ReportErrorInResult = 0, if the end marker isn't found, SQL will return an error of "Invalid length parameter passed to the LEFT or SUBSTRING function."
DECLARE #ReturnString VARCHAR(max) = ''
DECLARE #StartOfStartMarker INT = CHARINDEX(#StartMarker, #FullString)
DECLARE #StartOfTarget INT = CHARINDEX(#StartMarker, #FullString) + LEN(#StartMarker)
DECLARE #EndOfTarget INT = CHARINDEX(#EndMarker, #FullString, #StartOfTarget)
--If a marker wasn't found, put that into the
IF #ReportErrorInResult = 1
BEGIN
IF #EndOfTarget = 0 SET #ReturnString = '[ERROR: EndMarker not found.]'
IF #StartOfStartMarker = 0 SET #ReturnString = '[ERROR: StartMarker not found.]'
IF #StartOfStartMarker = 0 AND #EndOfTarget = 0 SET #ReturnString = '[ERROR: Both StartMarker and EndMarker not found.]'
END
--If not reporting errors, and start marker not found (i.e. CHARINDEX = 0) we would start our string at the LEN(#StartMarker).
-- This would give an odd result. Best to just provide from 0, i.e. the start of the #FullString.
IF #ReportErrorInResult = 0 AND #StartOfStartMarker = 0 SET #StartOfTarget = 0
--Main action
IF #ReturnString = '' SET #ReturnString = SUBSTRING(#FullString, #StartOfTarget, #EndOfTarget - #StartOfTarget)
IF #TrimLRWhiteSpace = 1 SET #ReturnString = LTRIM(RTRIM(#ReturnString))
RETURN #ReturnString
--Examples
-- SELECT '>' + dbo.GetStringBetweenMarkers('123456789','234','78',0,1) + '<' AS 'Result-Returns what is in between markers w/ white space'
-- SELECT '>' + dbo.GetStringBetweenMarkers('1234 56 789','234','78',0,1) + '<' AS 'Result-Without trimming white space'
-- SELECT '>' + dbo.GetStringBetweenMarkers('1234 56 789','234','78',1,1) + '<' AS 'Result-Will trim white space with a #TrimLRWhiteSpace = 1'
-- SELECT '>' + dbo.GetStringBetweenMarkers('abcdefgh','ABC','FG',0,1) + '<' AS 'Result-Not Case Sensitive'
-- SELECT '>' + dbo.GetStringBetweenMarkers('abc_de_fgh','_','_',0,1) + '<' AS 'Result-Using the same marker for start and end'
--Errors are returned if start or end marker are not found
-- SELECT '>' + dbo.GetStringBetweenMarkers('1234 56789','zz','78',0,1) + '<' AS 'Result-Start not found'
-- SELECT '>' + dbo.GetStringBetweenMarkers('1234 56789','234','zz',0,1) + '<' AS 'Result-End not found'
-- SELECT '>' + dbo.GetStringBetweenMarkers('1234 56789','zz','zz',0,1) + '<' AS 'Result-Niether found'
--If #ReportErrorInResult = 0
-- SELECT '>' + dbo.GetStringBetweenMarkers('123456789','zz','78',0,0) + '<' AS 'Result-Start not found-Returns from the start of the #FullString'
-- SELECT '>' + dbo.GetStringBetweenMarkers('123456789','34','zz',0,0) + '<' AS 'Result-End found-should get "Invalid length parameter passed to the LEFT or SUBSTRING function."'
END
GO
SELECT SUBSTRING('aaaaa$bbbbb$ccccc',instr('aaaaa$bbbbb$ccccc','$',1,1)+1, instr('aaaaa$bbbbb$ccccc','$',1,2)-1) -instr('aaaaa$bbbbb$ccccc','$',1,1)) as My_String
Hope this helps :
Declared a variable , in case of any changes need to be made thats only once .
declare #line varchar(100)
set #line ='Email_i-Julie#mail.com'
select SUBSTRING(#line ,(charindex('-',#line)+1), CHARINDEX('#',#line)-charindex('-',#line)-1)
I needed to get (099) 0000111-> (099) | 0000111 like two different columns.
SELECT
SUBSTRING(Phone, CHARINDEX('(', Phone) + 0, (2 + ((LEN(Phone)) - CHARINDEX(')', REVERSE(Phone))) - CHARINDEX('(', Phone))) AS CodePhone,
LTRIM(SUBSTRING(Phone, CHARINDEX(')', Phone) + 1, LEN(Phone))) AS NumberPhone
FROM
Suppliers
WHERE
Phone LIKE '%(%)%'
DECLARE #text VARCHAR(MAX)
SET #text = 'All I knew was that the dog had been very bad and required harsh punishment immediately regardless of what anyone else thought.'
DECLARE #pretext AS nvarchar(100) = 'the dog'
DECLARE #posttext AS nvarchar(100) = 'immediately'
SELECT
CASE
WHEN CHARINDEX(#posttext, #Text) - (CHARINDEX(#pretext, #Text) + len(#pretext)) < 0
THEN ''
ELSE SUBSTRING(#Text,
CHARINDEX(#pretext, #Text) + LEN(#pretext),
CHARINDEX(#posttext, #Text) - (CHARINDEX(#pretext, #Text) + LEN(#pretext)))
END AS betweentext
I'm a few years behind, but here's what I did to get a string between characters, that are not the same and also in the even you don't find the ending character, to still give you the substring
BEGIN
DECLARE #TEXT AS VARCHAR(20)
SET #TEXT='E101465445454-1'
SELECT SUBSTRING(#TEXT, CHARINDEX('E', #TEXT)+1, CHARINDEX('-',#TEXT)) as 'STR',
CAST(CHARINDEX('E', #TEXT)+1 AS INT) as 'val1', CAST(CHARINDEX('-', #TEXT) AS INT) as 'val2',
(CAST(CHARINDEX('-',#TEXT) AS INT) - CAST(CHARINDEX('E',#TEXT)+1 AS INT)) as 'SUBTR', LEN(#TEXT) as 'LEN'
SELECT CASE WHEN (CHARINDEX('-', #TEXT) > 0) THEN
SUBSTRING(#TEXT, CHARINDEX('E', #TEXT)+1, (CAST(CHARINDEX('-',#TEXT) AS INT) - CAST(CHARINDEX('E',#TEXT)+1 AS INT)))
ELSE
SUBSTRING(#TEXT, CHARINDEX('E', #TEXT)+1,LEN(#TEXT)- CHARINDEX('E', #TEXT))
END
END
Try it and comment for any improvements or if it does the job
select substring(#string,charindex('#first',#string)+1,charindex('#second',#string)-(charindex('#first',#string)+1))
Let us consider we have a string DUMMY_DATA_CODE_FILE and we want to find out the substring between 2nd and 3rd underscore(_). Then we use query something like this.
select SUBSTRING('DUMMY_DATA_CODE_FILE',charindex('_', 'DUMMY_DATA_CODE_FILE', (charindex('_','DUMMY_DATA_CODE_FILE', 1))+1)+1, (charindex('_', 'DUMMY_DATA_CODE_FILE', (charindex('_','DUMMY_DATA_CODE_FILE', (charindex('_','DUMMY_DATA_CODE_FILE', 1))+1))+1)- charindex('_', 'DUMMY_DATA_CODE_FILE', (charindex('_','DUMMY_DATA_CODE_FILE', 1))+1)-1)) as Code