I have a dilemma regarding how to extract a sub string from a larger string without using custom functions
The string is of this form:
" [1].[2]"
" [1].[2].[3]"
" [1].[2].[3].[4]"
Basically, it's a hierarchy that has 4 leading spaces for every child of a node. The task is to maintain those 4 leading spaces for every node, but get only the child in the end, without the full path
So, the final result should be something like this:
" [2]"
" [3]"
" [4]"
Could anyone help me?
Edit: or, if it's the case, should I modify the CTE that is building the result set?
The CTE is like the one below:
WITH Hierarchy(cid, cname, parent_id, Parents, nname)
AS
(
SELECT map_id, name, parent_id, CAST('' AS VARCHAR(MAX)), CAST(''+name AS VARCHAR(MAX))
FROM tblMapping AS tm1
WHERE parent_id = 0
UNION ALL
SELECT tm.map_id, tm.name, Parent.cid,
CAST(CASE WHEN Parent.Parents = ''
THEN(CAST(tm.parent_id AS VARCHAR(MAX)))
ELSE(Parent.Parents + '.' + CAST(tm.parent_id AS VARCHAR(MAX)))
END AS VARCHAR(MAX)),
-- add the formated column description
CAST(CASE WHEN Parent.Parents = ''
THEN(Parent.cname)
ELSE(' '+Parent.nname+ '.'+tm.name)
END AS VARCHAR(MAX))
FROM tblMapping AS tm
INNER JOIN Hierarchy AS Parent ON tm.parent_id = Parent.cid
)
SELECT *
FROM Hierarchy
ORDER BY ltrim(nname)
OPTION(MAXRECURSION 32767)
Thank you!
Try something like this
UPDATED: Inlcudes a root node now...
DECLARE #mockup TABLE(YourString VARCHAR(100));
INSERT INTO #mockup VALUES
(' [1]')
,(' [1].[2]')
,(' [1].[2].[3]')
,(' [1].[2].[3].[4]');
SELECT LEFT(YourString,CHARINDEX('[',YourString)-1)
+SUBSTRING(YourString,LEN(YourString)+1-CHARINDEX('[',REVERSE(YourString)),1000)
FROM #mockup;
I would concatenate the spaces part with the last index.
Something like
Select left(your_string, char_index(your_string, "[") - 1) +
right(your_string, 3) from your_table
The 3 is assuming there's always one digit. If not, this is a bit complex because no last index of in SQLServer:
Select left(your_string, char_index(your_string, "[") - 1) +
reverse(left(reverse(your_string), char_index(reverse(your_string), "[")))
from your_table
Related
I have a list of drug names that are stored in various upper and lower case combinations. I need to capitalize the first letter of each word in a string, excluding certain words. The string is separated by spaces, but can also be separated by a forward slash.
The following code works:
create table #exclusionlist (word varchar(25))
create table #drugnames (drugname varchar(50))
insert into #exclusionlist values ('ER')
insert into #exclusionlist values ('HCL')
insert into #drugnames values ('DRUGNAME ER')
insert into #drugnames values ('drugname hcl')
insert into #drugnames values ('ONEDRUG/OTHERDRUG')
select 'Product Name' = drugname
, 'Product Name 2' = STUFF((SELECT ' ' +
case when value in (select word from #exclusionlist) then upper(value)
else upper(left(value, 1)) + lower(substring(value, 2, len(value))) end
from STRING_SPLIT(drugname, ' ')
FOR XML PATH('')) ,1,1,'')
from #drugnames
The output looks like this:
Drugname ER
Drugname HCL
Onedrug/otherdrug
How can I get that last one to look like this:
Onedrug/Otherdrug
I did try STRING_SPLIT(replace(drugname, '/', ' '), ' ') but obviously replaces the slash with a space. And if the slash is at the end of the string like ONEDRUG/OTHERDRUG/ then the result looks like Onedrug Otherdrug
It's possible that the string may end in a forward slash due to the field only holding N number of characters. When data gets inserted into the table, only the first N characters of the drug name are inserted. If that Nth character is a slash, the string will end in a slash.
You can use a CASE expression for the separator parameter to the STRING_SPLIT function.
In the code below, a common table expression (CTE) uses STRING_SPLIT to split out all the drugname words and capitalize the first letter of each word as appropriate.
The CTE results are unioned together using STRING_AGG to join the drugnames back together. Note that separator parameter for STRING_AGG cannot be an expression. Using a CASE expression results in this error:
Msg 8733, Level 16, State 1, Line 14 Separator parameter for
STRING_AGG must be a string literal or variable.
For SQL Server, you would need to be on SQL 2017 or greater for the STRING_AGG function. (I added an id/identity column to the #drugnames temp table to assist with grouping.)
DROP TABLE IF EXISTS #exclusionlist;
DROP TABLE IF EXISTS #drugnames;
CREATE TABLE #exclusionlist (word VARCHAR(25))
CREATE TABLE #drugnames (id INT IDENTITY, drugname VARCHAR(50))
INSERT INTO #exclusionlist VALUES ('ER')
INSERT INTO #exclusionlist VALUES ('HCL')
INSERT INTO #drugnames VALUES ('DRUGNAME ER')
INSERT INTO #drugnames VALUES ('drugname hcl')
INSERT INTO #drugnames VALUES ('ONEDRUG/OTHERDRUG')
;WITH SomeStuff AS
(
select d.id, d.drugname,
sp.value AS SplitValue, el.word AS ExclusionListSpaceWord,
COALESCE(el.word,
UPPER(LEFT(sp.value, 1)) + LOWER(RIGHT(sp.value, LEN(sp.value) - 1))
) AS CapitalizedWord
from #drugnames d
CROSS APPLY STRING_SPLIT(d.drugname, CASE WHEN d.drugname LIKE '%/%' THEN '/' ELSE ' ' END ) sp
LEFT JOIN #exclusionlist el
ON el.word = sp.value
)
SELECT ss.id, STRING_AGG(ss.CapitalizedWord, '/') AS ReconstructedDrugname
FROM SomeStuff ss
WHERE ss.drugname LIKE '%/%'
GROUP BY ss.id
UNION
SELECT ss.id, STRING_AGG(ss.CapitalizedWord, ' ') AS ReconstructedDrugname
FROM SomeStuff ss
WHERE ss.drugname LIKE '% %'
GROUP BY ss.id
Output:
id ReconstructedDrugname
----------- ----------------------
1 Drugname ER
2 Drugname HCL
3 Onedrug/Otherdrug
If (hopefully) using SQL 2017+ you can somewhat compact the logic. first a single string split by using translate to create a common separator. THen apply the exclude word criteria forllowed by the upper-case criteria, then re-aggregate, noting which separator to use.
select *
from #drugnames
outer apply (
select case
when max(sep)=' ' then String_Agg(word,' ')
else String_Agg(word,'/') end NewName
from (
select
case
when exists (select * from #exclusionlist x where x.word = value)
then Upper(value)
else Stuff(Lower(value),1,1,Upper(Left(value,1)))
end word, Iif(drugname like '%/%','/',' ') sep
from String_Split(Translate(drugname,' /','**'),'*')
)w
)new;
Example DB<>Fiddle
Output:
Note - using string_split does not, according to the documentation, guarantee the ordering of the values. In practice, I've never seen this be the case and since you're already using the function I'm using here also. There's plenty of ways to split the string while retaining an ordering (using json for example) should it ever prove necessary.
If you are still on 2016 then the string_agg can just be replaced with the for xml implementation you're already using.
If the "/" is replaced to a " /", then the split can still happen on the space.
And the extra space can be removed afterwards.
SELECT
[Product Name] = d.drugname
, [Product Name 2] = ca.drugname2
FROM #drugnames d
CROSS APPLY (
SELECT
REPLACE(LTRIM(x.value('(./text())[1]','VARCHAR(MAX)')),' /','/') AS drugname2
FROM
(
SELECT ' '+
CASE
WHEN e.word IS NOT NULL THEN e.word
WHEN s.value LIKE '/%'
THEN STUFF(LOWER(s.value),1,2,UPPER(LEFT(s.value,2)))
ELSE STUFF(LOWER(s.value),1,1,UPPER(LEFT(s.value,1)))
END
FROM STRING_SPLIT(REPLACE(d.drugname,'/',' /'),' ') s
LEFT JOIN #exclusionlist e
ON e.word = s.value
FOR XML PATH(''), TYPE
) q(x)
) ca;
Product Name
Product Name 2
DRUGNAME ER
Drugname ER
drugname hcl
Drugname HCL
ONEDRUG/OTHERDRUG
Onedrug/Otherdrug
Test on db<>fiddle here
So I need to compare a string against another string to see if any parts of the string match. This would be useful for checking if a list of salespeople IDs against the ones that are listed to a specific GM or if falls outside of that GMs list of IDs:
ID_SP ID_GM NEEDED FIELD (overlap)
136,338,342 512,338,112 338
512,112,208 512,338,112 512,112
587,641,211 512,338,112 null
I'm struggling on how to achieve this. I'm guessing some sort of UDF?
I realize this would be much easier to have done prior to using the for XML path(''), but I'm hoping for a solution that doesn't require me to unravel the data as that will blow up the overall size of the dataset.
No, that is not how you do it. You would go back to the raw data. To get the ids in common:
select tbob.id
from t tbob join
t tmary
on tbob.id = tmary.id and tbob.manager = 'Bob' and tmary.manager = 'Mary';
Since the data set isn't two raw sources, but one 'concatenated field' and a hardcoded string field that is a list of GMIDs (same value for every row) then the correct answer (from the starting point of the question) is to use something like nodes('/M') as Split(a).
Then you get something like this:
ID_SP ID_GM
136 512,338,112
338 512,338,112
342 512,338,112
and can do something like this:
case when ID_GM not like '%'+ID_SP+'%'then 1 else 0 end as 'indicator'
From here you can aggregate back and sum the indicator field and say that if > 0 then the ID_SP exists in the list of ID_GMs
Hope this helps someone else.
-- Try This
Declare #String1 as varchar(100)='512,112,208';
Declare #String2 as varchar(100)='512,338,112';
WITH FirstStringSplit(S1) AS
(
SELECT CAST('<x>' + REPLACE(#String1,',','</x><x>') + '</x>' AS XML)
)
,SecondStringSplit(S2) AS
(
SELECT CAST('<x>' + REPLACE(#String2,',','</x><x>') + '</x>' AS XML)
)
SELECT STUFF(
(
SELECT ',' + part1.value('.','nvarchar(max)')
FROM FirstStringSplit
CROSS APPLY S1.nodes('/x') AS A(part1)
WHERE part1.value('.','nvarchar(max)') IN(SELECT B.part2.value('.','nvarchar(max)')
FROM SecondStringSplit
CROSS APPLY S2.nodes('/x') AS B(part2)
)
FOR XML PATH('')
),1,1,'')
Gordon is correct, that you should not do this. This ought do be done with the raw data. the following code will "go back to the raw data" and solve this with an easy INNER JOIN.
The CTEs will create derived tables (all the many rows you want to avoid) and check them for equality (Not using indexes! One more reason to do this in advance):
DECLARE #tbl TABLE(ID INT IDENTITY,ID_SP VARCHAR(100),ID_GM VARCHAR(100));
INSERT INTO #tbl VALUES
('136,338,342','512,338,112')
,('512,112,208','512,338,112')
,('587,641,211','512,338,112');
WITH Splitted AS
(
SELECT t.*
,CAST('<x>' + REPLACE(t.ID_SP,',','</x><x>') + '</x>' AS xml) AS PartedSP
,CAST('<x>' + REPLACE(t.ID_GM,',','</x><x>') + '</x>' AS xml) AS PartedGM
FROM #tbl AS t
)
,SetSP AS
(
SELECT Splitted.ID
,Splitted.ID_SP
,x.value('text()[1]','int') AS SP_ID
FROM Splitted
CROSS APPLY PartedSP.nodes('/x') AS A(x)
)
,SetGM AS
(
SELECT Splitted.ID
,Splitted.ID_GM
,x.value('text()[1]','int') AS GM_ID
FROM Splitted
CROSS APPLY PartedGM.nodes('/x') AS A(x)
)
,BackToYourRawData AS --Here is the point you should do this in advance!
(
SELECT SetSP.ID
,SetSP.SP_ID
,SetGM.GM_ID
FROM SetSP
INNER JOIN SetGM ON SetSP.ID=SetGM.ID
AND SetSP.SP_ID=SetGM.GM_ID
)
SELECT ID
,STUFF((
SELECT ',' + CAST(rd2.SP_ID AS VARCHAR(10))
FROM BackToYourRawData AS rd2
WHERE rd.ID=rd2.ID
ORDER BY rd2.SP_ID
FOR XML PATH('')),1,1,'') AS CommonID
FROM BackToYourRawData AS rd
GROUP BY ID;
The result
ID CommonID
1 338
2 112,512
Thank you in advance.
I want to remove the values after ., but the length is not defined it keeps changing but it should not take the values after the second FULL STOP.
1)Example:
Input:- ROL0602.E.DCM.5264403 and COK0105.F.SKE and CLT005.02A.FCM.65721
output: ROL0602.E and COK0105.F and CLT005.02A
2) example :
Input: SKE-5700-00211-000
output: SKE-5700-00211
These are the two columns i want some help with.
I tried using the charindex but as the length keeps on changing i wasn't able to do it.
Try it like this:
DECLARE #tbl TABLE(YourValue VARCHAR(100));
INSERT INTO #tbl VALUES
('ROL0602.E.DCM.5264403'),('COK0105.F.SKE'),('CLT005.02A.FCM.65721'),('SKE-5700-00211-000');
SELECT CASE WHEN CHARINDEX('.',YourValue)>0 THEN LEFT(YourValue,CHARINDEX('.',YourValue,CHARINDEX('.',YourValue,1)+1)-1)
ELSE YourValue END
FROM #tbl AS tbl
The result
ROL0602.E
COK0105.F
CLT005.02A
SKE-5700-00211-000
Please provide a rule for the last example. Don't know how to cut this...
This command uses LEFT to take a string starting at the beginning. The lenght is found by searching for a dot, starting at the position 1 after the first dot.
UPDATE: A more generic solution
The following fill first split the string in its parts (easy to use with other separators too). This is finally re-concatenated depending on a rule you can define yourself:
DECLARE #tbl TABLE(YourValue VARCHAR(100));
INSERT INTO #tbl VALUES
('ROL0602.E.DCM.5264403'),('COK0105.F.SKE'),('CLT005.02A.FCM.65721'),('SKE-5700-00211-000');
WITH Parted AS
(
SELECT YourValue
,CAST('<x>' + REPLACE(REPLACE(tbl.YourValue,'-','.'),'.','</x><x>') + '</x>' AS XML) AS Casted
,CASE WHEN CHARINDEX('.',YourValue)>0 THEN '.' ELSE CASE WHEN CHARINDEX('-',YourValue)>0 THEN '-' ELSE '?' END END AS SeparatorChar
FROM #tbl AS tbl
)
,SingleParts AS
(
SELECT SeparatorChar
,YourValue
,ISNULL(Casted.value('/x[1]','nvarchar(max)'),'') AS Part1
,ISNULL(Casted.value('/x[2]','nvarchar(max)'),'') AS Part2
,ISNULL(Casted.value('/x[3]','nvarchar(max)'),'') AS Part3
,ISNULL(Casted.value('/x[4]','nvarchar(max)'),'') AS Part4
,ISNULL(Casted.value('/x[5]','nvarchar(max)'),'') AS Part5
FROM Parted
)
SELECT CASE SeparatorChar
WHEN '.' THEN Part1 + '.' + Part2
WHEN '-' THEN Part1 + '-' + Part2 + '-' + Part3
ELSE YourValue
END
FROM SingleParts
The result
ROL0602.E
COK0105.F
CLT005.02A
SKE-5700-00211
SHnugo's solution is excellent but will fail if there is only one dot (.) in the string. Here's a tweaked version that will handle that (note my comments).
DECLARE #tbl TABLE(YourValue VARCHAR(100));
INSERT INTO #tbl VALUES
('ROL0602.E.DCM.5264403'),('COK0105.F.SKE'),('CLT005.02A.FCM.65721'),('CLT099.02ACFVVV721'), ('SKE-5700-00211-000');
SELECT
CASE
--If there are two or more dots(.) then return everything up to the second dot:
WHEN LEN(YourValue) - LEN(REPLACE(YourValue,'.','')) > 1
THEN LEFT(YourValue,CHARINDEX('.',YourValue,CHARINDEX('.',YourValue,1)+1)-1)
ELSE YourValue -- if there are 1 or 0 dots(.) then return the entire value
END
FROM #tbl AS tbl;
You can try as follows:
SELECT REPLACE('ROL0602.E.DCM.5264403',
( '.' + REVERSE(SUBSTRING(REVERSE('ROL0602.E.DCM.5264403'), 0,
CHARINDEX('.',
REVERSE('ROL0602.E.DCM.5264403')))) ),
'')
I want to convert numbers to 3 decimal places for each number between the character ".". For example:
1.1.5.2 -> 001.001.005.002
1.2 -> 001.002
4.0 -> 004.000
4.3 ->004.003
4.10 -> 004.010
This is my query:
SELECT ItemNo
FROM EstAsmTemp
This is fairly easy once you understand all the steps:
Split the string into the individual data points.
Convert the parsed values into the format you want.
Shove the new values back into a delimited list.
Ideally you shouldn't store data with multiple datapoints in a single intersection like this but sometimes you just have no choice.
I am using the string splitter from Jeff Moden and the community at Sql Server Central which can be found here. http://www.sqlservercentral.com/articles/Tally+Table/72993/. There are plenty of other decent string splitters out there. Here are some excellent examples of other options. http://sqlperformance.com/2012/07/t-sql-queries/split-strings.
Make sure you understand this code before you use it in your production system because it will be you that gets the phone call at 3am asking for it to be fixed.
with something(SomeValue) as
(
select '1.1.5.2' union all
select '1.2' union all
select '4.0' union all
select '4.3' union all
select '4.10'
)
, parsedValues as
(
select SomeValue
, right('000' + CAST(x.Item as varchar(3)), 3) as NewValue
, x.ItemNumber as SortOrder
from something s
cross apply dbo.DelimitedSplit8K(SomeValue, '.') x
)
select SomeValue
, STUFF((Select '.' + NewValue
from parsedValues pv2
where pv2.SomeValue = pv.SomeValue
order by pv2.SortOrder
FOR XML PATH('')), 1, 1, '') as Details
from parsedValues pv
group by pv.SomeValue
I decided to change it in the presentation layer, per Zohar Peled's comment.
You did not mention the number of '.' separator a column can have. I assume, the max is 4 and the solution is below.
SELECT STUFF(ISNULL('.' + RIGHT('000' + PARSENAME(STRVALUE,4),4),'') + ISNULL('.' + RIGHT('000' + PARSENAME(STRVALUE,3),4) ,'') + ISNULL('.' + RIGHT('000' + PARSENAME(STRVALUE,2),4) ,'') + ISNULL('.' + RIGHT('000' + PARSENAME(STRVALUE,1),4),''),1,1,'')
FROM (VALUES('1.1.5.2'), ('1.2'), ('4.0'),('4.3'), ('4.10')) A (STRVALUE)
I have a table that contains text field with placeholders. Something like this:
Row Notes
1. This is some notes ##placeholder130## this ##myPlaceholder##, #oneMore#. End.
2. Second row...just a ##test#.
(This table contains about 1-5k rows on average. Average number of placeholders in one row is 5-15).
Now, I have a lookup table that looks like this:
Name Value
placeholder130 Dog
myPlaceholder Cat
oneMore Cow
test Horse
(Lookup table will contain anywhere from 10k to 100k records)
I need to find the fastest way to join those placeholders from strings to a lookup table and replace with value. So, my result should look like this (1st row):
This is some notes Dog this Cat, Cow. End.
What I came up with was to split each row into multiple for each placeholder and then join it to lookup table and then concat records back to original row with new values, but it takes around 10-30 seconds on average.
You could try to split the string using a numbers table and rebuild it with for xml path.
select (
select coalesce(L.Value, T.Value)
from Numbers as N
cross apply (select substring(Notes.notes, N.Number, charindex('##', Notes.notes + '##', N.Number) - N.Number)) as T(Value)
left outer join Lookup as L
on L.Name = T.Value
where N.Number <= len(notes) and
substring('##' + notes, Number, 2) = '##'
order by N.Number
for xml path(''), type
).value('text()[1]', 'varchar(max)')
from Notes
SQL Fiddle
I borrowed the string splitting from this blog post by Aaron Bertrand
SQL Server is not very fast with string manipulation, so this is probably best done client-side. Have the client load the entire lookup table, and replace the notes as they arrived.
Having said that, it can of course be done in SQL. Here's a solution with a recursive CTE. It performs one lookup per recursion step:
; with Repl as
(
select row_number() over (order by l.name) rn
, Name
, Value
from Lookup l
)
, Recurse as
(
select Notes
, 0 as rn
from Notes
union all
select replace(Notes, '##' + l.name + '##', l.value)
, r.rn + 1
from Recurse r
join Repl l
on l.rn = r.rn + 1
)
select *
from Recurse
where rn =
(
select count(*)
from Lookup
)
option (maxrecursion 0)
Example at SQL Fiddle.
Another option is a while loop to keep replacing lookups until no more are found:
declare #notes table (notes varchar(max))
insert #notes
select Notes
from Notes
while 1=1
begin
update n
set Notes = replace(n.Notes, '##' + l.name + '##', l.value)
from #notes n
outer apply
(
select top 1 Name
, Value
from Lookup l
where n.Notes like '%##' + l.name + '##%'
) l
where l.name is not null
if ##rowcount = 0
break
end
select *
from #notes
Example at SQL Fiddle.
I second the comment that tsql is just not suited for this operation, but if you must do it in the db here is an example using a function to manage the multiple replace statements.
Since you have a relatively small number of tokens in each note (5-15) and a very large number of tokens (10k-100k) my function first extracts tokens from the input as potential tokens and uses that set to join to your lookup (dbo.Token below). It was far too much work to look for an occurrence of any of your tokens in each note.
I did a bit of perf testing using 50k tokens and 5k notes and this function runs really well, completing in <2 seconds (on my laptop). Please report back how this strategy performs for you.
note: In your example data the token format was not consistent (##_#, ##_##, #_#), I am guessing this was simply a typo and assume all tokens take the form of ##TokenName##.
--setup
if object_id('dbo.[Lookup]') is not null
drop table dbo.[Lookup];
go
if object_id('dbo.fn_ReplaceLookups') is not null
drop function dbo.fn_ReplaceLookups;
go
create table dbo.[Lookup] (LookupName varchar(100) primary key, LookupValue varchar(100));
insert into dbo.[Lookup]
select '##placeholder130##','Dog' union all
select '##myPlaceholder##','Cat' union all
select '##oneMore##','Cow' union all
select '##test##','Horse';
go
create function [dbo].[fn_ReplaceLookups](#input varchar(max))
returns varchar(max)
as
begin
declare #xml xml;
select #xml = cast(('<r><i>'+replace(#input,'##' ,'</i><i>')+'</i></r>') as xml);
--extract the potential tokens
declare #LookupsInString table (LookupName varchar(100) primary key);
insert into #LookupsInString
select distinct '##'+v+'##'
from ( select [v] = r.n.value('(./text())[1]', 'varchar(100)'),
[r] = row_number() over (order by n)
from #xml.nodes('r/i') r(n)
)d(v,r)
where r%2=0;
--tokenize the input
select #input = replace(#input, l.LookupName, l.LookupValue)
from dbo.[Lookup] l
join #LookupsInString lis on
l.LookupName = lis.LookupName;
return #input;
end
go
return
--usage
declare #Notes table ([Id] int primary key, notes varchar(100));
insert into #Notes
select 1, 'This is some notes ##placeholder130## this ##myPlaceholder##, ##oneMore##. End.' union all
select 2, 'Second row...just a ##test##.';
select *,
dbo.fn_ReplaceLookups(notes)
from #Notes;
Returns:
Tokenized
--------------------------------------------------------
This is some notes Dog this Cat, Cow. End.
Second row...just a Horse.
Try this
;WITH CTE (org, calc, [Notes], [level]) AS
(
SELECT [Notes], [Notes], CONVERT(varchar(MAX),[Notes]), 0 FROM PlaceholderTable
UNION ALL
SELECT CTE.org, CTE.[Notes],
CONVERT(varchar(MAX), REPLACE(CTE.[Notes],'##' + T.[Name] + '##', T.[Value])), CTE.[level] + 1
FROM CTE
INNER JOIN LookupTable T ON CTE.[Notes] LIKE '%##' + T.[Name] + '##%'
)
SELECT DISTINCT org, [Notes], level FROM CTE
WHERE [level] = (SELECT MAX(level) FROM CTE c WHERE CTE.org = c.org)
SQL FIDDLE DEMO
Check the below devioblog post for reference
devioblog post
To get speed, you can preprocess the note templates into a more efficient form. This will be a sequence of fragments, with each ending in a substitution. The substitution might be NULL for the last fragment.
Notes
Id FragSeq Text SubsId
1 1 'This is some notes ' 1
1 2 ' this ' 2
1 3 ', ' 3
1 4 '. End.' null
2 1 'Second row...just a ' 4
2 2 '.' null
Subs
Id Name Value
1 'placeholder130' 'Dog'
2 'myPlaceholder' 'Cat'
3 'oneMore' 'Cow'
4 'test' 'Horse'
Now we can do the substitutions with a simple join.
SELECT Notes.Text + COALESCE(Subs.Value, '')
FROM Notes LEFT JOIN Subs
ON SubsId = Subs.Id WHERE Notes.Id = ?
ORDER BY FragSeq
This produces a list of fragments with substitutions complete. I am not an MSQL user, but in most dialects of SQL you can concatenate these fragments in a variable quite easily:
DECLARE #Note VARCHAR(8000)
SELECT #Note = COALESCE(#Note, '') + Notes.Text + COALSCE(Subs.Value, '')
FROM Notes LEFT JOIN Subs
ON SubsId = Subs.Id WHERE Notes.Id = ?
ORDER BY FragSeq
Pre-processing a note template into fragments will be straightforward using the string splitting techniques of other posts.
Unfortunately I'm not at a location where I can test this, but it ought to work fine.
I really don't know how it will perform with 10k+ of lookups.
how does the old dynamic SQL performs?
DECLARE #sqlCommand NVARCHAR(MAX)
SELECT #sqlCommand = N'PlaceholderTable.[Notes]'
SELECT #sqlCommand = 'REPLACE( ' + #sqlCommand +
', ''##' + LookupTable.[Name] + '##'', ''' +
LookupTable.[Value] + ''')'
FROM LookupTable
SELECT #sqlCommand = 'SELECT *, ' + #sqlCommand + ' FROM PlaceholderTable'
EXECUTE sp_executesql #sqlCommand
Fiddle demo
And now for some recursive CTE.
If your indexes are correctly set up, this one should be very fast or very slow. SQL Server always surprises me with performance extremes when it comes to the r-CTE...
;WITH T AS (
SELECT
Row,
StartIdx = 1, -- 1 as first starting index
EndIdx = CAST(patindex('%##%', Notes) as int), -- first ending index
Result = substring(Notes, 1, patindex('%##%', Notes) - 1)
-- (first) temp result bounded by indexes
FROM PlaceholderTable -- **this is your source table**
UNION ALL
SELECT
pt.Row,
StartIdx = newstartidx, -- starting index (calculated in calc1)
EndIdx = EndIdx + CAST(newendidx as int) + 1, -- ending index (calculated in calc4 + total offset)
Result = Result + CAST(ISNULL(newtokensub, newtoken) as nvarchar(max))
-- temp result taken from subquery or original
FROM
T
JOIN PlaceholderTable pt -- **this is your source table**
ON pt.Row = T.Row
CROSS APPLY(
SELECT newstartidx = EndIdx + 2 -- new starting index moved by 2 from last end ('##')
) calc1
CROSS APPLY(
SELECT newtxt = substring(pt.Notes, newstartidx, len(pt.Notes))
-- current piece of txt we work on
) calc2
CROSS APPLY(
SELECT patidx = patindex('%##%', newtxt) -- current index of '##'
) calc3
CROSS APPLY(
SELECT newendidx = CASE
WHEN patidx = 0 THEN len(newtxt) + 1
ELSE patidx END -- if last piece of txt, end with its length
) calc4
CROSS APPLY(
SELECT newtoken = substring(pt.Notes, newstartidx, newendidx - 1)
-- get the new token
) calc5
OUTER APPLY(
SELECT newtokensub = Value
FROM LookupTable
WHERE Name = newtoken -- substitute the token if you can find it in **your lookup table**
) calc6
WHERE newstartidx + len(newtxt) - 1 <= len(pt.Notes)
-- do this while {new starting index} + {length of txt we work on} exceeds total length
)
,lastProcessed AS (
SELECT
Row,
Result,
rn = row_number() over(partition by Row order by StartIdx desc)
FROM T
) -- enumerate all (including intermediate) results
SELECT *
FROM lastProcessed
WHERE rn = 1 -- filter out intermediate results (display only last ones)