How to create a single string in SQL, based on Hierarchical Data - CTE and XML? - sql

I have a database table storing stock locations for various products.
One part number can have several Departments, each department can have several racks and so on (shelves and then rows).
I need to concatenate this information to fit the dataset on label (on a single line).
To retrieve the dataset my query is as follows.
select d.Department_Name, r.Rack_Name, s.Shelf_Name, rw.Row_Name from Part_Locator l
join Part_Numbers p
on l.Part_Id=p.Part_Id
join PL_Departments d
on l.Department_Id=d.Department_Id
join PL_Racks r
on l.Rack_Id=r.Rack_Id
join PL_Shelfs s
on l.Shelf_Id=s.Shelf_Id
join PL_Rows rw
on l.Row_Id=rw.Row_Id
where p.Part_Number='1BODY000997'
Results are as follows:
I can make a nested look to build a string, but I believe a combination of recursive CTE, Stuff and XML could so the same for me. So far I can find many examples, but none that I can understand well enough to make them work together application.
The desired output is as follows:
Assembly - Rack IA-10 - Shelf A - Row 1,2,3,4,5,6 - Shelf B - Row 1,2,3,4,5,6 - Shelf C - Row 1,2,3,4,5,6
I am not opposed to changing that format as long as it reads logically. Also please note there can multiple departments.
Any help to get me going in the right direction would be great.
Thanks in advance!
I was able to get this working using cursor. I also created a view to remove the joins from the query, much easier to read.
DECLARE
#partnumber varchar(50)='1BODY000997', #dept varchar(50), #rack varchar(50), #shelf varchar(50), #row varchar(50), #text varchar(500)='';
DECLARE department_cursor CURSOR FOR
SELECT DISTINCT Department_Name FROM Part_Locations WHERE Part_Number=#partnumber
OPEN department_cursor;
FETCH NEXT FROM department_cursor INTO #dept;
WHILE ##FETCH_STATUS = 0
BEGIN
SET #text=#text+CAST(#dept as varchar(50))+' - ';
DECLARE rack_cursor CURSOR FOR
SELECT DISTINCT Rack_Name FROM Part_Locations WHERE Part_Number=#partnumber AND Department_Name=#dept
OPEN rack_cursor;
FETCH NEXT FROM rack_cursor INTO #rack
WHILE ##FETCH_STATUS = 0
BEGIN
SET #text=#text+'Rack:'+CAST(#rack as varchar(50))+' - ';
DECLARE shelf_cursor CURSOR FOR
SELECT DISTINCT Shelf_Name FROM Part_Locations WHERE Part_Number=#partnumber AND Department_Name=#dept AND Rack_Name=#rack
OPEN shelf_cursor;
FETCH NEXT FROM shelf_cursor INTO #shelf
WHILE ##FETCH_STATUS = 0
BEGIN
SET #text=#text+'Shelf:'+CAST(#shelf as varchar(50))+' (Row ';
SET #text=#text+(SELECT abc=STUFF((SELECT DISTINCT ',' + Row_Name FROM Part_Locations WHERE Part_Number=#partnumber AND Department_Name=#dept AND Rack_Name=#rack AND Shelf_Name=#shelf FOR XML PATH ('')), 1, 1, ''))
SET #text=#text+') ';
FETCH NEXT FROM shelf_cursor INTO #shelf
END;
CLOSE shelf_cursor;
DEALLOCATE shelf_cursor;
FETCH NEXT FROM rack_cursor INTO #rack
END;
CLOSE rack_cursor;
DEALLOCATE rack_cursor;
FETCH NEXT FROM department_cursor INTO #dept;
END
CLOSE department_cursor;
DEALLOCATE department_cursor;
PRINT (#text)
this gives me good output of..
Assembly - Rack:IA-10 - Shelf:A (Row 1,2,3,4,5,6) Shelf:B (Row 1,2,3,4,5,6) Shelf:C (Row 1,2,3,4,5,6)

As mentioned, you could do a cte, or you could do a running field (similar to running total) , or xml path. Actually, forget the running field, too hard. Below are two examples , one with xml, followed by a cte. I used in that link I sent earlier to the the xml path. Due to the complexity, I needed to do two xml paths ( first one into a temp table)
Note that I created a table (ITEMS) and put your results into it ...
SELECT DISTINCT i.Dept, i.rack, i.shelf,
stuff( ( select ',' + i2.rowname
from ITEMS i2
where i2.Dept = i.Dept and
i2.rack = i.rack and
i2.shelf =i.shelf
FOR XML PATH('')),1,1,'') as RownumbersCommaSeparated
INTO #T1
FROM ITEMS i
SELECT
stuff( ( select ' ' + T.Dept + ' Rack ' + T.rack + ' Shelf ' + T.shelf + ' Rows ' + T.RownumbersCommaSeparated
from #T1 T
FOR XML PATH('')),1,1,'' ) as DesiredOutput
The common table expression is a bit longer ! There are TWO recursions ...
;
WITH
CTE0 AS
(
SELECT i.*,
ROW_NUMBER() OVER(ORDER BY i.Dept,i.rack,i.shelf, i.rowname) recordnumber,
RANK() OVER ( ORDER BY i.Dept,i.rack,i.shelf) Parent,
RANK() OVER (PARTITION BY i.Dept,i.rack,i.shelf ORDER BY i.Dept,i.rack,i.shelf,i.rowname) rowidForeachParent
FROM ITEMS i
),
CTE1 AS
( SELECT
i.Parent,
i.Dept + ' Rack ' +
i.rack + ' Shelf ' +
i.shelf + ' Rows ' as CoreData,
MAX(i.rowidforeachparent) maxrowidforeachparent
FROM cte0 i
group by i.Parent, i.Dept, i.rack, i.shelf
),
CTE2 AS
(
SELECT
i1.Parent,
i1.maxrowidforeachparent,
CAST( i1.CoreData + ',' + i0.rowname AS varchar(MAX) ) as outputX,
i0.rowidForeachParent
FROM cte1 i1
join cte0 i0 on
i0.Parent = i1.Parent AND
I0.rowidForeachParent = 1
UNION ALL
SELECT
i1.Parent,
i1.maxrowidforeachparent,
i1.outputX + ',' + i0.rowname,
i0.rowidForeachParent
FROM cte2 i1
join cte0 i0 on
i0.Parent = i1.Parent AND
I0.rowidForeachParent = I1.rowidForeachParent+1
) ,
CTE3 AS
(
SELECT i.outputX ,
ROW_NUMBER() OVER (ORDER BY Parent) ParentId
FROM cte2 i
where i.rowidForeachParent = i.maxrowidforeachparent
),
CTE4 AS
(
SELECT outputX, i3.ParentId FROM CTE3 i3
where i3.ParentId = 1
UNION ALL
SELECT i4.outputX + ' ' + i3.outputX, i3.ParentId FROM CTE3 i3
join CTE4 i4 on i3.ParentId = i4.ParentId + 1
)
select top 1 * from cte4 order by ParentId desc

Related

Generate a comma-separated list of numbers in a single string

Is there a way to generate a comma-separated string of a series of numbers where the "begin" and "end" numbers are provided?
For example, provide the numbers 1 and 10 and the output would be a single value of: 1,2,3,4,5,6,7,8,9,10
10/10/2019 edit explaining why I'm interested in this:
My workplace writes queries with several columns in the SELECT statement plus aggregate functions. Then a GROUP BY clause using the column numbers. I figured using a macro that creates a comma-separated list to copy/paste in would save some time.
SELECT t.colA
, t.colB
, t.colC
, t.colD
, t.colE
, t.colF
, t.colG
, t.colH
, t.colI
, t.colJ
, sum(t.colK) as sumK
, sum(t.colL) as sumL
, sum(t.colM) as sumM
FROM t
GROUP BY 1, 2, 3, 4, 5, 6, 7, 8, 9, 10
;
You can use a recursive CTE to generate your numbers, and xml_agg to generate your string:
with recursive nums (counter) as
( select * from (select cast(1 as bigint) as counter) t
union all
select
counter + 1
from nums
where counter between 1 and 9
)
select
trim(trailing ',' from cast(xmlagg(cast(counter as varchar(2)) || ',' order by counter) as varchar(100)))
from nums
Check these methods in SQL Server-
IF OBJECT_ID('TEMPDB..#Sample') IS NOT NULL
DROP TABLE #Sample
Create table #Sample
(
NUM int
)
declare #n int
select #n=10
insert into #Sample(NUM)
SELECT NUM FROM (select row_number() over (order by (select null)) AS NUM from sys.columns) A WHERE NUM<=#N
--Method 1 (For SQL SERVER -NEW VERSION Support)
SELECT STRING_AGG(NUM,',') AS EXPECTED_RESULT FROM #Sample
--Method 1 (For SQL SERVER -OLD VERSION Support)
select DISTINCT STUFF(CAST((
SELECT ' ,' +CAST(c.num AS VARCHAR(MAX))
FROM (
SELECT num
FROM #Sample
) c
FOR XML PATH(''), TYPE) AS VARCHAR(MAX)), 1, 2, '') AS EXPECTED_RESULT
from #Sample t
While loop seems appropriate
declare #begin int=1
declare #end int=11
declare #list varchar(500)
if #begin > #end
begin
select 'error, beginning number ' + convert(varchar(500),#begin)
+ ' must not be greater than ending number '
+ convert(varchar(500),#end) + '.' err
return
end
else
set #list = convert(varchar(500),#begin)
;
while #begin < #end
begin
set #begin += 1
set #list = #list + ',' + convert(varchar(500),#begin)
end
select #list
You might want to use varchar(5000) or something depending on how big you want it to get.
disclaimer -- I don't know if this works with teradata
I'm not sure there is a good direct way to generate a series in Teradata. You can fake it a few different ways though. Here's a comma separated list of numbers from 5 to 15, for example:
SELECT TRIM(TRAILING ',' FROM (XMLAGG(TRIM(rn)|| ',' ) (VARCHAR(10000))))
FROM (SELECT 4 + ROW_NUMBER() OVER (ORDER BY Sys_Calendar."CALENDAR".day_of_calendar) as rn FROM Sys_Calendar."CALENDAR" QUALIFY rn <= 15) t
I've only used sys_calendar.calendar here because it's a big table. Any big table would do here though.
Here's one way to do it in Teradata:
SELECT ARRAY_AGG(src.RowNum)
FROM (
SELECT ROW_NUMBER() OVER() AS RowNum
FROM sys_calendar.calendar
QUALIFY RowNum BETWEEN <begin_num> AND <end_num>
) src
This will give you the output as an ARRAY data type, which you can probably cast as a VARCHAR. It also assumes begin_num > 0 and <end_num> is less than the number of rows in the sys_calendar.calendar view. You can always fiddle with this to fit your required range of values.
There are also DelimitedBuild UDFs out there (if you can find one) that can be used to convert row values into delimited strings.
The cheapest way to achieve your goal is this one (no functions, or joins to tables required):
WITH RECURSIVE NumberRanges(TheNumber,TheString) AS
(
SELECT 1 AS TheNumber,casT(1 as VARCHAR(500)) as TheString
FROM
(
SELECT * FROM (SELECT NULL AS X) X
) DUMMYTABLE
UNION ALL
SELECT
TheNumber + 1 AS TheNumber,
TheString ||',' || TRIM(TheNumber+1)
FROM NumberRanges
WHERE
TheNumber < 10
)
SELECT TheString
FROM NumberRanges
QUALIFY ROW_NUMBER() OVER ( ORDER BY TheNumber DESC) = 1;
Result String: 1,2,3,4,5,6,7,8,9,10

Select values from multiple rows from 1 table as 1 record

I'm struggling to even explain what I need to do so please be patient with me.
I have the following table and rows in it:
TBNAME: Distances
Track, Person, Date, Distance
TRACK1, P1, 1/1/2014, 15
TRACK2, P1, 13/1/2014, 12
TRACK1, P1, 20/2/2014, 10
TRACK2, P1, 15/1/2014, 9
TRACK1, P2, 2/1/2014, 11
TRACK2, P2, 14/1/2014, 13
TRACK1, P2, 21/2/2014, 8
TRACK2, P2, 16/1/2014, 6
What I would, ideally, like to see as a result is something like this:
P1, TRACK1, 20/2/2014, 10, TRACK2, 15/1/2014, 9
P2, TRACK1, 21/2/2014, 8, TRACK2, 16/1/2014, 6
Or, in other words, for each person, the most recent date and distance for that date for each track in one row.
Hope someone can understand this and offer a solution too :)
Cheers,
Pierre
Try this:
SELECT T1.Person, T1.Track, MAX(T1.Date), MIN(T1.Distance),
T2.Track, MAX(T2.Date), MIN(T2.Distance)
FROM Distances AS T1 INNER JOIN
Distances AS T2 ON T1.Person = T2.Person
WHERE T1.Track <> T2.Track AND T1.Track = 'Track1'
GROUP BY T1.Track, T1.Person, T2.Track
The output result of the query is showing exactly the same of your expected result.
Try combining the table by itself and connecting them with the common column.
In your case you want Person.
Select t1.Person,
t1.Tract,
t1.Date,
t1.Distance,
t2.Tract,
t2.date,
t2.Distance
From table_name t1, table_name t2
WHERE t1.Person = t2.Person;
Try this:
SELECT DISTINCT ON ("Person", "Track") *
FROM "Table"
ORDER BY "Person", "Date" DESC NULLS LAST;
Here is a query to get the records needed. First get the maximum date per track and person. Then join with the table to get the complete record.
If you know beforehand which tracks you will get, you can use a pivot query for this. As I've never done this, I ask you to look this up yourself. However, as mentioned in my comment to your request, I would use a programming language (C#, Java, PHP or whatever) to care about that.
select d.track, d.person, d.date, d.distance
from
(
select track, person, max(distances.date) as `date`
from distances
group by track, person
) lastd
inner join distances d on d.track = lastd.track and d.person = lastd.person and d.date = lastd.date
order by d.track, d.person;
BTW: date is a reserved keyword. I would not recommend to use it for a column name. Whenever you use it without a qualifier you will have to use those strange quotes.
Look for ROW_NUMBER() and OVER PARITION BY.
Idea is something like (I did not try to run this query):
;WITH
data AS
(
SELECT
*,
-- returns number for each pair of person and track starting from most recent date
--Code enhanced at here
row_number() over (partition BY person, track order by dte DESC) nr
FROM distances
)
SELECT
*
FROM
data
WHERE
nr = 1 -- we want just the most recent one
ORDER BY
person, -- "group by" person
track ;
It's still doesn't support showing one row for each person...
I don't think you can do it with SQL (because of unknown number of tracks).
There is PIVOT/UNPIVOT, but I don't think it fits here.
WITH CTE AS
(
Select P1.Track,P1.Person,ROW_NUMBER() OVER (Partition by Person,Track Order by Date
Desc) AS RN1
,Date,Distance
from Distances P1
)Select T.Person,T.Track1,T.T1Date
,T.T1Distance,T.Track2,T.T2Date,T.T2Distance
From (
Select C1.Person,C1.Track AS 'Track1',C1.Date AS 'T1Date',
C1.Distance 'T1Distance',
C2.Track AS 'Track2',C2.Date As 'T2Date',C2.Distance 'T2Distance',
ROW_NUMBER() OVER (Partition BY C1.Person Order by C1.Date Desc) RNX
from
CTE C1
JOIN
CTE C2 ON C1.RN1=1 AND C2.RN1=1
AND C1.Person=C2.Person
AND C1.Track<>C2.Track
)t Where t.RNX=1
you may also use dynamic query to achieve your expected result :)
DECLARE #nCount INT
DECLARE #nStart INT
DECLARE #Query NVARCHAR(MAX) =' '
DECLARE #sPerson NVARCHAR(MAX)
DECLARE #sTrack NVARCHAR(MAX)
SET #nCount = (SELECT COUNT(DISTINCT(person)) FROM DISTANCES)
SET #nStart = 1
WHILE #nStart <= #nCount
BEGIN
SET #sPerson = (SELECT PERSON FROM (
SELECT PERSON, ROW_NUMBER() OVER (ORDER BY PERSON) RN FROM (
SELECT DISTINCT(PERSON) FROM DISTANCES
) T1
) T2 WHERE RN = #nStart
)
SET #Query = #Query + '
SELECT '''+#sPerson+''' + '','' + STUFF( '','' +(
SELECT TRACK + '', '' + DATE + '', '' + DISTANCE FROM (
SELECT TRACK, DATE,DISTANCE,
ROW_NUMBER() OVER (PARTITION BY TRACK ORDER BY DATE DESC) RN FROM (
SELECT TRACK,date,DISTANCE FROM DISTANCES WHERE PERSON = '''+#sPerson+'''
) T1
) T2
WHERE RN = 1 FOR XML PATH('''')
),1,1,''''
)
'
IF(#nStart != #nCount)
SET #Query = #Query + ' UNION ALL '
SET #nStart = #nStart + 1
END
EXEC SP_EXECUTESQL #Query
To have a general query it need to be dynamic
DECLARE #query AS NVARCHAR(MAX)
DECLARE #pivotCols AS NVARCHAR(MAX)
DECLARE #cols AS NVARCHAR(MAX)
SELECT #pivotCols = STUFF((SELECT DISTINCT ',' + QUOTENAME([Track])
FROM Distances
FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)') , 1, 1, '')
;WITH T AS (
SELECT Track
, _ID = CAST(Row_Number() OVER (ORDER BY Track) AS VARCHAR)
FROM Distances
GROUP BY Track
)
SELECT #Cols = STUFF((
SELECT ', Track_' + _ID + ' = ''' + Track + ''''
+ ', LastRun_' + _ID + ' = ' + QUOTENAME([Track])
+ ', Distance_' + _ID + '
= SUM(CASE WHEN d.Date = ' + QUOTENAME([Track]) + '
AND d.Track = ''' + Track + '''
THEN d.Distance ELSE NULL END)'
FROM T FOR XML PATH(''), TYPE
).value('.', 'NVARCHAR(MAX)') , 1, 1, '')
SELECT #query = '
With LR AS (
SELECT Person, ' + #pivotCols + '
FROM (SELECT Track, Person, [Date] FROM Distances) d
PIVOT (MAX([Date]) FOR Track IN (' + #pivotCols + ')) pvt
)
SELECT d.Person, ' + #Cols + '
FROM Distances d
INNER JOIN LR ON d.Person = LR.Person AND d.Date IN (' + #pivotCols + ')
GROUP BY d.Person, ' + #pivotCols + ''
execute(#query);
SQLFiddle demo
The first query generate the list of field for the PIVOT.
The second one generate the fields for the compound query.
The PIVOT is used to get, for every person, the last run on every track, that is than joined back to the base data to get the distance

replace value in varchar(max) field with join

I have a table that contains text field with placeholders. Something like this:
Row Notes
1. This is some notes ##placeholder130## this ##myPlaceholder##, #oneMore#. End.
2. Second row...just a ##test#.
(This table contains about 1-5k rows on average. Average number of placeholders in one row is 5-15).
Now, I have a lookup table that looks like this:
Name Value
placeholder130 Dog
myPlaceholder Cat
oneMore Cow
test Horse
(Lookup table will contain anywhere from 10k to 100k records)
I need to find the fastest way to join those placeholders from strings to a lookup table and replace with value. So, my result should look like this (1st row):
This is some notes Dog this Cat, Cow. End.
What I came up with was to split each row into multiple for each placeholder and then join it to lookup table and then concat records back to original row with new values, but it takes around 10-30 seconds on average.
You could try to split the string using a numbers table and rebuild it with for xml path.
select (
select coalesce(L.Value, T.Value)
from Numbers as N
cross apply (select substring(Notes.notes, N.Number, charindex('##', Notes.notes + '##', N.Number) - N.Number)) as T(Value)
left outer join Lookup as L
on L.Name = T.Value
where N.Number <= len(notes) and
substring('##' + notes, Number, 2) = '##'
order by N.Number
for xml path(''), type
).value('text()[1]', 'varchar(max)')
from Notes
SQL Fiddle
I borrowed the string splitting from this blog post by Aaron Bertrand
SQL Server is not very fast with string manipulation, so this is probably best done client-side. Have the client load the entire lookup table, and replace the notes as they arrived.
Having said that, it can of course be done in SQL. Here's a solution with a recursive CTE. It performs one lookup per recursion step:
; with Repl as
(
select row_number() over (order by l.name) rn
, Name
, Value
from Lookup l
)
, Recurse as
(
select Notes
, 0 as rn
from Notes
union all
select replace(Notes, '##' + l.name + '##', l.value)
, r.rn + 1
from Recurse r
join Repl l
on l.rn = r.rn + 1
)
select *
from Recurse
where rn =
(
select count(*)
from Lookup
)
option (maxrecursion 0)
Example at SQL Fiddle.
Another option is a while loop to keep replacing lookups until no more are found:
declare #notes table (notes varchar(max))
insert #notes
select Notes
from Notes
while 1=1
begin
update n
set Notes = replace(n.Notes, '##' + l.name + '##', l.value)
from #notes n
outer apply
(
select top 1 Name
, Value
from Lookup l
where n.Notes like '%##' + l.name + '##%'
) l
where l.name is not null
if ##rowcount = 0
break
end
select *
from #notes
Example at SQL Fiddle.
I second the comment that tsql is just not suited for this operation, but if you must do it in the db here is an example using a function to manage the multiple replace statements.
Since you have a relatively small number of tokens in each note (5-15) and a very large number of tokens (10k-100k) my function first extracts tokens from the input as potential tokens and uses that set to join to your lookup (dbo.Token below). It was far too much work to look for an occurrence of any of your tokens in each note.
I did a bit of perf testing using 50k tokens and 5k notes and this function runs really well, completing in <2 seconds (on my laptop). Please report back how this strategy performs for you.
note: In your example data the token format was not consistent (##_#, ##_##, #_#), I am guessing this was simply a typo and assume all tokens take the form of ##TokenName##.
--setup
if object_id('dbo.[Lookup]') is not null
drop table dbo.[Lookup];
go
if object_id('dbo.fn_ReplaceLookups') is not null
drop function dbo.fn_ReplaceLookups;
go
create table dbo.[Lookup] (LookupName varchar(100) primary key, LookupValue varchar(100));
insert into dbo.[Lookup]
select '##placeholder130##','Dog' union all
select '##myPlaceholder##','Cat' union all
select '##oneMore##','Cow' union all
select '##test##','Horse';
go
create function [dbo].[fn_ReplaceLookups](#input varchar(max))
returns varchar(max)
as
begin
declare #xml xml;
select #xml = cast(('<r><i>'+replace(#input,'##' ,'</i><i>')+'</i></r>') as xml);
--extract the potential tokens
declare #LookupsInString table (LookupName varchar(100) primary key);
insert into #LookupsInString
select distinct '##'+v+'##'
from ( select [v] = r.n.value('(./text())[1]', 'varchar(100)'),
[r] = row_number() over (order by n)
from #xml.nodes('r/i') r(n)
)d(v,r)
where r%2=0;
--tokenize the input
select #input = replace(#input, l.LookupName, l.LookupValue)
from dbo.[Lookup] l
join #LookupsInString lis on
l.LookupName = lis.LookupName;
return #input;
end
go
return
--usage
declare #Notes table ([Id] int primary key, notes varchar(100));
insert into #Notes
select 1, 'This is some notes ##placeholder130## this ##myPlaceholder##, ##oneMore##. End.' union all
select 2, 'Second row...just a ##test##.';
select *,
dbo.fn_ReplaceLookups(notes)
from #Notes;
Returns:
Tokenized
--------------------------------------------------------
This is some notes Dog this Cat, Cow. End.
Second row...just a Horse.
Try this
;WITH CTE (org, calc, [Notes], [level]) AS
(
SELECT [Notes], [Notes], CONVERT(varchar(MAX),[Notes]), 0 FROM PlaceholderTable
UNION ALL
SELECT CTE.org, CTE.[Notes],
CONVERT(varchar(MAX), REPLACE(CTE.[Notes],'##' + T.[Name] + '##', T.[Value])), CTE.[level] + 1
FROM CTE
INNER JOIN LookupTable T ON CTE.[Notes] LIKE '%##' + T.[Name] + '##%'
)
SELECT DISTINCT org, [Notes], level FROM CTE
WHERE [level] = (SELECT MAX(level) FROM CTE c WHERE CTE.org = c.org)
SQL FIDDLE DEMO
Check the below devioblog post for reference
devioblog post
To get speed, you can preprocess the note templates into a more efficient form. This will be a sequence of fragments, with each ending in a substitution. The substitution might be NULL for the last fragment.
Notes
Id FragSeq Text SubsId
1 1 'This is some notes ' 1
1 2 ' this ' 2
1 3 ', ' 3
1 4 '. End.' null
2 1 'Second row...just a ' 4
2 2 '.' null
Subs
Id Name Value
1 'placeholder130' 'Dog'
2 'myPlaceholder' 'Cat'
3 'oneMore' 'Cow'
4 'test' 'Horse'
Now we can do the substitutions with a simple join.
SELECT Notes.Text + COALESCE(Subs.Value, '')
FROM Notes LEFT JOIN Subs
ON SubsId = Subs.Id WHERE Notes.Id = ?
ORDER BY FragSeq
This produces a list of fragments with substitutions complete. I am not an MSQL user, but in most dialects of SQL you can concatenate these fragments in a variable quite easily:
DECLARE #Note VARCHAR(8000)
SELECT #Note = COALESCE(#Note, '') + Notes.Text + COALSCE(Subs.Value, '')
FROM Notes LEFT JOIN Subs
ON SubsId = Subs.Id WHERE Notes.Id = ?
ORDER BY FragSeq
Pre-processing a note template into fragments will be straightforward using the string splitting techniques of other posts.
Unfortunately I'm not at a location where I can test this, but it ought to work fine.
I really don't know how it will perform with 10k+ of lookups.
how does the old dynamic SQL performs?
DECLARE #sqlCommand NVARCHAR(MAX)
SELECT #sqlCommand = N'PlaceholderTable.[Notes]'
SELECT #sqlCommand = 'REPLACE( ' + #sqlCommand +
', ''##' + LookupTable.[Name] + '##'', ''' +
LookupTable.[Value] + ''')'
FROM LookupTable
SELECT #sqlCommand = 'SELECT *, ' + #sqlCommand + ' FROM PlaceholderTable'
EXECUTE sp_executesql #sqlCommand
Fiddle demo
And now for some recursive CTE.
If your indexes are correctly set up, this one should be very fast or very slow. SQL Server always surprises me with performance extremes when it comes to the r-CTE...
;WITH T AS (
SELECT
Row,
StartIdx = 1, -- 1 as first starting index
EndIdx = CAST(patindex('%##%', Notes) as int), -- first ending index
Result = substring(Notes, 1, patindex('%##%', Notes) - 1)
-- (first) temp result bounded by indexes
FROM PlaceholderTable -- **this is your source table**
UNION ALL
SELECT
pt.Row,
StartIdx = newstartidx, -- starting index (calculated in calc1)
EndIdx = EndIdx + CAST(newendidx as int) + 1, -- ending index (calculated in calc4 + total offset)
Result = Result + CAST(ISNULL(newtokensub, newtoken) as nvarchar(max))
-- temp result taken from subquery or original
FROM
T
JOIN PlaceholderTable pt -- **this is your source table**
ON pt.Row = T.Row
CROSS APPLY(
SELECT newstartidx = EndIdx + 2 -- new starting index moved by 2 from last end ('##')
) calc1
CROSS APPLY(
SELECT newtxt = substring(pt.Notes, newstartidx, len(pt.Notes))
-- current piece of txt we work on
) calc2
CROSS APPLY(
SELECT patidx = patindex('%##%', newtxt) -- current index of '##'
) calc3
CROSS APPLY(
SELECT newendidx = CASE
WHEN patidx = 0 THEN len(newtxt) + 1
ELSE patidx END -- if last piece of txt, end with its length
) calc4
CROSS APPLY(
SELECT newtoken = substring(pt.Notes, newstartidx, newendidx - 1)
-- get the new token
) calc5
OUTER APPLY(
SELECT newtokensub = Value
FROM LookupTable
WHERE Name = newtoken -- substitute the token if you can find it in **your lookup table**
) calc6
WHERE newstartidx + len(newtxt) - 1 <= len(pt.Notes)
-- do this while {new starting index} + {length of txt we work on} exceeds total length
)
,lastProcessed AS (
SELECT
Row,
Result,
rn = row_number() over(partition by Row order by StartIdx desc)
FROM T
) -- enumerate all (including intermediate) results
SELECT *
FROM lastProcessed
WHERE rn = 1 -- filter out intermediate results (display only last ones)

How to generate an update query of a dynamic query (automatically)?

I'm storing some queries in a table column so I can execute them later passing some parameters.
But it has been really annoying to format the query into an Update sentence, because of the special characters.
For Example:
SELECT * FROM MOUNTAINS WHERE MON_NAME='PALMA' AND MON_DESC LIKE '%TRANVULCANIA%'
Then I need the string just for the udpate query:
UPDATE QUERIES
SET QUE_SEL='SELECT * FROM MOUNTAINS WHERE MON_NAME='''+'PALMA'+''' AND MON_DESC LIKE '''+'%TRANVULCANIA%'+''' '
WHERE QUE_ID=1
as you can see the first ' must be replaced for '''+' but the next door ' must be replaced by '+'''
This is the query I'm working on:
DECLARE #QUERY VARCHAR(MAX)
SELECT #QUERY='SELECT * FROM QUERIES WHERE QUE_NOMBRE='''+'PRUEBA 1'+''' '
SELECT
t.r.value('.', 'varchar(255)') AS token
, ROW_NUMBER() OVER (ORDER BY (SELECT 1)) AS id
FROM (
SELECT myxml = CAST('<t>' + REPLACE(#QUERY, '''', '</t><t>''</t><t>') + '</t>' AS XML)
) p
CROSS APPLY myxml.nodes('/t') t(r)
this is the result:
token id
-------------------------------------------------- --------------------
SELECT * FROM QUERIES WHERE QUE_NOMBRE= 1
' 2
PRUEBA 1 3
' 4
5
Now I want a column that tell me when to open and when to close and then I can set the final replace.
Adapting the solution given by #rivarolle
DECLARE #QUERY VARCHAR(MAX)
DECLARE #FORMATTED varchar(max)
SELECT #QUERY='SELECT * FROM QUERIES WHERE QUE_NOMBRE='''+'PRUEBA 1'+''''
;WITH TOKENS AS(
SELECT
t.r.value('.', 'varchar(MAX)') AS token
, ROW_NUMBER() OVER (ORDER BY (SELECT 1)) AS Id
FROM (
SELECT myxml = CAST('<t>' + REPLACE(#QUERY, '''', '</t><t>''</t><t>') + '</t>' AS XML)
) p
CROSS APPLY myxml.nodes('/t') t(r)
)
,
Tokens2 as (
SELECT
TOKENS.token as token
,quotes.row%2 as tipoapostrofe
from Tokens
left join (select row_number() over( order by Id asc) as row, a.* FROM (SELECT * from Tokens) a where Token = '''') quotes
on quotes.Id = Tokens.Id
)
SELECT #FORMATTED = STUFF((
SELECT ' ' + REPLACE(token,'''',CASE tipoapostrofe WHEN 1 THEN '''''''+''' WHEN 0 THEN '''+''''''' ELSE '' END) AS [text()]
FROM Tokens2
FOR XML PATH('')
), 1, 1, '')
print #FORMATTED
This Works, just need a function for cleaning XML special characters and another for putting back, and the Dynamic queries are printed ready for an update.
I think its not necessary to replace an apostrophe with '''+' to open and '+''' to close, I made some probes and you can exec a query that you replace opening and closing apostrophes with the same.. for example '''+' for open and '''+' for close.
So the query would be:
DECLARE #QUERY VARCHAR(MAX)
DECLARE #FORMATTED varchar(max)
SELECT #QUERY='SELECT * FROM QUERIES WHERE QUE_NOMBRE='''+'PRUEBA 1'+''''
SELECT #FORMATTED= STUFF((
SELECT ' ' +
(SELECT
CASE
WHEN t.r.value('.', 'varchar(250)')='''' THEN REPLACE(t.r.value('.', 'varchar(250)'), '''','''''''+''')
ELSE t.r.value('.', 'varchar(250)')
END
) AS [text()]
-- , ROW_NUMBER() OVER (ORDER BY (SELECT 1)) AS id
FROM (
SELECT myxml = CAST('<t>' + REPLACE(#QUERY, '''', '</t><t>''</t><t>') + '</t>' AS XML)
) p
CROSS APPLY myxml.nodes('/t') t(r)
FOR XML PATH('')
), 1, 1, '')
SET #FORMATTED=REPLACE(#FORMATTED,' ','')
PRINT #FORMATTED
then I get:
SELECT * FROM QUERIES WHERE QUE_NOMBRE= '''+' PRUEBA 1 '''+'
then I copy into a variable and execute
DECLARE #VAR VARCHAR(500)
SET #VAR='SELECT * FROM QUERIES WHERE QUE_NOMBRE='''+'PRUEBA 1'''+' '
EXEC(#VAR)
It Works for very simple queries, but with longer and complicated queries it doesn't works..
Assuming your token table is Tokens(Token, Id, Position):
update Tokens
set position = quotes.row%2
from Tokens
left join (select row_number() over( order by Id asc) as row, a.* FROM (SELECT * from Tokens) a where Token = '''') quotes
on quotes.Id = Tokens.Id
The position column will have a value of 1 for starting quote and 0 for closing quote. NULL for the rest.

How can I pivot these key+values rows into a table of complete entries?

Maybe I demand too much from SQL but I feel like this should be possible. I start with a list of key-value pairs, like this:
'0:First, 1:Second, 2:Third, 3:Fourth'
etc. I can split this up pretty easily with a two-step parse that gets me a table like:
EntryNumber PairNumber Item
0 0 0
1 0 First
2 1 1
3 1 Second
etc.
Now, in the simple case of splitting the pairs into a pair of columns, it's fairly easy. I'm interested in the more advanced case where I might have multiple values per entry, like:
'0:First:Fishing, 1:Second:Camping, 2:Third:Hiking'
and such.
In that generic case, I'd like to find a way to take my 3-column result table and somehow pivot it to have one row per entry and one column per value-part.
So I want to turn this:
EntryNumber PairNumber Item
0 0 0
1 0 First
2 0 Fishing
3 1 1
4 1 Second
5 1 Camping
Into this:
Entry [1] [2] [3]
0 0 First Fishing
1 1 Second Camping
Is that just too much for SQL to handle, or is there a way? Pivots (even tricky dynamic pivots) seem like an answer, but I can't figure how to get that to work.
No, in SQL you can't infer columns dynamically based on the data found during the same query.
Even using the PIVOT feature in Microsoft SQL Server, you must know the columns when you write the query, and you have to hard-code them.
You have to do a lot of work to avoid storing the data in a relational normal form.
Alright, I found a way to accomplish what I was after. Strap in, this is going to get bumpy.
So the basic problem is to take a string with two kinds of delimiters: entries and values. Each entry represents a set of values, and I wanted to turn the string into a table with one column for each value per entry. I tried to make this a UDF, but the necessity for a temporary table and dynamic SQL meant it had to be a stored procedure.
CREATE PROCEDURE [dbo].[ParseValueList]
(
#parseString varchar(8000),
#itemDelimiter CHAR(1),
#valueDelimiter CHAR(1)
)
AS
BEGIN
SET NOCOUNT ON;
IF object_id('tempdb..#ParsedValues') IS NOT NULL
BEGIN
DROP TABLE #ParsedValues
END
CREATE TABLE #ParsedValues
(
EntryID int,
[Rank] int,
Pair varchar(200)
)
So that's just basic set up, establishing the temp table to hold my intermediate results.
;WITH
E1(N) AS (SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1),--Brute forces 10 rows
E2(N) AS (SELECT 1 FROM E1 a, E1 b), --Uses a cross join to generate 100 rows (10 * 10)
E4(N) AS (SELECT 1 FROM E2 a, E2 b), --Uses a cross join to generate 10,000 rows (100 * 100)
cteTally(N) AS (SELECT ROW_NUMBER() OVER (ORDER BY N) FROM E4)
That beautiful piece of SQL comes from SQL Server Central's Forums and is credited to "a guru." It's a great little 10,000 line tally table perfect for string splitting.
INSERT INTO #ParsedValues
SELECT ItemNumber AS EntryID, ROW_NUMBER() OVER (PARTITION BY ItemNumber ORDER BY ItemNumber) AS [Rank],
SUBSTRING(Items.Item, T1.N, CHARINDEX(#valueDelimiter, Items.Item + #valueDelimiter, T1.N) - T1.N) AS [Value]
FROM(
SELECT ROW_NUMBER() OVER (ORDER BY T2.N) AS ItemNumber,
SUBSTRING(#parseString, T2.N, CHARINDEX(#itemDelimiter, #parseString + #itemDelimiter, T2.N) - T2.N) AS Item
FROM cteTally T2
WHERE T2.N < LEN(#parseString) + 2 --Ensures we cut out once the entire string is done
AND SUBSTRING(#itemDelimiter + #parseString, T2.N, 1) = #itemDelimiter
) AS Items, cteTally T1
WHERE T1.N < LEN(#parseString) + 2 --Ensures we cut out once the entire string is done
AND SUBSTRING(#valueDelimiter + Items.Item, T1.N, 1) = #valueDelimiter
Ok, this is the first really dense meaty part. The inner select is breaking up my string along the item delimiter (the comma), using the guru's string splitting method. Then that table is passed up to the outer select which does the same thing, but this time using the value delimiter (the colon) to each row. The inner RowNumber (EntryID) and the outer RowNumber over Partition (Rank) are key to the pivot. EntryID show which Item the values belong to, and Rank shows the ordinal of the values.
DECLARE #columns varchar(200)
DECLARE #columnNames varchar(2000)
DECLARE #query varchar(8000)
SELECT #columns = COALESCE(#columns + ',[' + CAST([Rank] AS varchar) + ']', '[' + CAST([Rank] AS varchar)+ ']'),
#columnNames = COALESCE(#columnNames + ',[' + CAST([Rank] AS varchar) + '] AS Value' + CAST([Rank] AS varchar)
, '[' + CAST([Rank] AS varchar)+ '] AS Value' + CAST([Rank] AS varchar))
FROM (SELECT DISTINCT [Rank] FROM #ParsedValues) AS Ranks
SET #query = '
SELECT '+ #columnNames +'
FROM #ParsedValues
PIVOT
(
MAX([Value]) FOR [Rank]
IN (' + #columns + ')
) AS pvt'
EXECUTE(#query)
DROP TABLE #ParsedValues
END
And at last, the dynamic sql that makes it possible. By getting a list of Distinct Ranks, we set up our column list. This is then written into the dynamic pivot which tilts the values over and slots each value into the proper column, each with a generic "Value#" heading.
Thus by calling EXEC ParseValueList with a properly formatted string of values, we can break it up into a table to feed into our purposes! It works (but is probably overkill) for simple key:value pairs, and scales up to a fair number of columns (About 50 at most, I think, but that'd be really silly.)
Anyway, hope that helps anyone having a similar issue.
(Yeah, it probably could have been done in something like SQLCLR as well, but I find a great joy in solving problems with pure SQL.)
Though probably not optimal, here's a more condensed solution.
DECLARE #DATA varchar(max);
SET #DATA = '0:First:Fishing, 1:Second:Camping, 2:Third:Hiking';
SELECT
DENSE_RANK() OVER (ORDER BY [Data].[row]) AS [Entry]
, [Data].[row].value('(./B/text())[1]', 'int') as "[1]"
, [Data].[row].value('(./B/text())[2]', 'varchar(64)') as "[2]"
, [Data].[row].value('(./B/text())[3]', 'varchar(64)') as "[3]"
FROM
(
SELECT
CONVERT(XML, '<A><B>' + REPLACE(REPLACE(#DATA , ',', '</B></A><A><B>'), ':', '</B><B>') + '</B></A>').query('.')
) AS [T]([c])
CROSS APPLY [T].[c].nodes('/A') AS [Data]([row]);
Hope is not too late.
You can use the function RANK to know the position of each Item per PairNumber. And then use Pivot
SELECT PairNumber, [1] ,[2] ,[3]
FROM
(
SELECT PairNumber, Item, RANK() OVER (PARTITION BY PairNumber order by EntryNumber) as RANKing
from tabla) T
PIVOT
(MAX(Item)
FOR RANKing in ([1],[2],[3])
)as PVT