SQL Recursive CTE Replacing records in each recursion - sql

Question

I have a table like this:
ItemID ItemFormula
100 'ID_3+ID_5'
110 'ID_2+ID_6'
120 'ID_100+ID_110'
130 'ID_120+ID_4'
This is the simplified version of a formula table with nearly 1000 records and up to 40 levels of reference (items used in other items). The task is breaking down the formulas to just one level reference where no other items are in one item. For example in the table above for id=130 I should have '((ID_3+ID_5)+(ID_2+ID_6))+ID_4'
EDIT: The operations do not limit to "+" and items have a character between them to be recognizable. For the sake of simplicity, I removed that character.
I can use recursive CTE for that. but my problem is that due to high levels of reference, my recursive select has lots of records joining so it takes a lot to complete.
My question is that: Can I keep the previous recursion only each time the recursion happens?
Here is my CTE Code
WITH Formula
AS (SELECT A.ItemID
,'ID_' + CONVERT(VARCHAR(20), A.ItemID) AS ItemText
,CONVERT(VARCHAR(MAX), A.ItemFormula) AS ItemFormula
FROM (VALUES (100,'ID_3+ID_5'),
(110,'ID_2+ID_6'),
(120,'ID_100+ID_110'),
(130,'ID_120+ID_4')
) A (ItemID,ItemFormula)
)
,REC
AS
(
SELECT A.ItemID
,A.ItemText
,A.ItemFormula
,1 AS LevelID
FROM Formula A
UNION ALL
SELECT A.ItemID
,A.ItemText
,' '
+ TRIM (REPLACE (REPLACE (A.ItemFormula, B.ItemText, ' ( ' + B.ItemFormula + ' ) '), ' ', ' '))
+ ' ' AS ItemFormula
,A.LevelID + 1 AS LevelID
FROM REC A
CROSS APPLY
(
SELECT *
FROM
(
SELECT *
,ROW_NUMBER () OVER (ORDER BY GETDATE ()) AS RowNum
FROM Formula B2
WHERE CHARINDEX (B2.ItemText, A.ItemFormula) > 0
) B3
WHERE B3.RowNum = 1
) B
)
,FinalQ
AS
(
SELECT A2.ItemID
,A2.ItemFormula
,A2.LevelID
FROM
(
SELECT A.ItemID
,REPLACE (TRIM (A.ItemFormula), ' ', '') AS ItemFormula
,A.LevelID
,ROW_NUMBER () OVER (PARTITION BY A.ItemID ORDER BY A.LevelID DESC) AS RowNum
FROM REC A
) A2
WHERE A2.RowNum = 1
)
SELECT * FROM FinalQ A2 ORDER BY A2.ItemID;
Thanks in advance.

Answer 1

My question is that: Can I keep the previous recursion only each time the recursion happens?
No. The recursive CTE will keep adding rows to the ones found in previous iterations. You don't have some kind of control that would allow you to remove rows of the recursive CTE during its iterations.
You can, however, filter them out after the recursive CTE is complete, maybe on a secondary CTE that takes into account only the last meaninful rows (by some kind of rule to be defined).
The only vaguely similar idea is found in PostgreSQL where you can use the UNION clause in addition to UNION ALL, to avoid producing more identical rows. But this is different to what you need, anyway.

Answer 2

This is an enormously complicated problem. Here are the ideas:
Find which items do not need any insertions. These are the ones that have no references to any others.
Build an ordering for item insertion. An insertion can go into an item, assuming that the item is already defined. A recursive CTE can be used for this.
Enumerate the insertions. Everything from (1) gets a "1". The rest are in order.
Process the insertions in the insertion order.
Here is my solution:
with ordering as (
select itemid, itemtext, itemformula, convert(varchar(max), null) as otheritemtext, 1 as lev
from formula f
where not exists (select 1
from formula f2 join
string_split(f.itemformula, '+') s
on f2.itemtext = s.value
where f2.itemid <> f.itemid
)
union all
select f.itemid, f.itemtext, f.itemformula, convert(varchar(max), s.value), lev + 1
from formula f cross apply
string_split(f.itemformula, '+') s join
ordering o
on o.itemtext = s.value
-- where lev <= 2
),
ordered as (
select distinct o.*,
dense_rank() over (order by (case when lev = 1 then -1 else lev end), (case when lev = 1 then '' else otheritemtext end)) as seqnum
from ordering o
),
cte as (
select o.itemid, o.itemtext, o.itemformula, convert(varchar(max), o.otheritemtext) as otheritemtext,
o.itemformula as newformula, o.seqnum, 1 as lev
from ordered o
where seqnum = 1
union all
select cte.itemid, o.itemtext, o.itemformula, convert(varchar(max), cte.itemtext),
replace(o.itemformula, o.otheritemtext, concat('(', cte.newformula, ')')), o.seqnum, cte.lev + 1
from cte join
ordered o
on cte.itemtext = o.otheritemtext and cte.seqnum < o.seqnum
)
select *
from cte;
And the db<>fiddle.

Answer 3

You could take advantage of the logical order of the formulas if any (Item_100 can not reference Item_150) and process items in a descending order.
The following uses LIKE and it will not work for formulas which have overlapping patterns (eg ID_10 & ID_100) you could fix that by some string manipulation or by keeping ItemIDs of fixed length (eg. ID_10010 & ID_10100: start numbering of items from a high number like 10000)
declare #f table
(
ItemId int,
ItemFormula varchar(1000)
);
insert into #f(ItemId, ItemFormula)
values
(100, 'ID_3+ID_5'),
(110, 'ID_2+ID_6'),
(120, 'ID_100+ID_110'),
(130, 'ID_120+ID_4'),
(140, '(ID_130+ID_110)/ID_100'),
(150, 'sqrt(ID_140, ID_130)'),
(160, 'ID_150-ID_120+ID_140');
;with cte
as
(
select f.ItemId, replace(cast(f.ItemFormula as varchar(max)), isnull('ID_' + cast(r.ItemId as varchar(max)), ''), isnull('(' + r.ItemFormula+ ')', '')) as therepl, 1 as lvl
from #f as f
outer apply (
select *
from
(
select rr.*, row_number() over(order by rr.ItemId desc) as rownum
from #f as rr
where f.ItemFormula like '%ID_' + cast(rr.ItemId as varchar(1000)) + '%'
) as src
where rownum = 1
) as r
union all
select c.ItemId, replace(c.therepl, 'ID_' + cast(r.ItemId as varchar(max)), '(' + r.ItemFormula+ ')'), c.lvl+1
from cte as c
cross apply (
select *
from
(
select rr.*, row_number() over(order by rr.ItemId desc) as rownum
from #f as rr
where c.therepl like '%ID_' + cast(rr.ItemId as varchar(1000)) + '%'
) as src
where rownum = 1
) as r
),
rown
as
(
select *, row_number() over (partition by itemid order by lvl desc) as rownum
from cte
)
select *
from rown
where rownum = 1;