Sql Recursion via CTE over LINQ SelectMany - sql

Question

Firstly I'm fairly well versed in LINQ queries, but a complete novice at writing direct SQL queries.
I want to be able to do the following:
For any given ItemId, loop through it's child Items until the very base child items have been selected.
Each item belongs to a container, which has a base (or parent) container Id specified (or NULL if it is the base container). A container can only have one parent container, but it can have multiple child containers.
Currently I've been doing something like the following:
using (MyEntities db = new MyEntities())
{
var theItem = db.Find(itemId);
var theContainer = theItem.Container.BaseContainer;
var theBaseItems = theItem.BaseItems.Where(bi => bi.ContainerId == theContainer.ContainerId).ToList();
while (theContainer.BaseContainerId != null)
{
theContainer = theContainer.BaseContainer;
theBaseItems = theBaseItems.SelectMany(bi => bi.BaseItems.Where(i => i.ContainerId == theContainer.ContainerId)).ToList();
}
}
This runs fine and fairly speedy, however when the ContainerId is quite high up the chain, I've noticed the ridiculous number of queries to the database caused by the SelectMany. For example, if 1000 Items belonged across 100 Items in a parent container, and those 100 items belonged across 10 Items in that containers parent container and finally those 10 belong to 1 item at the top of the chain, the Select Many will run 10 + 100 queries, flattening the results each time to retrieve the base 1000 items - AFAIK to be expected.
I have therefore suspected (after much research) that a Sql CTE may be a better option, not only hitting the database a little more gently, but possibly faster too - is this a bad assumption?
I am however struggling to get to grips with the CTE syntax and am hoping someone out there can shed their wisdom on my problem and help me out.
Re-creating the scenario
USE [TestDatabase]
GO
/****** Object: Table [dbo].[Containers] Script Date: 20/05/2013 14:17:20 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[Containers](
[ContainerId] [int] IDENTITY(1,1) NOT NULL,
[BaseContainerId] [int] NULL,
[Name] [nvarchar](50) NOT NULL,
CONSTRAINT [PK_Containers] PRIMARY KEY CLUSTERED
(
[ContainerId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
/****** Object: Table [dbo].[ItemRelationships] Script Date: 20/05/2013 14:17:20 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[ItemRelationships](
[ChildItemId] [int] NOT NULL,
[ParentItemId] [int] NOT NULL,
CONSTRAINT [PK_ItemRelationships] PRIMARY KEY CLUSTERED
(
[ChildItemId] ASC,
[ParentItemId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
/****** Object: Table [dbo].[Items] Script Date: 20/05/2013 14:17:20 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[Items](
[ItemId] [int] IDENTITY(1,1) NOT NULL,
[ContainerId] [int] NOT NULL,
[Name] [nvarchar](50) NOT NULL,
CONSTRAINT [PK_Items] PRIMARY KEY CLUSTERED
(
[ItemId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET IDENTITY_INSERT [dbo].[Containers] ON
INSERT [dbo].[Containers] ([ContainerId], [BaseContainerId], [Name]) VALUES (1, NULL, N'Level 1')
INSERT [dbo].[Containers] ([ContainerId], [BaseContainerId], [Name]) VALUES (2, 1, N'Level 2')
INSERT [dbo].[Containers] ([ContainerId], [BaseContainerId], [Name]) VALUES (3, 1, N'Level 2b')
INSERT [dbo].[Containers] ([ContainerId], [BaseContainerId], [Name]) VALUES (4, 2, N'Level 3')
INSERT [dbo].[Containers] ([ContainerId], [BaseContainerId], [Name]) VALUES (5, NULL, N'TypeB')
SET IDENTITY_INSERT [dbo].[Containers] OFF
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (1, 13)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (2, 13)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (3, 13)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (4, 14)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (5, 14)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (6, 14)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (7, 15)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (8, 15)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (9, 15)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (10, 16)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (11, 16)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (12, 16)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (13, 17)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (14, 17)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (15, 17)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (1007, 13)
INSERT [dbo].[ItemRelationships] ([ChildItemId], [ParentItemId]) VALUES (1008, 17)
SET IDENTITY_INSERT [dbo].[Items] ON
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (1, 1, N'A')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (2, 1, N'B')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (3, 1, N'C')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (4, 1, N'D')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (5, 1, N'E')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (6, 1, N'F')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (7, 1, N'G')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (8, 1, N'H')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (9, 1, N'I')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (10, 1, N'J')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (11, 1, N'K')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (12, 1, N'L')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (13, 2, N'A2')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (14, 2, N'A2')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (15, 2, N'C2')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (16, 3, N'D2B')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (17, 4, N'A3')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (1007, 5, N'TypeB1')
INSERT [dbo].[Items] ([ItemId], [ContainerId], [Name]) VALUES (1008, 5, N'TypeB2')
SET IDENTITY_INSERT [dbo].[Items] OFF
ALTER TABLE [dbo].[Containers] WITH CHECK ADD CONSTRAINT [FK_Containers_Containers] FOREIGN KEY([BaseContainerId])
REFERENCES [dbo].[Containers] ([ContainerId])
GO
ALTER TABLE [dbo].[Containers] CHECK CONSTRAINT [FK_Containers_Containers]
GO
ALTER TABLE [dbo].[ItemRelationships] WITH CHECK ADD CONSTRAINT [FK_ItemRelationships_ChildItems] FOREIGN KEY([ParentItemId])
REFERENCES [dbo].[Items] ([ItemId])
GO
ALTER TABLE [dbo].[ItemRelationships] CHECK CONSTRAINT [FK_ItemRelationships_ChildItems]
GO
ALTER TABLE [dbo].[ItemRelationships] WITH CHECK ADD CONSTRAINT [FK_ItemRelationships_ParentItems] FOREIGN KEY([ChildItemId])
REFERENCES [dbo].[Items] ([ItemId])
GO
ALTER TABLE [dbo].[ItemRelationships] CHECK CONSTRAINT [FK_ItemRelationships_ParentItems]
GO
ALTER TABLE [dbo].[Items] WITH CHECK ADD CONSTRAINT [FK_Items_Containers] FOREIGN KEY([ContainerId])
REFERENCES [dbo].[Containers] ([ContainerId])
ON UPDATE CASCADE
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Items] CHECK CONSTRAINT [FK_Items_Containers]
GO
USE [master]
GO
ALTER DATABASE [TestDatabase] SET READ_WRITE
GO
The following SQL Query correctly returns the immediate child items:
DECLARE #itemId BIGINT = 17;
SELECT
Items.ItemId
FROM
[TestDatabase].[dbo].[ItemRelationships]
INNER JOIN
[TestDatabase].[dbo].Items
ON
ItemRelationships.ChildItemId = Items.ItemId
INNER JOIN
[TestDatabase].[dbo].[Containers]
ON
Items.ContainerId = Containers.ContainerId
WHERE
ItemRelationships.ParentItemId = #itemId
AND
Items.ContainerId =
(
SELECT
BaseContainerId
FROM
[TestDatabase].[dbo].[Items]
INNER JOIN
[TestDatabase].[dbo].[Containers]
ON
Items.ContainerId = Containers.ContainerId
WHERE
Items.ItemId = #itemId
)
Results
Item Id 17 belongs to container 4. Container 4's base Container is container 2, Container 2's base container is container 1, and container 1's base container is NULL.
The above query returns ItemIds 13, 14 and 15 (which is correct) within container 2. However I need this query to automatically then look for a base container for container 2 and get all the ItemId's for the base items of Items 13, 14 and 15 (which should yield Item Ids 1 to 9 in this scenario).
Notes
As an item can be attached (through itemrelationships) to items from unrelated containers, the check for the current item(s) container's base container MUST be present.
If the ItemId passed is within a base container then the query should simply return the ItemId passed.
A CTE is preferred as the results will actually be used as part of another query against another table (but that is outside of the scope of this question).
I hope someone can help and thank you in advance for your efforts.

Answer 1

I am not sure I understand completely your model and the place of ItemRelationships table, so the query might require some tweaking - however it should give you an idea how to use recursive CTE.
DECLARE #itemID INT
Set #itemID = 17
;WITH CTE_Containers AS
(
SELECT c.ContainerId, c.BaseContainerId, i.ItemID AS ChildItemId, NULL AS ParentItemID, i.Name
FROM Items i
INNER JOIN Containers c ON i.ContainerId = c.ContainerId
WHERE i.ItemId = #itemID
UNION ALL
SELECT c.ContainerId, c.BaseContainerId, ir.ChildItemId, ir.ParentItemId, i.Name
FROM CTE_Containers cte
INNER JOIN dbo.Containers c ON cte.BaseContainerId = c.ContainerId
INNER JOIN dbo.ItemRelationships ir ON ir.ParentItemId = cte.ChildItemId
INNER JOIN dbo.Items i ON ir.ChildItemId = i.ItemID
)
SELECT * FROM CTE_Containers
As you may see - recursive CTEs consists of two parts. First (base) part - you select your row for given #itemID and in second (recursive) part you join your base part to tables to get child item.
This will run until there is nothing selected in recursive part - or some other condition you may impose is met.