Make a copy of parent-child structure in SQL - sql

I have a table MODELS to which several ITEMS can belong. The ITEMS table is a hierarchical table with a self join on the PARENT column. Root level items will have Null in PARENT. Items can go to any level deep.
create table MODELS (
MODELID int identity,
MODELNAME nvarchar(200) not null,
constraint PK_MODELS primary key (MODELID)
)
go
create table ITEMS (
ITEMID int identity,
MODELID int not null,
PARENT int null,
ITEMNUM nvarchar(20) not null,
constraint PK_ITEMS primary key (ITEMID)
)
go
alter table ITEMS
add constraint FK_ITEMS_MODEL foreign key (MODELID)
references MODELS (MODELID)
go
alter table ITEMS
add constraint FK_ITEMS_ITEMS foreign key (PARENT)
references ITEMS (ITEMID)
go
I wish to create stored procedure to copy a row in the MODELS table into a new row and also copy the entire structure in ITEMS as well.
For example, if I have the following in ITEMS:
ITEMID MODELID PARENT ITEMNUM
1 1 Null A
2 1 Null B
3 1 Null C
4 1 1 A.A
5 1 2 B.B
6 1 4 A.A.A
7 1 4 A.A.B
8 1 3 C.A
9 1 3 C.B
10 1 9 C.B.A
I'd like to create new Model row and copies of the 10 Items that should be as follows:
ITEMID MODELID PARENT ITEMNUM
11 2 Null A
12 2 Null B
13 2 Null C
14 2 11 A.A
15 2 12 B.B
16 2 14 A.A.A
17 2 14 A.A.B
18 2 13 C.A
19 2 13 C.B
20 2 19 C.B.A
I will pass the MODELID to be copied as a parameter to the Stored Procedure. The tricky part is setting the PARENT column correctly. I think this will need to be done recursively.
Any suggestions?

The solution described here will work correctly in multi-user environment. You don't need to lock the whole table. You don't need to disable self-referencing foreign key. You don't need recursion.
(ab)use MERGE with OUTPUT clause.
MERGE can INSERT, UPDATE and DELETE rows. In our case we need only to INSERT. 1=0 is always false, so the NOT MATCHED BY TARGET part is always executed. In general, there could be other branches, see docs. WHEN MATCHED is usually used to UPDATE; WHEN NOT MATCHED BY SOURCE is usually used to DELETE, but we don't need them here.
This convoluted form of MERGE is equivalent to simple INSERT, but unlike simple INSERT its OUTPUT clause allows to refer to the columns that we need. It allows to retrieve columns from both source and destination tables thus saving a mapping between old and new IDs.
sample data
DECLARE #Items TABLE (
ITEMID int identity,
MODELID int not null,
PARENT int null,
ITEMNUM nvarchar(20) not null
)
INSERT INTO #Items (MODELID, PARENT, ITEMNUM) VALUES
(1, Null, 'A'),
(1, Null, 'B'),
(1, Null, 'C'),
(1, 1 , 'A.A'),
(1, 2 , 'B.B'),
(1, 4 , 'A.A.A'),
(1, 4 , 'A.A.B'),
(1, 3 , 'C.A'),
(1, 3 , 'C.B'),
(1, 9 , 'C.B.A');
I omit the code that duplicates the Model row. Eventually you'll have ID of original Model and new Model.
DECLARE #SrcModelID int = 1;
DECLARE #DstModelID int = 2;
Declare a table variable (or temp table) to hold the mapping between old and new item IDs.
DECLARE #T TABLE(OldItemID int, NewItemID int);
Make a copy of Items remembering the mapping of IDs in the table variable and keeping old PARENT values.
MERGE INTO #Items
USING
(
SELECT ITEMID, PARENT, ITEMNUM
FROM #Items AS I
WHERE MODELID = #SrcModelID
) AS Src
ON 1 = 0
WHEN NOT MATCHED BY TARGET THEN
INSERT (MODELID, PARENT, ITEMNUM)
VALUES
(#DstModelID
,Src.PARENT
,Src.ITEMNUM)
OUTPUT Src.ITEMID AS OldItemID, inserted.ITEMID AS NewItemID
INTO #T(OldItemID, NewItemID)
;
Update old PARENT values with new IDs
WITH
CTE
AS
(
SELECT I.ITEMID, I.PARENT, T.NewItemID
FROM
#Items AS I
INNER JOIN #T AS T ON T.OldItemID = I.PARENT
WHERE I.MODELID = #DstModelID
)
UPDATE CTE
SET PARENT = NewItemID
;
Check the results
SELECT * FROM #Items;

You can do it without recursion. But you need to lock the table first maybe to be sure that it works fine.
insert into items (Modelid, Parent, ITEMNUM)
select 2 as modelId,
MAP.currId as Parent,
MO.ITEMNUM
from (
( select * from items where MODELID = 1) MO
left join
( select IDENT_CURRENT('ITEMS') + ROW_NUMBER() OVER(ORDER BY itemid ) currID ,
i.ItemID
from ITEMS i
where modelid = 1 ) MAP
on MO.Parent= MAP.ItemID
) ORDER BY MO.ItemID
The idea behind it is that we select all rows from original model in ITEM table and we generate fake ID for them.
The fake ID is :
Row 1 = current identity + 1,
Row 2 = current identity + 2,
etc.
After that we have mapping : oldid -> newid
Then we insert original model to ITEM table like it is but we replace Parent by record from our mapping.
The issue that I can see is that some ItemID may still not exist for Parent when we insert rows (ie. we insert row that will have ItemID 20 but its Parent is 21). For that we may need to disable constraint on Parent for the time of execution of this insert. After that we supposed to enable it again. Data will be correct of course.

Related

How to I get distinct combinations of one XRef column related to any value in the other XRef column

I need to select the count of unique value combinations of column B in an XRef table which is grouped by column A.
Consider the following schema and data, which represents a simple family structure. Each child has a father and mother:
TABLE Father
FatherID
Name
1
Alex
2
Bob
TABLE Mother
MotherID
Name
1
Alice
2
Barbara
TABLE Child
ChildID
FatherID
MotherID
Name
1
1 (Alex)
1 (Alice)
Adam
2
1 (Alex)
1 (Alice)
Billy
3
1 (Alex)
2 (Barbara)
Celine
4
2 (Bob)
2 (Barbara)
Derek
The distinct combinations of mothers for each father are:
Alex (Alice, Barbara)
Bob (Barbara)
In all there are two distinct combinations of mothers:
Alice, Barbara
Barbara
The query I want to write would return the count of those distinct combinations of mother, regardless of which father they are associated with:
UniqueMotherGroups
2
I was able to do this successfully using the STRING_AGG function, but it feels clunky. It also needs to operate over millions of rows and is quite slow at the moment. Is there a more idiomatic way to do this with set operations instead?
Here is my working example:
-- Drop pre-existing tables
DROP TABLE IF EXISTS dbo.Child;
DROP TABLE IF EXISTS dbo.Father;
DROP TABLE IF EXISTS dbo.Mother;
-- Create family tables.
CREATE TABLE dbo.Father
(
FatherID INT NOT NULL
, Name VARCHAR(50) NOT NULL
);
ALTER TABLE dbo.Father
ADD CONSTRAINT PK_Father
PRIMARY KEY CLUSTERED (FatherID);
ALTER TABLE dbo.Father SET (LOCK_ESCALATION = TABLE);
CREATE TABLE dbo.Mother
(
MotherID INT NOT NULL
, Name VARCHAR(50) NOT NULL
);
ALTER TABLE dbo.Mother
ADD CONSTRAINT PK_Mother
PRIMARY KEY CLUSTERED (MotherID);
ALTER TABLE dbo.Mother SET (LOCK_ESCALATION = TABLE);
CREATE TABLE dbo.Child
(
ChildID INT NOT NULL
, FatherID INT NOT NULL
, MotherID INT NOT NULL
, Name VARCHAR(50) NOT NULL
);
ALTER TABLE dbo.Child
ADD CONSTRAINT PK_Child
PRIMARY KEY CLUSTERED (ChildID);
CREATE NONCLUSTERED INDEX IX_Parents ON dbo.Child (FatherID, MotherID);
ALTER TABLE dbo.Child
ADD CONSTRAINT FK_Child_Father
FOREIGN KEY (FatherID)
REFERENCES dbo.Father (FatherID);
ALTER TABLE dbo.Child
ADD CONSTRAINT FK_Child_Mother
FOREIGN KEY (MotherID)
REFERENCES dbo.Mother (MotherID);
-- Insert two children with the same parents
INSERT INTO dbo.Father
(
FatherID
, Name
)
VALUES
(1, 'Alex')
, (2, 'Bob')
, (3, 'Charlie')
INSERT INTO dbo.Mother
(
MotherID
, Name
)
VALUES
(1, 'Alice')
, (2, 'Barbara');
INSERT INTO dbo.Child
(
ChildID
, FatherID
, MotherID
, Name
)
VALUES
(1, 1, 1, 'Adam')
, (2, 1, 1, 'Billy')
, (3, 1, 2, 'Celine')
, (4, 2, 2, 'Derek')
, (5, 3, 1, 'Eric');
-- CTE Gets distinct combinations of parents
WITH distinctParentCombinations (FatherID, MotherID)
AS (SELECT children.FatherID
, children.MotherID
FROM dbo.Child as children
GROUP BY children.FatherID
, children.MotherID
)
-- CTE Gets uses STRING_AGG to get unique combinations of mothers.
, motherGroups (Mothers)
AS (SELECT STRING_AGG(CONVERT(VARCHAR(MAX), distinctParentCombinations.MotherID), '-') WITHIN GROUP (ORDER BY distinctParentCombinations.MotherID) AS Mothers
FROM distinctParentCombinations
GROUP BY distinctParentCombinations.FatherID
)
-- Remove the COUNT function to see the actual combinations
SELECT COUNT(motherGroups.Mothers) AS UniqueMotherGroups
FROM motherGroups
-- Clean up the example
DROP TABLE IF EXISTS dbo.Child;
DROP TABLE IF EXISTS dbo.Father;
DROP TABLE IF EXISTS dbo.Mother;
You have a great explanation and setup of your "problem case".
Your setup runs great in (for example) tempdb.
You have solved the problem in a nice way, and I don't think you can optimize it much further if you are going to calculate the mother groups every time you run the query.
There is one small mistake though; You must do a COUNT(DISTINCT motherGroups.Mothers) in your final count.
Since you mention milions of rows, I would suggest a slightly different approach.
If you aggregate the mother groups as soon as there is a change in the Child table, your query can run fast every time - even with millions of rows.
The kind of queries you want to run is seldom run only once, so it would be nice if the heavy work is already done.
Usually I prefer not to use triggers, because you get extra logic in a place where it could be hard to find and debug.
But sometimes triggers are nice to have, especially when you are not able to change the source code running on the clients.
So, my solution is to add a new column to the Father table and to create a trigger which (re)generates the mother group each time there is a change in the Child table.
This way, the hard aggregation work for each father is done as soon there is a change, and you don't have to aggregate when you run your query.
Since you already have millions of rows, we also have to update these existing rows.
I have used SQL Server 2019 for this solution.
*** The solution ***
Add 1 or 2 new columns to the Father table.
If you should add 1 or 2, it depends on what your preferences are:
"Do I want to see the aggregated mother groups for debugging purpose, or do I just trust the hashed values?"
Column 1: Hashed value of the aggregated mother group for each Father row.
The hashed value is VARBINARY and is at least 32 bytes, but we will use VARBINARY(1600):
1600 is less than 1700 which is the max nonclustered index size, so we will not have any problems indexing the column.
Since the hash value is in blocks of 32 bytes, a value of 1600 will cover a really, really, really long aggreated mother group.
-- Column 1: Hashed value of the aggregated mother group for each Father row.
alter table Father add MotherHash varbinary(1600)
create index IX_MotherHash on Father(MotherHash)
Column 2: This column is more optional, and depends on your preferences.
The column could be nice to have for debugging purpose if any questions are made about the result.
Which VARCHAR-length you should use depends on your real data.
MAX? Then you have no problems storing the mother groups, but you might have problems indexing it, since 1700 is the max for an unclustered index. But maybe you don't need to index it?
1700? Then you are able to index the column, but depending on your real data, will this cover the biggest mother group?
Why indexing? If you want to list the aggregated mother groups, it could be faster to read the index than the whole table.
As said; this depends on you (and your data). If we have no need to see the aggregated mother groups, then we don't need this column at all.
For this demo/solution we will add the column for debugging purpose, without any indexing.
-- Column 2: This column is more optional, and depends on your preferences.
alter table Father add MotherGroup varchar(MAX)
go
Create a trigger on the Child table.
It will handle all inserts, updates and deletes in the Child table.
create or alter trigger trIUD_Child on Child
after insert, update, delete
as
begin
set nocount on
-- Get all FatherIDs from the Inserted and Deleted table.
-- An ordinary Temp table is created with a clustered index to get SEEK performance later.
-- The table might also have more than 100 rows, where table variables are not recommended.
declare #numRowsInInsertedDeleted int
create table #rowsInInsertedDeleted(rowId int identity(1, 1), FatherID int)
create unique clustered index ix on #rowsInInsertedDeleted(rowId)
insert #rowsInInsertedDeleted(FatherID)
select distinct f.FatherID
from
(
select i.FatherID from inserted i
union all
select i.FatherID from deleted i
) f
select #numRowsInInsertedDeleted = max(rowId) from #rowsInInsertedDeleted
-- We have to loop each of the FatherIDs, since we might have several rows in the Inserted and Deleted tables.
declare #rowId int = 0
while (#rowId < #numRowsInInsertedDeleted)
begin
-- Get the father for the next row.
select #rowId += 1
declare #fatherId int
select #fatherId = r.FatherID
from #rowsInInsertedDeleted r
where r.rowId = #rowId
-- Aggregate the mothers for this father.
declare #motherGroup varchar(max) = ''
select #motherGroup += ',' + cast(c.MotherID as varchar)
from Child c
where c.FatherID = #fatherId
group by c.MotherID
order by c.MotherID
-- Update the father record.
-- Any empty strings are handled automatically, skip the leading ','.
update Father
set MotherGroup = substring(#motherGroup, 2, 2147483647),
MotherHash = HASHBYTES('SHA2_256', #motherGroup)
where FatherID = #fatherId
end
end
go
Updating existing rows
Since you already have millions of rows, we must aggregate the mother groups for these existing rows.
If you don't have the disk space for logging the update of the whole table, maybe you should take your database out of AG and switch to Simple recovery model for this task?
In that case you should also modify the update with a WHERE clause to update only parts of the table, and run the update for each part until the whole table is updated.
Example: update Child set FatherID = FatherID where FatherID between 1 and 1000000
Note: This update statement could block access to the Child table for other users.
-- Aggregate the mother groups for the existing rows.
-- This could takes minutes to complete, depending on the number of rows.
-- NOTE: This update statement could block access to the Child table for other users.
update Child set FatherID = FatherID
That's it!
You should now be able to quickly get the mother groups on existing rows, and also after future changes in the Child table.
-- Voila - now you can get the unique mother groups any time at a fast speed.
select count(distinct MotherHash) from Father
Thank you for posting such a comprehensive setup for the test data. However, I'm not running any CREATE/DROP statements against my DB so I converted those tables into table variables. Using your data, I came up with the following query. Just change the table names back to your dbo. names and you should be able to test in your environment. I basically concatenate every father/mother combo into a text string using FOR XML PATH. Then I count up all the distinct combos. If you find error in my logic, let me know. I'm just tossing this in the ring of possible solutions.
WITH distinctCombos AS (
SELECT DISTINCT
c.FatherID, c.MotherID
FROM #Child as c
) , motherComboCount AS (
SELECT
f.FatherID
, f.[Name]
, STUFF((
SELECT
',' + CAST(dc.MotherID as nvarchar)
FROM distinctCombos as dc
WHERE dc.FatherID = f.FatherID
ORDER BY dc.MotherID ASC
FOR XML PATH('')
),1,1,'') as motherList
FROM #Father as f
)
SELECT
COUNT(DISTINCT motherList) as UniqueMotherGroups
FROM motherComboCount as mcc
To save a bit of compute power, remove the STUFF function as it's not necessary for the comparison... it just makes the list nicer to look at if displaying... and I'm in the habit of using it.
It looks like the main differences between our methods is the use of FOR XML PATH vs STRING_AGG (I'm still on older SQL.) And I use DISTINCT twice instead of GROUP BY. If you have a larger dataset to test against, let me know how the 2 methods compare. I'm trying to think of a completely set-based method but I can't see it at the moment.
Update: Method 2.
Here's an idea I had using recursive CTEs to build the distinct mother combinations. In your example data, there are only 2 mothers per father. So there would be a total of 4 set-based queries performed (first CTE, 2 queries in the recursive CTE and the final SELECT).
WITH uniqueCombo as (
SELECT DISTINCT
c.FatherID
, c.MotherID
, ROW_NUMBER() OVER(PARTITION BY c.FatherID ORDER BY c.MotherID) as row_num
FROM #Child as c
), combos as (
SELECT
uc.FatherID
, uc.MotherID
, CAST(uc.MotherID as nvarchar(max)) as [path]
, row_num
, 0 as hierarchy_num
FROM uniqueCombo as uc
WHERE uc.row_num = 1
UNION ALL
SELECT
uc.FatherID
, uc.MotherID
, co.[path] + ',' + CAST(uc.MotherID as nvarchar(max))
, uc.row_num
, co.hierarchy_num + 1 as heirarchy_num
FROM uniqueCombo as uc
INNER JOIN combos as co
ON co.FatherID = uc.FatherID
--AND co.MotherID <> uc.MotherID
AND co.row_num + 1 = uc.row_num
), rankedCombos as (
SELECT
c.[path]
, ROW_NUMBER() OVER(PARTITION BY c.FatherID ORDER BY c.hierarchy_num DESC) as row_num
FROM combos as c
)
SELECT COUNT(DISTINCT rc.[path]) as UniqueMotherGroups
FROM rankedCombos as rc
WHERE rc.row_num = 1
Update 2:
I had another idea to use a PIVOT to transpose the records so that the FatherID would be in the left-most column with the MotherIDs as the column headers. To make that work with a dynamic list of MotherIDs, you have to use a dynamic PIVOT/dynamic SQL. (FatherID isn't really needed in the PIVOT so it's not included in the PIVOT query. I just had to describe what the goal is.) After the pivot, you can SELECT DISTINCT to get the unique mother combinations. Then the last SELECT is to get the COUNT. This one I ran in SQL Fiddle:
SQL Fiddle
MS SQL Server 2017 Schema Setup:
-- Create family tables.
CREATE TABLE dbo.Father
(
FatherID INT NOT NULL
, Name VARCHAR(50) NOT NULL
);
ALTER TABLE dbo.Father
ADD CONSTRAINT PK_Father
PRIMARY KEY CLUSTERED (FatherID);
ALTER TABLE dbo.Father SET (LOCK_ESCALATION = TABLE);
CREATE TABLE dbo.Mother
(
MotherID INT NOT NULL
, Name VARCHAR(50) NOT NULL
);
ALTER TABLE dbo.Mother
ADD CONSTRAINT PK_Mother
PRIMARY KEY CLUSTERED (MotherID);
ALTER TABLE dbo.Mother SET (LOCK_ESCALATION = TABLE);
CREATE TABLE dbo.Child
(
ChildID INT NOT NULL
, FatherID INT NOT NULL
, MotherID INT NOT NULL
, Name VARCHAR(50) NOT NULL
);
ALTER TABLE dbo.Child
ADD CONSTRAINT PK_Child
PRIMARY KEY CLUSTERED (ChildID);
CREATE NONCLUSTERED INDEX IX_Parents ON dbo.Child (FatherID, MotherID);
ALTER TABLE dbo.Child
ADD CONSTRAINT FK_Child_Father
FOREIGN KEY (FatherID)
REFERENCES dbo.Father (FatherID);
ALTER TABLE dbo.Child
ADD CONSTRAINT FK_Child_Mother
FOREIGN KEY (MotherID)
REFERENCES dbo.Mother (MotherID);
-- Insert two children with the same parents
INSERT INTO dbo.Father
(
FatherID
, Name
)
VALUES
(1, 'Alex')
, (2, 'Bob')
, (3, 'Charlie')
INSERT INTO dbo.Mother
(
MotherID
, Name
)
VALUES
(1, 'Alice')
, (2, 'Barbara');
INSERT INTO dbo.Child
(
ChildID
, FatherID
, MotherID
, Name
)
VALUES
(1, 1, 1, 'Adam')
, (2, 1, 1, 'Billy')
, (3, 1, 2, 'Celine')
, (4, 2, 2, 'Derek')
, (5, 3, 1, 'Eric');
Query 1:
DECLARE #cols AS nvarchar(MAX)
DECLARE #query AS nvarchar(MAX)
SET #cols = STUFF((
SELECT DISTINCT ',' + QUOTENAME(m.MotherID)
FROM Mother as m
FOR XML PATH(''))
,1,1,'')
SET #query = 'SELECT COUNT(mCount) as UniqueMotherGroups FROM (
SELECT DISTINCT ' + #cols + ', 1 as mCount FROM (
SELECT ' + #cols + '
FROM (
SELECT
c.FatherID
, c.MotherID
, 1 as mID
FROM child as c
) x
PIVOT
(
MAX(mID)
FOR MotherID in (' + #cols + ')
) p
) as m
) as mg'
--SELECT #query
Exec(#query)
Results:
| UniqueMotherGroups |
|--------------------|
| 3 |
UPDATE 3: Here's one other idea... create a results table with a unique constraint and with IGNORE_DUP_KEY=ON. You could use this in a function or stored procedure, or, setup a trigger to put the mother combinations into a unique-combo-holding-table. With IGNORE_DUP_KEY=ON, you can insert every combo and only the unique combos will remain. Then just do a count of all the rows.
--Create a table to hold the results:
CREATE TABLE results (
ChildID int not null
, UniqueCombos nvarchar(50) not null
PRIMARY KEY WITH (IGNORE_DUP_KEY = ON)
);
--Insert all combos into the results table. The unique constraint will cause only unique entries to remain.
INSERT INTO results (ChildID, UniqueCombos)
SELECT DISTINCT
c.ChildID
, (
SELECT ',' + CAST(MotherID as nvarchar(500))
FROM Child as c2
WHERE c2.ChildID = c.ChildID
ORDER BY c2.MotherID
FOR XML PATH('')
) as mother_combos
FROM Child as c
;
--Count up all the rows in the results table. Since these are all unique combinations, it should be fast to sum.
SELECT COUNT(*)
FROM results;
If you accept to define a maximum number of mothers per father (here 7) you may try:
select count(*) as UniqueMotherGroups from (
select distinct m1, m2, m3, m4, m5, m6, m7 from (
select FatherID, row_number() over(partition by FatherID order by motherid) as rn, motherid
from (
select distinct FatherID, MotherID
from t_Child
)
)
pivot (
max(motherid) for rn in (1 as m1,2 as m2,3 as m3,4 as m4,5 as m5,6 as m6,7 as m7)
)
)
;
UNIQUEMOTHERGROUPS
------------------
3
Here is one idea. Instead of using precise STRING_AGG you can calculate a hash / checksum of the group. You don't need to know the exact composition of the group, you just need to distinguish between different groups. Calculating of the hash may be faster than concatenating strings.
SQL Server has a function CHECKSUM_AGG
You can write your own hashing function with CLR.
Sample data
CREATE TABLE #Child
(
ChildID INT NOT NULL IDENTITY PRIMARY KEY
,FatherID INT NOT NULL
,MotherID INT NOT NULL
,Name VARCHAR(50) NOT NULL
);
INSERT INTO #Child
(
FatherID
,MotherID
,Name
)
VALUES
(1, 1, 'Adam')
,(1, 1, 'Billy')
,(1, 2, 'Celine')
,(2, 2, 'Derek')
,(3, 1, 'Eric')
,(4, 1, 'A')
,(4, 1, 'B')
,(4, 2, 'C')
,(4, 2, 'D')
,(4, 2, 'E')
,(5, 2, 'F')
,(6, 2, 'G')
;
Query
WITH
distinctParentCombinations
AS
(
SELECT
FatherID
,MotherID
FROM #Child
GROUP BY
FatherID
,MotherID
)
,motherGroups
AS
(
SELECT
FatherID
,CHECKSUM_AGG(MotherID) AS MotherGroup
FROM distinctParentCombinations
GROUP BY
FatherID
)
SELECT COUNT(DISTINCT MotherGroup) AS UniqueMotherGroups
FROM motherGroups
;
Result
+--------------------+
| UniqueMotherGroups |
+--------------------+
| 3 |
+--------------------+
You need to compare performance of all methods on your actual data.
Obviously, with CHECKSUM_AGG it is possible that some of the groups will be missed. There is a chance that two different groups will generate the same checksum.
You know better if this is acceptable.
General way to speed up calculations is to have some of the results already pre-calculated. In your case, for the first part you can create indexed view as follows:
CREATE OR ALTER VIEW vw_distinctParentCombinations WITH SCHEMABINDING AS
SELECT children.FatherID
, children.MotherID
,COUNT_BIG(*) AS [wifes_count]
FROM dbo.Child as children
GROUP BY children.FatherID
, children.MotherID
GO
CREATE UNIQUE CLUSTERED INDEX IX_vw_distinctParentCombinations ON vw_distinctParentCombinations
(
FatherID,MotherID
);
Then in your initial query, you can avoid the first CTE:
-- CTE Gets distinct combinations of parents
WITH motherGroups (Mothers)
AS
(SELECT STRING_AGG(CONVERT(VARCHAR(MAX), distinctParentCombinations.MotherID), '-') WITHIN GROUP (ORDER BY distinctParentCombinations.MotherID) AS Mothers
FROM vw_distinctParentCombinations distinctParentCombinations WITH(NOEXPAND)
GROUP BY distinctParentCombinations.FatherID
)
-- Remove the COUNT function to see the actual combinations
SELECT COUNT(motherGroups.Mothers) AS UniqueMotherGroups
FROM motherGroups;
This will avoid the initial read of the large table and depending the distinct combinations of the pairs (father - mother) it can reduce the view size significantly.
Unfortunately, there are a lot of limitations in order to create an indexed view, and you are not able to create such for the second CTE.
If we change our mind and look this issue in different view, simply we can get the count of mothers with this query:
SELECT Count(distinct ConcatMothers) UniqueMothersCount from(
SELECT FatherID, concat(FatherID,'-',SUM(MotherID)) ConcatMothers
FROM dbo.Child
GROUP BY FatherID) t;
Or even you can use Dense_Rank() like this:
SELECT Max(RankMothers) UniqueMothersCount from(
SELECT FatherID, DENSE_RANK() over (order by concat(FatherID,'-',SUM(MotherID))) RankMothers
FROM dbo.Child
GROUP BY FatherID) t;
For the performance it is hard to measure because dataset is small but since we have one column in the group by and the motherId is in the select maybe we can change index as below:
CREATE NONCLUSTERED INDEX IX_Parents ON dbo.Child (FatherID) Include(MotherID);
but you need to check it on your dataset.

ignoring last value if its the same as current value

I have an table where i would like to query the following:
The data comes in batches . This data is combined with an id.
This ID only gets send ones when the new batch comes in. After that the ID only changes when there is a new batch . In the mean time the value stays null
What i need to do is if new data comes in and it has the same id as the previous batch i have to continue the insert with null in the id field instead of pushing a new row with the same id value.
Beneath is a simplistic view of the table
ID Values
1 10
null 20
null 20
null 20
null 20
2 20
null 20
null 20
null 20
null 20
1 20
null 20
If you could help me point in a directions that would help me a lot.
Maybe to clearify the id value is a set of tags. So there are some definied tags(100 or more) and when a new batch comes the batch gets a tag with it. And if that tag is the same as the previous the null has to continue instead of inserting the same tag
You'll need to add an identity field (or a timestamp) in order to be able to query the latest ID.
ALTER TABLE MyTable ADD MyIdent INT IDENTITY(1, 1) NOT NULL
Then on your insert (if your Id value is NULL) you can call
INSERT INTO MyTable (Id, Values)
SELECT TOP 1 Id, #ValuesVariable
FROM MyTable
WHERE Id IS NOT NULL
ORDER BY MyIdent DESC
This below Sp may helps to inert data try this
IF OBJECT_ID('tempdb..#Temp')IS NOT NULL
DROP TABLE #Temp
CREATE TABLE #Temp (ID INT,[Values] INT)
CREATE PROCEDURE usp_Insert
(
#Id INT,
#Values INT
)
AS
BEGIN
IF NOT EXISTS (SELECT 1 FROM #Temp WHERE ID = #ID)
BEGIN
INSERT INTO #Temp(ID,[Values])
SELECT #Id,#Values
END
ELSE
INSERT INTO #Temp(ID,[Values])
SELECT NULL,#Values
END
EXEC usp_Insert 2,12
SELECT * FROM #Temp

SQL Server, self referencing foreign compound key

I have a table with columns task_id (pk), client_id, parent_task_id, title. In other words, tasks are owned by clients, and some tasks have child tasks.
For example, client 7 may have a task "wash the car," with child tasks "vacuum carpet" and "wipe dashboard."
I want a constraint so that a task and its children are always owned by the same client.
Through a bit of experimentation, to do this, I created a self-referencing foreign key (client_id, parent_task_id) referencing (client_id, task_id). At first I received an error (There are no primary or candidate keys in the referenced table that match the referencing column list in the foreign key.) So I added a unique key for columns task_id, client_id. Now it seems to work.
I am wondering if this is the best solution (or at least reasonable one) to enforce this constraint. Any thoughts would be appreciated. Thanks much!
A 'parent' record would not need a [parent_task_id]
TASK ID | CLIENT ID | PARENT TASK ID | TITLE
1 | 7 | NULL | wash the car
(To find all of your parent records, SELECT * FROM TABLE WHERE [parent_task_id] is null)
A 'child' record would need a [parent_task_id], but not a [client_id] (because, as you stipulate, a child has the same client as it's parent).
TASK ID | CLIENT ID | PARENT TASK ID | TITLE
2 | NULL | 1 | vacuum carpent
3 | NULL | 1 | wipe dashboard
In this way, your self-referencing foreign key is all the constraint you need. No constraint / rule concerning [client_id] on child records is necessary, because all [client_id] values on child records will be ignored, in favor of the [client_id] on the parent record.
For example, if you want to know what the [client_id] is for a child record:
SELECT
c.task_id,
p.client_id,
c.title
FROM
table p --parent
INNER JOIN table c --child
ON p.task_id = c.parent_task_id
UPDATE
(How to query for the client ID of a grand-child)
--Create and populate your table (using a table var in this sample)
DECLARE #table table (task_id int, client_id int, parent_task_id int, title varchar(50))
INSERT INTO #table VALUES (1,7,NULL,'wash the car')
INSERT INTO #table VALUES (2,NULL,1,'vacuum carpet')
INSERT INTO #table VALUES (3,NULL,1,'wipe dashboard')
INSERT INTO #table VALUES (4,NULL,2,'Step 1: plug-in the vacuum')
INSERT INTO #table VALUES (5,NULL,2,'Step 2: turn-on the vacuum')
INSERT INTO #table VALUES (6,NULL,2,'Step 3: use the vacuum')
INSERT INTO #table VALUES (7,NULL,2,'Step 4: turn-off the vacuum')
INSERT INTO #table VALUES (8,NULL,2,'Step 5: empty the vacuum')
INSERT INTO #table VALUES (9,NULL,2,'Step 6: put-away the vacuum')
INSERT INTO #table VALUES (10,NULL,3,'Step 1: spray cleaner on the rag')
INSERT INTO #table VALUES (11,NULL,3,'Step 2: use the rag')
INSERT INTO #table VALUES (12,NULL,3,'Step 3: put-away the cleaner')
INSERT INTO #table VALUES (13,NULL,3,'Step 4: toss the rag in the laundry bin')
--Determine which grandchild you want the client_id for
DECLARE #task_id int
SET #task_id = 8 -- grandchild's ID to use to find client_id
--Create your CTE (this is the recursive part)
;WITH myList (task_id, client_id, parent_task_id, title)
AS
(
SELECT a.task_id, a.client_id, a.parent_task_id, a.title
FROM #table a
WHERE a.task_id = #task_id
UNION ALL
SELECT a.task_id, a.client_id, a.parent_task_id, a.title
FROM #table a
INNER JOIN myList m
ON a.task_id = m.parent_task_id
)
--Query your CTE
SELECT task_id, client_id, title FROM myList WHERE client_id is not null
In this example, I used a granchild's task_id (8 -- 'empty the vacuum') to find it's highest-level parent, which holds the client_id.
You can remove the WHERE clause from the last step if you want to see each parent, parent's parent, and so on up to the first-parent's record.

How can I create a closure table using data from an adjacency list?

I have a database containing a hierarchy of categories stored using the adjacency list model.
The hierarchy is 3 levels deep (not including an imaginary root node) and contains approx 1700 nodes. Nodes in the 2nd and 3rd levels can have multiple parents. A additional table is used for the many-to-many relationship as below:
CREATE TABLE dbo.Category(
id int IDENTITY(1,1) NOT NULL,
name varchar(255) NOT NULL,
)
CREATE TABLE dbo.CategoryHierarchy(
relId int IDENTITY(1,1) NOT NULL,
catId int NOT NULL,
parentId int NOT NULL,
)
If I move to using the transitive closure table method (for the sake of data integrity etc) is there a relatively easy query I can execute that would generate the values for the closure table? (using SQL Server 2005)
I've look through articles and presentations such as Bill Karwin's Models for hierarchical data but that only has insertion queries for a single node and it would take forever for me to create my tree like that.
Thanks.
EDIT:
RelID in the CategoryHierarchy table is purely for the sake of a primary key, it has no bearing on the node ids of the Category table.
Also by closure table, I mean a table like this:
CREATE TABLE ClosureTable (
ancestor int NOT NULL,
descendant int NOT NULL,
[length] int NOT NULL,
)
Where the first two columns are a compound primary key, and are individually foreign keys to Category.id.
I was trying to figure out the same thing, but wanted it in a recursive CTE. This wouldn't have worked for you (SQL Server 2008+), but here's what I ended up with for anyone else looking.
The key is that the anchors aren't your root nodes (where parent_id IS NULL), but instead all your zero depth rows-to-be in the closure table.
Table
CREATE TABLE dbo.category (
id INT IDENTITY(1, 1) NOT NULL,
parent_id INT NULL
)
Data
INSERT INTO dbo.category (id, parent_id)
VALUES
(1, NULL),
(2, 1),
(3, 1),
(4, 2)
CTE
WITH category_cte AS
(
SELECT
id AS ancestor,
id AS descendant,
0 AS depth
FROM dbo.category
UNION ALL
SELECT
CTE.ancestor AS ancestor,
C.id AS descendant,
CTE.depth + 1 AS depth
FROM dbo.category AS C
JOIN category_cte AS CTE
ON C.parent_id = CTE.descendant
)
SELECT * FROM category_cte
Result
ancestor descendant depth
-------- ---------- -----
1 1 0 <- anchor query
2 2 0
3 3 0
4 4 0
2 4 1 <- first recursive query
1 2 1
1 3 1
1 4 2 <- second recursive query
I think I've been able to work out the solution myself.
If anyone has a better way of doing this, please comment.
IF OBJECT_ID('dbo.ClosureTable', 'U') IS NOT NULL
DROP TABLE dbo.ClosureTable
GO
CREATE TABLE dbo.ClosureTable (
ancestor int NOT NULL,
descendant int NOT NULL,
distance int NULL
)
GO
DECLARE #depth INT
SET #depth = 1
INSERT INTO dbo.ClosureTable (ancestor, descendant, distance)
SELECT catid, catid, 0 FROM dbo.Category -- insert all the self-referencing nodes
WHILE (#depth < 4) -- my tree is only 4 levels deep, i.e 0 - 3
BEGIN
INSERT INTO dbo.ClosureTable (ancestor, descendant, distance)
SELECT ct.ancestor, h.catid, #depth
FROM dbo.ClosureTable ct INNER JOIN dbo.CategoryHierarchy h ON ct.descendant = h.parentid
WHERE ct.distance = #depth - 1
SET #depth = #depth + 1
END
Cheers :)
WITH Children (id, name, iteration) AS (
SELECT id, name, 0
FROM Category
--WHERE id = #folderid -- if you want a startpoint
UNION ALL
SELECT b.id, b.name, a.iteration + 1
FROM Children AS a, Category AS b, CategoryHierarchy c
WHERE a.id = c.parentId
AND b.id = c.catId
)
SELECT id, name, iteration FROM Children
Not tested, but should work like this. At least a start how you do this fast without loops.
EDIT: I missred, not relId, it is parentId that has to link to Childrens table. But the result should be a table with all tables. So this is not what your originally wanted?

How to get result of LEFT join into another table as one field

Not sure how to describe this so I will show example:
table PAGES
id int
parent int
name nvarchar
status tinyint
table PAGES_MODULES
id int
id_parent int
module_type nvarchar
module_id int
status int
One page can have more than one linked modules. Example records:
id parent name status
1 -1 Xyz 1
2 -1 Yqw 1
id id_parent module_type module_id status
1 1 ARTICLE 1 1
2 1 GALLERY 2 1
3 2 CATEGORY 3 1
What I need is to create select which will not return 2 results if I do select left join page_modules.
I would like to have select which returns linked modules as this:
id parent name status modules
1 -1 Xyz 1 ARTICLE GALLERY
2 -1 Yqw 1 CATEGORY
Is that possible?
Thanks.
UPDATE
I have tried COALESE, CROSS APPLY and SELECT within SELECT methods and came to these conclusions:
http://blog.feronovak.com/2011/10/multiple-values-in-one-column-aka.html
Hope I can publish these here, not meaning to spam or something.
You'd need to create a custom aggregate function that could concatenate the strings together, there is no built-in SQL Server function that does this.
You can create a custom aggregate function (assuming your using the latest version of SQL) using a .Net assembly. Here's the MS reference on how to do this (the example in the article is actually for a CONCATENATE function just like you require): http://msdn.microsoft.com/en-us/library/ms182741.aspx
Use group_concat() to smoosh multiple rows' worth of data into a single field like that. Note that it does have a length limit (1024 chars by default), so if you're going to have a zillion records being group_concatted, you'll only get the first few lines worth unless you raise the limit.
SELECT ..., GROUP_CONCAT(modules SEPARATOR ' ')
FROM ...
GROUP BY ...
Note that it IS an aggregate function, so you must have a group-by clause.
-- ==================
-- sample data
-- ==================
declare #pages table
(
id int,
parent int,
name nvarchar(max),
status tinyint
)
declare #pages_modules table
(
id int,
id_parent int,
module_type nvarchar(max),
module_id int,
status int
)
insert into #pages values (1, -1, 'Xyz', 1)
insert into #pages values (2, -1, 'Yqw', 1)
insert into #pages_modules values (1, 1, 'ARTICLE', 1, 1)
insert into #pages_modules values (2, 1, 'GALLERY', 2, 1)
insert into #pages_modules values (3, 2, 'CATEGORY', 3, 1)
-- ==================
-- solution
-- ==================
select
*,
modules = (
select module_type + ' ' from #pages_modules pm
where pm.id_parent = p.id
for xml path('')
)
from #pages p
You need to join both tables and then GROUP BY by pages.id, pages.parent, pages.status, pages.name and pages.status. Your modules field in your resultset is then a string aggregate function, i.e in Oracle LISTAGG(pages_modules.modules, ' ') as modules.