Find exactly matched groups in a many to many relationship table - sql

Question

Table shown below maps the many-to-many relationship between courses and students.
CREATE Table CourseStudents
(
CourseId INT NOT NULL,
StudentId INT NOT NULL,
PRIMARY KEY (CourseId, StudentId)
);
INSERT INTO CourseStudents VALUES (1, 1), (1, 2), (2, 1), (2, 2), (3, 3), (3, 2),
(4, 3), (4, 2), (5, 1)
Example data
| CourseId | StudentId |
|----------|-----------|
| 1 | 1 |
| 1 | 2 |
| 2 | 1 |
| 2 | 2 |
| 3 | 2 |
| 3 | 3 |
| 4 | 2 |
| 4 | 3 |
| 5 | 1 |
I'm looking for a query that returns all courses that have the exact same students. I was able to come up with the query shown below.
WITH CourseGroups AS
(
SELECT c.CourseId,
STUFF ((
SELECT ',' + CAST(c2.StudentId AS VARCHAR)
FROM CourseStudents c2
WHERE c2.CourseId = c.CourseId
ORDER BY c2.StudentId
FOR XML PATH ('')), 1, 1, '') AS StudentList
FROM CourseStudents c
GROUP BY c.CourseId)
SELECT cg.StudentList,
STUFF ((
SELECT ',' + CAST(cg2.CourseId AS VARCHAR(10))
FROM CourseGroups cg2
WHERE cg2.StudentList = cg.StudentList
FOR XML PATH ('')), 1, 1, '') AS ExactMatchCourseList
FROM CourseGroups cg
GROUP BY cg.StudentList
HAVING COUNT(*) > 1
Query returns
| StudentList | ExactMatchCourseList |
|-------------|----------------------|
| 1,2 | 1,2 |
| 2,3 | 3,4 |
Above result is fine. But I only need the ExactMatchCourseList.
The table I'm dealing has more than a billion rows so I need an efficient query that can find any matched courses within a few minutes of run time. Appreciate any help.
SqlFiddle

Answer 1

This only does 2 runs over your CourseStudents table, instead of the 4 you're currently doing. And if you add an index on CourseId on the CourseStudents table, the first run will only be an index scan. It also only runs the original STUFF once for each course, instead of once for each student, then grouping by course. I left out the final STUFF that I wasn't sure if you wanted or if it was just a byproduct of how you were calculating it.
CREATE TABLE #Course
(
CourseId INT NOT NULL PRIMARY KEY
);
INSERT INTO #Course
SELECT CourseId
FROM
CourseStudents s
GROUP BY
CourseId
ORDER BY
CourseId;
CREATE TABLE #CourseStudentList
(
CourseId INT NOT NULL PRIMARY KEY,
StudentList VARCHAR(MAX) NOT NULL
);
INSERT INTO #CourseStudentList
SELECT
c.CourseId,
STUFF ((
SELECT ',' + CAST(c2.StudentId AS VARCHAR)
FROM CourseStudents c2
WHERE c2.CourseId = c.CourseId
ORDER BY c2.StudentId
FOR XML PATH ('')), 1, 1, '') AS StudentList
FROM
#Course c
ORDER BY
c.CourseId;
SELECT *
FROM
(
SELECT
l.CourseId,
l.StudentList,
COUNT(*) OVER (PARTITION BY l.StudentList) AS [Count]
FROM
#CourseStudentList l
) l
WHERE
l.[Count] > 1
ORDER BY
l.StudentList;

Answer 2

This will give you a list of course pairs, although if you are going to get triplicates (or more) then you'll end up with some extra results. I don't have time to toy with this further to correct that issue, but maybe it points you in the right direction:
WITH CTE_CourseMatches AS (
SELECT
CS1.CourseId AS CourseId_1,
CS2.CourseId AS CourseId_2,
COUNT(*) AS cnt
FROM
CourseStudents CS1
INNER JOIN CourseStudents CS2 ON CS2.StudentId = CS1.StudentId AND CS2.CourseId > CS1.CourseId
GROUP BY
CS1.CourseId,
CS2.CourseId
),
CTE_CourseCounts AS (SELECT CourseId, COUNT(*) AS cnt FROM CourseStudents GROUP BY CourseID)
SELECT
CM.CourseId_1,
CM.CourseId_2
FROM
CTE_CourseMatches CM
INNER JOIN CTE_CourseCounts CC1 ON CC1.CourseId = CM.CourseId_1 AND CC1.cnt = CM.cnt
INNER JOIN CTE_CourseCounts CC2 ON CC2.CourseId = CM.CourseId_2 AND CC2.cnt = CM.cnt