Why does the optimizer decide to self-join a table? - sql

Question

I'm analyzing my query that looks like this:
WITH Project_UnfinishedCount AS (
SELECT P.Title AS title, COUNT(T.ID) AS cnt
FROM PROJECT P LEFT JOIN TASK T on P.ID = T.ID_PROJECT AND T.ACTUAL_FINISH_DATE IS NULL
GROUP BY P.ID, P.TITLE
)
SELECT Title
FROM Project_UnfinishedCount
WHERE cnt = (
SELECT MAX(cnt)
FROM Project_UnfinishedCount
);
It returns a title of a project that has the biggest number of unfinished tasks in it.
Here is its execution plan:
I wonder why it has steps 6-8 that look like self-join of project table? And than it stores the result of the join as a view, but the view, according to rows and bytes columns is the same as project table. Why does he do it?
I'd also like to know what 2 and 1 steps stand for. I guess, 2 stores the result of my CTE to use it in steps 10-14 and 1 removes the rows from the view that don't have the 'cnt' value that was returned by the subquery, is this a correct guess?

Answer 1

In addition to the comments above, when you reference a CTE more than once, there is a heuristic that tells the optimizer to materialize the CTE, which is why you see the temp table transformation.
A few other comments/questions regarding this query. I'm assuming that the relationship is that a PROJECT can have 0 or more tasks, and each TASK is for one and only one PROJECT. In that case, I wonder why you have an outer join? Moreover, you are joining on the ACTUAL_FINISH_DATE column. This would mean that if you have a project, where all the task were complete, then the outer join would materialize the non-matching row, which would make your query results appear to indicate that there was 1 unfinished task. So I think your CTE should look more like:
SELECT P.Title AS title, COUNT(T.ID) AS cnt
FROM PROJECT P
JOIN TASK T on P.ID = T.ID_PROJECT
WHERE T.ACTUAL_FINISH_DATE IS NULL
GROUP BY P.ID, P.TITLE
With all that being said, these "find the match (count, max etc) within a group" type of queries are often more efficient when written as a window function. That way you can eliminate the self join. This can make a big performance difference when you have millions or billions of rows. So for example, your query could be re-written as:
SELECT TITLE, CNT
FROM (
SELECT P.Title AS title, COUNT(T.ID) AS cnt
, RANK() OVER( ORDER BY COUNT(*) DESC ) RNK
FROM PROJECT P
JOIN TASK T on P.ID = T.ID_PROJECT
WHERE T.ACTUAL_FINISH_DATE IS NULL
GROUP BY P.ID, P.TITLE
)
WHERE RNK=1