Changing comparator in WHERE clause has catastrophic results on query performance - sql

Question

I have a monster query that I'm running against a SQL SERVER 2005 database that is acting very strange. I have two conditions in the WHERE clause of the outermost select, comparing a field to a constant date. When the constant dates are either identical (down to the second) or their date parts are not equal, the query runs in under 2 seconds. When the date parts are the same but the time parts are different, the query takes around 7 minutes to complete. Specifically, having a WHERE clause of
WHERE
d.date >= '2011-11-07 00:00:00' AND
d.date <= '2011-11-08 11:59:59'
works well and as expected. Changing the WHERE clause to
WHERE
d.date >= '2011-11-07 00:00:00' AND
d.date <= '2011-11-07 11:59:59'
causes the query to take many minutes.
I also noticed that when I turned off the index on the Agent_Hours table that the bad case of having the same dates the same reduces the query time to 25 seconds, still far longer than when they dates are different, but not by as much.
Below is the full query for reference (the WHERE clause in question is at the very end):
SELECT
s.transaction_id AS 'transaction',
s.created_on AS transaction_date,
s.first_name + ' ' + s.Last_Name AS customer_name,
a.name AS agent_name,
a.phantom AS phantom,
a.team AS agent_team,
a.id AS agent_number,
h.hours,
h2.hours_today,
d.*
FROM
(SELECT
agents.first_name + ' ' + agents.last_name AS name,
agents.id AS id,
agents.phantom AS phantom,
transient.value AS team,
transient.start_date AS team_start_date,
transient.end_date AS team_end_date
FROM
Agents.dbo.Agent_Static AS agents
JOIN
Agents.dbo.Agent_Transient AS transient
ON transient.agent = agents.id
WHERE
transient.field = 'team') AS a
LEFT JOIN Agents.dbo.Agent_Daily AS d
ON d.agent = a.id
LEFT JOIN (SELECT
agent_hours.agent AS agent,
dates.date AS date,
CAST(COUNT(*) AS FLOAT) / 4 AS hours
FROM
Agents.dbo.Agent_Hours AS agent_hours
JOIN
(SELECT
DISTINCT CONVERT(
VARCHAR(10),
hour_worked,
101)
AS date
FROM
Agents.dbo.Agent_Hours) AS dates
ON dates.date = CONVERT(
VARCHAR(10),
agent_hours.hour_worked,
101)
WHERE
(status = 'Phone' OR
status = 'Meeting')
GROUP BY
agent_hours.agent,
dates.date) AS h
ON h.agent = a.id AND
h.date = d.date
LEFT JOIN (SELECT
agent_hours.agent AS agent,
dates.date AS date,
CAST(COUNT(*) AS FLOAT) / 4 AS hours_today
FROM
Agents.dbo.Agent_Hours AS agent_hours
JOIN
(SELECT
DISTINCT CONVERT(
VARCHAR(10),
hour_worked,
101)
AS date
FROM
Agents.dbo.Agent_Hours) AS dates
ON dates.date = CONVERT(
VARCHAR(10),
agent_hours.hour_worked,
101)
WHERE
(status = 'Phone' OR
status = 'Meeting') AND
CONVERT(
VARCHAR(10),
CAST('11/09/2011 13:01' AS DATETIME),
101) = CONVERT(
VARCHAR(10),
agent_hours.hour_worked,
101) AND
CONVERT(
VARCHAR(10),
CAST('11/09/2011 13:01' AS DATETIME),
114) > CONVERT(
VARCHAR(10),
agent_hours.hour_worked,
114)
GROUP BY
agent_hours.agent,
dates.date) AS h2
ON h2.agent = a.id AND
h2.date = d.date
LEFT JOIN sale_transactions AS s
ON a.id = s.agent_hermes_id AND
s.created_on >= a.team_start_date AND
s.created_on <= a.team_end_date AND
CONVERT(
VARCHAR(10),
d.date,
101) = CONVERT(
VARCHAR(10),
s.created_on,
101)
LEFT JOIN sold_phrases AS p
ON s.Transaction_ID = p.transaction_id
WHERE
d.date >= '2011-11-07 00:00:00' AND
d.date <= '2011-11-07 11:59:59'

Answer 1

As a general rule, always post your exact table definition, including all indexes, when asking performance problems in SQL.
I cannot see any difference between the two cases, but considering your explanation, this is what likely happens: the cardinality estimates for the date range may trigger the index tipping point and you get wildly different execution plans. Such issues are best addressed by using plan guides, see Optimizing Queries in Deployed Applications by Using Plan Guides. You should be able to confirm if the problem is indeed the plan, see Displaying Graphical Execution Plans (SQL Server Management Studio).

Answer 2

This is maybe a micro optimization but have you consider changing the way you get the date part from datetime to DATEADD(dd, 0, DATEDIFF(dd, 0, datetime_format)). It's usually faster way than convert function.
SELECT
s.transaction_id AS 'transaction',
s.created_on AS transaction_date,
s.first_name + ' ' + s.Last_Name AS customer_name,
a.name AS agent_name,
a.phantom AS phantom,
a.team AS agent_team,
a.id AS agent_number,
h.hours,
h2.hours_today,
d.*
FROM (SELECT
agents.first_name + ' ' + agents.last_name AS name,
agents.id AS id,
agents.phantom AS phantom,
transient.value AS team,
transient.start_date AS team_start_date,
transient.end_date AS team_end_date
FROM
Agents.dbo.Agent_Static AS agents
JOIN
Agents.dbo.Agent_Transient AS transient
ON transient.agent = agents.id
WHERE
transient.field = 'team'
) AS a
LEFT JOIN Agents.dbo.Agent_Daily AS d ON d.agent = a.id
LEFT JOIN (
SELECT
agent_hours.agent AS agent,
dates.date AS date,
COUNT(*) / 4.0 AS hours
FROM Agents.dbo.Agent_Hours AS agent_hours
JOIN (
SELECT DATEADD(dd, 0, DATEDIFF(dd, 0, hour_worked)) as date
FROM Agents.dbo.Agent_Hours GROUP BY DATEADD(dd, 0, DATEDIFF(dd, 0, hour_worked))
) AS dates ON dates.date = DATEADD(dd, 0, DATEDIFF(dd, 0, agent_hours.hour_worked))
WHERE (status = 'Phone' OR status = 'Meeting')
GROUP BY agent_hours.agent, dates.date
) AS h ON h.agent = a.id AND h.date = d.date
LEFT JOIN (
SELECT
agent_hours.agent AS agent,
dates.date AS date,
COUNT(*) / 4.0 AS hours_today
FROM Agents.dbo.Agent_Hours AS agent_hours
JOIN (
SELECT DATEADD(dd, 0, DATEDIFF(dd, 0, hour_worked)) as date
FROM Agents.dbo.Agent_Hours GROUP BY DATEADD(dd, 0, DATEDIFF(dd, 0, hour_worked))
) AS dates ON dates.date = DATEADD(dd, 0, DATEDIFF(dd, 0, agent_hours.hour_worked))
WHERE
(status = 'Phone' OR status = 'Meeting') AND
agent_hours.hour_worked >=
DATEADD(dd, 0, DATEDIFF(dd, 0, CAST('11/09/2011 13:01' AS DATETIME)))
AND
agent_hours.hour_worked <
CAST('11/09/2011 13:01' AS DATETIME)
GROUP BY agent_hours.agent, dates.date
) AS h2 ON h2.agent = a.id AND h2.date = d.date
LEFT JOIN sale_transactions AS s
ON a.id = s.agent_hermes_id AND
s.created_on >= a.team_start_date AND
s.created_on <= a.team_end_date AND
DATEADD(dd, 0, DATEDIFF(dd, 0, d.date))
=
DATEADD(dd, 0, DATEDIFF(dd, 0, s.created_on))
LEFT JOIN sold_phrases AS p
ON s.Transaction_ID = p.transaction_id
WHERE
d.date >= '2011-11-07 00:00:00' AND
d.date <= '2011-11-07 11:59:59'
The more important (as Remus Rusanu already wrote) are indexes. Execute both queries and check which indexes are used in faster query and force SQL Server to use them always. You can do it using with(index(index_name)).