I have a table containing debug messages defined as follows:
id INT NOT NULL IDENTITY
message TEXT NOT NULL
date DATETIME NOT NULL
Against that, I run the following query:
SELECT * FROM (
SELECT ROW_NUMBER() OVER (ORDER BY ID DESC) AS RowNum, *
FROM debug WHERE message LIKE #Filter
) AS tbl
WHERE RowNum > (#Page*200-200)
AND RowNum <= (#Page * 200)
ORDER BY ID DESC
This gives me a single page of 200 results; but with 220k entries in the table (and counting!), the filtering and row numbering takes a considerable amount of time (>30 secs).
Is there anything I can do to speed up the process of getting that page of results?
EDIT: So I have switched to VARCHAR(MAX) as proposed by Gordon; but this didn't speed the things up. The resulting execution plan of the query is as follows (I try to repaint the graphics as ASCII-Art):
Sequence Project Clustered Index Scan
SELECT <- Sort <- Filter <- Top <- (Compute Scalar) <- Segment <- Filter <- [debug].[PK_debug]
Cost: 0% 26% 0% 0% 0% 0% 2% 71%
or in XML:
<?xml version="1.0" encoding="utf-16"?>
<ShowPlanXML xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Version="1.2" Build="12.0.2569.0" xmlns="http://schemas.microsoft.com/sqlserver/2004/07/showplan">
<BatchSequence>
<Batch>
<Statements>
<StmtSimple StatementCompId="4" StatementEstRows="16.4317" StatementId="1" StatementOptmLevel="FULL" StatementOptmEarlyAbortReason="GoodEnoughPlanFound" CardinalityEstimationModelVersion="120" StatementSubTreeCost="0.0438369" StatementText="SELECT * FROM (SELECT ROW_NUMBER() OVER (ORDER BY ID DESC) AS RowNum, * FROM debug WHERE message LIKE #Filter) AS tbl WHERE RowNum > (#Page*200-200) AND RowNum <= (#Page * 200) ORDER BY ID DESC" StatementType="SELECT" QueryHash="0x535D30B424597DBB" QueryPlanHash="0x17ABB516DFF1A1A7" RetrievedFromCache="true">
<StatementSetOptions ANSI_NULLS="true" ANSI_PADDING="true" ANSI_WARNINGS="true" ARITHABORT="true" CONCAT_NULL_YIELDS_NULL="true" NUMERIC_ROUNDABORT="false" QUOTED_IDENTIFIER="true" />
<QueryPlan DegreeOfParallelism="0" NonParallelPlanReason="EstimatedDOPIsOne" MemoryGrant="1024" CachedPlanSize="32" CompileTime="4" CompileCPU="4" CompileMemory="248">
<MemoryGrantInfo SerialRequiredMemory="512" SerialDesiredMemory="672" RequiredMemory="512" DesiredMemory="672" RequestedMemory="1024" GrantWaitTime="0" GrantedMemory="1024" MaxUsedMemory="56" />
<OptimizerHardwareDependentProperties EstimatedAvailableMemoryGrant="256000" EstimatedPagesCached="16000" EstimatedAvailableDegreeOfParallelism="1" />
<RelOp AvgRowSize="4055" EstimateCPU="0.000207573" EstimateIO="0.0112613" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="16.4317" LogicalOp="Sort" NodeId="0" Parallel="false" PhysicalOp="Sort" EstimatedTotalSubtreeCost="0.0438369">
<OutputList>
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="id" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="message" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="date" />
<ColumnReference Column="Expr1002" />
</OutputList>
<MemoryFractions Input="1" Output="1" />
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRebinds="1" ActualRewinds="0" ActualRows="147" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<Sort Distinct="false">
<OrderBy>
<OrderByColumn Ascending="false">
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="id" />
</OrderByColumn>
</OrderBy>
<RelOp AvgRowSize="4055" EstimateCPU="8.8E-05" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="16.4317" LogicalOp="Filter" NodeId="1" Parallel="false" PhysicalOp="Filter" EstimatedTotalSubtreeCost="0.0323681">
<OutputList>
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="id" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="message" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="date" />
<ColumnReference Column="Expr1002" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="147" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<Filter StartupExpression="false">
<RelOp AvgRowSize="4055" EstimateCPU="1E-05" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="100" LogicalOp="Top" NodeId="2" Parallel="false" PhysicalOp="Top" EstimatedTotalSubtreeCost="0.0322801">
<OutputList>
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="id" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="message" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="date" />
<ColumnReference Column="Expr1002" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="147" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<Top RowCount="false" IsPercent="false" WithTies="false">
<TopExpression>
<ScalarOperator ScalarString="CASE WHEN CONVERT_IMPLICIT(bigint,[#Page]*(200),0) IS NULL OR CONVERT_IMPLICIT(bigint,[#Page]*(200),0)<(0) THEN (0) ELSE CONVERT_IMPLICIT(bigint,[#Page]*(200),0) END">
<IF>
<Condition>
<ScalarOperator>
<Logical Operation="OR">
<ScalarOperator>
<Compare CompareOp="IS">
<ScalarOperator>
<Identifier>
<ColumnReference Column="ConstExpr1007">
<ScalarOperator>
<Convert DataType="bigint" Style="0" Implicit="true">
<ScalarOperator>
<Arithmetic Operation="MULT">
<ScalarOperator>
<Identifier>
<ColumnReference Column="#Page" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="(200)" />
</ScalarOperator>
</Arithmetic>
</ScalarOperator>
</Convert>
</ScalarOperator>
</ColumnReference>
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="NULL" />
</ScalarOperator>
</Compare>
</ScalarOperator>
<ScalarOperator>
<Compare CompareOp="LT">
<ScalarOperator>
<Identifier>
<ColumnReference Column="ConstExpr1007">
<ScalarOperator>
<Convert DataType="bigint" Style="0" Implicit="true">
<ScalarOperator>
<Arithmetic Operation="MULT">
<ScalarOperator>
<Identifier>
<ColumnReference Column="#Page" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="(200)" />
</ScalarOperator>
</Arithmetic>
</ScalarOperator>
</Convert>
</ScalarOperator>
</ColumnReference>
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="(0)" />
</ScalarOperator>
</Compare>
</ScalarOperator>
</Logical>
</ScalarOperator>
</Condition>
<Then>
<ScalarOperator>
<Const ConstValue="(0)" />
</ScalarOperator>
</Then>
<Else>
<ScalarOperator>
<Identifier>
<ColumnReference Column="ConstExpr1007">
<ScalarOperator>
<Convert DataType="bigint" Style="0" Implicit="true">
<ScalarOperator>
<Arithmetic Operation="MULT">
<ScalarOperator>
<Identifier>
<ColumnReference Column="#Page" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="(200)" />
</ScalarOperator>
</Arithmetic>
</ScalarOperator>
</Convert>
</ScalarOperator>
</ColumnReference>
</Identifier>
</ScalarOperator>
</Else>
</IF>
</ScalarOperator>
</TopExpression>
<RelOp AvgRowSize="4055" EstimateCPU="0.00284444" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="100" LogicalOp="Compute Scalar" NodeId="3" Parallel="false" PhysicalOp="Sequence Project" EstimatedTotalSubtreeCost="0.0322701">
<OutputList>
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="id" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="message" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="date" />
<ColumnReference Column="Expr1002" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="147" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<SequenceProject>
<DefinedValues>
<DefinedValue>
<ColumnReference Column="Expr1002" />
<ScalarOperator ScalarString="row_number">
<Sequence FunctionName="row_number" />
</ScalarOperator>
</DefinedValue>
</DefinedValues>
<RelOp AvgRowSize="4055" EstimateCPU="0.00071111" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="100" LogicalOp="Segment" NodeId="4" Parallel="false" PhysicalOp="Segment" EstimatedTotalSubtreeCost="0.0322621">
<OutputList>
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="id" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="message" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="date" />
<ColumnReference Column="Segment1008" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="147" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<Segment>
<GroupBy />
<SegmentColumn>
<ColumnReference Column="Segment1008" />
</SegmentColumn>
<RelOp AvgRowSize="4047" EstimateCPU="0.347654" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="100" LogicalOp="Filter" NodeId="5" Parallel="false" PhysicalOp="Filter" EstimatedTotalSubtreeCost="0.0322601">
<OutputList>
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="id" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="message" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="date" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="147" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<Filter StartupExpression="false">
<RelOp AvgRowSize="4047" EstimateCPU="0.434724" EstimateIO="9.53275" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="1111.11" LogicalOp="Clustered Index Scan" NodeId="6" Parallel="false" PhysicalOp="Clustered Index Scan" EstimatedTotalSubtreeCost="0.0312823" TableCardinality="395061">
<OutputList>
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="id" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="message" />
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="date" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="395061" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<IndexScan Ordered="true" ScanDirection="BACKWARD" ForcedIndex="false" ForceSeek="false" ForceScan="false" NoExpandHint="false" Storage="RowStore">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="id" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="message" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="date" />
</DefinedValue>
</DefinedValues>
<Object Database="[timeflex]" Schema="[dbo]" Table="[debug]" Index="[PK_debug]" IndexKind="Clustered" Storage="RowStore" />
</IndexScan>
</RelOp>
<Predicate>
<ScalarOperator ScalarString="[timeflex].[dbo].[debug].[message] like [#Filter]">
<Intrinsic FunctionName="like">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[timeflex]" Schema="[dbo]" Table="[debug]" Column="message" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Column="#Filter" />
</Identifier>
</ScalarOperator>
</Intrinsic>
</ScalarOperator>
</Predicate>
</Filter>
</RelOp>
</Segment>
</RelOp>
</SequenceProject>
</RelOp>
</Top>
</RelOp>
<Predicate>
<ScalarOperator ScalarString="[Expr1002]>CONVERT_IMPLICIT(bigint,[#Page]*(200)-(200),0) AND [Expr1002]<=CONVERT_IMPLICIT(bigint,[#Page]*(200),0)">
<Logical Operation="AND">
<ScalarOperator>
<Compare CompareOp="GT">
<ScalarOperator>
<Identifier>
<ColumnReference Column="Expr1002" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Column="ConstExpr1006">
<ScalarOperator>
<Convert DataType="bigint" Style="0" Implicit="true">
<ScalarOperator>
<Arithmetic Operation="SUB">
<ScalarOperator>
<Arithmetic Operation="MULT">
<ScalarOperator>
<Identifier>
<ColumnReference Column="#Page" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="(200)" />
</ScalarOperator>
</Arithmetic>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="(200)" />
</ScalarOperator>
</Arithmetic>
</ScalarOperator>
</Convert>
</ScalarOperator>
</ColumnReference>
</Identifier>
</ScalarOperator>
</Compare>
</ScalarOperator>
<ScalarOperator>
<Compare CompareOp="LE">
<ScalarOperator>
<Identifier>
<ColumnReference Column="Expr1002" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Column="ConstExpr1007">
<ScalarOperator>
<Convert DataType="bigint" Style="0" Implicit="true">
<ScalarOperator>
<Arithmetic Operation="MULT">
<ScalarOperator>
<Identifier>
<ColumnReference Column="#Page" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="(200)" />
</ScalarOperator>
</Arithmetic>
</ScalarOperator>
</Convert>
</ScalarOperator>
</ColumnReference>
</Identifier>
</ScalarOperator>
</Compare>
</ScalarOperator>
</Logical>
</ScalarOperator>
</Predicate>
</Filter>
</RelOp>
</Sort>
</RelOp>
<ParameterList>
<ColumnReference Column="#Page" ParameterRuntimeValue="(1)" />
<ColumnReference Column="#Filter" ParameterRuntimeValue="'%ZUser888%'" />
</ParameterList>
</QueryPlan>
</StmtSimple>
</Statements>
</Batch>
</BatchSequence>
</ShowPlanXML>
First, don't use text, use varchar(max) or nvarchar(max). It is officially deprecated in SQL Server 2016, so it might disappear in a future version.
For pagination, use OFFSET/FETCH. This was introduced in SQL Server 2012, so it is somewhat back compatible. There is no SQL Server 2010. Read about it here.
Let me eliminate table/page/row contention as the cause of the performance problem. My best guess is the like on a very wide column. If message really is a bunch of free form text, then use a full text index. The place to start is the documentation. For some reason, my links to Microsoft documentation aren't working well right now, but it is should be easy to find.
If the row width really is very wide, then you may have an additional problem with the select *. If you are doing paging, then this is likely for a user interface, so you might want to limit the columns to those that you really need -- and even shorten really long ones if you don't intend to show the entire column.
Related
In short: I created a new index that was recommended by a hint in the query plan. But it isn't used after it is created. Why?
I use the Microsoft SQL Server 14.0.3401.7.
I have a simple query:
SELECT COUNT(*) FROM [viewAS0003_PJ0003] AS _tbl_
The view is defined as
CREATE VIEW [dbo].[viewAS0003_PJ0003] AS
SELECT p.* FROM [tblAS0003_PJ0003] AS p
JOIN [tblAS0003] AS a ON a.[Id]=p.[IdAddress] WHERE p.[IdUserDelete] IS NULL AND a.[IdUserDelete] IS NULL
Indexes are defined as:
CREATE INDEX [idxAS0003_000058] ON [dbo].[tblAS0003] ([IdUserDelete] ASC)
CREATE INDEX [idxAS0003_PJ0003_001002] ON [dbo].[tblAS0003_PJ0003] ([IdAddress] ASC) INCLUDE([IdUserDelete])
CREATE INDEX [idxAS0003_PJ0003_001021] ON [dbo].[tblAS0003_PJ0003] ([IdUserDelete] ASC)
The query plan shows this:
<?xml version="1.0" encoding="utf-16"?>
<ShowPlanXML xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Version="1.481" Build="14.0.3401.7" xmlns="http://schemas.microsoft.com/sqlserver/2004/07/showplan">
<BatchSequence>
<Batch>
<Statements>
<StmtSimple StatementCompId="1" StatementEstRows="1" StatementId="1" StatementOptmLevel="FULL" StatementOptmEarlyAbortReason="GoodEnoughPlanFound" CardinalityEstimationModelVersion="140" StatementSubTreeCost="0.674259" StatementText="SELECT COUNT(*) FROM [viewAS0003_PJ0003] AS _tbl_" StatementType="SELECT" QueryHash="0x6C6609E0F75DADC3" QueryPlanHash="0x3E3661E36DB25D4C" RetrievedFromCache="true" SecurityPolicyApplied="false">
<StatementSetOptions ANSI_NULLS="true" ANSI_PADDING="true" ANSI_WARNINGS="true" ARITHABORT="true" CONCAT_NULL_YIELDS_NULL="true" NUMERIC_ROUNDABORT="false" QUOTED_IDENTIFIER="true" />
<QueryPlan DegreeOfParallelism="1" CachedPlanSize="32" CompileTime="11" CompileCPU="11" CompileMemory="584">
<MissingIndexes>
<MissingIndexGroup Impact="34.0898">
<MissingIndex Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]">
<ColumnGroup Usage="EQUALITY">
<Column Name="[IdUserDelete]" ColumnId="42" />
</ColumnGroup>
<ColumnGroup Usage="INCLUDE">
<Column Name="[IdAddress]" ColumnId="2" />
</ColumnGroup>
</MissingIndex>
</MissingIndexGroup>
</MissingIndexes>
<MemoryGrantInfo SerialRequiredMemory="0" SerialDesiredMemory="0" />
<OptimizerHardwareDependentProperties EstimatedAvailableMemoryGrant="209715" EstimatedPagesCached="52428" EstimatedAvailableDegreeOfParallelism="2" MaxCompileMemory="2518336" />
<QueryTimeStats CpuTime="54" ElapsedTime="54" />
<RelOp AvgRowSize="11" EstimateCPU="0" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="1" LogicalOp="Compute Scalar" NodeId="0" Parallel="false" PhysicalOp="Compute Scalar" EstimatedTotalSubtreeCost="0.674259">
<OutputList>
<ColumnReference Column="Expr1002" />
</OutputList>
<ComputeScalar>
<DefinedValues>
<DefinedValue>
<ColumnReference Column="Expr1002" />
<ScalarOperator ScalarString="CONVERT_IMPLICIT(int,[Expr1005],0)">
<Convert DataType="int" Style="0" Implicit="true">
<ScalarOperator>
<Identifier>
<ColumnReference Column="Expr1005" />
</Identifier>
</ScalarOperator>
</Convert>
</ScalarOperator>
</DefinedValue>
</DefinedValues>
<RelOp AvgRowSize="11" EstimateCPU="0.0176833" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="1" LogicalOp="Aggregate" NodeId="1" Parallel="false" PhysicalOp="Stream Aggregate" EstimatedTotalSubtreeCost="0.674259">
<OutputList>
<ColumnReference Column="Expr1005" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="1" Batches="0" ActualEndOfScans="1" ActualExecutions="1" ActualExecutionMode="Row" ActualElapsedms="54" ActualCPUms="54" />
</RunTimeInformation>
<StreamAggregate>
<DefinedValues>
<DefinedValue>
<ColumnReference Column="Expr1005" />
<ScalarOperator ScalarString="Count(*)">
<Aggregate AggType="countstar" Distinct="false" />
</ScalarOperator>
</DefinedValue>
</DefinedValues>
<RelOp AvgRowSize="9" EstimateCPU="0.230984" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="29471.3" LogicalOp="Inner Join" NodeId="2" Parallel="false" PhysicalOp="Merge Join" EstimatedTotalSubtreeCost="0.656576">
<OutputList />
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="48538" Batches="0" ActualEndOfScans="1" ActualExecutions="1" ActualExecutionMode="Row" ActualElapsedms="50" ActualCPUms="50" ActualScans="0" ActualLogicalReads="0" ActualPhysicalReads="0" ActualReadAheads="0" ActualLobLogicalReads="0" ActualLobPhysicalReads="0" ActualLobReadAheads="0" />
</RunTimeInformation>
<Merge ManyToMany="false">
<InnerSideJoinColumns>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]" Alias="[p]" Column="IdAddress" />
</InnerSideJoinColumns>
<OuterSideJoinColumns>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003]" Alias="[a]" Column="Id" />
</OuterSideJoinColumns>
<Residual>
<ScalarOperator ScalarString="[Grutzeck-AGVIP].[dbo].[tblAS0003_PJ0003].[IdAddress] as [p].[IdAddress]=[Grutzeck-AGVIP].[dbo].[tblAS0003].[Id] as [a].[Id]">
<Compare CompareOp="EQ">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]" Alias="[p]" Column="IdAddress" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003]" Alias="[a]" Column="Id" />
</Identifier>
</ScalarOperator>
</Compare>
</ScalarOperator>
</Residual>
<RelOp AvgRowSize="11" EstimateCPU="0.0511395" EstimateIO="0.0925812" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="46347.8" EstimatedRowsRead="46347.8" LogicalOp="Index Seek" NodeId="3" Parallel="false" PhysicalOp="Index Seek" EstimatedTotalSubtreeCost="0.143721" TableCardinality="55572">
<OutputList>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003]" Alias="[a]" Column="Id" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="40628" ActualRowsRead="40628" Batches="0" ActualEndOfScans="1" ActualExecutions="1" ActualExecutionMode="Row" ActualElapsedms="12" ActualCPUms="12" ActualScans="1" ActualLogicalReads="103" ActualPhysicalReads="0" ActualReadAheads="0" ActualLobLogicalReads="0" ActualLobPhysicalReads="0" ActualLobReadAheads="0" />
</RunTimeInformation>
<IndexScan Ordered="true" ScanDirection="FORWARD" ForcedIndex="false" ForceSeek="false" ForceScan="false" NoExpandHint="false" Storage="RowStore">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003]" Alias="[a]" Column="Id" />
</DefinedValue>
</DefinedValues>
<Object Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003]" Index="[idxAS0003_000058]" Alias="[a]" IndexKind="NonClustered" Storage="RowStore" />
<SeekPredicates>
<SeekPredicateNew>
<SeekKeys>
<Prefix ScanType="EQ">
<RangeColumns>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003]" Alias="[a]" Column="IdUserDelete" />
</RangeColumns>
<RangeExpressions>
<ScalarOperator ScalarString="NULL">
<Const ConstValue="NULL" />
</ScalarOperator>
</RangeExpressions>
</Prefix>
</SeekKeys>
</SeekPredicateNew>
</SeekPredicates>
</IndexScan>
</RelOp>
<RelOp AvgRowSize="15" EstimateCPU="0.0656983" EstimateIO="0.187569" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="59574.7" EstimatedRowsRead="59583" LogicalOp="Index Scan" NodeId="4" Parallel="false" PhysicalOp="Index Scan" EstimatedTotalSubtreeCost="0.253268" TableCardinality="59583">
<OutputList>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]" Alias="[p]" Column="IdAddress" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="59335" ActualRowsRead="59583" Batches="0" ActualEndOfScans="1" ActualExecutions="1" ActualExecutionMode="Row" ActualElapsedms="21" ActualCPUms="20" ActualScans="1" ActualLogicalReads="252" ActualPhysicalReads="0" ActualReadAheads="0" ActualLobLogicalReads="0" ActualLobPhysicalReads="0" ActualLobReadAheads="0" />
</RunTimeInformation>
<IndexScan Ordered="true" ScanDirection="FORWARD" ForcedIndex="false" ForceSeek="false" ForceScan="false" NoExpandHint="false" Storage="RowStore">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]" Alias="[p]" Column="IdAddress" />
</DefinedValue>
</DefinedValues>
<Object Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]" Index="[idxAS0003_PJ0003_001002]" Alias="[p]" IndexKind="NonClustered" Storage="RowStore" />
<Predicate>
<ScalarOperator ScalarString="[Grutzeck-AGVIP].[dbo].[tblAS0003_PJ0003].[IdUserDelete] as [p].[IdUserDelete] IS NULL">
<Compare CompareOp="IS">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]" Alias="[p]" Column="IdUserDelete" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="NULL" />
</ScalarOperator>
</Compare>
</ScalarOperator>
</Predicate>
</IndexScan>
</RelOp>
</Merge>
</RelOp>
</StreamAggregate>
</RelOp>
</ComputeScalar>
</RelOp>
</QueryPlan>
</StmtSimple>
</Statements>
</Batch>
</BatchSequence>
</ShowPlanXML>
Now I create the new required index.
CREATE INDEX idxTest ON [dbo].[tblAS0003_PJ0003] ([IdUserDelete]) INCLUDE ([IdAddress])
After executing:
CHECKPOINT;
GO
DBCC DROPCLEANBUFFERS;
GO
I repeat the query. Now the query plan shows this:
<?xml version="1.0" encoding="utf-16"?>
<ShowPlanXML xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Version="1.481" Build="14.0.3401.7" xmlns="http://schemas.microsoft.com/sqlserver/2004/07/showplan">
<BatchSequence>
<Batch>
<Statements>
<StmtSimple StatementCompId="1" StatementEstRows="1" StatementId="1" StatementOptmLevel="FULL" StatementOptmEarlyAbortReason="GoodEnoughPlanFound" CardinalityEstimationModelVersion="140" StatementSubTreeCost="0.673673" StatementText="SELECT COUNT(*) FROM [viewAS0003_PJ0003] AS _tbl_" StatementType="SELECT" QueryHash="0x6C6609E0F75DADC3" QueryPlanHash="0x3E3661E36DB25D4C" RetrievedFromCache="true" SecurityPolicyApplied="false">
<StatementSetOptions ANSI_NULLS="true" ANSI_PADDING="true" ANSI_WARNINGS="true" ARITHABORT="true" CONCAT_NULL_YIELDS_NULL="true" NUMERIC_ROUNDABORT="false" QUOTED_IDENTIFIER="true" />
<QueryPlan DegreeOfParallelism="1" CachedPlanSize="40" CompileTime="16" CompileCPU="13" CompileMemory="584">
<MemoryGrantInfo SerialRequiredMemory="0" SerialDesiredMemory="0" />
<OptimizerHardwareDependentProperties EstimatedAvailableMemoryGrant="209715" EstimatedPagesCached="52428" EstimatedAvailableDegreeOfParallelism="2" MaxCompileMemory="2467808" />
<WaitStats>
<Wait WaitType="PAGEIOLATCH_SH" WaitTimeMs="2" WaitCount="3" />
<Wait WaitType="SOS_SCHEDULER_YIELD" WaitTimeMs="38" WaitCount="12" />
</WaitStats>
<QueryTimeStats CpuTime="54" ElapsedTime="94" />
<RelOp AvgRowSize="11" EstimateCPU="0" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="1" LogicalOp="Compute Scalar" NodeId="0" Parallel="false" PhysicalOp="Compute Scalar" EstimatedTotalSubtreeCost="0.673673">
<OutputList>
<ColumnReference Column="Expr1002" />
</OutputList>
<ComputeScalar>
<DefinedValues>
<DefinedValue>
<ColumnReference Column="Expr1002" />
<ScalarOperator ScalarString="CONVERT_IMPLICIT(int,[Expr1005],0)">
<Convert DataType="int" Style="0" Implicit="true">
<ScalarOperator>
<Identifier>
<ColumnReference Column="Expr1005" />
</Identifier>
</ScalarOperator>
</Convert>
</ScalarOperator>
</DefinedValue>
</DefinedValues>
<RelOp AvgRowSize="11" EstimateCPU="0.0176121" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="1" LogicalOp="Aggregate" NodeId="1" Parallel="false" PhysicalOp="Stream Aggregate" EstimatedTotalSubtreeCost="0.673673">
<OutputList>
<ColumnReference Column="Expr1005" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="1" Batches="0" ActualEndOfScans="1" ActualExecutions="1" ActualExecutionMode="Row" ActualElapsedms="94" ActualCPUms="54" />
</RunTimeInformation>
<StreamAggregate>
<DefinedValues>
<DefinedValue>
<ColumnReference Column="Expr1005" />
<ScalarOperator ScalarString="Count(*)">
<Aggregate AggType="countstar" Distinct="false" />
</ScalarOperator>
</DefinedValue>
</DefinedValues>
<RelOp AvgRowSize="9" EstimateCPU="0.230469" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="29352.7" LogicalOp="Inner Join" NodeId="2" Parallel="false" PhysicalOp="Merge Join" EstimatedTotalSubtreeCost="0.65606">
<OutputList />
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="48538" Batches="0" ActualEndOfScans="1" ActualExecutions="1" ActualExecutionMode="Row" ActualElapsedms="90" ActualCPUms="50" ActualScans="0" ActualLogicalReads="0" ActualPhysicalReads="0" ActualReadAheads="0" ActualLobLogicalReads="0" ActualLobPhysicalReads="0" ActualLobReadAheads="0" />
</RunTimeInformation>
<Merge ManyToMany="false">
<InnerSideJoinColumns>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]" Alias="[p]" Column="IdAddress" />
</InnerSideJoinColumns>
<OuterSideJoinColumns>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003]" Alias="[a]" Column="Id" />
</OuterSideJoinColumns>
<Residual>
<ScalarOperator ScalarString="[Grutzeck-AGVIP].[dbo].[tblAS0003_PJ0003].[IdAddress] as [p].[IdAddress]=[Grutzeck-AGVIP].[dbo].[tblAS0003].[Id] as [a].[Id]">
<Compare CompareOp="EQ">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]" Alias="[p]" Column="IdAddress" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003]" Alias="[a]" Column="Id" />
</Identifier>
</ScalarOperator>
</Compare>
</ScalarOperator>
</Residual>
<RelOp AvgRowSize="11" EstimateCPU="0.0511395" EstimateIO="0.0925812" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="46347.8" EstimatedRowsRead="46347.8" LogicalOp="Index Seek" NodeId="3" Parallel="false" PhysicalOp="Index Seek" EstimatedTotalSubtreeCost="0.143721" TableCardinality="55572">
<OutputList>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003]" Alias="[a]" Column="Id" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="40628" ActualRowsRead="40628" Batches="0" ActualEndOfScans="1" ActualExecutions="1" ActualExecutionMode="Row" ActualElapsedms="26" ActualCPUms="11" ActualScans="1" ActualLogicalReads="103" ActualPhysicalReads="1" ActualReadAheads="101" ActualLobLogicalReads="0" ActualLobPhysicalReads="0" ActualLobReadAheads="0" />
</RunTimeInformation>
<IndexScan Ordered="true" ScanDirection="FORWARD" ForcedIndex="false" ForceSeek="false" ForceScan="false" NoExpandHint="false" Storage="RowStore">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003]" Alias="[a]" Column="Id" />
</DefinedValue>
</DefinedValues>
<Object Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003]" Index="[idxAS0003_000058]" Alias="[a]" IndexKind="NonClustered" Storage="RowStore" />
<SeekPredicates>
<SeekPredicateNew>
<SeekKeys>
<Prefix ScanType="EQ">
<RangeColumns>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003]" Alias="[a]" Column="IdUserDelete" />
</RangeColumns>
<RangeExpressions>
<ScalarOperator ScalarString="NULL">
<Const ConstValue="NULL" />
</ScalarOperator>
</RangeExpressions>
</Prefix>
</SeekKeys>
</SeekPredicateNew>
</SeekPredicates>
</IndexScan>
</RelOp>
<RelOp AvgRowSize="15" EstimateCPU="0.0656983" EstimateIO="0.187569" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="59335" EstimatedRowsRead="59583" LogicalOp="Index Scan" NodeId="4" Parallel="false" PhysicalOp="Index Scan" EstimatedTotalSubtreeCost="0.253268" TableCardinality="59583">
<OutputList>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]" Alias="[p]" Column="IdAddress" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="59335" ActualRowsRead="59583" Batches="0" ActualEndOfScans="1" ActualExecutions="1" ActualExecutionMode="Row" ActualElapsedms="48" ActualCPUms="22" ActualScans="1" ActualLogicalReads="252" ActualPhysicalReads="0" ActualReadAheads="249" ActualLobLogicalReads="0" ActualLobPhysicalReads="0" ActualLobReadAheads="0" />
</RunTimeInformation>
<IndexScan Ordered="true" ScanDirection="FORWARD" ForcedIndex="false" ForceSeek="false" ForceScan="false" NoExpandHint="false" Storage="RowStore">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]" Alias="[p]" Column="IdAddress" />
</DefinedValue>
</DefinedValues>
<Object Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]" Index="[idxAS0003_PJ0003_001002]" Alias="[p]" IndexKind="NonClustered" Storage="RowStore" />
<Predicate>
<ScalarOperator ScalarString="[Grutzeck-AGVIP].[dbo].[tblAS0003_PJ0003].[IdUserDelete] as [p].[IdUserDelete] IS NULL">
<Compare CompareOp="IS">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[Grutzeck-AGVIP]" Schema="[dbo]" Table="[tblAS0003_PJ0003]" Alias="[p]" Column="IdUserDelete" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="NULL" />
</ScalarOperator>
</Compare>
</ScalarOperator>
</Predicate>
</IndexScan>
</RelOp>
</Merge>
</RelOp>
</StreamAggregate>
</RelOp>
</ComputeScalar>
</RelOp>
</QueryPlan>
</StmtSimple>
</Statements>
</Batch>
</BatchSequence>
</ShowPlanXML>
As you can see the new index isn't used at all. The execution plan didn't change. Why?
I am unable to create clustered index on my view for a derived table.
Please help me to optimize my view where I can create an index as its my requirement to optimize. If indexing is not possible for my case please help me to write a faster view.
Here is my code:
ALTER VIEW [dbo].[VW_FGDOrderByMaxTimeAdded]
WITH SCHEMABINDING
AS
SELECT t.id,
t.Customer,
t.Scanner,
t.custType,
FDGOrder,
StatusID,
TimeAdded,
cntID,
empID,
effectChildren
FROM dbo.smsFDGOrder t
INNER JOIN dbo.smsFDGOrderStatus ON t.id = dbo.smsFDGOrderStatus.FDGOrder
INNER JOIN (
SELECT smsFDGOrder.id,
MAX(DISTINCT TimeAdded) AS MAX_TIME
FROM dbo.smsFDGOrder AS smsFDGOrder
INNER JOIN dbo.smsFDGOrderStatus ON smsFDGOrder.id = FDGOrder
GROUP BY smsFDGOrder.id
) a ON a.id = t.id
AND a.MAX_TIME = TimeAdded
I tried to add the index by this :
CREATE UNIQUE CLUSTERED INDEX IDX_VW_FGDOrderByMaxTimeAdded_ID
ON dbo.VW_FGDOrderByMaxTimeAdded (id);
but got error:
Msg 10109, Level 16, State 1, Line 1
Cannot create index on view "sharedmsdb-LIVE.dbo.VW_FGDOrderByMaxTimeAdded" because it references derived table "a" (defined by SELECT statement in FROM clause).
Consider removing the reference to the derived table or not indexing
the view.
Execution Plan for Devart Answer :
<?xml version="1.0" encoding="utf-16"?>
<ShowPlanXML xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Version="1.2" Build="11.0.5343.0" xmlns="http://schemas.microsoft.com/sqlserver/2004/07/showplan">
<BatchSequence>
<Batch>
<Statements>
<StmtSimple StatementCompId="1" StatementId="1" StatementText="CREATE /*UNIQUE*/ NONCLUSTERED INDEX ixd ON dbo.smsFDGOrderStatus (FDGOrder, TimeAdded DESC)
" StatementType="CREATE INDEX" RetrievedFromCache="false" />
</Statements>
</Batch>
<Batch>
<Statements>
<StmtSimple StatementCompId="1" StatementEstRows="72653.9" StatementId="1" StatementOptmLevel="FULL" StatementSubTreeCost="10.5795" StatementText="
SELECT t.id,
t.Customer,
t.Scanner,
t.custType,
s.FDGOrder,
s.StatusID,
s.TimeAdded,
s.cntID,
s.empID,
s.effectChildren
FROM dbo.smsFDGOrder t
JOIN dbo.smsFDGOrderStatus s ON t.id = s.FDGOrder
JOIN (
SELECT FDGOrder, MAX(TimeAdded) AS MAX_TIME
FROM dbo.smsFDGOrderStatus
GROUP BY FDGOrder
) a ON a.FDGOrder = s.FDGOrder
AND a.MAX_TIME = TimeAdded" StatementType="SELECT" QueryHash="0x75C062BEC9148C5B" QueryPlanHash="0xF019E4D78D6C40" RetrievedFromCache="true">
<StatementSetOptions ANSI_NULLS="true" ANSI_PADDING="true" ANSI_WARNINGS="true" ARITHABORT="true" CONCAT_NULL_YIELDS_NULL="true" NUMERIC_ROUNDABORT="false" QUOTED_IDENTIFIER="true" />
<QueryPlan NonParallelPlanReason="NoParallelPlansInDesktopOrExpressEdition" CachedPlanSize="32" CompileTime="36" CompileCPU="36" CompileMemory="488">
<MemoryGrantInfo SerialRequiredMemory="0" SerialDesiredMemory="0" />
<OptimizerHardwareDependentProperties EstimatedAvailableMemoryGrant="102297" EstimatedPagesCached="7864" EstimatedAvailableDegreeOfParallelism="2" />
<RelOp AvgRowSize="49" EstimateCPU="0.317657" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="72653.9" LogicalOp="Inner Join" NodeId="0" Parallel="false" PhysicalOp="Merge Join" EstimatedTotalSubtreeCost="10.5795">
<OutputList>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="ID" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="Customer" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="Scanner" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="custType" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="FDGOrder" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="StatusID" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="TimeAdded" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="cntID" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="empID" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="effectChildren" />
</OutputList>
<Merge ManyToMany="false">
<InnerSideJoinColumns>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Column="FDGOrder" />
</InnerSideJoinColumns>
<OuterSideJoinColumns>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="ID" />
</OuterSideJoinColumns>
<Residual>
<ScalarOperator ScalarString="[sharedmsdb-LIVE].[dbo].[smsFDGOrder].[ID] as [t].[ID]=[sharedmsdb-LIVE].[dbo].[smsFDGOrderStatus].[FDGOrder]">
<Compare CompareOp="EQ">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="ID" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Column="FDGOrder" />
</Identifier>
</ScalarOperator>
</Compare>
</ScalarOperator>
</Residual>
<RelOp AvgRowSize="23" EstimateCPU="0.0798894" EstimateIO="0.193495" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="72484" LogicalOp="Clustered Index Scan" NodeId="1" Parallel="false" PhysicalOp="Clustered Index Scan" EstimatedTotalSubtreeCost="0.273385" TableCardinality="72484">
<OutputList>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="ID" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="Customer" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="Scanner" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="custType" />
</OutputList>
<IndexScan Ordered="true" ScanDirection="FORWARD" ForcedIndex="false" ForceSeek="false" ForceScan="false" NoExpandHint="false" Storage="RowStore">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="ID" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="Customer" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="Scanner" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Alias="[t]" Column="custType" />
</DefinedValue>
</DefinedValues>
<Object Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrder]" Index="[PK_FDGOrder]" Alias="[t]" IndexKind="Clustered" />
</IndexScan>
</RelOp>
<RelOp AvgRowSize="36" EstimateCPU="1.67094" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="72654.8" LogicalOp="Inner Join" NodeId="2" Parallel="false" PhysicalOp="Merge Join" EstimatedTotalSubtreeCost="9.9885">
<OutputList>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="FDGOrder" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="StatusID" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="TimeAdded" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="cntID" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="empID" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="effectChildren" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Column="FDGOrder" />
</OutputList>
<Merge ManyToMany="false">
<InnerSideJoinColumns>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="FDGOrder" />
</InnerSideJoinColumns>
<OuterSideJoinColumns>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Column="FDGOrder" />
</OuterSideJoinColumns>
<Residual>
<ScalarOperator ScalarString="[sharedmsdb-LIVE].[dbo].[smsFDGOrderStatus].[FDGOrder]=[sharedmsdb-LIVE].[dbo].[smsFDGOrderStatus].[FDGOrder] as [s].[FDGOrder] AND [Expr1007]=[sharedmsdb-LIVE].[dbo].[smsFDGOrderStatus].[TimeAdded] as [s].[TimeAdded]">
<Logical Operation="AND">
<ScalarOperator>
<Compare CompareOp="EQ">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Column="FDGOrder" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="FDGOrder" />
</Identifier>
</ScalarOperator>
</Compare>
</ScalarOperator>
<ScalarOperator>
<Compare CompareOp="EQ">
<ScalarOperator>
<Identifier>
<ColumnReference Column="Expr1007" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="TimeAdded" />
</Identifier>
</ScalarOperator>
</Compare>
</ScalarOperator>
</Logical>
</ScalarOperator>
</Residual>
<RelOp AvgRowSize="19" EstimateCPU="0.445733" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="72440" LogicalOp="Aggregate" NodeId="3" Parallel="false" PhysicalOp="Stream Aggregate" EstimatedTotalSubtreeCost="2.57312">
<OutputList>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Column="FDGOrder" />
<ColumnReference Column="Expr1007" />
</OutputList>
<StreamAggregate>
<DefinedValues>
<DefinedValue>
<ColumnReference Column="Expr1007" />
<ScalarOperator ScalarString="MAX([sharedmsdb-LIVE].[dbo].[smsFDGOrderStatus].[TimeAdded])">
<Aggregate AggType="MAX" Distinct="false">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Column="TimeAdded" />
</Identifier>
</ScalarOperator>
</Aggregate>
</ScalarOperator>
</DefinedValue>
</DefinedValues>
<GroupBy>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Column="FDGOrder" />
</GroupBy>
<RelOp AvgRowSize="19" EstimateCPU="0.750931" EstimateIO="1.37646" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="682522" LogicalOp="Index Scan" NodeId="4" Parallel="false" PhysicalOp="Index Scan" EstimatedTotalSubtreeCost="2.12739" TableCardinality="682522">
<OutputList>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Column="FDGOrder" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Column="TimeAdded" />
</OutputList>
<IndexScan Ordered="true" ScanDirection="FORWARD" ForcedIndex="false" ForceSeek="false" ForceScan="false" NoExpandHint="false" Storage="RowStore">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Column="FDGOrder" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Column="TimeAdded" />
</DefinedValue>
</DefinedValues>
<Object Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Index="[ixd]" IndexKind="NonClustered" />
</IndexScan>
</RelOp>
</StreamAggregate>
</RelOp>
<RelOp AvgRowSize="32" EstimateCPU="0.750931" EstimateIO="4.9935" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="682522" LogicalOp="Index Scan" NodeId="8" Parallel="false" PhysicalOp="Index Scan" EstimatedTotalSubtreeCost="5.74443" TableCardinality="682522">
<OutputList>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="FDGOrder" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="StatusID" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="TimeAdded" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="cntID" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="empID" />
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="effectChildren" />
</OutputList>
<IndexScan Ordered="true" ScanDirection="FORWARD" ForcedIndex="false" ForceSeek="false" ForceScan="false" NoExpandHint="false" Storage="RowStore">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="FDGOrder" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="StatusID" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="TimeAdded" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="cntID" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="empID" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Alias="[s]" Column="effectChildren" />
</DefinedValue>
</DefinedValues>
<Object Database="[sharedmsdb-LIVE]" Schema="[dbo]" Table="[smsFDGOrderStatus]" Index="[_dta_index_smsFDGOrderStatus_5_1938105945__K2_K3_K1_K4_5_6_7_8]" Alias="[s]" IndexKind="NonClustered" />
</IndexScan>
</RelOp>
</Merge>
</RelOp>
</Merge>
</RelOp>
</QueryPlan>
</StmtSimple>
</Statements>
</Batch>
</BatchSequence>
</ShowPlanXML>
Variant #1:
CREATE /*UNIQUE*/ NONCLUSTERED INDEX ix ON dbo.smsFDGOrderStatus (FDGOrder, TimeAdded DESC)
GO
SELECT t.id,
t.Customer,
t.Scanner,
t.custType,
s.FDGOrder,
s.StatusID,
s.TimeAdded,
s.cntID,
s.empID,
s.effectChildren
FROM dbo.smsFDGOrder t
JOIN dbo.smsFDGOrderStatus s ON t.id = s.FDGOrder
JOIN (
SELECT FDGOrder, MAX(TimeAdded) AS MAX_TIME
FROM dbo.smsFDGOrderStatus
GROUP BY FDGOrder
) a ON a.FDGOrder = s.FDGOrder
AND a.MAX_TIME = s.TimeAdded
Variant #2:
CREATE /*UNIQUE*/ NONCLUSTERED INDEX ix2
ON dbo.smsFDGOrderStatus(FDGOrder, TimeAdded DESC)
INCLUDE (StatusID, cntID, empID, effectChildren)
GO
SELECT t.id,
t.Customer,
t.Scanner,
t.custType,
s.FDGOrder,
s.StatusID,
s.TimeAdded,
s.cntID,
s.empID,
s.effectChildren
FROM dbo.smsFDGOrder t
JOIN (
SELECT FDGOrder,
StatusID,
TimeAdded,
cntID,
empID,
effectChildren,
RowNum = ROW_NUMBER() OVER (PARTITION BY FDGOrder ORDER BY TimeAdded DESC)
FROM dbo.smsFDGOrderStatus
) t ON t.id = s.FDGOrder
AND t.RowNum = 1
Variant #3 ... the long way - aggregated table:
IF OBJECT_ID('dbo.smsFDGOrderStatus_Last', 'U') IS NOT NULL
DROP TABLE dbo.smsFDGOrderStatus_Last
GO
SELECT FDGOrder, MAX(TimeAdded) AS MAX_TIME
INTO dbo.smsFDGOrderStatus_Last
FROM dbo.smsFDGOrderStatus
GROUP BY FDGOrder
GO
CREATE UNIQUE CLUSTERED INDEX ix_l ON dbo.table_name (FDGOrder)
GO
CREATE /*UNIQUE*/ NONCLUSTERED INDEX ix2
ON dbo.smsFDGOrderStatus(FDGOrder, TimeAdded DESC)
INCLUDE (StatusID, cntID, empID, effectChildren)
GO
CREATE TRIGGER dbo.trg_ID
ON dbo.smsFDGOrderStatus
AFTER INSERT
AS
BEGIN
SET NOCOUNT ON;
;WITH cte AS (
SELECT *
FROM dbo.smsFDGOrderStatus
WHERE FDGOrder IN (SELECT i.FDGOrder FROM INSERTED i)
)
MERGE cte t
USING (
SELECT FDGOrder, MAX(TimeAdded) AS MAX_TIME
FROM INSERTED
GROUP BY FDGOrder
) s ON s.ManufacturerName = t.ManufacturerString
WHEN MATCHED AND (t.MAX_TIME != s.MAX_TIME)
THEN
UPDATE SET MAX_TIME = s.MAX_TIME
WHEN NOT MATCHED BY TARGET
THEN
INSERT (FDGOrder, MAX_TIME)
VALUES (s.ManufacturerName, s.MAX_TIME);
END
GO
SELECT t.id,
t.Customer,
t.Scanner,
t.custType,
s.FDGOrder,
s.StatusID,
s.TimeAdded,
s.cntID,
s.empID,
s.effectChildren
FROM dbo.smsFDGOrder t
JOIN dbo.smsFDGOrderStatus s ON t.id = s.FDGOrder
JOIN dbo.smsFDGOrderStatus_Last a ON a.FDGOrder = s.FDGOrder AND a.MAX_TIME = s.TimeAdded
I have a table with 450,000 rows that has a varchar column which is of variable length (between 6 and 13 characters, unevenly distributed). I need to join to another table using criteria that says a column in the target table starts with the value of the column of the first table.
In my current test sample, I know that all the matches will be 6 characters, so I'm doing a join using t1.Digits = left(t2.Number, 6), which is very fast (a few seconds to run my large query). My test sample is 10,000 records, though in production the query will need to operate on hundreds of thousands.
I also know that the vast majority of records will always be 6-character matches, but I need to support matching on more, or I'll sometimes get duplicate records returned. The problem is that I've tried all of the following ways of doing this, and each one is much slower than my simpler join on the left six characters. I've never let them run longer than five minutes, but they show no signs of ever terminating:
t1.Digits = left(t2.Number, datalength(t1.Digits))
charindex(t1.Digits, t2.Number) = 1
Adding a pre-calculated DigitLength int column to t1 and then using t1.Digits = left(t2.Number, t1.DigitLength)
t2.Number like t1.Digits + '%'
Each of the above four solutions achieves what I want theoretically, but runs much too slowly for my purposes.
Even though the values in these columns are numeric, I'm using varchar because in many cases there are leading zeroes that need to be preserved. In any case, there ought to be a fast solution even for cases where the data is alphanumeric.
Is anyone aware of a very fast "starts with" logic that would be comparable in performance to my too-simplistic join?
I do have a clustered index on the t1.Digits column?
Here's an execution plan running with method #4 above:
<?xml version="1.0" encoding="utf-16"?>
<ShowPlanXML xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Version="1.0" Build="9.00.5000.00" xmlns="http://schemas.microsoft.com/sqlserver/2004/07/showplan">
<BatchSequence>
<Batch>
<Statements>
<StmtSimple StatementCompId="1" StatementEstRows="10720" StatementId="1" StatementOptmLevel="FULL" StatementSubTreeCost="7471.7" StatementText="select c.FromNumber, c.ToNumber, d.Destination, d.Digits
from Converting c
--join CASH.CASH.dbo.DestinationLookup d on d.Digits = left(c.FromNumber, 6)
join CASH.CASH.dbo.DestinationLookup d on c.FromNumber like d.Digits + '%'
" StatementType="SELECT">
<StatementSetOptions ANSI_NULLS="false" ANSI_PADDING="false" ANSI_WARNINGS="false" ARITHABORT="true" CONCAT_NULL_YIELDS_NULL="false" NUMERIC_ROUNDABORT="false" QUOTED_IDENTIFIER="false" />
<QueryPlan DegreeOfParallelism="1" MemoryGrant="114" CachedPlanSize="99" CompileTime="36" CompileCPU="35" CompileMemory="312">
<RelOp AvgRowSize="77" EstimateCPU="174.861" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="10720" LogicalOp="Inner Join" NodeId="0" Parallel="false" PhysicalOp="Nested Loops" EstimatedTotalSubtreeCost="7471.7">
<OutputList>
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="FromNumber" />
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="ToNumber" />
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Digits" />
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Destination" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="10720" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<NestedLoops Optimized="false">
<OuterReferences>
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="FromNumber" />
</OuterReferences>
<RelOp AvgRowSize="38" EstimateCPU="0.164714" EstimateIO="0.00281532" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="10720" LogicalOp="Sort" NodeId="1" Parallel="false" PhysicalOp="Sort" EstimatedTotalSubtreeCost="0.340338">
<OutputList>
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="FromNumber" />
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="ToNumber" />
</OutputList>
<MemoryFractions Input="1" Output="1" />
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRebinds="1" ActualRewinds="0" ActualRows="10720" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<Sort Distinct="false">
<OrderBy>
<OrderByColumn Ascending="true">
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="FromNumber" />
</OrderByColumn>
</OrderBy>
<RelOp AvgRowSize="38" EstimateCPU="0.00296763" EstimateIO="0.126907" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="10720" LogicalOp="Table Scan" NodeId="2" Parallel="false" PhysicalOp="Table Scan" EstimatedTotalSubtreeCost="0.129875">
<OutputList>
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="FromNumber" />
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="ToNumber" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="10720" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<TableScan Ordered="false" ForcedIndex="false" NoExpandHint="false">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="FromNumber" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="ToNumber" />
</DefinedValue>
</DefinedValues>
<Object Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" />
</TableScan>
</RelOp>
</Sort>
</RelOp>
<RelOp AvgRowSize="48" EstimateCPU="0.00290986" EstimateIO="0.01" EstimateRebinds="1390" EstimateRewinds="9329" EstimateRows="15609.2" LogicalOp="Lazy Spool" NodeId="3" Parallel="false" PhysicalOp="Table Spool" EstimatedTotalSubtreeCost="7296.5">
<OutputList>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Digits" />
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Destination" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRebinds="1391" ActualRewinds="9329" ActualRows="10720" ActualEndOfScans="10720" ActualExecutions="10720" />
</RunTimeInformation>
<Spool>
<RelOp AvgRowSize="48" EstimateCPU="5.21308" EstimateIO="0" EstimateRebinds="1390" EstimateRewinds="0" EstimateRows="15609.2" LogicalOp="Compute Scalar" NodeId="4" Parallel="false" PhysicalOp="Compute Scalar" EstimatedTotalSubtreeCost="7251.4">
<OutputList>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Digits" />
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Destination" />
</OutputList>
<ComputeScalar>
<DefinedValues>
<DefinedValue>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Digits" />
<ScalarOperator ScalarString="[CASH].[CASH].[dbo].[DestinationLookup].[Digits] as [d].[Digits]">
<Identifier>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Digits" />
</Identifier>
</ScalarOperator>
</DefinedValue>
<DefinedValue>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Destination" />
<ScalarOperator ScalarString="[CASH].[CASH].[dbo].[DestinationLookup].[Destination] as [d].[Destination]">
<Identifier>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Destination" />
</Identifier>
</ScalarOperator>
</DefinedValue>
</DefinedValues>
<RelOp AvgRowSize="48" EstimateCPU="5.21308" EstimateIO="0" EstimateRebinds="1390" EstimateRewinds="0" EstimateRows="15609.2" LogicalOp="Remote Query" NodeId="5" Parallel="false" PhysicalOp="Remote Query" EstimatedTotalSubtreeCost="7251.4">
<OutputList>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Digits" />
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Destination" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRebinds="1391" ActualRewinds="0" ActualRows="1391" ActualEndOfScans="1391" ActualExecutions="1391" />
</RunTimeInformation>
<RemoteQuery RemoteSource="CASH" RemoteQuery="SELECT "Tbl1004"."Digits" "Col1021","Tbl1004"."Destination" "Col1022" FROM "CASH"."dbo"."DestinationLookup" "Tbl1004" WHERE ? like "Tbl1004"."Digits"+'%'" />
</RelOp>
</ComputeScalar>
</RelOp>
</Spool>
</RelOp>
</NestedLoops>
</RelOp>
</QueryPlan>
</StmtSimple>
</Statements>
</Batch>
</BatchSequence>
</ShowPlanXML>
And here's the plan when joining with a simple left(t2.Number, 6):
<?xml version="1.0" encoding="utf-16"?>
<ShowPlanXML xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Version="1.0" Build="9.00.5000.00" xmlns="http://schemas.microsoft.com/sqlserver/2004/07/showplan">
<BatchSequence>
<Batch>
<Statements>
<StmtSimple StatementCompId="1" StatementEstRows="10720" StatementId="1" StatementOptmLevel="FULL" StatementSubTreeCost="15.1845" StatementText="select c.FromNumber, c.ToNumber, d.Destination, d.Digits
from Converting c
join CASH.CASH.dbo.DestinationLookup d on d.Digits = left(c.FromNumber, 6) " StatementType="SELECT">
<StatementSetOptions ANSI_NULLS="false" ANSI_PADDING="false" ANSI_WARNINGS="false" ARITHABORT="true" CONCAT_NULL_YIELDS_NULL="false" NUMERIC_ROUNDABORT="false" QUOTED_IDENTIFIER="false" />
<QueryPlan DegreeOfParallelism="1" CachedPlanSize="105" CompileTime="60" CompileCPU="58" CompileMemory="360">
<RelOp AvgRowSize="77" EstimateCPU="0.0448096" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="10720" LogicalOp="Inner Join" NodeId="0" Parallel="false" PhysicalOp="Nested Loops" EstimatedTotalSubtreeCost="15.1845">
<OutputList>
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="FromNumber" />
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="ToNumber" />
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Digits" />
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Destination" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="10720" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<NestedLoops Optimized="false">
<OuterReferences>
<ColumnReference Column="Expr1005" />
</OuterReferences>
<RelOp AvgRowSize="43" EstimateCPU="0.001072" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="10720" LogicalOp="Compute Scalar" NodeId="1" Parallel="false" PhysicalOp="Compute Scalar" EstimatedTotalSubtreeCost="0.13985">
<OutputList>
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="FromNumber" />
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="ToNumber" />
<ColumnReference Column="Expr1005" />
</OutputList>
<ComputeScalar>
<DefinedValues>
<DefinedValue>
<ColumnReference Column="Expr1005" />
<ScalarOperator ScalarString="substring([CASH].[dbo].[Converting].[FromNumber] as [c].[FromNumber],(1),(6))">
<Intrinsic FunctionName="substring">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="FromNumber" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="(1)" />
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="(6)" />
</ScalarOperator>
</Intrinsic>
</ScalarOperator>
</DefinedValue>
</DefinedValues>
<RelOp AvgRowSize="38" EstimateCPU="0.011949" EstimateIO="0.126829" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="10720" LogicalOp="Table Scan" NodeId="2" Parallel="false" PhysicalOp="Table Scan" EstimatedTotalSubtreeCost="0.138778">
<OutputList>
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="FromNumber" />
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="ToNumber" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="10720" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<TableScan Ordered="false" ForcedIndex="false" NoExpandHint="false">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="FromNumber" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" Column="ToNumber" />
</DefinedValue>
</DefinedValues>
<Object Database="[CASH]" Schema="[dbo]" Table="[Converting]" Alias="[c]" />
</TableScan>
</RelOp>
</ComputeScalar>
</RelOp>
<RelOp AvgRowSize="48" EstimateCPU="0.000258212" EstimateIO="0.003125" EstimateRebinds="10580.9" EstimateRewinds="138.124" EstimateRows="1" LogicalOp="Lazy Spool" NodeId="6" Parallel="false" PhysicalOp="Index Spool" EstimatedTotalSubtreeCost="14.9998">
<OutputList>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Digits" />
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Destination" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRebinds="830" ActualRewinds="9890" ActualRows="10720" ActualEndOfScans="0" ActualExecutions="10720" />
</RunTimeInformation>
<Spool>
<SeekPredicate>
<Prefix ScanType="EQ">
<RangeColumns>
<ColumnReference Column="Expr1005" />
</RangeColumns>
<RangeExpressions>
<ScalarOperator ScalarString="[Expr1005]">
<Identifier>
<ColumnReference Column="Expr1005" />
</Identifier>
</ScalarOperator>
</RangeExpressions>
</Prefix>
</SeekPredicate>
<RelOp AvgRowSize="48" EstimateCPU="0.0103333" EstimateIO="0" EstimateRebinds="1180" EstimateRewinds="0" EstimateRows="1" LogicalOp="Compute Scalar" NodeId="7" Parallel="false" PhysicalOp="Compute Scalar" EstimatedTotalSubtreeCost="12.2037">
<OutputList>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Digits" />
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Destination" />
</OutputList>
<ComputeScalar>
<DefinedValues>
<DefinedValue>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Digits" />
<ScalarOperator ScalarString="[CASH].[CASH].[dbo].[DestinationLookup].[Digits] as [d].[Digits]">
<Identifier>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Digits" />
</Identifier>
</ScalarOperator>
</DefinedValue>
<DefinedValue>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Destination" />
<ScalarOperator ScalarString="[CASH].[CASH].[dbo].[DestinationLookup].[Destination] as [d].[Destination]">
<Identifier>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Destination" />
</Identifier>
</ScalarOperator>
</DefinedValue>
</DefinedValues>
<RelOp AvgRowSize="48" EstimateCPU="0.0103333" EstimateIO="0" EstimateRebinds="1180" EstimateRewinds="0" EstimateRows="1" LogicalOp="Remote Query" NodeId="8" Parallel="false" PhysicalOp="Remote Query" EstimatedTotalSubtreeCost="12.2037">
<OutputList>
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Digits" />
<ColumnReference Server="[CASH]" Database="[CASH]" Schema="[dbo]" Table="[DestinationLookup]" Alias="[d]" Column="Destination" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRebinds="456" ActualRewinds="0" ActualRows="456" ActualEndOfScans="0" ActualExecutions="456" />
</RunTimeInformation>
<RemoteQuery RemoteSource="CASH" RemoteQuery="SELECT "Tbl1004"."Digits" "Col1015","Tbl1004"."Destination" "Col1016" FROM "CASH"."dbo"."DestinationLookup" "Tbl1004" WHERE "Tbl1004"."Digits"=?" />
</RelOp>
</ComputeScalar>
</RelOp>
</Spool>
</RelOp>
</NestedLoops>
</RelOp>
</QueryPlan>
</StmtSimple>
</Statements>
</Batch>
</BatchSequence>
</ShowPlanXML>
UPDATE: I've been unable to find an ideal solution, but I have discovered a next-best thing. It seems that a trivially simple query on these two tables using "like" completes in about five seconds. So rather than try to cram the join into my monster query, where it never completes, I've used it to create a temporary lookup table, which my monster query then uses. In all, the big query completes in nine seconds now, and I have support for variable length strings in my varchar join.
Another thing that helped speed this up was changing the fill factor for the column in t1 from 80 to 100. This fill factor is ideal for the table because it's a static reference table that only changes once a year.
The most performant solution of those four is the fourth one.
Let's set up the test environment:
create table #t1 (digits varchar(10), filler char(5000) default(''))
create table #t2 (number varchar(10), filler char(5000) default(''))
go
insert #t1 (digits) values
('123'),('234'),('345'),('456'),('567')
insert #t2 (number) values
('1234'),('234'),('345689'),('45'),('567890')
go
create index ix_t2 on #t2(number);
go
Now, let's execute four semantically identical queries, but with Query --> Include Actual Execution Plan enabled, and also with SET STATISTICS IO ON:
-- 1
select *
from #t1
inner join #t2
on #t1.digits = left(#t2.number, datalength(#t1.digits))
-- 2
select *
from #t1
inner join #t2
on charindex(#t1.Digits, #t2.Number) = 1
-- 3
select *
from #t1
inner join #t2
on charindex(#t1.digits, #t2.number) = 1
-- 4
select *
from #t1
inner join #t2
on #t2.number like #t1.digits + '%'
As you can see, execution plans for 1, 2 & 3 include the table scan operators on both tables (including an additional Compute Scalar operator for the first one), but the fourth query does index seek on our index on #t2. Moreover, if you check the output of statistics io, you'll see the logical reads measure on #t2 (the table with the index) for the 1, 2 & 3 as 25, but for the fourth one as only 14 (of course, with more rows, the numbers go high up).
Build an index on table1.digits. Then try the following:
select t2.*, t1.<whatever>
from table1 t2 cross apply
(select top 1 <whatever>
from table1 t1
where t1.digits <= t2.number
order by t1.digits desc
) t1;
SQL Server is sometimes better at optimizing "apply" queries than regular joins. In this case, it might spot that the index is useful both for the where and order by and proceed efficiently. (I also think the same would apply to a correlated subquery.)
Could anyone advice on how do I improve the performance of the following query. Note, the problem seems to be caused by where clause.
Data (table contains a huge set of rows - 500K+, the set of parameters it's called with assums the return of 2-5K records per query, which takes 8-10 minutes currently):
USE [SomeDb]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[Data](
[x] [money] NOT NULL,
[y] [money] NOT NULL,
CONSTRAINT [PK_Data] PRIMARY KEY CLUSTERED
(
[x] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
The Query
select top 10000
s.x as sx,
e.x as ex,
s.y as sy,
e.y as ey,
e.y - s.y as y_delta,
e.x - s.x as x_delta
from Data s
inner join Data e
on e.x > s.x and e.x - s.x between xFrom and xTo
--where e.y - s.y > #yDelta -- when uncommented causes a huge delay
Update 1 - Execution Plan
<?xml version="1.0" encoding="utf-16"?>
<ShowPlanXML xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Version="1.2" Build="11.0.2100.60" xmlns="http://schemas.microsoft.com/sqlserver/2004/07/showplan">
<BatchSequence>
<Batch>
<Statements>
<StmtSimple StatementCompId="1" StatementEstRows="100" StatementId="1" StatementOptmLevel="FULL" StatementOptmEarlyAbortReason="GoodEnoughPlanFound" StatementSubTreeCost="0.0263655" StatementText="select top 100
s.x as sx,
e.x as ex,
s.y as sy,
e.y as ey,
e.y - s.y as y_delta,
e.x - s.x as x_delta
from Data s
inner join Data e
on e.x > s.x and e.x - s.x between 100 and 105
where e.y - s.y > 0.01
" StatementType="SELECT" QueryHash="0xAAAC02AC2D78CB56" QueryPlanHash="0x747994153CB2D637" RetrievedFromCache="true">
<StatementSetOptions ANSI_NULLS="true" ANSI_PADDING="true" ANSI_WARNINGS="true" ARITHABORT="true" CONCAT_NULL_YIELDS_NULL="true" NUMERIC_ROUNDABORT="false" QUOTED_IDENTIFIER="true" />
<QueryPlan DegreeOfParallelism="0" NonParallelPlanReason="NoParallelPlansInDesktopOrExpressEdition" CachedPlanSize="24" CompileTime="13" CompileCPU="13" CompileMemory="424">
<MemoryGrantInfo SerialRequiredMemory="0" SerialDesiredMemory="0" />
<OptimizerHardwareDependentProperties EstimatedAvailableMemoryGrant="52199" EstimatedPagesCached="14561" EstimatedAvailableDegreeOfParallelism="4" />
<RelOp AvgRowSize="55" EstimateCPU="1E-05" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="100" LogicalOp="Compute Scalar" NodeId="0" Parallel="false" PhysicalOp="Compute Scalar" EstimatedTotalSubtreeCost="0.0263655">
<OutputList>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="x" />
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="y" />
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="x" />
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="y" />
<ColumnReference Column="Expr1004" />
<ColumnReference Column="Expr1005" />
</OutputList>
<ComputeScalar>
<DefinedValues>
<DefinedValue>
<ColumnReference Column="Expr1004" />
<ScalarOperator ScalarString="[SomeDb].[dbo].[Data].[y] as [e].[y]-[SomeDb].[dbo].[Data].[y] as [s].[y]">
<Arithmetic Operation="SUB">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="y" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="y" />
</Identifier>
</ScalarOperator>
</Arithmetic>
</ScalarOperator>
</DefinedValue>
<DefinedValue>
<ColumnReference Column="Expr1005" />
<ScalarOperator ScalarString="[SomeDb].[dbo].[Data].[x] as [e].[x]-[SomeDb].[dbo].[Data].[x] as [s].[x]">
<Arithmetic Operation="SUB">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="x" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="x" />
</Identifier>
</ScalarOperator>
</Arithmetic>
</ScalarOperator>
</DefinedValue>
</DefinedValues>
<RelOp AvgRowSize="39" EstimateCPU="1E-05" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="100" LogicalOp="Top" NodeId="1" Parallel="false" PhysicalOp="Top" EstimatedTotalSubtreeCost="0.0263555">
<OutputList>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="x" />
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="y" />
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="x" />
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="y" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="100" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<Top RowCount="false" IsPercent="false" WithTies="false">
<TopExpression>
<ScalarOperator ScalarString="(100)">
<Const ConstValue="(100)" />
</ScalarOperator>
</TopExpression>
<RelOp AvgRowSize="39" EstimateCPU="151828" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="100" LogicalOp="Inner Join" NodeId="2" Parallel="false" PhysicalOp="Nested Loops" EstimatedTotalSubtreeCost="0.0263455">
<OutputList>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="x" />
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="y" />
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="x" />
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="y" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="100" ActualEndOfScans="0" ActualExecutions="1" />
</RunTimeInformation>
<NestedLoops Optimized="false">
<OuterReferences>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="x" />
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="y" />
</OuterReferences>
<RelOp AvgRowSize="23" EstimateCPU="1.80448" EstimateIO="3.76461" EstimateRebinds="0" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="1" LogicalOp="Clustered Index Scan" NodeId="3" Parallel="false" PhysicalOp="Clustered Index Scan" EstimatedTotalSubtreeCost="0.0032831" TableCardinality="1640290">
<OutputList>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="x" />
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="y" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="15225" ActualEndOfScans="0" ActualExecutions="1" />
</RunTimeInformation>
<IndexScan Ordered="false" ForcedIndex="false" ForceScan="false" NoExpandHint="false">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="x" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="y" />
</DefinedValue>
</DefinedValues>
<Object Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Index="[PK_Data]" Alias="[e]" IndexKind="Clustered" />
</IndexScan>
</RelOp>
<RelOp AvgRowSize="23" EstimateCPU="0.902317" EstimateIO="1.88387" EstimateRebinds="1" EstimateRewinds="0" EstimatedExecutionMode="Row" EstimateRows="100" LogicalOp="Clustered Index Seek" NodeId="4" Parallel="false" PhysicalOp="Clustered Index Seek" EstimatedTotalSubtreeCost="0.0263655" TableCardinality="1640290">
<OutputList>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="x" />
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="y" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="100" ActualEndOfScans="15224" ActualExecutions="15225" />
</RunTimeInformation>
<IndexScan Ordered="true" ScanDirection="FORWARD" ForcedIndex="false" ForceSeek="false" ForceScan="false" NoExpandHint="false" Storage="RowStore">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="x" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="y" />
</DefinedValue>
</DefinedValues>
<Object Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Index="[PK_Data]" Alias="[s]" IndexKind="Clustered" />
<SeekPredicates>
<SeekPredicateNew>
<SeekKeys>
<EndRange ScanType="LT">
<RangeColumns>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="x" />
</RangeColumns>
<RangeExpressions>
<ScalarOperator ScalarString="[SomeDb].[dbo].[Data].[x] as [e].[x]">
<Identifier>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="x" />
</Identifier>
</ScalarOperator>
</RangeExpressions>
</EndRange>
</SeekKeys>
</SeekPredicateNew>
</SeekPredicates>
<Predicate>
<ScalarOperator ScalarString="([SomeDb].[dbo].[Data].[x] as [e].[x]-[SomeDb].[dbo].[Data].[x] as [s].[x])>=($100.0000) AND ([SomeDb].[dbo].[Data].[x] as [e].[x]-[SomeDb].[dbo].[Data].[x] as [s].[x])<=($105.0000) AND ([SomeDb].[dbo].[Data].[y] as [e].[y]-[SomeDb].[dbo].[Data].[y] as [s].[y])>(0.01)">
<Logical Operation="AND">
<ScalarOperator>
<Compare CompareOp="GE">
<ScalarOperator>
<Arithmetic Operation="SUB">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="x" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="x" />
</Identifier>
</ScalarOperator>
</Arithmetic>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="($100.0000)" />
</ScalarOperator>
</Compare>
</ScalarOperator>
<ScalarOperator>
<Compare CompareOp="LE">
<ScalarOperator>
<Arithmetic Operation="SUB">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="x" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="x" />
</Identifier>
</ScalarOperator>
</Arithmetic>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="($105.0000)" />
</ScalarOperator>
</Compare>
</ScalarOperator>
<ScalarOperator>
<Compare CompareOp="GT">
<ScalarOperator>
<Arithmetic Operation="SUB">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[e]" Column="y" />
</Identifier>
</ScalarOperator>
<ScalarOperator>
<Identifier>
<ColumnReference Database="[SomeDb]" Schema="[dbo]" Table="[Data]" Alias="[s]" Column="y" />
</Identifier>
</ScalarOperator>
</Arithmetic>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="(0.01)" />
</ScalarOperator>
</Compare>
</ScalarOperator>
</Logical>
</ScalarOperator>
</Predicate>
</IndexScan>
</RelOp>
</NestedLoops>
</RelOp>
</Top>
</RelOp>
</ComputeScalar>
</RelOp>
</QueryPlan>
</StmtSimple>
</Statements>
</Batch>
</BatchSequence>
</ShowPlanXML>
I have often seen big performance gains by inserting the results of the first query (in your case without the where clause) into a TEMP table or a table variable, and selecting from this afterwards (which basically helps the query optimiser to select an appropriate execution plan).
Also just noticed you don't have an INDEX on column Y, which may speed up a bit.
EDIT
Also, try the following (gives me slightly better performance):
SELECT * FROM
(SELECT
s.x as sx,
e.x as ex,
s.y as sy,
e.y as ey,
e.y - s.y as y_delta,
e.x - s.x as x_delta
FROM Data s
JOIN Data e
ON e.x > s.x
) data
WHERE data.y_delta > #yDelta AND data.x_delta BETWEEN #xFrom AND #xTo
The main issues are that the where clause gives a cross join (not what I would call a simple join) (giving many rows is that the join compares many rows in e for a row in s) and also the .y comparison has to use a table scan (over at least temporary data if not the whole table).
If the query is common then there is a possible fix in doing the join once and copying the data into a pre calculated table and indexing the differences.
I can think of an odd strategy to improve performance. This involves adding an auto-incrementing id into the table, then finding the bounds for Xfrom and Xto in terms of the id, and then looking for anything in the range.
The following query suggests what I mean:
with e1 as (
select e.*,
(select min(id) from data s where e.x between s.x + #xfrom and s.x + #xto) as idstart,
(select max(id) from data s where e.x between s.x + #xfrom and s.x + #xto) as idend
from data e
)
select <whatever>
from e1 join
e1 s
on e1.idstart = s.id
where e1.idstart <= e1.idend union all
select <whatever>
from e1 join
e1 s
on e1.idstart+1 = s.id
where e1.idstart+1 <= e1.idend
. . .
I have had good luck with the correlated subquery on an indexed field returning the next value. After that, the joins are equijoins, which should be very fast. In the end, you would want to modify the query so it can do the comparisons using a table of enumerated values.
I have an index that contains ConChainNum, Control, and TransDate, and I just rebuilt the indexes. The table I'm working with has over 100 million records and the following query returns no results:
select * from sbt_pos_sales
where conchainnum = 810 and
control = 852013688 and
transdate = 20120712
Given that I have an index made specifically with the three fields in the where clause, I really expected an almost immediate result, but it took about half a minute to tell me there where no results.
Can someone explain to me why this index doesn't make it any faster than that, and if I can somehow do it differently to make it faster?
Execution Plan:
<?xml version="1.0" encoding="utf-16"?>
<ShowPlanXML xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" Version="1.1.1" Build="10.0.5500.0" xmlns="http://schemas.microsoft.com/sqlserver/2004/07/showplan">
<BatchSequence>
<Batch>
<Statements>
<StmtSimple StatementCompId="1" StatementEstRows="20.2265" StatementId="1" StatementOptmLevel="FULL" StatementSubTreeCost="348.687" StatementText="SELECT * FROM [ngdwdb]..[sbt_pos_sales] WHERE [conchainnum]=#1 AND [control]=#2 AND [transdate]=#3" StatementType="SELECT" QueryHash="0x11B117D9A8C6DF58" QueryPlanHash="0xBA3EFC04623601F6">
<StatementSetOptions ANSI_NULLS="true" ANSI_PADDING="true" ANSI_WARNINGS="true" ARITHABORT="true" CONCAT_NULL_YIELDS_NULL="true" NUMERIC_ROUNDABORT="false" QUOTED_IDENTIFIER="true" />
<QueryPlan DegreeOfParallelism="8" MemoryGrant="5504" CachedPlanSize="48" CompileTime="95" CompileCPU="4" CompileMemory="368">
<RelOp AvgRowSize="193" EstimateCPU="0.0655713" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="7330.87" LogicalOp="Gather Streams" NodeId="1" Parallel="true" PhysicalOp="Parallelism" EstimatedTotalSubtreeCost="348.686">
<OutputList>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="RowIdent" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="ConChainNum" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Control" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="TransDate" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="ControlGS" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="UPC_UCP" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="DateSold" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Price_UCP" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Store" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="QtySold" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="FileName" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="IssueCode" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="DealerNum" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="0" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
</RunTimeInformation>
<Parallelism>
<RelOp AvgRowSize="193" EstimateCPU="0.00766076" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="7330.87" LogicalOp="Inner Join" NodeId="2" Parallel="true" PhysicalOp="Nested Loops" EstimatedTotalSubtreeCost="348.621">
<OutputList>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="RowIdent" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="ConChainNum" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Control" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="TransDate" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="ControlGS" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="UPC_UCP" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="DateSold" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Price_UCP" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Store" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="QtySold" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="FileName" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="IssueCode" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="DealerNum" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="6" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="2" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="3" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="7" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="5" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="4" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="8" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="1" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="0" ActualRows="0" ActualEndOfScans="0" ActualExecutions="0" />
</RunTimeInformation>
<NestedLoops Optimized="true" WithUnorderedPrefetch="true">
<OuterReferences>
<ColumnReference Column="Bmk1000" />
<ColumnReference Column="Expr1009" />
</OuterReferences>
<RelOp AvgRowSize="58" EstimateCPU="14.9196" EstimateIO="0" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="7330.87" LogicalOp="Compute Scalar" NodeId="5" Parallel="true" PhysicalOp="Compute Scalar" EstimatedTotalSubtreeCost="324.579">
<OutputList>
<ColumnReference Column="Bmk1000" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="ConChainNum" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Control" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="TransDate" />
<ColumnReference Column="Expr1008" />
</OutputList>
<ComputeScalar>
<DefinedValues>
<DefinedValue>
<ColumnReference Column="Expr1008" />
<ScalarOperator ScalarString="BmkToPage([Bmk1000])">
<Intrinsic FunctionName="BmkToPage">
<ScalarOperator>
<Identifier>
<ColumnReference Column="Bmk1000" />
</Identifier>
</ScalarOperator>
</Intrinsic>
</ScalarOperator>
</DefinedValue>
</DefinedValues>
<RelOp AvgRowSize="58" EstimateCPU="15.196" EstimateIO="294.463" EstimateRebinds="0" EstimateRewinds="0" EstimateRows="7330.87" LogicalOp="Index Seek" NodeId="6" Parallel="true" PhysicalOp="Index Seek" EstimatedTotalSubtreeCost="309.659" TableCardinality="107302000">
<OutputList>
<ColumnReference Column="Bmk1000" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="ConChainNum" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Control" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="TransDate" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="6" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="3" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="2" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="7" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="8" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="5" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="1" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="4" ActualRows="0" ActualEndOfScans="1" ActualExecutions="1" />
<RunTimeCountersPerThread Thread="0" ActualRows="0" ActualEndOfScans="0" ActualExecutions="0" />
</RunTimeInformation>
<IndexScan Ordered="true" ScanDirection="FORWARD" ForcedIndex="false" ForceSeek="false" NoExpandHint="false">
<DefinedValues>
<DefinedValue>
<ColumnReference Column="Bmk1000" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="ConChainNum" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Control" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="TransDate" />
</DefinedValue>
</DefinedValues>
<Object Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Index="[idxLoad]" IndexKind="NonClustered" />
<SeekPredicates>
<SeekPredicateNew>
<SeekKeys>
<Prefix ScanType="EQ">
<RangeColumns>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="ConChainNum" />
</RangeColumns>
<RangeExpressions>
<ScalarOperator ScalarString="(810)">
<Const ConstValue="(810)" />
</ScalarOperator>
</RangeExpressions>
</Prefix>
</SeekKeys>
</SeekPredicateNew>
</SeekPredicates>
<Predicate>
<ScalarOperator ScalarString="CONVERT_IMPLICIT(int,[NGDWDB].[dbo].[SBT_POS_SALES].[Control],0)=(852013688) AND CONVERT_IMPLICIT(int,[NGDWDB].[dbo].[SBT_POS_SALES].[TransDate],0)=(20120712)">
<Logical Operation="AND">
<ScalarOperator>
<Compare CompareOp="EQ">
<ScalarOperator>
<Convert DataType="int" Style="0" Implicit="true">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Control" />
</Identifier>
</ScalarOperator>
</Convert>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="(852013688)" />
</ScalarOperator>
</Compare>
</ScalarOperator>
<ScalarOperator>
<Compare CompareOp="EQ">
<ScalarOperator>
<Convert DataType="int" Style="0" Implicit="true">
<ScalarOperator>
<Identifier>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="TransDate" />
</Identifier>
</ScalarOperator>
</Convert>
</ScalarOperator>
<ScalarOperator>
<Const ConstValue="(20120712)" />
</ScalarOperator>
</Compare>
</ScalarOperator>
</Logical>
</ScalarOperator>
</Predicate>
</IndexScan>
</RelOp>
</ComputeScalar>
</RelOp>
<RelOp AvgRowSize="136" EstimateCPU="0.0001581" EstimateIO="0.003125" EstimateRebinds="7329.87" EstimateRewinds="0" EstimateRows="1" LogicalOp="RID Lookup" NodeId="11" Parallel="true" PhysicalOp="RID Lookup" EstimatedTotalSubtreeCost="24.0341" TableCardinality="107302000">
<OutputList>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="RowIdent" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="ControlGS" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="UPC_UCP" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="DateSold" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Price_UCP" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Store" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="QtySold" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="FileName" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="IssueCode" />
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="DealerNum" />
</OutputList>
<RunTimeInformation>
<RunTimeCountersPerThread Thread="6" ActualRows="0" ActualEndOfScans="0" ActualExecutions="0" />
<RunTimeCountersPerThread Thread="2" ActualRows="0" ActualEndOfScans="0" ActualExecutions="0" />
<RunTimeCountersPerThread Thread="3" ActualRows="0" ActualEndOfScans="0" ActualExecutions="0" />
<RunTimeCountersPerThread Thread="7" ActualRows="0" ActualEndOfScans="0" ActualExecutions="0" />
<RunTimeCountersPerThread Thread="5" ActualRows="0" ActualEndOfScans="0" ActualExecutions="0" />
<RunTimeCountersPerThread Thread="4" ActualRows="0" ActualEndOfScans="0" ActualExecutions="0" />
<RunTimeCountersPerThread Thread="8" ActualRows="0" ActualEndOfScans="0" ActualExecutions="0" />
<RunTimeCountersPerThread Thread="1" ActualRows="0" ActualEndOfScans="0" ActualExecutions="0" />
<RunTimeCountersPerThread Thread="0" ActualRows="0" ActualEndOfScans="0" ActualExecutions="0" />
</RunTimeInformation>
<IndexScan Lookup="true" Ordered="true" ScanDirection="FORWARD" ForcedIndex="false" ForceSeek="false" NoExpandHint="false">
<DefinedValues>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="RowIdent" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="ControlGS" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="UPC_UCP" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="DateSold" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Price_UCP" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="Store" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="QtySold" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="FileName" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="IssueCode" />
</DefinedValue>
<DefinedValue>
<ColumnReference Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" Column="DealerNum" />
</DefinedValue>
</DefinedValues>
<Object Database="[NGDWDB]" Schema="[dbo]" Table="[SBT_POS_SALES]" TableReferenceId="-1" IndexKind="Heap" />
<SeekPredicates>
<SeekPredicateNew>
<SeekKeys>
<Prefix ScanType="EQ">
<RangeColumns>
<ColumnReference Column="Bmk1000" />
</RangeColumns>
<RangeExpressions>
<ScalarOperator ScalarString="[Bmk1000]">
<Identifier>
<ColumnReference Column="Bmk1000" />
</Identifier>
</ScalarOperator>
</RangeExpressions>
</Prefix>
</SeekKeys>
</SeekPredicateNew>
</SeekPredicates>
</IndexScan>
</RelOp>
</NestedLoops>
</RelOp>
</Parallelism>
</RelOp>
<ParameterList>
<ColumnReference Column="#3" ParameterCompiledValue="(20120712)" ParameterRuntimeValue="(20120712)" />
<ColumnReference Column="#2" ParameterCompiledValue="(852013688)" ParameterRuntimeValue="(852013688)" />
<ColumnReference Column="#1" ParameterCompiledValue="(810)" ParameterRuntimeValue="(810)" />
</ParameterList>
</QueryPlan>
</StmtSimple>
</Statements>
</Batch>
</BatchSequence>
</ShowPlanXML>
Rather than keep filling up the comments, I'll formalize this as an answer...
Based on your execution plan, I recommend adding a clustered index on the table. It's hard to be 100% sure what's going on, but RID lookups are a problem.
RID lookups are what happens when there's not a clustering key associated with the table. In a nonclustered index, the clustering key gets replicated in the nonclustered index if there is one, speeding up any row access required after the index entries are found. In a heap (a table with a nonclustered index) you don't have that clustering key, so you have to look it up in the heap.
That's why you have an INNER JOIN in the execution plan, even though there's no JOIN in your SQL. Applying clustering to the primary key in the table should eliminate this join, and leave it with a simple index seek.
Although the current plan shows it burning 89% of your time right now, I'll bet that with a clustered index on the table you'll simplify it down to a simple (and much faster) index seek.
EDIT:
Let me amend this after looking at your execution plan XML. Although the clustering key should still help, the real problem is the data types in your indexes. On the index seek I'm showing
CONVERT_IMPLICIT(int,[NGDWDB].[dbo].[SBT_POS_SALES].[Control],0)=(852013688) AND CONVERT_IMPLICIT(int,[NGDWDB].[dbo].[SBT_POS_SALES].[TransDate],0)=(20120712)
, which is telling me that Control and TransDate are not the same data types as you are providing in your query. If you fix this up such that no conversions are required, you will very likely see a huge speedup in the query.