Filestream column in the WHERE clause locks the server - sql

I have implemented filestream in an existing database on SQL Server 2008 r2.
Now I have a very urgent problem as my site is practically down now:
With a very simple table like this:
CREATE TABLE [dbo].[Table1](
[Id] [int] IDENTITY(1,1),
[rowguid] [uniqueidentifier] ROWGUIDCOL NOT NULL,
[Image] [varbinary](max) FILESTREAM NULL,
CONSTRAINT [PK_Table1] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF,
ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY],
CONSTRAINT [Table1RowguidUnique] UNIQUE NONCLUSTERED
(
[rowguid] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF,
ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
ALTER TABLE [dbo].[Table1] ADD CONSTRAINT [DF_table1_rowguid] DEFAULT (newid()) FOR [rowguid]
GO
ALTER TABLE dbo.Table1
SET ( FILESTREAM_ON = fsfg_LiveWebsite )
GO
If I run:
select * from Table1 where Id = 1
it runs very quickly and give the correct result.
If I run anything with the "Varbinary(max) FILESTREAM" field in the where clause the whole table locks down.
So for example any of those 2 queries:
select Id from Table1 where Id = 1 and [Image] is null
select Id from Table1 where Id = 1 and [Image] = convert(varbinary(max), 'a')
What could this be?
Please reply asap with any suggestion!
Thank you

First and foremost if you want to query a VARBINARY column you'll need to enable and use Full-Text Search.
Reference Articles.
http://technet.microsoft.com/en-us/library/ms142531.aspx (this article will give you an overview of Full-Text Search and the VARBINARY column)
http://technet.microsoft.com/en-us/library/ms187787.aspx (this article will show you how to use the CONTAINS T-SQL command to query the field)

Related

Violation of primary key after switch partition (stage>dbo)

Would someone help, please, to get rid of the error:
Violation of PRIMARY KEY constraint 'PK_stmp_tst1'. Cannot insert duplicate key in object 'dbo.stmp_tst'. The duplicate key value is (1).
which occurs after switch partition of the table.
Full SQL Script below:
I. Create 2 equal tables in different schemas:
CREATE TABLE dbo.stmp_tst(
[inn] [varchar](20) NULL,
[id] [bigint] IDENTITY(1,1) NOT NULL,
CONSTRAINT [PK_stmp_tst] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
go
CREATE TABLE stage.stmp_tst(
[inn] [varchar](20) NULL,
[id] [bigint] IDENTITY(1,1) NOT NULL,
CONSTRAINT [PK_stmp_tst] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
II. Insert data into stage table.
insert into stage.stmp_tst (inn)
select '1111'
III. Switch-partition stage to dbo.
alter table stage.stmp_tst switch partition 1 to dbo.stmp_tst partition 1;
IV. Add data to dbo table.
insert into dbo.stmp_tst (inn)
select '1111'
V. We have the error:
Violation of PRIMARY KEY constraint 'PK_stmp_tst1'. Cannot insert
duplicate key in object 'dbo.stmp_tst'. The duplicate key value is
(1).
IV. Drop temporary tables:
drop table dbo.stmp_tst
drop table stage.stmp_tst
It can be solved by the query:
DBCC CHECKIDENT ('dbo.stmp_tst', RESEED);
but reseeding takes time.
Is it possible to do a switch-partition correctly without reseed?
Thank you.

TRUNCATE Partitions from table failed (SQL Server)

I have a table Log:
CREATE TABLE [dbo].[Log]
(
[Id] [INT] IDENTITY(1,1) NOT NULL,
[Date] [DATETIME] NOT NULL,
[Thread] [VARCHAR](255) NOT NULL,
[Level] [VARCHAR](50) NOT NULL,
[Logger] [VARCHAR](255) NOT NULL,
[Message] [VARCHAR](4000) NOT NULL,
[Exception] [VARCHAR](2000) NULL,
CONSTRAINT [PK_Log]
PRIMARY KEY NONCLUSTERED ([Id] ASC)
)
The PK is Id, and we partitioned column [Date] after create an index on it and change PK to non-clustered:
ALTER TABLE [dbo].[Log]
ADD CONSTRAINT [PK_Log]
PRIMARY KEY NONCLUSTERED ([Id] ASC) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
CREATE CLUSTERED INDEX [IX_Log_Date]
ON [dbo].[Log]([Date] ASC) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
The partitions are created successfully.
Now, we want to use Truncate to remove partitions:
TRUNCATE TABLE [dbo].[Log]
WITH (PARTITIONS (1 TO 2));
But get this error:
TRUNCATE TABLE statement failed. Index 'PK_Log' is not partitioned, but table 'Log' uses partition function 'myDateRangePF'. Index and table must use an equivalent partition function.
Does this mean partitioned table can only have one index? if the existing table has multiple index, in order to truncate it, we have to remove all indexes first?
Thanks
The issue is that you created the index PK_Log...ON [PRIMARY], which made it a non-partitioned index on a partitioned table. You'll need to drop that index (and any other non-partitioned indexes, probably) and recreate it. Either specify the partitioning filegroup explicitly, or leave the ON clause out and let SQL Server pick the filegroup. By default, it will create the index on the same filegroup as the underlying table and with the same partitioning as the table.
See Partitioned Indexes in BOL for additional information.

Filtered Index expression does not work on Bit

I want a bool constraint on a certain value for column IsDefaultLanguage:
Id...ISO639_ISO3166...ApplicationId...IsDefaultLanguage
1....de-de............1...............1
2....fr-fr............1...............1
The second datarow insertion should result in a unique error because IsDefaultLanguage should only be allowed ONE for True (1) for an application.
Also should a language be available only ONE for an application.
The Filter index does not work on my side (Sql Server 2014)
What do I wrong?
TABLE
CREATE TABLE [dbo].[Languages](
[Id] [int] IDENTITY(1,1) NOT NULL,
[ISO639_ISO3166] [char](5) NOT NULL,
[ApplicationId] [int] NOT NULL,
[IsDefaultLanguage] [bit] NOT NULL,
CONSTRAINT [PK_dbo.Languages] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
GO
ALTER TABLE [dbo].[Languages] WITH CHECK ADD CONSTRAINT [FK_dbo.Languages_dbo.Applications_ApplicationId] FOREIGN KEY([ApplicationId])
REFERENCES [dbo].[Applications] ([Id])
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Languages] CHECK CONSTRAINT [FK_dbo.Languages_dbo.Applications_ApplicationId]
GO
INDEX
CREATE UNIQUE NONCLUSTERED INDEX [IX_IsoCodeApplicationId] ON [dbo].[Languages]
(
[ISO639_ISO3166] ASC,
[ApplicationId] ASC,
[IsDefaultLanguage] ASC
)
WHERE ([IsDefaultLanguage]=(1))
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
GO
The second datarow insertion should result in a unique error because IsDefaultLanguage should only be allowed ONE for True (1) for an application. Also should a language be available only ONE for an application.
The above is not very clear to me. Here is what I think you are trying to say:
IsDefaultLanguage can only have the value 1 once per ApplicationId.
The value for ISO639_ISO3166 must be unique per ApplicationId
If that's the case, split your index in 2 separate indexes to take care of your 2 different requirements:
CREATE UNIQUE NONCLUSTERED INDEX [IX_IsoCodeApplicationId] ON [dbo].[Languages]
(
[ISO639_ISO3166] ASC,
[ApplicationId] ASC
)
CREATE UNIQUE NONCLUSTERED INDEX [IX_DefaultLanguageApplicationId] ON [dbo].[Languages]
(
[IsDefaultLanguage] ASC,
[ApplicationId] ASC,
)
WHERE ([IsDefaultLanguage]=(1))
Your index is defined incorrectly for what you're trying to do. Let's look at it another way. You'd expect that the following query would return 0 rows:
select [ApplicationID], count(*)
from [dbo].[Languages]
where [IsDefaultLanguage] = 1
group by [ApplicationID]
having count(*) > 1
That is, per ApplicationID, there is only one row with a value of IsDefaultLanguage = 1. So, your index should be:
CREATE UNIQUE NONCLUSTERED INDEX [IX_IsoCodeApplicationId]
ON [dbo].[Languages]
(
[ApplicationId] ASC
)
WHERE ([IsDefaultLanguage]=(1))
Completing the analogy with the query, there will (read: can) only ever be one row per ApplicationID in the index (because it's a unique index).

EF 6 Bridge Table Insert Not Working

I am still trying to learn how to use EF and running into a problem with my bridge table.
When I try to create a new Order with associated Resources, I get the following SQL error:
{"Violation of PRIMARY KEY constraint 'PK_Resource_Type'. Cannot insert duplicate key in object 'dbo.Resource_Type'. The duplicate key value is (2).\r\nThe statement has been terminated."}
Code Looks like this
ResourceType resource = new ResourceType();
resource.ID = 2;
resource.Name = "Van"
order.resourceType().Add(resource)
db.Orders.Add(order);
db.SaveChanges();
Tables look like this
--Order table
CREATE TABLE [dbo].[Orders](
[Order_ID] [int] IDENTITY(1,1) NOT NULL,
[OrdernName] [varchar](100) NOT NULL,
CONSTRAINT [PK_Orders] PRIMARY KEY CLUSTERED
(
[Order_ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
--Resource Type table
CREATE TABLE [dbo].[Resource_Type](
[ResourceType_ID] [int] NOT NULL,
[ResourceType] [varchar] (30) NOT NULL
CONSTRAINT [PK_Resource_Type] PRIMARY KEY CLUSTERED
(
[ResourceType_ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
--Resource Type To Order table
CREATE TABLE [dbo].[Resource_Type_Order](
[ResourceType_ID] [int] NOT NULL,
[Order_ID] [int] NOT NULL
CONSTRAINT [PK_Resource_Type_Order] PRIMARY KEY CLUSTERED
(
[ResourceType_ID] ASC,
[Order_ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[Resource_Type_Order] WITH CHECK ADD CONSTRAINT FK_Order_Resource_Type_Order FOREIGN KEY([Order_ID])
REFERENCES [dbo].[Orders]([Order_ID])
GO
ALTER TABLE [dbo].[Resource_Type_Order] WITH CHECK ADD CONSTRAINT FK_Resource_Type_Order_Resource_Type FOREIGN KEY([ReosurceType_ID])
REFERENCES [dbo].[Resource_Type]([Resource_ID])
GO
Am I using the bridge table correctly? I want my bridge table to look like this, after I add an order (ID=1) with two resources (car ID=1 and van ID=2).

SQL Table Performance - Large Table

I have a very large table ~55,000,000 records.
Indexes have been added to the most commonly used columns, but the table is still very slow.
Are there any suggestions as to how the tables performance could be improved?
I have thought about partitioning the table, but was not sure it was necessary.
--Table
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[EngineRecord](
[Id] [uniqueidentifier] NOT NULL,
[CreateDate] [datetime] NOT NULL,
[ChangeDate] [datetime] NOT NULL,
[CompanyId] [uniqueidentifier] NOT NULL,
[DriverEmployeeId] [uniqueidentifier] NOT NULL,
[EobrDeviceId] [uniqueidentifier] NOT NULL,
[EobrTimestampUtc] [datetime] NOT NULL,
[EobrOverallStatus] [int] NOT NULL,
[Speedometer] [decimal](14, 4) NOT NULL,
[Odometer] [decimal](14, 4) NOT NULL,
[Tachometer] [decimal](14, 4) NOT NULL,
[GpsTimestampUtc] [datetime] NULL,
[GpsLatitude] [decimal](18, 8) NULL,
[GPSLongitude] [decimal](18, 8) NULL,
[RecordType] [int] NOT NULL,
[FuelEconomyAverage] [decimal](8, 4) NOT NULL,
[FuelEconomyInstant] [decimal](8, 4) NOT NULL,
[FuelUseTotal] [decimal](14, 4) NOT NULL,
[BrakePressure] [decimal](8, 4) NOT NULL,
[CruiseControlSet] [bit] NOT NULL,
[TransmissionAttained] [nvarchar](2) NULL,
[TransmissionSelected] [nvarchar](2) NULL,
[IsProcessed] [bit] NOT NULL,
[LastChangedByUserId] [uniqueidentifier] NOT NULL,
CONSTRAINT [PK_EngineRecord] PRIMARY KEY NONCLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = ON, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY],
CONSTRAINT [NK_EngineRecord] UNIQUE CLUSTERED
(
[CompanyId] ASC,
[EobrDeviceId] ASC,
[EobrTimestampUtc] ASC
)WITH (PAD_INDEX = ON, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[EngineRecord] WITH NOCHECK ADD CONSTRAINT [FK_EngineRecord_CompanyLevel] FOREIGN KEY([CompanyId])
REFERENCES [dbo].[CompanyLevel] ([Id])
GO
ALTER TABLE [dbo].[EngineRecord] CHECK CONSTRAINT [FK_EngineRecord_CompanyLevel]
GO
ALTER TABLE [dbo].[EngineRecord] WITH NOCHECK ADD CONSTRAINT [FK_EngineRecord_Employee] FOREIGN KEY([DriverEmployeeId])
REFERENCES [dbo].[Employee] ([Id])
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[EngineRecord] CHECK CONSTRAINT [FK_EngineRecord_Employee]
GO
ALTER TABLE [dbo].[EngineRecord] WITH NOCHECK ADD CONSTRAINT [FK_EngineRecord_EobrDevice] FOREIGN KEY([EobrDeviceId])
REFERENCES [dbo].[EobrDevice] ([Id])
GO
ALTER TABLE [dbo].[EngineRecord] CHECK CONSTRAINT [FK_EngineRecord_EobrDevice]
GO
---------------------
--Indexes/Constraints
---------------------
ALTER TABLE [dbo].[EngineRecord] ADD CONSTRAINT [PK_EngineRecord] PRIMARY KEY NONCLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = ON, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [NC_EngineRecord_Employee] ON [dbo].[EngineRecord]
(
[DriverEmployeeId] ASC
)WITH (PAD_INDEX = ON, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [NC_RecordType] ON [dbo].[EngineRecord]
(
[RecordType] ASC
)WITH (PAD_INDEX = ON, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
GO
ALTER TABLE [dbo].[EngineRecord] ADD CONSTRAINT [NK_EngineRecord] UNIQUE CLUSTERED
(
[CompanyId] ASC,
[EobrDeviceId] ASC,
[EobrTimestampUtc] ASC
)WITH (PAD_INDEX = ON, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [IX_EngineRecord_DBA] ON [dbo].[EngineRecord]
(
[CompanyId] ASC,
[GpsLatitude] ASC,
[GPSLongitude] ASC
)
INCLUDE ( [EobrDeviceId],
[EobrTimestampUtc]) WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 90) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX [NC_IsProcessed] ON [dbo].[EngineRecord]
(
[IsProcessed] ASC
)WITH (PAD_INDEX = ON, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
GO
EDIT:
Here is a sproc that takes some time to run that is used often.
CREATE PROCEDURE [dbo].[EngineRecord__GetEobrListToProcessByRecordType]
#RecordTypeEnum int
AS
DECLARE #ChangeHistory bit -- dummy variable for VS 2008 database project
SET NOCOUNT ON
SELECT EobrDevice.[Id] as EobrDeviceId,
EobrDevice.[UnitId],
CompanyGroupRoot.[Id] as CGRootId,
CompanyGroup.[Id] as CompanyGroupId,
EobrDevice.[CompanyId]
FROM dbo.EobrDevice
INNER JOIN dbo.CompanyLevel ON EobrDevice.[CompanyId] = CompanyLevel.[Id]
INNER JOIN dbo.CompanyGroup ON CompanyLevel.ParentGroupId = CompanyGroup.[Id]
INNER JOIN dbo.CompanyGroupRoot ON CompanyGroup.CGRootId = CompanyGroupRoot.[Id]
WHERE EobrDevice.[Id] IN ( SELECT DISTINCT EngineRecord.EobrDeviceId FROM dbo.EngineRecord WHERE IsProcessed = 0 AND RecordType = #RecordTypeEnum )
AND EobrDevice.UnitId IS NOT NULL
EDIT 2:
This is something we run every night to purge out old records. This always takes a lot of time.
DECLARE #dt6MonthsPrior datetime
SET #dt6MonthsPrior = DATEADD(m, -6, getdate())
SELECT * FROM EngineRecord
WHERE EngineRecord.EobrTimeStampUtc < #dt6MonthsPrior
ORDER BY EobrTimestampUtc ASC
None of the fields in your WHERE criteria are contained in an index. Indexing those fields will help. The efficacy of your other indices is impossible to determine without a more thorough understanding of how the table is used.
If you really wanted this query to fly you could have a clustered index on Odometer and Tachometer, but that's probably not reasonable given the table's other uses.
Update:
Your 2nd stored proc doesn't seem like it should be terribly slow, it does seem like the only thing that would help that is an index on the date.
55 million records isn't that big these days, I'm no expert on partitioning, but I don't think you'd see much if any improvement by partitioning your table, I usually don't bother unless I expect a table to exceed a few hundred million records, but in a production environment there are other benefits of partitioning.
Are you certain hardware is not responsible for the poor performance you're seeing? There are a host of settings/features in SQL Server that affect performance as well.
An index like this might help this specific query:
CREATE INDEX x ON dbo.EngineRecord(Odometer, Tachometer) WHERE FuelUseTotal IS NOT NULL;
This will help most if you stop ordering by the timestamp.
Do you know how to get Execution Plans? You have no index on tach or odo or FuelUse, so your sample query will result in a full table scan. From Sql Management Studio, right click in the query window, select "Include Actual Execution Plan" and then run your query. You will see an output that explains to you the steps SQL server will has to perform to actually run your query. This can be very instructive once you take the time to understand an execution plan.
Also, you might want to investigate covering indexes. These can be a dramatic difference if you have some queries that you use frequently. Of course, like any index, there is more overhead when you add/delete
Indexing the fields in the WHERE like Goat CO suggests is a good start, I would also advise to move the WHERE condition to the first INNER JOIN, that way the temporary table created for further handling is already much smaller after the first INNER JOIN (I've seen it do performance wonders)
SELECT EobrDevice.[Id] as EobrDeviceId,
EobrDevice.[UnitId],
CompanyGroupRoot.[Id] as CGRootId,
CompanyGroup.[Id] as CompanyGroupId,
EobrDevice.[CompanyId]
FROM dbo.EobrDevice
INNER JOIN dbo.CompanyLevel
ON EobrDevice.UnitId IS NOT NULL
AND EobrDevice.[CompanyId] = CompanyLevel.[Id]
AND EobrDevice.[Id] IN (
SELECT DISTINCT EngineRecord.EobrDeviceId
FROM dbo.EngineRecord
WHERE IsProcessed = 0
AND RecordType = #RecordTypeEnum
)
INNER JOIN dbo.CompanyGroup ON CompanyLevel.ParentGroupId = CompanyGroup.[Id]
INNER JOIN dbo.CompanyGroupRoot ON CompanyGroup.CGRootId = CompanyGroupRoot.[Id]
I've also moved the EobrDevice.UnitId IS NOT NULL condition to be checked first so that checking with other tables and running the subquery only happens when that condition is met.
PARTITIONNING INDEXES should have an impact in your performance. But they have to be done within appropriate separate drives. You give no information of your hardwares (what are you using, NAS ? SAS Drives ? ... )
Also, Normalization is not always the best choice regarding the objectives of your process, especially for Analytics purpose. Some fields (CompanyLevel, CompanyGroup) denormalized within your main table would have a better impact in your selection -
Well, every master chef have his own Kitchen, so let's skip this discussion....
The built of your indexes do not fit the way you purge your data. You will get a better performance if you decide to change your
[EobrTimestampUtc] ASC
change to
[EobrTimestampUtc] DESC
will impact the seek index for EngineRecord.EobrTimeStampUtc < #dt6MonthsPrior