I have been sniffing out forums and blogs lately, but need some help with a long running query. It's part of a system of stored procedures. This specified statement used to run in about 5 minutes, but lately has been running up to 72 hours!
Here's the setup:
SQL Server 2005 with 28GB memory. Two mountpoints to a SAN with shared disks consisting of 10 spindles. Data is on one mountpoint, Log on another, tempdb on data space. Just one userdatabase on this server.
Here's two tables, condit and condmod. Condit contains 800K records, condmod is initially empty. I issue a truncate mcmain.condmod before the process starts, for testing purposes.
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF__condit__con_notm__000AF8CF]') AND type = 'D')
BEGIN
ALTER TABLE [mcmain].[condit] DROP CONSTRAINT [DF__condit__con_notm__000AF8CF]
END
GO
/****** Object: Table [mcmain].[condit] Script Date: 02/07/2012 11:57:47 ******/
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[mcmain].[condit]') AND type in (N'U'))
DROP TABLE [mcmain].[condit]
GO
/****** Object: Table [mcmain].[condit] Script Date: 02/07/2012 11:57:49 ******/
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[mcmain].[condit]') AND type in (N'U'))
BEGIN
CREATE TABLE [mcmain].[condit](
[con_levgln] [char](13) NULL,
[con_stat] [char](4) NULL,
[con_dscgrp] [char](35) NULL,
[con_levart] [char](20) NULL,
[con_desc] [char](50) NULL,
[con_disc1] [numeric](5, 0) NULL,
[con_disc2] [numeric](5, 0) NULL,
[con_disc3] [numeric](5, 0) NULL,
[con_ntprce] [numeric](9, 0) NULL,
[con_dtstrt] [datetime] NULL,
[con_dtend] [datetime] NULL,
[con_volc] [char](8) NULL,
[con_updnmr] [char](20) NULL,
[con_notmod] [bit] NULL,
[con_ascver] [char](5) NULL,
[con_prddat] [datetime] NULL,
[con_cusgln] [char](13) NULL,
[con_cusdeb] [char](40) NULL,
[con_rowid] [int] IDENTITY(1,1) NOT NULL
) ON [PRIMARY]
END
GO
IF NOT EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF__condit__con_notm__000AF8CF]') AND type = 'D')
BEGIN
ALTER TABLE [mcmain].[condit] ADD DEFAULT ((0)) FOR [con_notmod]
END
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF__condmod__com_not__7E22B05D]') AND type = 'D')
BEGIN
ALTER TABLE [mcmain].[condmod] DROP CONSTRAINT [DF__condmod__com_not__7E22B05D]
END
GO
/****** Object: Table [mcmain].[condmod] Script Date: 02/07/2012 11:57:56 ******/
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[mcmain].[condmod]') AND type in (N'U'))
DROP TABLE [mcmain].[condmod]
GO
/****** Object: Table [mcmain].[condmod] Script Date: 02/07/2012 11:57:58 ******/
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[mcmain].[condmod]') AND type in (N'U'))
BEGIN
CREATE TABLE [mcmain].[condmod](
[com_levgln] [char](13) NULL,
[com_stat] [char](4) NULL,
[com_dscgrp] [char](35) NULL,
[com_levart] [char](20) NULL,
[com_desc] [char](50) NULL,
[com_disc1] [numeric](5, 0) NULL,
[com_disc2] [numeric](5, 0) NULL,
[com_disc3] [numeric](5, 0) NULL,
[com_ntprce] [numeric](9, 0) NULL,
[com_dtstrt] [datetime] NULL,
[com_dtend] [datetime] NULL,
[com_volc] [char](8) NULL,
[com_updnmr] [char](20) NULL,
[com_notmod] [bit] NULL,
[com_ascver] [char](8) NULL,
[com_prddat] [datetime] NULL,
[com_cusgln] [char](13) NULL,
[com_cusdeb] [char](40) NULL,
[com_rowid] [int] IDENTITY(1,1) NOT NULL
) ON [PRIMARY]
END
GO
IF NOT EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF__condmod__com_not__7E22B05D]') AND type = 'D')
BEGIN
ALTER TABLE [mcmain].[condmod] ADD DEFAULT ((0)) FOR [com_notmod]
END
GO
Here's the isolated code that runs a long time:
DECLARE #TempIdTable TABLE ([com_rowid] Int PRIMARY KEY)
INSERT #TempIdTable([com_rowid])
SELECT cmd.[com_rowid]
FROM [mcmain].[condmod] AS cmd
LEFT OUTER JOIN [mcmain].[condit] AS cdt
ON con_levgln = com_levgln
AND IsNull(con_dscgrp,'') = IsNull(com_dscgrp,'')
AND IsNull(con_levart,'') = IsNull(com_levart,'')
AND IsNull(con_volc,'') = IsNull(com_volc,'')
AND IsNull(con_cusgln,'') = IsNull(com_cusgln,'')
AND IsNull(con_cusdeb,'') = IsNull(com_cusdeb,'')
WHERE con_levgln is NULL
--select * from #TempIdTable
INSERT INTO mcmain.condit(con_levgln
,con_stat
,con_dscgrp
,con_levart
,con_desc
,con_disc1
,con_disc2
,con_disc3
,con_ntprce
,con_dtstrt
,con_dtend
,con_volc
,con_notmod
,con_updnmr
,con_ascver
,con_cusgln
,con_cusdeb)
SELECT com_levgln
,com_stat
,com_dscgrp
,com_levart
,com_desc
,com_disc1
,com_disc2
,com_disc3
,com_ntprce
,com_dtstrt
,com_dtend
,com_volc
,com_notmod
,com_updnmr
,com_ascver
,com_cusgln
,com_cusdeb
FROM [mcmain].[condmod] AS cmd
INNER JOIN #TempIdTable AS tit
ON tit.com_rowid = cmd.com_rowid
The insert into the #TempIdTable takes forever. What can I do to speed up this process?
TIA
Cees Cappelle
p.s. I have clustered indexes on both tables, like:
/****** Object: Index [condmodTest] Script Date: 02/07/2012 13:24:34 ******/
IF NOT EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[mcmain].[condmod]') AND name = N'condmodTest')
CREATE CLUSTERED INDEX [condmodTest] ON [mcmain].[condmod]
(
[com_levgln] ASC,
[com_dscgrp] ASC,
[com_levart] ASC,
[com_volc] ASC,
[com_cusgln] ASC,
[com_cusdeb] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
If I do a straight select (just now), it takes 2 seconds.
Here's the code:
SELECT com_levgln
,com_stat
,com_dscgrp
,com_levart
,com_desc
,com_disc1
,com_disc2
,com_disc3
,com_ntprce
,com_dtstrt
,com_dtend
,com_volc
,com_notmod
,com_updnmr
,com_ascver
,com_cusgln
,com_cusdeb
FROM mcmain.condmod
LEFT OUTER JOIN mcmain.condit
ON con_levgln = com_levgln
AND IsNull(con_dscgrp,'') = IsNull(com_dscgrp,'')
AND IsNull(con_levart,'') = IsNull(com_levart,'')
AND IsNull(con_volc,'') = IsNull(com_volc,'')
AND IsNull(con_cusgln,'') = IsNull(com_cusgln,'')
AND IsNull(con_cusdeb,'') = IsNull(com_cusdeb,'')
WHERE con_levgln is NULL
I just did a Actial Execution plan with the following code. It took 3'16'' ???
truncate table mcmain.condit
CREATE TABLE #TempIdTable ([com_rowid] Int PRIMARY KEY)
-- DECLARE #TempIdTable TABLE
-- ([com_rowid] Int PRIMARY KEY)
INSERT #TempIdTable
([com_rowid])
SELECT cmd.[com_rowid]
FROM [mcmain].[condmod] AS cmd
LEFT OUTER JOIN [mcmain].[condit] AS cdt
ON con_levgln = com_levgln
AND IsNull(con_dscgrp,'') = IsNull(com_dscgrp,'')
AND IsNull(con_levart,'') = IsNull(com_levart,'')
AND IsNull(con_volc,'') = IsNull(com_volc,'')
AND IsNull(con_cusgln,'') = IsNull(com_cusgln,'')
AND IsNull(con_cusdeb,'') = IsNull(com_cusdeb,'')
WHERE con_levgln is NULL
-- AND com_updnmr = #plannummer
INSERT INTO mcmain.condit
(con_levgln
,con_stat
,con_dscgrp
,con_levart
,con_desc
,con_disc1
,con_disc2
,con_disc3
,con_ntprce
,con_dtstrt
,con_dtend
,con_volc
,con_notmod
,con_updnmr
,con_ascver
,con_cusgln
,con_cusdeb)
SELECT com_levgln
,com_stat
,com_dscgrp
,com_levart
,com_desc
,com_disc1
,com_disc2
,com_disc3
,com_ntprce
,com_dtstrt
,com_dtend
,com_volc
,com_notmod
,com_updnmr
,com_ascver
,com_cusgln
,com_cusdeb
FROM [mcmain].[condmod] AS cmd
INNER JOIN #TempIdTable AS tit
ON tit.com_rowid = cmd.com_rowid
Same statement but using a tablevariable took 1'39''
When I run the sp, the same statement takes hours and hours. Still don't get it.
If the SELECT itself takes a long time
You could consider using NOT EXISTS rather than OUTER JOIN ... NULL as this is often more efficient.
Also I would get rid of the non sargable ISNULL comparisons.
SELECT cmd.[com_rowid]
FROM [mcmain].[condmod] AS cmd
WHERE NOT EXISTS (SELECT *
FROM [mcmain].[condit] AS cdt
WHERE con_levgln = com_levgln
AND EXISTS (SELECT con_dscgrp,
con_levart,
con_volc,
con_cusgln,
con_cusdeb
INTERSECT
SELECT com_dscgrp,
com_levart,
com_volc,
com_cusgln,
com_cusdeb))
If the SELECT runs quite quickly on its own but just not when inserting to the table variable then check whether in the quick case you get a parallel plan.
Queries that insert to table variables do not get parallelised so if that is the issue you could consider changing to a #temp table.
If neither of these suggestions help then I suggest you start monitoring the wait types when this procedure runs. See the paper "SQL Server 2005 Waits and Queues"
Note: the use of #tables uses tempdb.
You should try increasing the number of temp db files... you can start with the number of tempdb files equal to the number of procs you have... but as a good starting point just jump to 10... The problem you could be having is contention for tempdb files. You should look for page latching in tempdb.
This article discusses how the generalization I wrote above is a misconception (but not necessarily wrong) but in doing so does a good job of explaining the issue. It also points you to other articles which give informatoin on how to look for page latching.
A SQL Server DBA myth a day: (12/30) tempdb should always have one data file per processor core
Related
I have a table as below with a default value column as [IsProcessed]
IF NOT EXISTS (SELECT * FROM sys.objects
WHERE object_id = OBJECT_ID(N'[dbo].[Kovair_JileEventHistory]')
AND type in (N'U'))
BEGIN
CREATE TABLE [dbo].[Kovair_JileEventHistory]
(
[Id] [bigint] IDENTITY(1,1) NOT NULL,
[GeneratedActionId] [char](36) NOT NULL,
[BaseUrl] [nvarchar](255) NOT NULL,
[ProjectName] [nvarchar](255) NOT NULL,
[EntityName] [varchar](255) NOT NULL,
[EntityId] [varchar](255) NOT NULL,
[RelatedEntityName] [varchar](255) NOT NULL,
[RelatedEntityId] [varchar](255) NOT NULL,
[ActionName] [varchar](255) NOT NULL,
[LoopbackEventName] [varchar](255) NOT NULL,
[LastUpdatedOnGMT] [varchar](50) NOT NULL,
[LastUpdatedBy] [nvarchar](255) NOT NULL,
[IsProcessed] [char](1) NULL,
PRIMARY KEY CLUSTERED ([Id] ASC)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF,
IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON,
ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
END
IF NOT EXISTS (SELECT * FROM sys.objects
WHERE object_id = OBJECT_ID(N'[dbo].[DF__Kovair_Ji__IsPro__5224328E]')
AND type = 'D')
BEGIN
ALTER TABLE [dbo].[Kovair_JileEventHistory]
ADD DEFAULT ('N') FOR [IsProcessed]
END
When I am executing this lines in a different system if I execute it once its working fine but on the second time when I am executing it its giving me an error as below.
Msg 1781, Level 16, State 1, Line 26
Column already has a DEFAULT bound to it.
Msg 1750, Level 16, State 0, Line 26
Could not create constraint. See previous errors.
But when ever I am doing the same in my system from where I have generated this script I am able to execute it multiple time without any error.
I think
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[DF__Kovair_Ji__IsPro__5224328E]') AND type = 'D')
BEGIN
ALTER TABLE [dbo].[Kovair_JileEventHistory] ADD DEFAULT ('N') FOR [IsProcessed]
END
in this lines the part where OBJECT_ID(N'[dbo].[DF__Kovair_Ji__IsPro__5224328E]') is system specific.
Is there any way to solve this issue?
This is flawed:
IF NOT EXISTS
(
SELECT * FROM sys.objects
WHERE object_id = OBJECT_ID(N'[dbo].[DF__Kovair_Ji__IsPro__5224328E]')
AND type = 'D'
)
For a few reasons:
You let the system name the constraint, which means every time you create the table, that system will come up with a new name. You should always explicitly name your constraints so that scripts like this can be made more predictable and so that you can have a sane naming convention.
ALTER TABLE dbo.tablename
ADD CONSTRAINT DF_SomethingSmarter
DEFAULT('N') FOR IsProcessed;
You're basically just saying "is there a default constraint on this table with this name" but you've hard-coded the constraint name. It's possible this could behave differently on a different system for a different reason - say there is another column that starts with IsPro... and it happens to get the same quasi-random system-generated constraint name.
If you know what column and what type of constraint, a much safer (and admittedly more verbose) check is to ignore the constraint name and go after the table + column name:
IF NOT EXISTS
(
SELECT 1 FROM sys.default_constraints AS dc
WHERE parent_object_id = OBJECT_ID(N'dbo.Kovair_JileEventHistory')
AND EXISTS
(
SELECT 1 FROM sys.columns AS c
WHERE c.object_id = dc.parent_object_id
AND c.column_id = dc.parent_column_id
AND c.name = N'IsProcessed')
)
)
BEGIN
-- now I know there is no default constraint
-- on this column, no matter what it's named
END
I would even argue for something more verbose, since at scale some of the metadata functions do not obey isolation semantics, which might be problematic depending on the scope of the change and the recoverability of the entire script:
IF NOT EXISTS
(
SELECT 1 FROM sys.default_constraints AS dc
INNER JOIN sys.tables AS t
ON t.[object_id] = dc.parent_object_id
AND t.name = N'Kovair_JileEventHistory'
AND t.[schema_id] = 1
WHERE EXISTS
...
Problem
We are trying to understand why executing the same code by calling an stored procedure versus executing the store procedure contents in a query window is showing very different execution times but returns exactly the same result set of 183 rows.
Test1
Executing the following SP from SSMS takes 5 minutes to return the results.
EXEC uspFleetSummaryReportSelectByDateCommand #UserID = 1468, #Date = '2015-09-28'
For reference this is the SP detail:
CREATE PROCEDURE [dbo].[uspFleetSummaryReportSelectByDateCommand]
(
#UserID int,
#Date DateTime
)
AS
DECLARE #CustomerID int
SET #CustomerID = (Select CustomerID FROM [User] WHERE UserID = #UserID)
SELECT j.JourneyID,
j.DeviceID,
j.StartDate,
j.EndDate,
ISNULL(JourneyDistance, 0.0) AS [JourneyDistance],
CONVERT(VARCHAR(8), DATEADD(SECOND, DATEDIFF(SECOND, j.StartDate, j.EndDate), 0), 114) AS [Duration],
v.Registration,
v.InitOdometer,
jt.Name AS [JourneyType],
dt.Name AS [DeviceType],
PrivateJourney = (dbo.fxIsPrivateJourney(j.JourneyTypeID, j.DeviceID, #UserID)),
CONVERT(VARCHAR(8), DATEADD(SECOND, ISNULL(e.IdleTime, 0), 0), 114) AS [IdleTime]
FROM Journey j WITH (NOLOCK)
INNER JOIN Vehicle v WITH (NOLOCK) ON v.DeviceID = j.DeviceID
INNER JOIN JourneyType jt WITH (NOLOCK) ON jt.JourneyTypeID = j.JourneyTypeID
INNER JOIN Device d WITH (NOLOCK) ON d.DeviceID = j.DeviceID
INNER JOIN Configuration config WITH (NOLOCK) ON config.ConfigurationID = d.ConfigurationID
INNER JOIN DeviceType dt WITH (NOLOCK) ON dt.DeviceTypeID = config.DeviceTypeID
LEFT OUTER JOIN (
SELECT
e.JourneyId,
SUM(DATEDIFF(SECOND, e.StartDateTime, e.EndDateTime)) AS [IdleTime]
FROM [Event] e WITH (NOLOCK)
WHERE e.JourneyId = JourneyID AND e.EventType = 4/*Idle Event*/
GROUP BY e.JourneyId
) e ON e.JourneyId = j.JourneyID
WHERE j.StartDate BETWEEN #Date AND DATEADD(DAY,1,#Date)
AND (j.JourneyDistance IS NOT NULL)
AND DATEDIFF(MINUTE,j.StartDate,ISNULL(j.EndDate,getdate())) > 0
AND j.DeviceID IN (Select v.DeviceID
FROM Vehicle v WITH (NOLOCK)
INNER JOIN Customer c WITH (NOLOCK) ON c.CustomerID = v.CustomerID
INNER JOIN [User] u ON u.CustomerID = c.CustomerID
WHERE v.CustomerID = #CustomerID AND u.UserID = #UserID
AND (v.LevelOneID = u.LevelOneID Or u.LevelOneID is null)
AND (v.LevelTwoID = u.LevelTwoID Or u.LevelTwoID is null)
AND (v.LevelThreeID = u.LevelThreeID Or u.LevelThreeID is null)
AND (v.LevelFourID = u.LevelFourID Or u.LevelFourID is null)
AND (v.LevelFiveID = u.LevelFiveID Or u.LevelFiveID is null)
AND (v.DriverID = u.LevelSixID Or u.LevelSixID is null)
AND ISNULL(v.HideFromCustomer,0) != 1
)
ORDER BY Registration,j.JourneyID
Test2
But executing the same SP code and setting the variables take 10 seconds to return the results.
Please find below the same SP with the variables set. The following script is executed from SSMS query window.
DECLARE #UserID INT = 13651
DECLARE #Date DATETIME = '2015-09-28'
DECLARE #CustomerID int
SET #CustomerID = (Select CustomerID FROM [User] WHERE UserID = #UserID)
SELECT j.JourneyID,
j.DeviceID,
j.StartDate,
j.EndDate,
ISNULL(JourneyDistance, 0.0) AS [JourneyDistance],
CONVERT(VARCHAR(8), DATEADD(SECOND, DATEDIFF(SECOND, j.StartDate, j.EndDate), 0), 114) AS [Duration],
v.Registration,
v.InitOdometer,
jt.Name AS [JourneyType],
dt.Name AS [DeviceType],
PrivateJourney = (dbo.fxIsPrivateJourney(j.JourneyTypeID, j.DeviceID, #UserID)),
CONVERT(VARCHAR(8), DATEADD(SECOND, ISNULL(e.IdleTime, 0), 0), 114) AS [IdleTime]
FROM Journey j WITH (NOLOCK)
INNER JOIN Vehicle v WITH (NOLOCK) ON v.DeviceID = j.DeviceID
INNER JOIN JourneyType jt WITH (NOLOCK) ON jt.JourneyTypeID = j.JourneyTypeID
INNER JOIN Device d WITH (NOLOCK) ON d.DeviceID = j.DeviceID
INNER JOIN Configuration config WITH (NOLOCK) ON config.ConfigurationID = d.ConfigurationID
INNER JOIN DeviceType dt WITH (NOLOCK) ON dt.DeviceTypeID = config.DeviceTypeID
LEFT OUTER JOIN (
SELECT
e.JourneyId,
SUM(DATEDIFF(SECOND, e.StartDateTime, e.EndDateTime)) AS [IdleTime]
FROM [Event] e WITH (NOLOCK)
WHERE e.JourneyId = JourneyID AND e.EventType = 4/*Idle Event*/
GROUP BY e.JourneyId
) e ON e.JourneyId = j.JourneyID
WHERE j.StartDate BETWEEN #Date AND DATEADD(DAY,1,#Date)
AND (j.JourneyDistance IS NOT NULL)
AND DATEDIFF(MINUTE,j.StartDate,ISNULL(j.EndDate,getdate())) > 0
AND j.DeviceID IN (Select v.DeviceID
FROM Vehicle v WITH (NOLOCK)
INNER JOIN Customer c WITH (NOLOCK) ON c.CustomerID = v.CustomerID
INNER JOIN [User] u ON u.CustomerID = c.CustomerID
WHERE v.CustomerID = #CustomerID AND u.UserID = #UserID
AND (v.LevelOneID = u.LevelOneID Or u.LevelOneID is null)
AND (v.LevelTwoID = u.LevelTwoID Or u.LevelTwoID is null)
AND (v.LevelThreeID = u.LevelThreeID Or u.LevelThreeID is null)
AND (v.LevelFourID = u.LevelFourID Or u.LevelFourID is null)
AND (v.LevelFiveID = u.LevelFiveID Or u.LevelFiveID is null)
AND (v.DriverID = u.LevelSixID Or u.LevelSixID is null)
AND ISNULL(v.HideFromCustomer,0) != 1
)
ORDER BY Registration,j.JourneyID
Debugging to date
Comparing the two statements side by side they are identical bar the setting of the variables.
Comparing the result sets side by side they are identical.
Selecting the variable CUSTOMERID in isolation takes milliseconds.
Dates variables passed are in the same format.
We have run this test multiple times to rule out cache related issue.
Query execution plan was examined on both tests. When executing the SP it was clear an index was missing on table EVENT when executing TEST1.
Index added
CREATE NONCLUSTERED INDEX [290915_EventTypeJourneyID, EventTypeJID,>] ON [dbo].[Event]
(
[EventType] ASC,
[JourneyId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
Result
Execution time for TEST1 dropped to 1 second.
Question
Ok so the principle issue was resolved but fundamentally I don't understand why the performance gap between the two tests that essentially are running the same code ? its same same code , should be using the same indexes , execution times should be similar.
Thank you for any insights into this behaviour.
Reference
Sql server 2008 64bit standard edition.
Table.JOURNEY (350m rows)
CREATE TABLE [dbo].[Journey](
[JourneyID] [int] IDENTITY(1,1) NOT NULL,
[StartAddress] [varchar](500) NULL,
[StartPostcode] [varchar](50) NULL,
[EndAddress] [varchar](500) NULL,
[EndPostcode] [varchar](50) NULL,
[JourneyTypeID] [int] NULL,
[Comment] [varchar](500) NULL,
[DriverID] [int] NULL,
[StartDate] [datetime] NULL,
[EndDate] [datetime] NULL,
[IdleTimeEngineOn] [int] NULL,
[TimeSinceLastJourney] [int] NULL,
[JourneyDistance] [decimal](8, 2) NULL,
[DeviceID] [int] NOT NULL,
[tempJourneyID] [int] NULL,
[tempCustomerID] [int] NULL,
CONSTRAINT [Journey_PK] PRIMARY KEY CLUSTERED
(
[JourneyID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
GO
ALTER TABLE [dbo].[Journey] WITH CHECK ADD CONSTRAINT [Device_Journey_FK1] FOREIGN KEY([DeviceID])
REFERENCES [dbo].[Device] ([DeviceID])
GO
ALTER TABLE [dbo].[Journey] CHECK CONSTRAINT [Device_Journey_FK1]
GO
ALTER TABLE [dbo].[Journey] WITH CHECK ADD CONSTRAINT [Driver_Journey_FK1] FOREIGN KEY([DriverID])
REFERENCES [dbo].[Driver] ([DriverID])
GO
ALTER TABLE [dbo].[Journey] CHECK CONSTRAINT [Driver_Journey_FK1]
GO
ALTER TABLE [dbo].[Journey] WITH NOCHECK ADD CONSTRAINT [JourneyType_Journey_FK1] FOREIGN KEY([JourneyTypeID])
REFERENCES [dbo].[JourneyType] ([JourneyTypeID])
GO
ALTER TABLE [dbo].[Journey] CHECK CONSTRAINT [JourneyType_Journey_FK1]
GO
Table.EVENT (36m rows)
CREATE TABLE [dbo].[Event](
[EventID] [int] IDENTITY(1,1) NOT NULL,
[StartDateTime] [datetime] NULL,
[EndDateTime] [datetime] NULL,
[StartLocationID] [int] NOT NULL,
[EndLocationID] [int] NULL,
[AlertRaised] [bit] NULL,
[EventRuleID] [int] NULL,
[DeviceID] [int] NOT NULL,
[EventMessage] [varchar](max) NULL,
[TopSpeed] [decimal](4, 1) NULL,
[SpeedZone] [int] NULL,
[EventType] [int] NULL,
[ImpactId] [int] NULL,
[NotificationStatus] [bit] NULL,
[CableBreakZone0] [int] NULL,
[CableBreakDistance0] [int] NULL,
[CableBreakZone1] [int] NULL,
[CableBreakDistance1] [int] NULL,
[AdValue] [int] NULL,
[DriverId] [int] NULL,
[VehicleId] [int] NULL,
[JourneyId] [int] NULL,
CONSTRAINT [Event_PK] PRIMARY KEY CLUSTERED
(
[EventID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
GO
ALTER TABLE [dbo].[Event] WITH CHECK ADD CONSTRAINT [Device_Event_FK1] FOREIGN KEY([DeviceID])
REFERENCES [dbo].[Device] ([DeviceID])
GO
ALTER TABLE [dbo].[Event] CHECK CONSTRAINT [Device_Event_FK1]
GO
ALTER TABLE [dbo].[Event] WITH CHECK ADD CONSTRAINT [Event_Impact_FK] FOREIGN KEY([ImpactId])
REFERENCES [dbo].[Impact] ([ImpactID])
GO
ALTER TABLE [dbo].[Event] CHECK CONSTRAINT [Event_Impact_FK]
GO
ALTER TABLE [dbo].[Event] WITH CHECK ADD CONSTRAINT [EventRule_Event_FK1] FOREIGN KEY([EventRuleID])
REFERENCES [dbo].[EventRule] ([EventRuleID])
GO
ALTER TABLE [dbo].[Event] CHECK CONSTRAINT [EventRule_Event_FK1]
GO
ALTER TABLE [dbo].[Event] WITH CHECK ADD CONSTRAINT [FK_Event_Driver] FOREIGN KEY([DriverId])
REFERENCES [dbo].[Driver] ([DriverID])
GO
ALTER TABLE [dbo].[Event] CHECK CONSTRAINT [FK_Event_Driver]
GO
ALTER TABLE [dbo].[Event] WITH CHECK ADD CONSTRAINT [FK_Event_Journey] FOREIGN KEY([JourneyId])
REFERENCES [dbo].[Journey] ([JourneyID])
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Event] CHECK CONSTRAINT [FK_Event_Journey]
GO
ALTER TABLE [dbo].[Event] WITH CHECK ADD CONSTRAINT [FK_Event_Vehicle] FOREIGN KEY([VehicleId])
REFERENCES [dbo].[Vehicle] ([VehicleID])
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Event] CHECK CONSTRAINT [FK_Event_Vehicle]
GO
There's several things that affect query plan creation in SQL Server which might cause really strange things seem to happen.
The plan for stored procedure is (normally) created in the first execution, using the parameters from that time. That plan is saved and used for all the future executions, even if the parameters change.
The procedure can get a new plan if for example statistics change.
If the procedure is such that the optimal plan is totally different depending on the values that are passed in or the procedure for example has a lot of so called optional parameters (for example field = #variable or #variable is NULL -style of coding) -- this can lead to really bad situations and this is usually what is referred as parameter sniffing.
The parameters used to compile the plan can be seen in the properties of the leftmost object in the plan.
If you run the same statement in management studio, but the parameters are local variables that are assigned in the beginning, the statement will be optimized with unknown values, because the values are not known at that point, even if it seems obvious what the values will be.
This is the same thing with procedures that use local variable defined inside the procedure.
If you are running applications that have different session settings, the plans created in step 1. might not be usable, and a different plan will be stored and used. This can cause a situation where it seems that the same procedure behaves different when executed with the same parameters in the application and management studio.
The attributes can be checked from sys.dm_exec_plan_attributes.
For more details, you could check out for example Slow in the Application, Fast in SSMS? Understanding Performance Mysteries by Erland Sommarskog.
Edit: And to understand what's happening, always look at both actual execution plan and statistics IO output. Those should tell you why something is slower than the other (unless it's blocking, waiting etc related)
Source
Difference between running a query in a batch of scripts in SSMS vs running a query in a Stored Procedure is at least in two way:
Before creating your SP use SET ANSI_NULLS ON.
Use inner variable instead arguments of SP in your query like this:
DECLARE #pUserID INT, #pDate DATETIME;
SELECT #pUserID = #UserID, #pDate = #Date;
SELECT ... #pUserID ...;
Had the same problem on an sql server 2008
Running this once fixed it for me :
USE master
ALTER DATABASE [dbname] SET ARITHABORT ON WITH NO_WAIT;
ssms always sets ARITHABORT on, clients written in dotnet do not, that is why both are using a different plan with different statistics.
I have a rather old and very long query written in the stone age of SQL Server 2000. As you may suspect this uses the old joins like c.address_id =* b.address_id
When I try to run it, an error message pops up that I have to set the compatibility level to 80 to enable this type of join. Which I have tried this way:
ALTER DATABASE SandBox
SET compatibility_level = 80
But it seems like that this is being ignored. The script is around 800 rows - so is there any way to run this query in MSSQL 2008 without rewriting it?
It would be required to run overnight twice a week, so for this time I won't care about the performance of the script, etc.
Any suggestions are welcome.
I would rewrite it.
Having said that, this worked for me.
ALTER DATABASE ScratchPadDB
SET compatibility_level = 80
Select ##Version /* Microsoft SQL Server 2008 R2 (RTM) - 10.50.1600.1 (Intel X86) Apr 2 2010 15:53:02 Copyright (c) Microsoft Corporation Developer Edition on Windows NT 6.1 <X64> (Build 7601: Service Pack 1) (WOW64) (Hypervisor) */
/**/
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'Employee' and TABLE_TYPE = N'BASE TABLE' )
BEGIN
DROP TABLE [dbo].[Employee]
END
GO
IF EXISTS ( SELECT * FROM INFORMATION_SCHEMA.TABLES where TABLE_SCHEMA = N'dbo' and TABLE_NAME = N'Department' and TABLE_TYPE = N'BASE TABLE' )
BEGIN
DROP TABLE [dbo].[Department]
END
GO
/**/
CREATE TABLE [dbo].[Department](
[DepartmentUUID] [uniqueidentifier] NOT NULL,
[TheVersionProperty] [timestamp] NOT NULL,
[DepartmentName] [nvarchar](80) NULL,
[CreateDate] [datetime] NOT NULL
)
ALTER TABLE dbo.[Department] ADD CONSTRAINT PK_Department PRIMARY KEY NONCLUSTERED ([DepartmentUUID])
GO
ALTER TABLE [dbo].[Department] ADD CONSTRAINT CK_DepartmentName_Unique UNIQUE ([DepartmentName])
GO
CREATE TABLE [dbo].[Employee] (
[EmployeeUUID] [uniqueidentifier] NOT NULL,
[ParentDepartmentUUID] [uniqueidentifier] NOT NULL,
[TheVersionProperty] [timestamp] NOT NULL,
[SSN] [nvarchar](11) NOT NULL,
[LastName] [varchar](64) NOT NULL,
[FirstName] [varchar](64) NOT NULL,
[CreateDate] [datetime] NOT NULL,
[HireDate] [datetime] NOT NULL
)
GO
ALTER TABLE dbo.Employee ADD CONSTRAINT PK_Employee PRIMARY KEY NONCLUSTERED (EmployeeUUID)
GO
ALTER TABLE [dbo].[Employee] ADD CONSTRAINT CK_SSN_Unique UNIQUE (SSN)
GO
ALTER TABLE [dbo].[Employee] ADD CONSTRAINT FK_EmployeeToDepartment FOREIGN KEY (ParentDepartmentUUID) REFERENCES dbo.Department (DepartmentUUID)
GO
Select * from
dbo.Department d, dbo.Employee e
Where
d.DepartmentUUID =* e.ParentDepartmentUUID
Not exactly sure on what did you try so far, but executing it within the current script will not necessarily work. If you still struggle with setting the compatibility level, try opening a new query window and run only that portion.
I want to transfer data between databases automatically. For example in every one hour, data in one table in first db will be copied into a table in second db.
What do you recommend for me to do ?
By using jobs maybe?
Thanks in advance.
You can probably look at SQL Server Database Mirroring if you are already on SQL Server 2008. Avoids the need for an additional ETL tool.
Create a store procedure and schedule it for one hours:-
Create procedure copydata
AS
INSERT INTO DB1.dbo.TempTable
SELECT * FROM DB2.dbo.TempTable
Have you considered using Integration Services?
You can set up a package doing what you want it to do and then add it to a job with a scheduler.
Here is the "poor man's method".
This logic will you eliminate the need to truncate the destination table every time.
You can schedule a job to call the stored procedure every hour.
OrganizationDB, that is my db name. Yours may be different.
If your source data is on a different database, use that db's name ....
USE [OrganizationDB]
GO
DROP TABLE [dbo].[EmployeeSource]
GO
DROP TABLE [dbo].[EmployeeDestination]
GO
CREATE TABLE [dbo].[EmployeeSource](
[EmployeeUUID] [uniqueidentifier] NOT NULL,
[SSN] [char](9) NOT NULL,
[LastName] [varchar](40) NOT NULL,
[FirstName] [varchar](40) NOT NULL,
[HireDate] [smalldatetime] NOT NULL,
CONSTRAINT [PK_EmployeeSource] PRIMARY KEY NONCLUSTERED
(
[EmployeeUUID] ASC
),
CONSTRAINT [CK_EmployeeSource_SSN_Unique] UNIQUE NONCLUSTERED
(
[SSN] ASC
)
)
GO
ALTER TABLE [dbo].[EmployeeSource] ADD DEFAULT (newsequentialid()) FOR [EmployeeUUID]
GO
ALTER TABLE [dbo].[EmployeeSource] ADD DEFAULT (getdate()) FOR [HireDate]
GO
CREATE TABLE [dbo].[EmployeeDestination](
[EmployeeUUID] [uniqueidentifier] NOT NULL,
[SSN] [char](9) NOT NULL,
[LastName] [varchar](40) NOT NULL,
[FirstName] [varchar](40) NOT NULL,
[HireDate] [smalldatetime] NOT NULL,
CONSTRAINT [PK_EmployeeDestination] PRIMARY KEY NONCLUSTERED
(
[EmployeeUUID] ASC
),
CONSTRAINT [CK_EmployeeDestination_SSN_Unique] UNIQUE NONCLUSTERED
(
[SSN] ASC
)
)
GO
ALTER TABLE [dbo].[EmployeeDestination] ADD DEFAULT (newsequentialid()) FOR [EmployeeUUID]
GO
ALTER TABLE [dbo].[EmployeeDestination] ADD DEFAULT (getdate()) FOR [HireDate]
GO
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[uspPoorMansDataCopy]') AND type in (N'P', N'PC'))
DROP PROCEDURE [dbo].[uspPoorMansDataCopy]
Go
/*
exec [dbo].[uspPoorMansDataCopy]
*/
CREATE PROCEDURE [dbo].[uspPoorMansDataCopy]
AS
SET NOCOUNT ON
/* USE SURROGATE KEY */
INSERT INTO dbo.EmployeeDestination
(
[EmployeeUUID]
, [SSN]
, [LastName]
, [FirstName]
, [HireDate]
)
Select
[EmployeeUUID]
, [SSN]
, [LastName]
, [FirstName]
, [HireDate]
From
OrganizationDB.dbo.EmployeeSource es
where
not exists (select null from dbo.EmployeeDestination innerDestination where innerDestination.EmployeeUUID = es.EmployeeUUID)
/* OR USE UNIQUE CONSTRAINT */
INSERT INTO dbo.EmployeeDestination
(
[EmployeeUUID]
, [SSN]
, [LastName]
, [FirstName]
, [HireDate]
)
Select
[EmployeeUUID]
, [SSN]
, [LastName]
, [FirstName]
, [HireDate]
From
OrganizationDB.dbo.EmployeeSource es
where
not exists (select null from dbo.EmployeeDestination innerDestination where UPPER(innerDestination.SSN) = upper(es.SSN))
SET NOCOUNT OFF
GO
GRANT EXECUTE ON [dbo].[uspPoorMansDataCopy] TO public
GO
If this is only data from one table and if both databases are on the same server you can also consider creating INSERT trigger on that table that will automatically catch all new data and copy it into secondary database.
I wouldn’t do this though if databases are not on the same server and if servers are not in the same physical location because it might cause performance issues.
I have the following SQL Query:
select Subjects.S_ID as ID,
Subjects.S_ParentID as ParentID,
Subjects.S_Name as Name,
Subjects.S_Order as [Order],
subjects.Sbj_IsVisible
from Subjects
left join KPI_SubjectDetails k on Subjects.S_ID = k.S_ID
where
subjects.Sbj_CourseID = 7594
and subjects.Sbj_Type=2
and subjects.Sbj_IsVisible=1
order by subjects.S_Level,
k.SD_Order
Each Subject has a s_ParentID. The most top subjects have a s_ParnetID of 0.
I want to add a SQL Join, which will do the following:
If a parent Subject is set to Sbj_IsVisible = 0 (any subject can be a parent), then the SQL should not output it or any of its children. However, if s_ParentID is set to 0, I don't want to do the Sbj_IsVisible check as this is the top most subject.
Here's what I got:
select Subjects.S_ID as ID,
Subjects.S_ParentID as ParentID,
Subjects.S_Name as Name,
Subjects.S_Order as [Order],
subjects.Sbj_IsVisible
from Subjects
join Subjects_tbl st on Subjects.S_ParentID = st.S_ID and subjects.S_ParentID <> 0
left join KPI_SubjectDetails k on Subjects.S_ID = k.S_ID
where
subjects.Sbj_CourseID = 7594
and subjects.Sbj_Type=2
and subjects.Sbj_IsVisible=1
and st.Sbj_IsVisible = 1
order by subjects.S_Level,
k.SD_Order
This partly works. When a parent subject is set to sbj_Isvisible 0, it does not return its children.
However, if the top most subject is set to sbj_IsVisible 1, the top most subject does not output, but its children do.
BTW, This is one a SQL Server 2008.
//edit
adding some example data.
This is the output of the original query:
ID ParentID Name Order Sbj_IsVisible
9017 0 'Boot Camp' 18 1
9033 9017 1 4 1
9049 9017 test 1 8 1
9050 9049 test 2 1 1
and this is the output of my query:
ID ParentID Name Order Sbj_IsVisible
9033 9017 1 4 1
9049 9017 test 1 8 1
9050 9049 test 2 1 1
here's the create table output:
USE [Fox8]
GO
/****** Object: Table [dbo].[Subjects_tbl] Script Date: 02/22/2012 16:25:12 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[Subjects_tbl](
[S_ID] [int] IDENTITY(1,1) NOT NULL,
[S_TopID] [int] NULL,
[S_ParentID] [int] NULL,
[S_Name] [nvarchar](255) NULL,
[S_Order] [int] NULL,
[S_ItemCount] [int] NOT NULL,
[S_Level] [int] NULL,
[S_IsInherited] [int] NOT NULL,
[S_SortType] [nvarchar](50) NULL,
[S_SortOrder] [nvarchar](50) NULL,
[OriginalSbj_CourseID] [int] NULL,
[Sbj_CourseID] [int] NOT NULL,
[Sbj_IsVisible] [int] NULL,
[Sbj_SkinType] [int] NULL,
[CopyOf_SubjectID] [int] NULL,
[Sbj_GUID] [uniqueidentifier] NULL,
[Sbj_type] [int] NULL,
[s_OriginalSubjectID] [int] NULL,
[OriginalEvalTree_SbjId] [int] NULL,
[S_IsDeleted] [smallint] NOT NULL,
[S_DateDeleted] [datetime] NULL,
[S_IsPrimary] [bit] NULL,
CONSTRAINT [PK_Subjects] PRIMARY KEY CLUSTERED
(
[S_ID] ASC,
[S_ItemCount] ASC,
[Sbj_CourseID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY],
CONSTRAINT [UX_Subjects_S_ID_Sbj_CourseID] UNIQUE NONCLUSTERED
(
[S_ID] ASC,
[Sbj_CourseID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
EXEC sys.sp_addextendedproperty #name=N'MS_Description', #value=N'bitwise field 1 for regular subject 2 for weighted Subject 4 for X of Y Subject' , #level0type=N'SCHEMA',#level0name=N'dbo', #level1type=N'TABLE',#level1name=N'Subjects_tbl', #level2type=N'COLUMN',#level2name=N'Sbj_type'
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD CONSTRAINT [DF_Subjects_S_ItemCount] DEFAULT ((0)) FOR [S_ItemCount]
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD CONSTRAINT [DF_Subjects_S_IsInherited] DEFAULT ((1)) FOR [S_IsInherited]
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD CONSTRAINT [DF_Subjects_Sbj_CourseID] DEFAULT ((-1)) FOR [Sbj_CourseID]
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD DEFAULT ((0)) FOR [Sbj_SkinType]
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD CONSTRAINT [DF_Subjects_Sbj_IsEvaluation] DEFAULT ((1)) FOR [Sbj_type]
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD DEFAULT ((0)) FOR [S_IsDeleted]
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD DEFAULT ((0)) FOR [S_IsPrimary]
GO
Your question is a little confusing to me but let me suggest using an OR clause, as in:
SELECT s.S_ID AS ID, s.S_ParentID AS ParentID, s.S_Name AS Name,
s.S_Order AS [Order], s.Sbj_IsVisible
FROM Subjects s
LEFT JOIN Subjects_tbl st ON s.S_ParentID = st.S_ID
LEFT JOIN KPI_SubjectDetails k ON s.S_ID = k.S_ID
WHERE s.Sbj_CourseID = 7594
AND s.Sbj_Type=2
AND s.Sbj_IsVisible = 1
AND (st.Sbj_IsVisible = 0 OR s.S_ParentID = 0)
ORDER BY s.S_Level, k.SD_Order
Essentially, select information from the subjects table if either it's corresponding parent is not visible or it does not have a corresponding parent (along with whatever your other conditions mean).
Hope that helps!