Related
So I have been tasked with converting an Oracle DB to SQL Server. I have never worked with DB's before so I am just running MS SQL Server Migration Assistant to start and only received one error regarding foreign keys. I have done some research but don't understand the principal enough to know how the change needs to be made. Below are the Oracle and SQL Tables from the migration. At the very bottom is the error on the SQL Server side. If anyone could lead me in the right direction I would appreciate it.
Oracle Table:
CREATE TABLE XYZUSER.XYZCMDARGS
(
CMDROWKEY NUMBER(10, 0) NOT NULL,
CMDARGNUM NUMBER(3, 0) NOT NULL,
MEMBERNAME VARCHAR2(50) NOT NULL,
MEMBERTYPE NUMBER(1, 0) NOT NULL,
ARGFORMAT NUMBER(1, 0) NOT NULL,
STARTBIT NUMBER(2, 0) NOT NULL,
TOTALBITS NUMBER(2, 0) NOT NULL,
MSBORLSBFIRST CHAR(1) NOT NULL,
MEMBERVALUE NUMBER(20, 0) NOT NULL,
STATELIST VARCHAR2(200) NOT NULL,
RANGEHIGHFLOAT FLOAT(52) NOT NULL,
RANGELOWFLOAT FLOAT(52) NOT NULL,
RANGEHIGHINT NUMBER(20, 0) NOT NULL,
RANGELOWINT NUMBER(20, 0) NOT NULL,
UNITS CHAR(8) NOT NULL,
INCRPERBIT FLOAT(52) NOT NULL,
ARGPROCESSING NUMBER(1, 0) NOT NULL
);
ALTER TABLE XYZUSER.XYZCMDARGS ADD CONSTRAINT XYZCMDARGS_PK
PRIMARY KEY (CMDROWKEY, CMDARGNUM);
ALTER TABLE XYZUSER.XYZCMDARGS ADD CONSTRAINT XYZCMDARGS_CMDROWKEY_FK
FOREIGN KEY (CMDROWKEY)
REFERENCES XYZUSER.XYZCMDS (CMDROWKEY);
SQL Server:
IF EXISTS (SELECT * FROM sys.objects so JOIN sys.schemas sc ON so.schema_id = sc.schema_id WHERE so.name = N'XYZCMDARGS' AND sc.name = N'XYZUSER' AND type in (N'U'))
BEGIN
DECLARE #drop_statement nvarchar(500)
DECLARE drop_cursor CURSOR FOR
SELECT 'alter table '+quotename(schema_name(ob.schema_id))+
'.'+quotename(object_name(ob.object_id))+ ' drop constraint ' + quotename(fk.name)
FROM sys.objects ob INNER JOIN sys.foreign_keys fk ON fk.parent_object_id = ob.object_id
WHERE fk.referenced_object_id =
(
SELECT so.object_id
FROM sys.objects so JOIN sys.schemas sc
ON so.schema_id = sc.schema_id
WHERE so.name = N'XYZCMDARGS' AND sc.name = N'XYZUSER' AND type in (N'U')
)
OPEN drop_cursor
FETCH NEXT FROM drop_cursor
INTO #drop_statement
WHILE ##FETCH_STATUS = 0
BEGIN
EXEC (#drop_statement)
FETCH NEXT FROM drop_cursor
INTO #drop_statement
END
CLOSE drop_cursor
DEALLOCATE drop_cursor
DROP TABLE [XYZUSER].[XYZCMDARGS]
END
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE
[XYZUSER].[XYZCMDARGS]
(
[CMDROWKEY] numeric(10, 0) NOT NULL,
[CMDARGNUM] numeric(3, 0) NOT NULL,
[MEMBERNAME] varchar(50) NOT NULL,
[MEMBERTYPE] numeric(1, 0) NOT NULL,
[ARGFORMAT] numeric(1, 0) NOT NULL,
[STARTBIT] numeric(2, 0) NOT NULL,
[TOTALBITS] numeric(2, 0) NOT NULL,
[MSBORLSBFIRST] char(1) NOT NULL,
[MEMBERVALUE] numeric(20, 0) NOT NULL,
[STATELIST] varchar(200) NOT NULL,
[RANGEHIGHFLOAT] float(52) NOT NULL,
[RANGELOWFLOAT] float(52) NOT NULL,
[RANGEHIGHINT] numeric(20, 0) NOT NULL,
[RANGELOWINT] numeric(20, 0) NOT NULL,
[UNITS] char(8) NOT NULL,
[INCRPERBIT] float(52) NOT NULL,
[ARGPROCESSING] numeric(1, 0) NOT NULL
)
WITH (DATA_COMPRESSION = NONE)
GO
IF EXISTS (SELECT * FROM sys.objects so JOIN sys.schemas sc ON so.schema_id = sc.schema_id WHERE so.name = N'XYZCMDARGS_PK' AND sc.name = N'XYZUSER' AND type in (N'PK'))
ALTER TABLE [XYZUSER].[XYZCMDARGS] DROP CONSTRAINT [XYZCMDARGS_PK]
GO
ALTER TABLE [XYZUSER].[XYZCMDARGS]
ADD CONSTRAINT [XYZCMDARGS_PK]
PRIMARY KEY
CLUSTERED ([CMDROWKEY] ASC, [CMDARGNUM] ASC)
GO
IF EXISTS (SELECT * FROM sys.objects so JOIN sys.schemas sc ON so.schema_id = sc.schema_id WHERE so.name = N'XYZCMDARGS_CMDROWKEY_FK' AND sc.name = N'XYZUSER' AND type in (N'F'))
ALTER TABLE [XYZUSER].[XYZCMDARGS] DROP CONSTRAINT [XYZCMDARGS_CMDROWKEY_FK]
GO
/*
* SSMA error messages:
* O2SS0231: Foreign keys with different types of columns and referenced columns cannot be converted:XYZCMDARGS_CMDROWKEY_FK.
ALTER TABLE [XYZUSER].[XYZCMDARGS]
ADD CONSTRAINT [XYZCMDARGS_CMDROWKEY_FK]
FOREIGN KEY
([CMDROWKEY])
REFERENCES
[ACEDB1A].[XYZUSER].[XYZCMDS] ([CMDROWKEY])
ON DELETE NO ACTION
ON UPDATE NO ACTION
*/
The issue appears to be that the parent table column XYZUSER.XYZCMDS (CMDROWKEY) is declared as a NUMBER(9,0) while the child table column XYZUSER.XYZCMDARGS (CMDROWKEY) is declared as a NUMBER(10,0). Since the child table declared a larger column than the parent table allows, it is probably safe to change the child table column to a NUMBER(9,0) (assuming nothing weird is going on where the child table was allowed to store values that are larger than what the parent table allows). Alternately, you could redefine the parent table column as a NUMBER(10,0) but then you'd need to redefine the column in any other child tables that reference this parent.
Problem
We are trying to understand why executing the same code by calling an stored procedure versus executing the store procedure contents in a query window is showing very different execution times but returns exactly the same result set of 183 rows.
Test1
Executing the following SP from SSMS takes 5 minutes to return the results.
EXEC uspFleetSummaryReportSelectByDateCommand #UserID = 1468, #Date = '2015-09-28'
For reference this is the SP detail:
CREATE PROCEDURE [dbo].[uspFleetSummaryReportSelectByDateCommand]
(
#UserID int,
#Date DateTime
)
AS
DECLARE #CustomerID int
SET #CustomerID = (Select CustomerID FROM [User] WHERE UserID = #UserID)
SELECT j.JourneyID,
j.DeviceID,
j.StartDate,
j.EndDate,
ISNULL(JourneyDistance, 0.0) AS [JourneyDistance],
CONVERT(VARCHAR(8), DATEADD(SECOND, DATEDIFF(SECOND, j.StartDate, j.EndDate), 0), 114) AS [Duration],
v.Registration,
v.InitOdometer,
jt.Name AS [JourneyType],
dt.Name AS [DeviceType],
PrivateJourney = (dbo.fxIsPrivateJourney(j.JourneyTypeID, j.DeviceID, #UserID)),
CONVERT(VARCHAR(8), DATEADD(SECOND, ISNULL(e.IdleTime, 0), 0), 114) AS [IdleTime]
FROM Journey j WITH (NOLOCK)
INNER JOIN Vehicle v WITH (NOLOCK) ON v.DeviceID = j.DeviceID
INNER JOIN JourneyType jt WITH (NOLOCK) ON jt.JourneyTypeID = j.JourneyTypeID
INNER JOIN Device d WITH (NOLOCK) ON d.DeviceID = j.DeviceID
INNER JOIN Configuration config WITH (NOLOCK) ON config.ConfigurationID = d.ConfigurationID
INNER JOIN DeviceType dt WITH (NOLOCK) ON dt.DeviceTypeID = config.DeviceTypeID
LEFT OUTER JOIN (
SELECT
e.JourneyId,
SUM(DATEDIFF(SECOND, e.StartDateTime, e.EndDateTime)) AS [IdleTime]
FROM [Event] e WITH (NOLOCK)
WHERE e.JourneyId = JourneyID AND e.EventType = 4/*Idle Event*/
GROUP BY e.JourneyId
) e ON e.JourneyId = j.JourneyID
WHERE j.StartDate BETWEEN #Date AND DATEADD(DAY,1,#Date)
AND (j.JourneyDistance IS NOT NULL)
AND DATEDIFF(MINUTE,j.StartDate,ISNULL(j.EndDate,getdate())) > 0
AND j.DeviceID IN (Select v.DeviceID
FROM Vehicle v WITH (NOLOCK)
INNER JOIN Customer c WITH (NOLOCK) ON c.CustomerID = v.CustomerID
INNER JOIN [User] u ON u.CustomerID = c.CustomerID
WHERE v.CustomerID = #CustomerID AND u.UserID = #UserID
AND (v.LevelOneID = u.LevelOneID Or u.LevelOneID is null)
AND (v.LevelTwoID = u.LevelTwoID Or u.LevelTwoID is null)
AND (v.LevelThreeID = u.LevelThreeID Or u.LevelThreeID is null)
AND (v.LevelFourID = u.LevelFourID Or u.LevelFourID is null)
AND (v.LevelFiveID = u.LevelFiveID Or u.LevelFiveID is null)
AND (v.DriverID = u.LevelSixID Or u.LevelSixID is null)
AND ISNULL(v.HideFromCustomer,0) != 1
)
ORDER BY Registration,j.JourneyID
Test2
But executing the same SP code and setting the variables take 10 seconds to return the results.
Please find below the same SP with the variables set. The following script is executed from SSMS query window.
DECLARE #UserID INT = 13651
DECLARE #Date DATETIME = '2015-09-28'
DECLARE #CustomerID int
SET #CustomerID = (Select CustomerID FROM [User] WHERE UserID = #UserID)
SELECT j.JourneyID,
j.DeviceID,
j.StartDate,
j.EndDate,
ISNULL(JourneyDistance, 0.0) AS [JourneyDistance],
CONVERT(VARCHAR(8), DATEADD(SECOND, DATEDIFF(SECOND, j.StartDate, j.EndDate), 0), 114) AS [Duration],
v.Registration,
v.InitOdometer,
jt.Name AS [JourneyType],
dt.Name AS [DeviceType],
PrivateJourney = (dbo.fxIsPrivateJourney(j.JourneyTypeID, j.DeviceID, #UserID)),
CONVERT(VARCHAR(8), DATEADD(SECOND, ISNULL(e.IdleTime, 0), 0), 114) AS [IdleTime]
FROM Journey j WITH (NOLOCK)
INNER JOIN Vehicle v WITH (NOLOCK) ON v.DeviceID = j.DeviceID
INNER JOIN JourneyType jt WITH (NOLOCK) ON jt.JourneyTypeID = j.JourneyTypeID
INNER JOIN Device d WITH (NOLOCK) ON d.DeviceID = j.DeviceID
INNER JOIN Configuration config WITH (NOLOCK) ON config.ConfigurationID = d.ConfigurationID
INNER JOIN DeviceType dt WITH (NOLOCK) ON dt.DeviceTypeID = config.DeviceTypeID
LEFT OUTER JOIN (
SELECT
e.JourneyId,
SUM(DATEDIFF(SECOND, e.StartDateTime, e.EndDateTime)) AS [IdleTime]
FROM [Event] e WITH (NOLOCK)
WHERE e.JourneyId = JourneyID AND e.EventType = 4/*Idle Event*/
GROUP BY e.JourneyId
) e ON e.JourneyId = j.JourneyID
WHERE j.StartDate BETWEEN #Date AND DATEADD(DAY,1,#Date)
AND (j.JourneyDistance IS NOT NULL)
AND DATEDIFF(MINUTE,j.StartDate,ISNULL(j.EndDate,getdate())) > 0
AND j.DeviceID IN (Select v.DeviceID
FROM Vehicle v WITH (NOLOCK)
INNER JOIN Customer c WITH (NOLOCK) ON c.CustomerID = v.CustomerID
INNER JOIN [User] u ON u.CustomerID = c.CustomerID
WHERE v.CustomerID = #CustomerID AND u.UserID = #UserID
AND (v.LevelOneID = u.LevelOneID Or u.LevelOneID is null)
AND (v.LevelTwoID = u.LevelTwoID Or u.LevelTwoID is null)
AND (v.LevelThreeID = u.LevelThreeID Or u.LevelThreeID is null)
AND (v.LevelFourID = u.LevelFourID Or u.LevelFourID is null)
AND (v.LevelFiveID = u.LevelFiveID Or u.LevelFiveID is null)
AND (v.DriverID = u.LevelSixID Or u.LevelSixID is null)
AND ISNULL(v.HideFromCustomer,0) != 1
)
ORDER BY Registration,j.JourneyID
Debugging to date
Comparing the two statements side by side they are identical bar the setting of the variables.
Comparing the result sets side by side they are identical.
Selecting the variable CUSTOMERID in isolation takes milliseconds.
Dates variables passed are in the same format.
We have run this test multiple times to rule out cache related issue.
Query execution plan was examined on both tests. When executing the SP it was clear an index was missing on table EVENT when executing TEST1.
Index added
CREATE NONCLUSTERED INDEX [290915_EventTypeJourneyID, EventTypeJID,>] ON [dbo].[Event]
(
[EventType] ASC,
[JourneyId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
Result
Execution time for TEST1 dropped to 1 second.
Question
Ok so the principle issue was resolved but fundamentally I don't understand why the performance gap between the two tests that essentially are running the same code ? its same same code , should be using the same indexes , execution times should be similar.
Thank you for any insights into this behaviour.
Reference
Sql server 2008 64bit standard edition.
Table.JOURNEY (350m rows)
CREATE TABLE [dbo].[Journey](
[JourneyID] [int] IDENTITY(1,1) NOT NULL,
[StartAddress] [varchar](500) NULL,
[StartPostcode] [varchar](50) NULL,
[EndAddress] [varchar](500) NULL,
[EndPostcode] [varchar](50) NULL,
[JourneyTypeID] [int] NULL,
[Comment] [varchar](500) NULL,
[DriverID] [int] NULL,
[StartDate] [datetime] NULL,
[EndDate] [datetime] NULL,
[IdleTimeEngineOn] [int] NULL,
[TimeSinceLastJourney] [int] NULL,
[JourneyDistance] [decimal](8, 2) NULL,
[DeviceID] [int] NOT NULL,
[tempJourneyID] [int] NULL,
[tempCustomerID] [int] NULL,
CONSTRAINT [Journey_PK] PRIMARY KEY CLUSTERED
(
[JourneyID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
GO
ALTER TABLE [dbo].[Journey] WITH CHECK ADD CONSTRAINT [Device_Journey_FK1] FOREIGN KEY([DeviceID])
REFERENCES [dbo].[Device] ([DeviceID])
GO
ALTER TABLE [dbo].[Journey] CHECK CONSTRAINT [Device_Journey_FK1]
GO
ALTER TABLE [dbo].[Journey] WITH CHECK ADD CONSTRAINT [Driver_Journey_FK1] FOREIGN KEY([DriverID])
REFERENCES [dbo].[Driver] ([DriverID])
GO
ALTER TABLE [dbo].[Journey] CHECK CONSTRAINT [Driver_Journey_FK1]
GO
ALTER TABLE [dbo].[Journey] WITH NOCHECK ADD CONSTRAINT [JourneyType_Journey_FK1] FOREIGN KEY([JourneyTypeID])
REFERENCES [dbo].[JourneyType] ([JourneyTypeID])
GO
ALTER TABLE [dbo].[Journey] CHECK CONSTRAINT [JourneyType_Journey_FK1]
GO
Table.EVENT (36m rows)
CREATE TABLE [dbo].[Event](
[EventID] [int] IDENTITY(1,1) NOT NULL,
[StartDateTime] [datetime] NULL,
[EndDateTime] [datetime] NULL,
[StartLocationID] [int] NOT NULL,
[EndLocationID] [int] NULL,
[AlertRaised] [bit] NULL,
[EventRuleID] [int] NULL,
[DeviceID] [int] NOT NULL,
[EventMessage] [varchar](max) NULL,
[TopSpeed] [decimal](4, 1) NULL,
[SpeedZone] [int] NULL,
[EventType] [int] NULL,
[ImpactId] [int] NULL,
[NotificationStatus] [bit] NULL,
[CableBreakZone0] [int] NULL,
[CableBreakDistance0] [int] NULL,
[CableBreakZone1] [int] NULL,
[CableBreakDistance1] [int] NULL,
[AdValue] [int] NULL,
[DriverId] [int] NULL,
[VehicleId] [int] NULL,
[JourneyId] [int] NULL,
CONSTRAINT [Event_PK] PRIMARY KEY CLUSTERED
(
[EventID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
GO
ALTER TABLE [dbo].[Event] WITH CHECK ADD CONSTRAINT [Device_Event_FK1] FOREIGN KEY([DeviceID])
REFERENCES [dbo].[Device] ([DeviceID])
GO
ALTER TABLE [dbo].[Event] CHECK CONSTRAINT [Device_Event_FK1]
GO
ALTER TABLE [dbo].[Event] WITH CHECK ADD CONSTRAINT [Event_Impact_FK] FOREIGN KEY([ImpactId])
REFERENCES [dbo].[Impact] ([ImpactID])
GO
ALTER TABLE [dbo].[Event] CHECK CONSTRAINT [Event_Impact_FK]
GO
ALTER TABLE [dbo].[Event] WITH CHECK ADD CONSTRAINT [EventRule_Event_FK1] FOREIGN KEY([EventRuleID])
REFERENCES [dbo].[EventRule] ([EventRuleID])
GO
ALTER TABLE [dbo].[Event] CHECK CONSTRAINT [EventRule_Event_FK1]
GO
ALTER TABLE [dbo].[Event] WITH CHECK ADD CONSTRAINT [FK_Event_Driver] FOREIGN KEY([DriverId])
REFERENCES [dbo].[Driver] ([DriverID])
GO
ALTER TABLE [dbo].[Event] CHECK CONSTRAINT [FK_Event_Driver]
GO
ALTER TABLE [dbo].[Event] WITH CHECK ADD CONSTRAINT [FK_Event_Journey] FOREIGN KEY([JourneyId])
REFERENCES [dbo].[Journey] ([JourneyID])
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Event] CHECK CONSTRAINT [FK_Event_Journey]
GO
ALTER TABLE [dbo].[Event] WITH CHECK ADD CONSTRAINT [FK_Event_Vehicle] FOREIGN KEY([VehicleId])
REFERENCES [dbo].[Vehicle] ([VehicleID])
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Event] CHECK CONSTRAINT [FK_Event_Vehicle]
GO
There's several things that affect query plan creation in SQL Server which might cause really strange things seem to happen.
The plan for stored procedure is (normally) created in the first execution, using the parameters from that time. That plan is saved and used for all the future executions, even if the parameters change.
The procedure can get a new plan if for example statistics change.
If the procedure is such that the optimal plan is totally different depending on the values that are passed in or the procedure for example has a lot of so called optional parameters (for example field = #variable or #variable is NULL -style of coding) -- this can lead to really bad situations and this is usually what is referred as parameter sniffing.
The parameters used to compile the plan can be seen in the properties of the leftmost object in the plan.
If you run the same statement in management studio, but the parameters are local variables that are assigned in the beginning, the statement will be optimized with unknown values, because the values are not known at that point, even if it seems obvious what the values will be.
This is the same thing with procedures that use local variable defined inside the procedure.
If you are running applications that have different session settings, the plans created in step 1. might not be usable, and a different plan will be stored and used. This can cause a situation where it seems that the same procedure behaves different when executed with the same parameters in the application and management studio.
The attributes can be checked from sys.dm_exec_plan_attributes.
For more details, you could check out for example Slow in the Application, Fast in SSMS? Understanding Performance Mysteries by Erland Sommarskog.
Edit: And to understand what's happening, always look at both actual execution plan and statistics IO output. Those should tell you why something is slower than the other (unless it's blocking, waiting etc related)
Source
Difference between running a query in a batch of scripts in SSMS vs running a query in a Stored Procedure is at least in two way:
Before creating your SP use SET ANSI_NULLS ON.
Use inner variable instead arguments of SP in your query like this:
DECLARE #pUserID INT, #pDate DATETIME;
SELECT #pUserID = #UserID, #pDate = #Date;
SELECT ... #pUserID ...;
Had the same problem on an sql server 2008
Running this once fixed it for me :
USE master
ALTER DATABASE [dbname] SET ARITHABORT ON WITH NO_WAIT;
ssms always sets ARITHABORT on, clients written in dotnet do not, that is why both are using a different plan with different statistics.
A member of my team has gotten some strange behaviour which can be recreated both in the development environment and the system test environment MS SQL databases.
If he runs this query directly it returns 517 rows, which is the correct and expected result:
SELECT
p.package_id, la.CODE_KID
FROM package p with (nolock), Strength s with (nolock),
ProductCODE la with (nolock), CODE a with (nolock)
where p.Strength_ID = s.Strength_ID
and la.Product_ID = s.Product_ID
AND la.CODE_KID = a.CODE_ID
except
select p.package_ID, p.CODE_KID from package p
However, if he puts the same query in a view it wrongly returns 311 rows - 206 rows less than if he runs the query directly.
If we run the Query Analyser for both the direct query and the view query we see that the two query plans are quite different, but we don't understand why.
He also tried to dump the query into a temporary table:
insert into MyDB.CODE_PACKAGE
SELECT
p.package_id, la.CODE_KID
FROM package p with (nolock), Strength s with (nolock),
ProductCODE la with (nolock), CODE a with (nolock)
where p.Strength_ID = s.Strength_ID
and la.Product_ID = s.Product_ID
AND la.CODE_KID = a.CODE_ID
except
select p.package_ID, p.CODE_KID from package p
, which correctly creates a table that has 517 rows. However, if he puts the same SQL in a stored procedure it wrongly returns 311 rows.
It seems that once the query is encapsulated within a database object it returns too few rows.
As mentioned, he has recreated the error on other database systems too.
Any ideas what can cause this strange behaviour?
He has also tried the following without any success:
Remove the nolock
Set transaction isolation level to read uncommitted
Update
I'm not sure if the SSMS Wizard or Template was used to create the view, but if I select "Script View as -> CREATE to -> New Query Editor Window" then this is the output:
USE [TestUtv]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE view [MyDBviews].[CODE_PACKAGE]
as
SELECT
p.package_id, la.CODE_KID
FROM package p with (nolock), Strength s with (nolock),
ProductCODE la with (nolock), CODE a with (nolock)
where p.Strength_ID = s.Strength_ID
and la.Product_ID = s.Product_ID
AND la.CODE_KID = a.CODE_ID
except
select p.package_ID, p.CODE_KID from package p
GO
Here is one of the tables that is used, unfortunately there database is quite huge with hundreds of tables and views so I can't post everything here.
CREATE TABLE [dbo].[Package](
[Package_ID] [uniqueidentifier] NOT NULL,
[Multiple] [int] NULL,
[Multiple2] [int] NULL,
[OutProdnum] [varchar](6) NULL,
[OutProdnumDate] [datetime] NULL,
[zzzPackage_KID] [uniqueidentifier] NULL,
[Strength_ID] [uniqueidentifier] NULL,
[Indi] [varchar](4096) NULL,
[CreatedDate] [datetime] NULL,
[CreatedBy] [varchar](255) NULL,
[LastChangedDate] [datetime] NULL,
[LastChangedBy] [varchar](255) NULL,
[CODE_KID] [uniqueidentifier] NULL,
[MarkDate] [datetime] NULL,
[Amount] [int] NULL,
[KIPackage_ID] [uniqueidentifier] NULL,
[xyz] [bit] NULL,
[Ean] [varchar](255) NULL,
[D_ID] [uniqueidentifier] NULL,
[abc_ID] [uniqueidentifier] NULL,
[DDD] [decimal](18, 4) NULL,
[era_KID] [uniqueidentifier] NULL,
[uuu] [decimal](18, 4) NULL,
[ueer_KID] [uniqueidentifier] NULL,
[abcIdString] [varchar](4095) NULL,
[ExternalId] [varchar](255) NULL,
[Dpack_KID] [uniqueidentifier] NULL,
[tttpacks_KID] [uniqueidentifier] NULL,
CONSTRAINT [Package_PK] PRIMARY KEY CLUSTERED
(
[Package_ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
Try incapsulating your query in an SP and/or a Function and compare the numbers of records returned.
What you are seeing is likely impacted by connection ANSI settings. SSMS and SQL Server sets some by default on each connection and Objects like views have them set at creation time and persisted at execution time.
Check the options currently enabled for your sessions:
https://www.mssqltips.com/sqlservertip/1415/determining-set-options-for-a-current-session-in-sql-server/
Check options that are persisted with your view:
SELECT * FROM sys.sql_modules
Try the same query using JOIN statements. you can end up with different results from different query plans on queries that use this older joining technique.
SELECT
p.package_id,
la.CODE_KID
FROM package p with (nolock)
INNER JOIN Strength s with (nolock)
ON p.Strength_ID = s.Strength_ID
INNER JOIN ProductCODE la with (nolock)
ON la.Product_ID = s.Product_ID
INNER JOIN CODE a with (nolock)
ON la.CODE_KID = a.CODE_ID
EXCEPT
SELECT
p.package_ID,
p.CODE_KID
FROM package p
I recommend the following syntax for what you are trying to accomplish. My guess is the old ANSI JOIN Syntax you are using breaks somehow when creating a stored procedure.
SELECT
p.package_id
,la.CODE_KID
FROM package AS p
JOIN Strength AS s ON p.Strength_ID = s.Strength_ID
JOIN ProductCODE AS la ON la.Product_ID = s.Product_ID
JOIN CODE AS a ON la.CODE_KID = a.CODE_ID
EXCEPT
SELECT p.package_ID, p.CODE_KID from package p
The really question is, which recods are filtered out if the query is incapsulated into a view?
so, as #Alex suggested before, you should compare results from your query and the same query in the view.
in SSMS try something like:
SELECT *
FROM (
SELECT p.package_id, la.CODE_KID
FROM package p with (nolock), Strength s with (nolock),
ProductCODE la with (nolock), CODE a with (nolock)
where p.Strength_ID = s.Strength_ID
and la.Product_ID = s.Product_ID
AND la.CODE_KID = a.CODE_ID
except
select p.package_ID, p.CODE_KID from package p
) AS DIRECT_Q
FULL JOIN
(
SELECT *
FROM YOUR_VIEW_CREATED_FROM_QUERY /* <-- THE NAME OF YOUR VIEW */
) AS VIEWED_Q
ON DIRECT_Q.package_id=VIEWED_Q.package_id
AND DIRECT_Q.CODE_KID=VIEWED_Q.CODE_KID
in this way you will see which records are missing and try to understand the reason (nulls, accents, particular characters ecc ecc)
I will also give a check if datatype and collation for package.CODE_KID and ProductCODE.CODE_KID are the same.
I have the following SQL Query:
select Subjects.S_ID as ID,
Subjects.S_ParentID as ParentID,
Subjects.S_Name as Name,
Subjects.S_Order as [Order],
subjects.Sbj_IsVisible
from Subjects
left join KPI_SubjectDetails k on Subjects.S_ID = k.S_ID
where
subjects.Sbj_CourseID = 7594
and subjects.Sbj_Type=2
and subjects.Sbj_IsVisible=1
order by subjects.S_Level,
k.SD_Order
Each Subject has a s_ParentID. The most top subjects have a s_ParnetID of 0.
I want to add a SQL Join, which will do the following:
If a parent Subject is set to Sbj_IsVisible = 0 (any subject can be a parent), then the SQL should not output it or any of its children. However, if s_ParentID is set to 0, I don't want to do the Sbj_IsVisible check as this is the top most subject.
Here's what I got:
select Subjects.S_ID as ID,
Subjects.S_ParentID as ParentID,
Subjects.S_Name as Name,
Subjects.S_Order as [Order],
subjects.Sbj_IsVisible
from Subjects
join Subjects_tbl st on Subjects.S_ParentID = st.S_ID and subjects.S_ParentID <> 0
left join KPI_SubjectDetails k on Subjects.S_ID = k.S_ID
where
subjects.Sbj_CourseID = 7594
and subjects.Sbj_Type=2
and subjects.Sbj_IsVisible=1
and st.Sbj_IsVisible = 1
order by subjects.S_Level,
k.SD_Order
This partly works. When a parent subject is set to sbj_Isvisible 0, it does not return its children.
However, if the top most subject is set to sbj_IsVisible 1, the top most subject does not output, but its children do.
BTW, This is one a SQL Server 2008.
//edit
adding some example data.
This is the output of the original query:
ID ParentID Name Order Sbj_IsVisible
9017 0 'Boot Camp' 18 1
9033 9017 1 4 1
9049 9017 test 1 8 1
9050 9049 test 2 1 1
and this is the output of my query:
ID ParentID Name Order Sbj_IsVisible
9033 9017 1 4 1
9049 9017 test 1 8 1
9050 9049 test 2 1 1
here's the create table output:
USE [Fox8]
GO
/****** Object: Table [dbo].[Subjects_tbl] Script Date: 02/22/2012 16:25:12 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[Subjects_tbl](
[S_ID] [int] IDENTITY(1,1) NOT NULL,
[S_TopID] [int] NULL,
[S_ParentID] [int] NULL,
[S_Name] [nvarchar](255) NULL,
[S_Order] [int] NULL,
[S_ItemCount] [int] NOT NULL,
[S_Level] [int] NULL,
[S_IsInherited] [int] NOT NULL,
[S_SortType] [nvarchar](50) NULL,
[S_SortOrder] [nvarchar](50) NULL,
[OriginalSbj_CourseID] [int] NULL,
[Sbj_CourseID] [int] NOT NULL,
[Sbj_IsVisible] [int] NULL,
[Sbj_SkinType] [int] NULL,
[CopyOf_SubjectID] [int] NULL,
[Sbj_GUID] [uniqueidentifier] NULL,
[Sbj_type] [int] NULL,
[s_OriginalSubjectID] [int] NULL,
[OriginalEvalTree_SbjId] [int] NULL,
[S_IsDeleted] [smallint] NOT NULL,
[S_DateDeleted] [datetime] NULL,
[S_IsPrimary] [bit] NULL,
CONSTRAINT [PK_Subjects] PRIMARY KEY CLUSTERED
(
[S_ID] ASC,
[S_ItemCount] ASC,
[Sbj_CourseID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY],
CONSTRAINT [UX_Subjects_S_ID_Sbj_CourseID] UNIQUE NONCLUSTERED
(
[S_ID] ASC,
[Sbj_CourseID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
EXEC sys.sp_addextendedproperty #name=N'MS_Description', #value=N'bitwise field 1 for regular subject 2 for weighted Subject 4 for X of Y Subject' , #level0type=N'SCHEMA',#level0name=N'dbo', #level1type=N'TABLE',#level1name=N'Subjects_tbl', #level2type=N'COLUMN',#level2name=N'Sbj_type'
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD CONSTRAINT [DF_Subjects_S_ItemCount] DEFAULT ((0)) FOR [S_ItemCount]
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD CONSTRAINT [DF_Subjects_S_IsInherited] DEFAULT ((1)) FOR [S_IsInherited]
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD CONSTRAINT [DF_Subjects_Sbj_CourseID] DEFAULT ((-1)) FOR [Sbj_CourseID]
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD DEFAULT ((0)) FOR [Sbj_SkinType]
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD CONSTRAINT [DF_Subjects_Sbj_IsEvaluation] DEFAULT ((1)) FOR [Sbj_type]
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD DEFAULT ((0)) FOR [S_IsDeleted]
GO
ALTER TABLE [dbo].[Subjects_tbl] ADD DEFAULT ((0)) FOR [S_IsPrimary]
GO
Your question is a little confusing to me but let me suggest using an OR clause, as in:
SELECT s.S_ID AS ID, s.S_ParentID AS ParentID, s.S_Name AS Name,
s.S_Order AS [Order], s.Sbj_IsVisible
FROM Subjects s
LEFT JOIN Subjects_tbl st ON s.S_ParentID = st.S_ID
LEFT JOIN KPI_SubjectDetails k ON s.S_ID = k.S_ID
WHERE s.Sbj_CourseID = 7594
AND s.Sbj_Type=2
AND s.Sbj_IsVisible = 1
AND (st.Sbj_IsVisible = 0 OR s.S_ParentID = 0)
ORDER BY s.S_Level, k.SD_Order
Essentially, select information from the subjects table if either it's corresponding parent is not visible or it does not have a corresponding parent (along with whatever your other conditions mean).
Hope that helps!
I have been sniffing out forums and blogs lately, but need some help with a long running query. It's part of a system of stored procedures. This specified statement used to run in about 5 minutes, but lately has been running up to 72 hours!
Here's the setup:
SQL Server 2005 with 28GB memory. Two mountpoints to a SAN with shared disks consisting of 10 spindles. Data is on one mountpoint, Log on another, tempdb on data space. Just one userdatabase on this server.
Here's two tables, condit and condmod. Condit contains 800K records, condmod is initially empty. I issue a truncate mcmain.condmod before the process starts, for testing purposes.
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF__condit__con_notm__000AF8CF]') AND type = 'D')
BEGIN
ALTER TABLE [mcmain].[condit] DROP CONSTRAINT [DF__condit__con_notm__000AF8CF]
END
GO
/****** Object: Table [mcmain].[condit] Script Date: 02/07/2012 11:57:47 ******/
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[mcmain].[condit]') AND type in (N'U'))
DROP TABLE [mcmain].[condit]
GO
/****** Object: Table [mcmain].[condit] Script Date: 02/07/2012 11:57:49 ******/
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[mcmain].[condit]') AND type in (N'U'))
BEGIN
CREATE TABLE [mcmain].[condit](
[con_levgln] [char](13) NULL,
[con_stat] [char](4) NULL,
[con_dscgrp] [char](35) NULL,
[con_levart] [char](20) NULL,
[con_desc] [char](50) NULL,
[con_disc1] [numeric](5, 0) NULL,
[con_disc2] [numeric](5, 0) NULL,
[con_disc3] [numeric](5, 0) NULL,
[con_ntprce] [numeric](9, 0) NULL,
[con_dtstrt] [datetime] NULL,
[con_dtend] [datetime] NULL,
[con_volc] [char](8) NULL,
[con_updnmr] [char](20) NULL,
[con_notmod] [bit] NULL,
[con_ascver] [char](5) NULL,
[con_prddat] [datetime] NULL,
[con_cusgln] [char](13) NULL,
[con_cusdeb] [char](40) NULL,
[con_rowid] [int] IDENTITY(1,1) NOT NULL
) ON [PRIMARY]
END
GO
IF NOT EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF__condit__con_notm__000AF8CF]') AND type = 'D')
BEGIN
ALTER TABLE [mcmain].[condit] ADD DEFAULT ((0)) FOR [con_notmod]
END
GO
IF EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF__condmod__com_not__7E22B05D]') AND type = 'D')
BEGIN
ALTER TABLE [mcmain].[condmod] DROP CONSTRAINT [DF__condmod__com_not__7E22B05D]
END
GO
/****** Object: Table [mcmain].[condmod] Script Date: 02/07/2012 11:57:56 ******/
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[mcmain].[condmod]') AND type in (N'U'))
DROP TABLE [mcmain].[condmod]
GO
/****** Object: Table [mcmain].[condmod] Script Date: 02/07/2012 11:57:58 ******/
IF NOT EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[mcmain].[condmod]') AND type in (N'U'))
BEGIN
CREATE TABLE [mcmain].[condmod](
[com_levgln] [char](13) NULL,
[com_stat] [char](4) NULL,
[com_dscgrp] [char](35) NULL,
[com_levart] [char](20) NULL,
[com_desc] [char](50) NULL,
[com_disc1] [numeric](5, 0) NULL,
[com_disc2] [numeric](5, 0) NULL,
[com_disc3] [numeric](5, 0) NULL,
[com_ntprce] [numeric](9, 0) NULL,
[com_dtstrt] [datetime] NULL,
[com_dtend] [datetime] NULL,
[com_volc] [char](8) NULL,
[com_updnmr] [char](20) NULL,
[com_notmod] [bit] NULL,
[com_ascver] [char](8) NULL,
[com_prddat] [datetime] NULL,
[com_cusgln] [char](13) NULL,
[com_cusdeb] [char](40) NULL,
[com_rowid] [int] IDENTITY(1,1) NOT NULL
) ON [PRIMARY]
END
GO
IF NOT EXISTS (SELECT * FROM dbo.sysobjects WHERE id = OBJECT_ID(N'[DF__condmod__com_not__7E22B05D]') AND type = 'D')
BEGIN
ALTER TABLE [mcmain].[condmod] ADD DEFAULT ((0)) FOR [com_notmod]
END
GO
Here's the isolated code that runs a long time:
DECLARE #TempIdTable TABLE ([com_rowid] Int PRIMARY KEY)
INSERT #TempIdTable([com_rowid])
SELECT cmd.[com_rowid]
FROM [mcmain].[condmod] AS cmd
LEFT OUTER JOIN [mcmain].[condit] AS cdt
ON con_levgln = com_levgln
AND IsNull(con_dscgrp,'') = IsNull(com_dscgrp,'')
AND IsNull(con_levart,'') = IsNull(com_levart,'')
AND IsNull(con_volc,'') = IsNull(com_volc,'')
AND IsNull(con_cusgln,'') = IsNull(com_cusgln,'')
AND IsNull(con_cusdeb,'') = IsNull(com_cusdeb,'')
WHERE con_levgln is NULL
--select * from #TempIdTable
INSERT INTO mcmain.condit(con_levgln
,con_stat
,con_dscgrp
,con_levart
,con_desc
,con_disc1
,con_disc2
,con_disc3
,con_ntprce
,con_dtstrt
,con_dtend
,con_volc
,con_notmod
,con_updnmr
,con_ascver
,con_cusgln
,con_cusdeb)
SELECT com_levgln
,com_stat
,com_dscgrp
,com_levart
,com_desc
,com_disc1
,com_disc2
,com_disc3
,com_ntprce
,com_dtstrt
,com_dtend
,com_volc
,com_notmod
,com_updnmr
,com_ascver
,com_cusgln
,com_cusdeb
FROM [mcmain].[condmod] AS cmd
INNER JOIN #TempIdTable AS tit
ON tit.com_rowid = cmd.com_rowid
The insert into the #TempIdTable takes forever. What can I do to speed up this process?
TIA
Cees Cappelle
p.s. I have clustered indexes on both tables, like:
/****** Object: Index [condmodTest] Script Date: 02/07/2012 13:24:34 ******/
IF NOT EXISTS (SELECT * FROM sys.indexes WHERE object_id = OBJECT_ID(N'[mcmain].[condmod]') AND name = N'condmodTest')
CREATE CLUSTERED INDEX [condmodTest] ON [mcmain].[condmod]
(
[com_levgln] ASC,
[com_dscgrp] ASC,
[com_levart] ASC,
[com_volc] ASC,
[com_cusgln] ASC,
[com_cusdeb] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, IGNORE_DUP_KEY = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
If I do a straight select (just now), it takes 2 seconds.
Here's the code:
SELECT com_levgln
,com_stat
,com_dscgrp
,com_levart
,com_desc
,com_disc1
,com_disc2
,com_disc3
,com_ntprce
,com_dtstrt
,com_dtend
,com_volc
,com_notmod
,com_updnmr
,com_ascver
,com_cusgln
,com_cusdeb
FROM mcmain.condmod
LEFT OUTER JOIN mcmain.condit
ON con_levgln = com_levgln
AND IsNull(con_dscgrp,'') = IsNull(com_dscgrp,'')
AND IsNull(con_levart,'') = IsNull(com_levart,'')
AND IsNull(con_volc,'') = IsNull(com_volc,'')
AND IsNull(con_cusgln,'') = IsNull(com_cusgln,'')
AND IsNull(con_cusdeb,'') = IsNull(com_cusdeb,'')
WHERE con_levgln is NULL
I just did a Actial Execution plan with the following code. It took 3'16'' ???
truncate table mcmain.condit
CREATE TABLE #TempIdTable ([com_rowid] Int PRIMARY KEY)
-- DECLARE #TempIdTable TABLE
-- ([com_rowid] Int PRIMARY KEY)
INSERT #TempIdTable
([com_rowid])
SELECT cmd.[com_rowid]
FROM [mcmain].[condmod] AS cmd
LEFT OUTER JOIN [mcmain].[condit] AS cdt
ON con_levgln = com_levgln
AND IsNull(con_dscgrp,'') = IsNull(com_dscgrp,'')
AND IsNull(con_levart,'') = IsNull(com_levart,'')
AND IsNull(con_volc,'') = IsNull(com_volc,'')
AND IsNull(con_cusgln,'') = IsNull(com_cusgln,'')
AND IsNull(con_cusdeb,'') = IsNull(com_cusdeb,'')
WHERE con_levgln is NULL
-- AND com_updnmr = #plannummer
INSERT INTO mcmain.condit
(con_levgln
,con_stat
,con_dscgrp
,con_levart
,con_desc
,con_disc1
,con_disc2
,con_disc3
,con_ntprce
,con_dtstrt
,con_dtend
,con_volc
,con_notmod
,con_updnmr
,con_ascver
,con_cusgln
,con_cusdeb)
SELECT com_levgln
,com_stat
,com_dscgrp
,com_levart
,com_desc
,com_disc1
,com_disc2
,com_disc3
,com_ntprce
,com_dtstrt
,com_dtend
,com_volc
,com_notmod
,com_updnmr
,com_ascver
,com_cusgln
,com_cusdeb
FROM [mcmain].[condmod] AS cmd
INNER JOIN #TempIdTable AS tit
ON tit.com_rowid = cmd.com_rowid
Same statement but using a tablevariable took 1'39''
When I run the sp, the same statement takes hours and hours. Still don't get it.
If the SELECT itself takes a long time
You could consider using NOT EXISTS rather than OUTER JOIN ... NULL as this is often more efficient.
Also I would get rid of the non sargable ISNULL comparisons.
SELECT cmd.[com_rowid]
FROM [mcmain].[condmod] AS cmd
WHERE NOT EXISTS (SELECT *
FROM [mcmain].[condit] AS cdt
WHERE con_levgln = com_levgln
AND EXISTS (SELECT con_dscgrp,
con_levart,
con_volc,
con_cusgln,
con_cusdeb
INTERSECT
SELECT com_dscgrp,
com_levart,
com_volc,
com_cusgln,
com_cusdeb))
If the SELECT runs quite quickly on its own but just not when inserting to the table variable then check whether in the quick case you get a parallel plan.
Queries that insert to table variables do not get parallelised so if that is the issue you could consider changing to a #temp table.
If neither of these suggestions help then I suggest you start monitoring the wait types when this procedure runs. See the paper "SQL Server 2005 Waits and Queues"
Note: the use of #tables uses tempdb.
You should try increasing the number of temp db files... you can start with the number of tempdb files equal to the number of procs you have... but as a good starting point just jump to 10... The problem you could be having is contention for tempdb files. You should look for page latching in tempdb.
This article discusses how the generalization I wrote above is a misconception (but not necessarily wrong) but in doing so does a good job of explaining the issue. It also points you to other articles which give informatoin on how to look for page latching.
A SQL Server DBA myth a day: (12/30) tempdb should always have one data file per processor core