how i can improve the performance of the sq statement which contains Varbinary(Max) datatype
Table Structure
CREATE TABLE [dbo].[Table_1](
[EmpID] [numeric](18, 0) NOT NULL,
[SrNo] [numeric](18, 0) NOT NULL,
[Type] [varchar](10) NULL,
[FileName] [varchar](100) NULL,
[D1] [varchar](50) NULL,
[D2] [varchar](50) NULL,
[Data] [varbinary](max) NULL,
[CreatedBy] [varchar](50) NULL,
[CreatedOn] [datetime] NULL,
CONSTRAINT [PK_Table_1] PRIMARY KEY CLUSTERED
(
[EmpID] ASC,
[SrNo] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
the Select Query takes around 2 to 3 seconds to select a single record with
Data
Checked Indexing of the Table
I have two tables in which I need to check whether UCCID is tied to only one BUYID as part of testing. UCCID should be unique for my table. But BUyid can have multiple UCCID. But UCCID should be tied to unique (only one) BUYID.
CREATE TABLE [dbo].[EquipmentBase](
[EDAEquipmentId] [uniqueidentifier] NOT NULL,
[BuyID] [nvarchar](100) NULL,
[EqtAE] [nvarchar](100) NULL,
[EqtSn] [nvarchar](100) NULL,
[EqtSz] [nvarchar](100) NULL,
[EqtUccYr] [nvarchar](100) NULL,
[EqtUnit] [nvarchar](100) NULL,
[EqtValue] [nvarchar](100) NULL,
[UccID] [NVARCHAR](100) NULL,
CONSTRAINT [PK__EquipmentBase] PRIMARY KEY CLUSTERED
(
[_EDAEquipmentId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE TABLE [dbo].[activityBase](
[activityId] [UNIQUEIDENTIFIER] NOT NULL,
[UccID] [NVARCHAR](100) NULL,
[ASCity] [NVARCHAR](100) NULL,
[ASClass] [NVARCHAR](100) NULL,
[ASComp] [NVARCHAR](100) NULL,
[ASID] [NVARCHAR](100) NULL,
[ASState] [NVARCHAR](100) NULL,
[BuyADR1] [NVARCHAR](100) NULL,
[BuyADR2] [NVARCHAR](100) NULL,
[BuyID] [NVARCHAR](100) NULL,
[UccCom] [NVARCHAR](100) NULL,
[UccDate] [DATETIME] NULL,
[UccFips2] [NVARCHAR](100) NULL,
[UccStatus] [NVARCHAR](100) NULL,
[UccTransID] [NVARCHAR](100) NULL,
CONSTRAINT [PK__activityBase] PRIMARY KEY CLUSTERED
(
[activityId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
) ON [PRIMARY]
GO
I tried the below query but I am not getting the result for what I am looking into? Can anyone help pls? I expect the below query to return 0 so that UCCID is tied to single BUYID.
SELECT COUNT(*) FROM dbo.activityBase ab
JOIN (SELECT eb.BuyID FROM dbo.EquipmentBase eb
WHERE eb.UccID IS NOT NULL
GROUP BY eb.BuyID
HAVING COUNT(eb.BuyID)=1)c
ON c.BuyID = ab.BuyID
If I'm understanding correctly (and your question is not super clear), if you want to identify any values of UccID for which records exist with more than one different value of BuyID(across both of your tables), you could use:
SELECT UccId, COUNT(*)
FROM
(
SELECT DISTINCT BuyID, UccID FROM EquipmentBase
UNION
SELECT DISTINCT BuyID, UccID FROM activityBase
) AS DistinctCombos
GROUP BY UccId HAVING COUNT(*) > 1
This uses a subquery in the FROM clause to return all combinations of the two fields across both tables, then inspects those results to find any UccID values which have more than one distinct BuyID value against them.
I know this is a common topic but i stil feel my scenario requires some custom advice. I was selecting from a table the other day and it took me an AGE to select on a datetime column that has not been indexed. I want to index this, only problem ebing is that the "production" 2005 box i am unfamiliar with and how it will handle the index creation. With that in mind i am wondering what the safest way for me to create an index on the table is. All the details i think people will need are below (i hope) :
Index to be created on field 5 (possibly field4 also)
Table 1 definition:
CREATE TABLE [dbo].[TABLE 1](
[ID] [uniqueidentifier] NOT NULL CONSTRAINT [DF_BLA] DEFAULT (newid()),
[field1] [nvarchar](7) NULL,
[field2] [nvarchar](10) NULL,
[field3] [nvarchar](2) NULL,
[field4] [datetime] NULL,
[field5] [datetime] NULL,
[field6] [smallint] NULL,
[field7] [nvarchar](1) NULL,
[field8] [nvarchar](7) NULL,
[field9] [nvarchar](60) NULL,
[field10] [smallint] NULL,
[field11] [nvarchar](15) NULL,
[field12] [datetime] NULL,
CONSTRAINT [PK_ID] PRIMARY KEY CLUSTERED
(
[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
Total count of rows in Table 1 : 2926836
I have a SQL database that has two tables that hold a queue of work to be done and the history of that work:
CREATE TABLE [dbo].[WorkQueue](
[WorkQueueID] [int] IDENTITY(1,1) NOT NULL,
[BatchID] [int] NOT NULL,
[AdHocProjectID] [int] NOT NULL,
[TaskID] [int] NOT NULL,
[CreateDate] [datetime] NOT NULL,
[CompletedDate] [datetime] NULL,
[AlertMessage] [varchar](1024) NULL,
CONSTRAINT [PK_WorkQueue] PRIMARY KEY CLUSTERED
(
[WorkQueueID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[WorkQueueHistory](
[WorkQueueHistoryID] [int] IDENTITY(1,1) NOT NULL,
[WorkQueueID] [int] NOT NULL,
[EmployeeID] [int] NOT NULL,
[BatchBoxID] [int] NOT NULL,
[StartTime] [datetime] NOT NULL,
[EndTime] [datetime] NULL,
[PercentageCompleted] [int] NOT NULL,
[FinishedTask] [bit] NOT NULL,
[comment] [varchar](255) NULL,
CONSTRAINT [PK_WorkQueueHistory] PRIMARY KEY CLUSTERED
(
[WorkQueueHistoryID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
) ON [PRIMARY]
The WorkQueue table contains tasks from batches (Jobs) that need to be worked. Users insert into the WorkQueueHistory table when they start working on a job, and when they finish working on a job that row in the WorkQueueHistory table is updated with their end time and the FinishedTask bit is set if that task was indeed completed 100%.
My question is what is the best way to prevent two users from starting the same task? Right now we don't have enough users for this to be a problem, but as things grow I know there is going to be a concurrency issue if I don't address this.
You should use a transaction with the appropriate isolation level set.
See http://www.blackwasp.co.uk/SQLTransactions.aspx
begin tran
-- select next job to do
-- update job as started
commit
The problem
During the passed few months the below described procedure has worked without any problems a vast majority of the times it has run (on 2008 r2). We have, however, three instances of incorrectly connected data. The question is, what is causing this and how do I remedy it?
DATA_PreImp
sourceid col01 col02 col03 col04 col...
100001 John Smith
100002 Calvin Klein
100003 Peter Parker
100004 Moe Greene
Usually the rendered result is that the attribute is connected to the Items_Main correctly but sometimes (less than 1%) the order is scrambled so that the value of col01 is not connected to the same Items_Main as the value of the rest of the columns.
Any insights as to what is causing this would be most appreciated.
The data moving procedure
We have an SSIS package that transfers data from a flat table called DATA_PreImp to a structure consisting of three related tables (attribute based).
Items_Main should contains one row for each row in DATA_PreImp
Items_Featurevalues contains one row for each column value of a row in DATA_PreImp
Items_MainRel contains the connection between Items_Main and Items_FeatureValues
The first step in the SSIS package inserts the data from DATA_PreImp to Items_Main and inserts the generated identifier into the TARGET_ID column in the empty DATA_PreImpMappingTMP table.
insert into items_main(creationdate, status)
output inserted.itemid into DATA_PreImpMappingTMP(TARGET_ID)
select getdate(), '0' from data_preimp
order by sourceid asc;
The second step in the SSIS package fill the Items_MainRel table with TARGET_ID (Itemid originally) and an identifier for the feature (in this case a 5).
insert into items_mainrel(itemid, featureid)
output inserted.itemrelid into DATA_PreImpMapping2TMP(INDREL_ID)
select TARGET_ID, 5 from DATA_PreImpMappingTMP
order by TARGET_ID asc;
The third step is to fill the SOURCE_ID column in the DATA_PreImpMapping2TMP table with the SOURCE_ID from DATA_PreImp.
with cte as (select sourceid, row_number() over (order by sourceid asc) as row from data_preimp)
update m set m.SOURCE_ID = s.sourceid, m.FEAT_ID = 5
from DATA_PreImpMapping2TMP as m
join cte as s on s.row = m.ROW;
The last step is to fill the Items_FeatureValues table with data from DATA_PreImpMapping2TMP and DATA_PreImp.
insert into items_featurevalues(itemrelid, languageid, fnvarchar)
select DATA_PreImpMapping2TMP.INDREL_ID, 0, data_preimp.col01
from DATA_PreImpMapping2TMP
join data_preimp on (DATA_PreImpMapping2TMP.SOURCE_ID = data_preimp.sourceid)
where FEAT_ID = 5
Data table structure
Here is what is needed to create the scenario:
CREATE TABLE [dbo].[DATA_PreImp](
[sourceid] [bigint] IDENTITY(1,1) NOT NULL,
[col01] [nvarchar](500) NULL,
[col02] [nvarchar](500) NULL,
[col03] [nvarchar](500) NULL,
[col04] [nvarchar](500) NULL,
[col05] [nvarchar](500) NULL,
[col06] [nvarchar](500) NULL,
[col07] [nvarchar](500) NULL,
[col08] [nvarchar](500) NULL,
[col09] [nvarchar](500) NULL,
[col10] [nvarchar](500) NULL,
CONSTRAINT [PK_DATA_PreImp] PRIMARY KEY CLUSTERED
(
[sourceid] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[DATA_PreImpMappingTMP](
[ROW] [int] IDENTITY(1,1) NOT NULL,
[TARGET_ID] [int] NULL,
PRIMARY KEY CLUSTERED
(
[ROW] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[Items_Main](
[Itemid] [int] IDENTITY(1,1) NOT NULL,
[creationDate] [smalldatetime] NOT NULL,
[status] [int] NOT NULL,
[purchdate] [smalldatetime] NULL,
[logindate] [smalldatetime] NULL,
CONSTRAINT [PK_Items_Main] PRIMARY KEY CLUSTERED
(
[Itemid] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[DATA_PreImpMapping2TMP](
[ROW] [int] IDENTITY(1,1) NOT NULL,
[SOURCE_ID] [int] NULL,
[INDREL_ID] [int] NULL,
[FEAT_ID] [int] NULL,
PRIMARY KEY CLUSTERED
(
[ROW] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[Items_Features](
[featureId] [int] IDENTITY(1,1) NOT NULL,
[featureRef] [varchar](15) NOT NULL,
[featureName] [varchar](50) NOT NULL,
[creationDate] [smalldatetime] NOT NULL,
[status] [int] NOT NULL,
[fieldType] [varchar](50) NOT NULL,
[featureType] [int] NOT NULL,
[featureDesc] [varchar](500) NULL,
CONSTRAINT [PK_Items_Features] PRIMARY KEY CLUSTERED
(
[featureId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]) ON [PRIMARY]
CREATE TABLE [dbo].[Items_MainRel](
[ItemRelId] [int] IDENTITY(1,1) NOT NULL,
[Itemid] [int] NOT NULL,
[featureId] [int] NOT NULL,
CONSTRAINT [PK_Items_MainRel] PRIMARY KEY CLUSTERED
(
[ItemRelId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE TABLE [dbo].[Items_FeatureValues](
[valueId] [int] IDENTITY(1,1) NOT NULL,
[ItemRelId] [int] NOT NULL,
[languageId] [int] NOT NULL,
[FnVarChar] [nvarchar](250) NULL,
[FInt] [int] NULL,
[FImage] [int] NULL,
[FNText] [ntext] NULL,
[FSmallDateTime] [smalldatetime] NULL,
CONSTRAINT [PK_Items_FeatureValues] PRIMARY KEY CLUSTERED
(
[valueId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
ALTER TABLE [dbo].[Items_MainRel] WITH CHECK ADD CONSTRAINT [FK_Items_MainRel_Items_Features] FOREIGN KEY([featureId])
REFERENCES [dbo].[Items_Features] ([featureId])
GO
ALTER TABLE [dbo].[Items_MainRel] CHECK CONSTRAINT [FK_Items_MainRel_Items_Features]
GO
ALTER TABLE [dbo].[Items_MainRel] WITH CHECK ADD CONSTRAINT [FK_Items_MainRel_Items_Main] FOREIGN KEY([Itemid])
REFERENCES [dbo].[Items_Main] ([Itemid])
GO
ALTER TABLE [dbo].[Items_MainRel] CHECK CONSTRAINT [FK_Items_MainRel_Items_Main]
GO
ALTER TABLE [dbo].[Items_FeatureValues] WITH CHECK ADD CONSTRAINT [FK_Items_FeatureValues_Items_MainRel] FOREIGN KEY([ItemRelId])
REFERENCES [dbo].[Items_MainRel] ([ItemRelId])
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Items_FeatureValues] CHECK CONSTRAINT [FK_Items_FeatureValues_Items_MainRel]
GO
INSERT INTO DATA_PreImp (col01,col02,col03,col04)
VALUES('John', 'Smith', '1964', 'NewYork'),
('Calvin', 'Klein', '1960', 'Washington D. C.'),
('Peter', 'Parker', '1974', 'Losangles'),
('Moe', 'Greene', '1928', 'Lasvegas')
INSERT INTO Items_Features (featureRef, featureName, creationDate, [status], fieldType, featureType, featureDesc)
VALUES ('firstname','First_Name', GETDATE(), 0, 'FnVarChar', 3, 'FirstName'),
('lastname','Last_Name', GETDATE(), 0, 'FnVarChar', 3, 'LastName'),
('Birth','Birth', GETDATE(), 0, 'FnVarChar', 3, 'Birth'),
('City','City', GETDATE(), 0, 'FnVarChar', 3, 'City')
The problem was the computed column CUST_CD. After a lot of researching, it seems that the BULK INSERT does not like complex computed types (see Using SQL Server spatial types in SSIS data load). The solution is to removed the computed column and just make it a varchar(20) NULL. Then I created a new Execute SQL Task that updates any NULL rows with the computed value.