how i can improve the performance of the sq statement which contains Varbinary(Max) datatype
Table Structure
CREATE TABLE [dbo].[Table_1](
[EmpID] [numeric](18, 0) NOT NULL,
[SrNo] [numeric](18, 0) NOT NULL,
[Type] [varchar](10) NULL,
[FileName] [varchar](100) NULL,
[D1] [varchar](50) NULL,
[D2] [varchar](50) NULL,
[Data] [varbinary](max) NULL,
[CreatedBy] [varchar](50) NULL,
[CreatedOn] [datetime] NULL,
CONSTRAINT [PK_Table_1] PRIMARY KEY CLUSTERED
(
[EmpID] ASC,
[SrNo] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
the Select Query takes around 2 to 3 seconds to select a single record with
Data
Checked Indexing of the Table
Related
I am on SQL Server 11. I have the following table that stores the boundary of all LGA in NSW
CREATE TABLE [dbo].[nsw_lga_polygon_shp](
[id] [int] IDENTITY(1,1) NOT NULL,
[geom] [geometry] NULL,
[lg_ply_pid] [nvarchar](15) NULL,
[dt_create] [date] NULL,
[dt_retire] [date] NULL,
[lga_pid] [nvarchar](15) NULL,
[nsw_lga_sh] [date] NULL,
[nsw_lga__1] [date] NULL,
[nsw_lga__2] [nvarchar](100) NULL,
[nsw_lga__3] [nvarchar](100) NULL,
[nsw_lga__4] [date] NULL,
[nsw_lga__5] [nvarchar](15) NULL,
CONSTRAINT [PK_nsw_lga_polygon_shp] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
END
GO
SET IDENTITY_INSERT [dbo].[nsw_lga_polygon_shp] ON;
The table only have 198 rows
When I try to map the result against one of my table that have 36,531 rows, I notice that it have very serious performance issue, it told 5 mins to generate 500 rows.
select
* ,
(select top 1
nsw_lga__2
from
[nsw_lga_polygon_shp]
where
Geom.Filter(geometry::STGeomFromText('point(' +
convert(varchar(100),longitude_GIS )+
' ' +
cast(latitude_GIS as varchar(100))+ ')',4283)) = 1
) LGA
from
Report_A
Is there anyway I can may it run faster? Can I index the geometry column?
Thanks in advance!
I have a financial app where I store ticker data on a daily basis, so i create a primary key from the Ticker and Date.
I also need to make Ticker (which is part of the above composite key) a foreign key of another table called Tickers.
However when I go to add the Foreign Key relationship within SQL Server Management Studio, it is asking me to provide link for both columns Ticker and Date, whereas the Ticker table only contains Ticker (which is what I only want to bind on).
Any ideas on what I am doing wrong please or is this not possible?
My tables are as below:
CREATE TABLE [dbo].[Tickers](
[Ticker] [nvarchar](8) NOT NULL,
[Name] [nvarchar](50) NOT NULL,
[Market] [int] NOT NULL,
[Locale] [int] NOT NULL,
[Type] [int] NOT NULL,
[Active] [bit] NOT NULL,
[PrimaryExch] [nvarchar](50) NOT NULL,
[Updated] [datetime2](7) NOT NULL,
[Currency] [int] NOT NULL,
CONSTRAINT [PK_Tickers] PRIMARY KEY CLUSTERED
(
[Ticker] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE TABLE [dbo].[AggregateDay](
[Ticker] [nchar](8) NOT NULL,
[Date] [date] NOT NULL,
[Volume] [decimal](18, 0) NULL,
[Open] [decimal](18, 0) NULL,
[Close] [decimal](18, 0) NULL,
[High] [decimal](18, 0) NULL,
[Low] [decimal](18, 0) NULL,
[Samples] [int] NULL,
CONSTRAINT [PK_AggregateDay] PRIMARY KEY CLUSTERED
(
[Ticker] ASC,
[Date] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
One problem with your set-up is that the parent and child columns do not have the same datatype: [dbo].[Tickers]([Ticker]) is [nvarchar](8), while [dbo].[AggregateDay](Ticket) is [nchar](8).
If you align both datatypes, then you can create the relationship, like so:
CREATE TABLE [dbo].[AggregateDay](
[Ticker] [nvarchar](8) NOT NULL
REFERENCES [dbo].[Tickers]([Ticker]), -- foreign key declaration
[Date] [date] NOT NULL,
[Volume] [decimal](18, 0) NULL,
[Open] [decimal](18, 0) NULL,
[Close] [decimal](18, 0) NULL,
[High] [decimal](18, 0) NULL,
[Low] [decimal](18, 0) NULL,
[Samples] [int] NULL,
CONSTRAINT [PK_AggregateDay] PRIMARY KEY CLUSTERED
([Ticker] ASC, [Date] ASC)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
ON [PRIMARY]
) ON [PRIMARY]
Demo on DB Fiddle
I know this is a common topic but i stil feel my scenario requires some custom advice. I was selecting from a table the other day and it took me an AGE to select on a datetime column that has not been indexed. I want to index this, only problem ebing is that the "production" 2005 box i am unfamiliar with and how it will handle the index creation. With that in mind i am wondering what the safest way for me to create an index on the table is. All the details i think people will need are below (i hope) :
Index to be created on field 5 (possibly field4 also)
Table 1 definition:
CREATE TABLE [dbo].[TABLE 1](
[ID] [uniqueidentifier] NOT NULL CONSTRAINT [DF_BLA] DEFAULT (newid()),
[field1] [nvarchar](7) NULL,
[field2] [nvarchar](10) NULL,
[field3] [nvarchar](2) NULL,
[field4] [datetime] NULL,
[field5] [datetime] NULL,
[field6] [smallint] NULL,
[field7] [nvarchar](1) NULL,
[field8] [nvarchar](7) NULL,
[field9] [nvarchar](60) NULL,
[field10] [smallint] NULL,
[field11] [nvarchar](15) NULL,
[field12] [datetime] NULL,
CONSTRAINT [PK_ID] PRIMARY KEY CLUSTERED
(
[ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
Total count of rows in Table 1 : 2926836
I have a SQL database that has two tables that hold a queue of work to be done and the history of that work:
CREATE TABLE [dbo].[WorkQueue](
[WorkQueueID] [int] IDENTITY(1,1) NOT NULL,
[BatchID] [int] NOT NULL,
[AdHocProjectID] [int] NOT NULL,
[TaskID] [int] NOT NULL,
[CreateDate] [datetime] NOT NULL,
[CompletedDate] [datetime] NULL,
[AlertMessage] [varchar](1024) NULL,
CONSTRAINT [PK_WorkQueue] PRIMARY KEY CLUSTERED
(
[WorkQueueID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[WorkQueueHistory](
[WorkQueueHistoryID] [int] IDENTITY(1,1) NOT NULL,
[WorkQueueID] [int] NOT NULL,
[EmployeeID] [int] NOT NULL,
[BatchBoxID] [int] NOT NULL,
[StartTime] [datetime] NOT NULL,
[EndTime] [datetime] NULL,
[PercentageCompleted] [int] NOT NULL,
[FinishedTask] [bit] NOT NULL,
[comment] [varchar](255) NULL,
CONSTRAINT [PK_WorkQueueHistory] PRIMARY KEY CLUSTERED
(
[WorkQueueHistoryID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
) ON [PRIMARY]
The WorkQueue table contains tasks from batches (Jobs) that need to be worked. Users insert into the WorkQueueHistory table when they start working on a job, and when they finish working on a job that row in the WorkQueueHistory table is updated with their end time and the FinishedTask bit is set if that task was indeed completed 100%.
My question is what is the best way to prevent two users from starting the same task? Right now we don't have enough users for this to be a problem, but as things grow I know there is going to be a concurrency issue if I don't address this.
You should use a transaction with the appropriate isolation level set.
See http://www.blackwasp.co.uk/SQLTransactions.aspx
begin tran
-- select next job to do
-- update job as started
commit
The problem
During the passed few months the below described procedure has worked without any problems a vast majority of the times it has run (on 2008 r2). We have, however, three instances of incorrectly connected data. The question is, what is causing this and how do I remedy it?
DATA_PreImp
sourceid col01 col02 col03 col04 col...
100001 John Smith
100002 Calvin Klein
100003 Peter Parker
100004 Moe Greene
Usually the rendered result is that the attribute is connected to the Items_Main correctly but sometimes (less than 1%) the order is scrambled so that the value of col01 is not connected to the same Items_Main as the value of the rest of the columns.
Any insights as to what is causing this would be most appreciated.
The data moving procedure
We have an SSIS package that transfers data from a flat table called DATA_PreImp to a structure consisting of three related tables (attribute based).
Items_Main should contains one row for each row in DATA_PreImp
Items_Featurevalues contains one row for each column value of a row in DATA_PreImp
Items_MainRel contains the connection between Items_Main and Items_FeatureValues
The first step in the SSIS package inserts the data from DATA_PreImp to Items_Main and inserts the generated identifier into the TARGET_ID column in the empty DATA_PreImpMappingTMP table.
insert into items_main(creationdate, status)
output inserted.itemid into DATA_PreImpMappingTMP(TARGET_ID)
select getdate(), '0' from data_preimp
order by sourceid asc;
The second step in the SSIS package fill the Items_MainRel table with TARGET_ID (Itemid originally) and an identifier for the feature (in this case a 5).
insert into items_mainrel(itemid, featureid)
output inserted.itemrelid into DATA_PreImpMapping2TMP(INDREL_ID)
select TARGET_ID, 5 from DATA_PreImpMappingTMP
order by TARGET_ID asc;
The third step is to fill the SOURCE_ID column in the DATA_PreImpMapping2TMP table with the SOURCE_ID from DATA_PreImp.
with cte as (select sourceid, row_number() over (order by sourceid asc) as row from data_preimp)
update m set m.SOURCE_ID = s.sourceid, m.FEAT_ID = 5
from DATA_PreImpMapping2TMP as m
join cte as s on s.row = m.ROW;
The last step is to fill the Items_FeatureValues table with data from DATA_PreImpMapping2TMP and DATA_PreImp.
insert into items_featurevalues(itemrelid, languageid, fnvarchar)
select DATA_PreImpMapping2TMP.INDREL_ID, 0, data_preimp.col01
from DATA_PreImpMapping2TMP
join data_preimp on (DATA_PreImpMapping2TMP.SOURCE_ID = data_preimp.sourceid)
where FEAT_ID = 5
Data table structure
Here is what is needed to create the scenario:
CREATE TABLE [dbo].[DATA_PreImp](
[sourceid] [bigint] IDENTITY(1,1) NOT NULL,
[col01] [nvarchar](500) NULL,
[col02] [nvarchar](500) NULL,
[col03] [nvarchar](500) NULL,
[col04] [nvarchar](500) NULL,
[col05] [nvarchar](500) NULL,
[col06] [nvarchar](500) NULL,
[col07] [nvarchar](500) NULL,
[col08] [nvarchar](500) NULL,
[col09] [nvarchar](500) NULL,
[col10] [nvarchar](500) NULL,
CONSTRAINT [PK_DATA_PreImp] PRIMARY KEY CLUSTERED
(
[sourceid] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[DATA_PreImpMappingTMP](
[ROW] [int] IDENTITY(1,1) NOT NULL,
[TARGET_ID] [int] NULL,
PRIMARY KEY CLUSTERED
(
[ROW] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[Items_Main](
[Itemid] [int] IDENTITY(1,1) NOT NULL,
[creationDate] [smalldatetime] NOT NULL,
[status] [int] NOT NULL,
[purchdate] [smalldatetime] NULL,
[logindate] [smalldatetime] NULL,
CONSTRAINT [PK_Items_Main] PRIMARY KEY CLUSTERED
(
[Itemid] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[DATA_PreImpMapping2TMP](
[ROW] [int] IDENTITY(1,1) NOT NULL,
[SOURCE_ID] [int] NULL,
[INDREL_ID] [int] NULL,
[FEAT_ID] [int] NULL,
PRIMARY KEY CLUSTERED
(
[ROW] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[Items_Features](
[featureId] [int] IDENTITY(1,1) NOT NULL,
[featureRef] [varchar](15) NOT NULL,
[featureName] [varchar](50) NOT NULL,
[creationDate] [smalldatetime] NOT NULL,
[status] [int] NOT NULL,
[fieldType] [varchar](50) NOT NULL,
[featureType] [int] NOT NULL,
[featureDesc] [varchar](500) NULL,
CONSTRAINT [PK_Items_Features] PRIMARY KEY CLUSTERED
(
[featureId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]) ON [PRIMARY]
CREATE TABLE [dbo].[Items_MainRel](
[ItemRelId] [int] IDENTITY(1,1) NOT NULL,
[Itemid] [int] NOT NULL,
[featureId] [int] NOT NULL,
CONSTRAINT [PK_Items_MainRel] PRIMARY KEY CLUSTERED
(
[ItemRelId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE TABLE [dbo].[Items_FeatureValues](
[valueId] [int] IDENTITY(1,1) NOT NULL,
[ItemRelId] [int] NOT NULL,
[languageId] [int] NOT NULL,
[FnVarChar] [nvarchar](250) NULL,
[FInt] [int] NULL,
[FImage] [int] NULL,
[FNText] [ntext] NULL,
[FSmallDateTime] [smalldatetime] NULL,
CONSTRAINT [PK_Items_FeatureValues] PRIMARY KEY CLUSTERED
(
[valueId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
ALTER TABLE [dbo].[Items_MainRel] WITH CHECK ADD CONSTRAINT [FK_Items_MainRel_Items_Features] FOREIGN KEY([featureId])
REFERENCES [dbo].[Items_Features] ([featureId])
GO
ALTER TABLE [dbo].[Items_MainRel] CHECK CONSTRAINT [FK_Items_MainRel_Items_Features]
GO
ALTER TABLE [dbo].[Items_MainRel] WITH CHECK ADD CONSTRAINT [FK_Items_MainRel_Items_Main] FOREIGN KEY([Itemid])
REFERENCES [dbo].[Items_Main] ([Itemid])
GO
ALTER TABLE [dbo].[Items_MainRel] CHECK CONSTRAINT [FK_Items_MainRel_Items_Main]
GO
ALTER TABLE [dbo].[Items_FeatureValues] WITH CHECK ADD CONSTRAINT [FK_Items_FeatureValues_Items_MainRel] FOREIGN KEY([ItemRelId])
REFERENCES [dbo].[Items_MainRel] ([ItemRelId])
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Items_FeatureValues] CHECK CONSTRAINT [FK_Items_FeatureValues_Items_MainRel]
GO
INSERT INTO DATA_PreImp (col01,col02,col03,col04)
VALUES('John', 'Smith', '1964', 'NewYork'),
('Calvin', 'Klein', '1960', 'Washington D. C.'),
('Peter', 'Parker', '1974', 'Losangles'),
('Moe', 'Greene', '1928', 'Lasvegas')
INSERT INTO Items_Features (featureRef, featureName, creationDate, [status], fieldType, featureType, featureDesc)
VALUES ('firstname','First_Name', GETDATE(), 0, 'FnVarChar', 3, 'FirstName'),
('lastname','Last_Name', GETDATE(), 0, 'FnVarChar', 3, 'LastName'),
('Birth','Birth', GETDATE(), 0, 'FnVarChar', 3, 'Birth'),
('City','City', GETDATE(), 0, 'FnVarChar', 3, 'City')
The problem was the computed column CUST_CD. After a lot of researching, it seems that the BULK INSERT does not like complex computed types (see Using SQL Server spatial types in SSIS data load). The solution is to removed the computed column and just make it a varchar(20) NULL. Then I created a new Execute SQL Task that updates any NULL rows with the computed value.