I am on SQL Server 11. I have the following table that stores the boundary of all LGA in NSW
CREATE TABLE [dbo].[nsw_lga_polygon_shp](
[id] [int] IDENTITY(1,1) NOT NULL,
[geom] [geometry] NULL,
[lg_ply_pid] [nvarchar](15) NULL,
[dt_create] [date] NULL,
[dt_retire] [date] NULL,
[lga_pid] [nvarchar](15) NULL,
[nsw_lga_sh] [date] NULL,
[nsw_lga__1] [date] NULL,
[nsw_lga__2] [nvarchar](100) NULL,
[nsw_lga__3] [nvarchar](100) NULL,
[nsw_lga__4] [date] NULL,
[nsw_lga__5] [nvarchar](15) NULL,
CONSTRAINT [PK_nsw_lga_polygon_shp] PRIMARY KEY CLUSTERED
(
[id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
END
GO
SET IDENTITY_INSERT [dbo].[nsw_lga_polygon_shp] ON;
The table only have 198 rows
When I try to map the result against one of my table that have 36,531 rows, I notice that it have very serious performance issue, it told 5 mins to generate 500 rows.
select
* ,
(select top 1
nsw_lga__2
from
[nsw_lga_polygon_shp]
where
Geom.Filter(geometry::STGeomFromText('point(' +
convert(varchar(100),longitude_GIS )+
' ' +
cast(latitude_GIS as varchar(100))+ ')',4283)) = 1
) LGA
from
Report_A
Is there anyway I can may it run faster? Can I index the geometry column?
Thanks in advance!
Related
how i can improve the performance of the sq statement which contains Varbinary(Max) datatype
Table Structure
CREATE TABLE [dbo].[Table_1](
[EmpID] [numeric](18, 0) NOT NULL,
[SrNo] [numeric](18, 0) NOT NULL,
[Type] [varchar](10) NULL,
[FileName] [varchar](100) NULL,
[D1] [varchar](50) NULL,
[D2] [varchar](50) NULL,
[Data] [varbinary](max) NULL,
[CreatedBy] [varchar](50) NULL,
[CreatedOn] [datetime] NULL,
CONSTRAINT [PK_Table_1] PRIMARY KEY CLUSTERED
(
[EmpID] ASC,
[SrNo] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
the Select Query takes around 2 to 3 seconds to select a single record with
Data
Checked Indexing of the Table
I have two tables as below:
1) UserFavouriteCurrencies
CREATE TABLE [dbo].[UserFavouriteCurrencies](
[Id] [bigint] IDENTITY(1,1) NOT NULL,
[UserId] [int] NULL,
[CurrencyId] [int] NULL,
[EntryDate] [datetime] NULL,
CONSTRAINT [PK_UserFavouriteCurrencies] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
Above table is holding data of user favorite companies
2) CurrencyRates
CREATE TABLE [dbo].[CurrencyRates](
[Id] [bigint] IDENTITY(1,1) NOT NULL,
[CurrencyId] [int] NULL,
[Price] [float] NULL,
[Open_24h] [float] NULL,
[Volume_24h] [float] NULL,
[Low_24h] [float] NULL,
[High_24h] [float] NULL,
[Volume_30d] [float] NULL,
[BestBid] [float] NULL,
[BestAsk] [float] NULL,
[TradeId] [bigint] NULL,
[PriceDate] [datetime] NULL,
[PriceDateTimestamp] [bigint] NULL,
[OpenInterest] [float] NULL,
CONSTRAINT [PK_CoinbaseTickerRatesMaster] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
Above table holds rates details of currencies.
Call : EXEC sp_Getdata 1(Here 1 in UserId of UserFavouriteCurrencies table)
I need all records for particular currencies last latest records in one query as below
DECLARE #Today DATETIME = GETDATE(), #PrevDate DATETIME = GETDATE()<br>
SELECT TOP 1 Price, PriceDate, Volume_24h, PriceDateTimestamp
FROM CurrencyRates
WHERE CurrencyId = #CurrencyId AND PriceDate <= #Today
ORDER BY PriceDateTimestamp DESC
Above query is only select data for one particular currency but I need the data related to user which added on UserFavouriteCurrencies table.
Please help me to write store procedure for this query
Thanks in advance.
For each currency in UserFavouriteCurrencies Table, consider the query below
SELECT
uc.Id -- This is the userId
,cr.Price
,cr.PriceDate,
,cr.Volume_24h
,cr.PriceDateTimestamp
FROM
UserFavouriteCurrencies AS uc
INNER JOIN CurrencyRates AS cr -- See Hints : 1
ON cr.CurrencyId = uc.CurrencyId
WHERE
PriceDate <= GETDATE()
AND
uc.Id = #Id
-- You can add other Predicates Here...
Hints:
Consider using LEFT OUTER JOIN if user's currency may not exist in CurrencyRates
!This will however introduce NULLs which you must handle
I have a very simple SQL query:
select o.Visit_ID
from Datamart.dbo.ww_Orders o
inner join Datamart.dbo.ww_Order_Details on o.Visit_ID = ww_Order_Details.Visit_ID
where o.runstamp = '20160422'
this query takes < 0 seconds to return 11173 rows
When I add the GROUP BY statement:
select o.Visit_ID
from Datamart.dbo.ww_Orders o
inner join Datamart.dbo.ww_Order_Details on o.Visit_ID = ww_Order_Details.Visit_ID
where o.runstamp = '20160422'
group by o.Visit_ID
the server takes 6min 30 sec to retrieve the 3047 rows.
I would expect the GROUP BY query to take not that much longer than the original. How do I go about finding what the issues are? thanks
here are the table definitions:
Orders:
CREATE TABLE [dbo].[ww_Orders](
[Visit_ID] [int] NOT NULL,
[Member_ID] [int] NOT NULL,
[Membership_no] [varchar](20) NULL,
[Member_Card_Num_Orig] [varchar](16) NULL,
[SCV_ID] [int] NULL,
[Meeting_No] [int] NULL,
[Location_Name] [varchar](128) NULL,
[Leader_No] [int] NULL,
[CashAmt] [decimal](18, 2) NULL,
[EFTAmt] [decimal](18, 2) NULL,
[VouchAmt] [decimal](18, 2) NULL,
[Meet_Date] [datetime] NULL,
[runstamp] [varchar](50) NULL,
CONSTRAINT [PK_dbo.ww_Orders] PRIMARY KEY CLUSTERED
(
[Visit_ID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
SET ANSI_PADDING OFF
GO
Order Details:
CREATE TABLE [dbo].[ww_Order_Details](
[ord_det_pk] [int] IDENTITY(1,1) NOT NULL,
[Visit_ID] [int] NOT NULL,
[Item_Code] [nvarchar](50) NULL,
[Item_Name] [nvarchar](50) NULL,
[Qty] [int] NULL,
[Amt] [decimal](18, 2) NULL,
[Category_Code] [nvarchar](20) NULL,
CONSTRAINT [PK_ww_Order_Details] PRIMARY KEY CLUSTERED
(
[ord_det_pk] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
ALTER TABLE [dbo].[ww_Order_Details] WITH CHECK ADD CONSTRAINT [FK_ww_Order_Details_ww_Orders] FOREIGN KEY([Visit_ID])
REFERENCES [dbo].[ww_Orders] ([Visit_ID])
GO
ALTER TABLE [dbo].[ww_Order_Details] CHECK CONSTRAINT [FK_ww_Order_Details_ww_Orders]
GO
I'd add an index onto ww_Order_Details Visit_ID, probably make it the clustered index and drop the index on ord_det_pk. Also it might make more sense as an exists?
select o.Visit_ID
from Datamart.dbo.ww_Orders o
where exists (select 0 from Datamart.dbo.ww_Order_Details where o.Visit_ID = ww_Order_Details.Visit_ID)
and o.runstamp = '20160422'
I usually try to use a common table expression in cases like these, when a part of the query is really fast but an addition of a simple operation makes it really slow, usually it helps.
Try:
WITH CTE AS (
select o.Visit_ID
from Datamart.dbo.ww_Orders o
inner join Datamart.dbo.ww_Order_Details od on o.Visit_ID = od.Visit_ID
where o.runstamp = '20160422'
)
SELECT Visit_ID FROM CTE
group by Visit_ID
If this helps you can try to compare execution plans for your original query and this version to see whats going on
I have a SQL database that has two tables that hold a queue of work to be done and the history of that work:
CREATE TABLE [dbo].[WorkQueue](
[WorkQueueID] [int] IDENTITY(1,1) NOT NULL,
[BatchID] [int] NOT NULL,
[AdHocProjectID] [int] NOT NULL,
[TaskID] [int] NOT NULL,
[CreateDate] [datetime] NOT NULL,
[CompletedDate] [datetime] NULL,
[AlertMessage] [varchar](1024) NULL,
CONSTRAINT [PK_WorkQueue] PRIMARY KEY CLUSTERED
(
[WorkQueueID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[WorkQueueHistory](
[WorkQueueHistoryID] [int] IDENTITY(1,1) NOT NULL,
[WorkQueueID] [int] NOT NULL,
[EmployeeID] [int] NOT NULL,
[BatchBoxID] [int] NOT NULL,
[StartTime] [datetime] NOT NULL,
[EndTime] [datetime] NULL,
[PercentageCompleted] [int] NOT NULL,
[FinishedTask] [bit] NOT NULL,
[comment] [varchar](255) NULL,
CONSTRAINT [PK_WorkQueueHistory] PRIMARY KEY CLUSTERED
(
[WorkQueueHistoryID] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, FILLFACTOR = 80) ON [PRIMARY]
) ON [PRIMARY]
The WorkQueue table contains tasks from batches (Jobs) that need to be worked. Users insert into the WorkQueueHistory table when they start working on a job, and when they finish working on a job that row in the WorkQueueHistory table is updated with their end time and the FinishedTask bit is set if that task was indeed completed 100%.
My question is what is the best way to prevent two users from starting the same task? Right now we don't have enough users for this to be a problem, but as things grow I know there is going to be a concurrency issue if I don't address this.
You should use a transaction with the appropriate isolation level set.
See http://www.blackwasp.co.uk/SQLTransactions.aspx
begin tran
-- select next job to do
-- update job as started
commit
The problem
During the passed few months the below described procedure has worked without any problems a vast majority of the times it has run (on 2008 r2). We have, however, three instances of incorrectly connected data. The question is, what is causing this and how do I remedy it?
DATA_PreImp
sourceid col01 col02 col03 col04 col...
100001 John Smith
100002 Calvin Klein
100003 Peter Parker
100004 Moe Greene
Usually the rendered result is that the attribute is connected to the Items_Main correctly but sometimes (less than 1%) the order is scrambled so that the value of col01 is not connected to the same Items_Main as the value of the rest of the columns.
Any insights as to what is causing this would be most appreciated.
The data moving procedure
We have an SSIS package that transfers data from a flat table called DATA_PreImp to a structure consisting of three related tables (attribute based).
Items_Main should contains one row for each row in DATA_PreImp
Items_Featurevalues contains one row for each column value of a row in DATA_PreImp
Items_MainRel contains the connection between Items_Main and Items_FeatureValues
The first step in the SSIS package inserts the data from DATA_PreImp to Items_Main and inserts the generated identifier into the TARGET_ID column in the empty DATA_PreImpMappingTMP table.
insert into items_main(creationdate, status)
output inserted.itemid into DATA_PreImpMappingTMP(TARGET_ID)
select getdate(), '0' from data_preimp
order by sourceid asc;
The second step in the SSIS package fill the Items_MainRel table with TARGET_ID (Itemid originally) and an identifier for the feature (in this case a 5).
insert into items_mainrel(itemid, featureid)
output inserted.itemrelid into DATA_PreImpMapping2TMP(INDREL_ID)
select TARGET_ID, 5 from DATA_PreImpMappingTMP
order by TARGET_ID asc;
The third step is to fill the SOURCE_ID column in the DATA_PreImpMapping2TMP table with the SOURCE_ID from DATA_PreImp.
with cte as (select sourceid, row_number() over (order by sourceid asc) as row from data_preimp)
update m set m.SOURCE_ID = s.sourceid, m.FEAT_ID = 5
from DATA_PreImpMapping2TMP as m
join cte as s on s.row = m.ROW;
The last step is to fill the Items_FeatureValues table with data from DATA_PreImpMapping2TMP and DATA_PreImp.
insert into items_featurevalues(itemrelid, languageid, fnvarchar)
select DATA_PreImpMapping2TMP.INDREL_ID, 0, data_preimp.col01
from DATA_PreImpMapping2TMP
join data_preimp on (DATA_PreImpMapping2TMP.SOURCE_ID = data_preimp.sourceid)
where FEAT_ID = 5
Data table structure
Here is what is needed to create the scenario:
CREATE TABLE [dbo].[DATA_PreImp](
[sourceid] [bigint] IDENTITY(1,1) NOT NULL,
[col01] [nvarchar](500) NULL,
[col02] [nvarchar](500) NULL,
[col03] [nvarchar](500) NULL,
[col04] [nvarchar](500) NULL,
[col05] [nvarchar](500) NULL,
[col06] [nvarchar](500) NULL,
[col07] [nvarchar](500) NULL,
[col08] [nvarchar](500) NULL,
[col09] [nvarchar](500) NULL,
[col10] [nvarchar](500) NULL,
CONSTRAINT [PK_DATA_PreImp] PRIMARY KEY CLUSTERED
(
[sourceid] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[DATA_PreImpMappingTMP](
[ROW] [int] IDENTITY(1,1) NOT NULL,
[TARGET_ID] [int] NULL,
PRIMARY KEY CLUSTERED
(
[ROW] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[Items_Main](
[Itemid] [int] IDENTITY(1,1) NOT NULL,
[creationDate] [smalldatetime] NOT NULL,
[status] [int] NOT NULL,
[purchdate] [smalldatetime] NULL,
[logindate] [smalldatetime] NULL,
CONSTRAINT [PK_Items_Main] PRIMARY KEY CLUSTERED
(
[Itemid] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[DATA_PreImpMapping2TMP](
[ROW] [int] IDENTITY(1,1) NOT NULL,
[SOURCE_ID] [int] NULL,
[INDREL_ID] [int] NULL,
[FEAT_ID] [int] NULL,
PRIMARY KEY CLUSTERED
(
[ROW] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
CREATE TABLE [dbo].[Items_Features](
[featureId] [int] IDENTITY(1,1) NOT NULL,
[featureRef] [varchar](15) NOT NULL,
[featureName] [varchar](50) NOT NULL,
[creationDate] [smalldatetime] NOT NULL,
[status] [int] NOT NULL,
[fieldType] [varchar](50) NOT NULL,
[featureType] [int] NOT NULL,
[featureDesc] [varchar](500) NULL,
CONSTRAINT [PK_Items_Features] PRIMARY KEY CLUSTERED
(
[featureId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]) ON [PRIMARY]
CREATE TABLE [dbo].[Items_MainRel](
[ItemRelId] [int] IDENTITY(1,1) NOT NULL,
[Itemid] [int] NOT NULL,
[featureId] [int] NOT NULL,
CONSTRAINT [PK_Items_MainRel] PRIMARY KEY CLUSTERED
(
[ItemRelId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
GO
CREATE TABLE [dbo].[Items_FeatureValues](
[valueId] [int] IDENTITY(1,1) NOT NULL,
[ItemRelId] [int] NOT NULL,
[languageId] [int] NOT NULL,
[FnVarChar] [nvarchar](250) NULL,
[FInt] [int] NULL,
[FImage] [int] NULL,
[FNText] [ntext] NULL,
[FSmallDateTime] [smalldatetime] NULL,
CONSTRAINT [PK_Items_FeatureValues] PRIMARY KEY CLUSTERED
(
[valueId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
ALTER TABLE [dbo].[Items_MainRel] WITH CHECK ADD CONSTRAINT [FK_Items_MainRel_Items_Features] FOREIGN KEY([featureId])
REFERENCES [dbo].[Items_Features] ([featureId])
GO
ALTER TABLE [dbo].[Items_MainRel] CHECK CONSTRAINT [FK_Items_MainRel_Items_Features]
GO
ALTER TABLE [dbo].[Items_MainRel] WITH CHECK ADD CONSTRAINT [FK_Items_MainRel_Items_Main] FOREIGN KEY([Itemid])
REFERENCES [dbo].[Items_Main] ([Itemid])
GO
ALTER TABLE [dbo].[Items_MainRel] CHECK CONSTRAINT [FK_Items_MainRel_Items_Main]
GO
ALTER TABLE [dbo].[Items_FeatureValues] WITH CHECK ADD CONSTRAINT [FK_Items_FeatureValues_Items_MainRel] FOREIGN KEY([ItemRelId])
REFERENCES [dbo].[Items_MainRel] ([ItemRelId])
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Items_FeatureValues] CHECK CONSTRAINT [FK_Items_FeatureValues_Items_MainRel]
GO
INSERT INTO DATA_PreImp (col01,col02,col03,col04)
VALUES('John', 'Smith', '1964', 'NewYork'),
('Calvin', 'Klein', '1960', 'Washington D. C.'),
('Peter', 'Parker', '1974', 'Losangles'),
('Moe', 'Greene', '1928', 'Lasvegas')
INSERT INTO Items_Features (featureRef, featureName, creationDate, [status], fieldType, featureType, featureDesc)
VALUES ('firstname','First_Name', GETDATE(), 0, 'FnVarChar', 3, 'FirstName'),
('lastname','Last_Name', GETDATE(), 0, 'FnVarChar', 3, 'LastName'),
('Birth','Birth', GETDATE(), 0, 'FnVarChar', 3, 'Birth'),
('City','City', GETDATE(), 0, 'FnVarChar', 3, 'City')
The problem was the computed column CUST_CD. After a lot of researching, it seems that the BULK INSERT does not like complex computed types (see Using SQL Server spatial types in SSIS data load). The solution is to removed the computed column and just make it a varchar(20) NULL. Then I created a new Execute SQL Task that updates any NULL rows with the computed value.