This query is taking 2.17 sec of time and no. of rows in output are 128000.All the source tables have approx 100k rows.But the same query some times takes more time during Production hours, which is the cause of slowness in system, as seen in Top 5 highly time taking queries in Oracle 11g. Can we optimize it further.
Note : Indexes are created for all the columns used in Where and On conditions. Also find attached query plan.
SELECT (ROWNUM)ROWNUMBER, RES.*
FROM
(SELECT PIDS,
FILENUM,
FILETYPE,
ATT,
FILESTA,
WSNAME,
CMP5.CLT_NAME AS CLIENT,
CLT_CODE,
CLTREF,
TMTYPE,
ASSOCNAME,
FFCOUNTRY,
APPLICATIONNO,
FF_APPLICATIONNO,
FILING_TRADEMARKTITLE,
FF_TRADEMARKTITLE,
QUERY_TRADEMARKTITLE,
CMP6.DOCNAME AS SEARCH_TRADEMARKTITLE,
CMP7.DOCUMENTNAME as FF_SEARCH_TRADEMARK,
CMP.CLASS AS CLASSES,
CMP4.CLASS AS CLASSESFF,
CMP2.NAME AS PROPRIETER,
CMP2.CODE AS PROPRIETERCODE,
CMP5.CNTRY AS CNTRY,
CMP5.STATE AS STATE,
CMP5.CITY AS CITY
FROM
(SELECT EXT.PID AS PIDS,
EXT.FILE_NUMBER AS FILENUM,
EXT.proceeding_assignmenttype AS FILETYPE,
EXT.ATTORNEY AS ATT,
EXT.FILE_STATUS AS FILESTA,
CURRENTWS AS WSNAME,
EXT.CLIENT_CODE AS CLT_CODE,
EXT.client_ref AS CLTREF,
EXT.FP_APP_TYPE AS TMTYPE,
EXT.ff_nameofassociate AS ASSOCNAME,
EXT.ff_nameofcountry AS FFCOUNTRY,
EXT.filing_trademarkno AS APPLICATIONNO,
EXT.ff_trademarknumber AS FF_APPLICATIONNO,
EXT.FILING_TRADEMARK AS FILING_TRADEMARKTITLE,
EXT.FF_TRADMARK AS FF_TRADEMARKTITLE,
EXT.Q_TRADEMARK AS QUERY_TRADEMARKTITLE
FROM EXT_TRADEMARK EXT
WHERE CURRENTWSSTATUS = '1'
ORDER BY EXT.FILE_NUMBER DESC
)EXTTABLE
LEFT JOIN COMPLEX_TM_CLASS CMP
ON EXTTABLE.PIDS = CMP.PROCESSINSTANCEID
LEFT JOIN COMPLEX_TM_PROPRIEATOR CMP2
ON EXTTABLE.PIDS = CMP2.PROCESSINSTANCEID
LEFT JOIN COMPLEX_TM_FF_CLASS CMP4
ON EXTTABLE.PIDS = CMP4.PROCESSINSTANCEID
LEFT JOIN FA_CLT_MSTR CMP5
ON EXTTABLE.CLT_CODE = CMP5.CLT_COD
LEFT JOIN COMPLEX_TM_SEARCH CMP6
ON EXTTABLE.PIDS = CMP6.PID
LEFT JOIN COMPLEX_TM_FF_SEARCH CMP7
ON EXTTABLE.PIDS = CMP7.PID order by FILENUM desc
) RES ;
Related
My SQL query is taking a large amount of time to run. I wrote a similar query and pit them against each other and this one runs FASTER when a small dataset (10K lines) is used, but about 20-30x slower than the other one when a LARGE dataset (500K+ lines) is used. My first query however does not have ONE column that I need, and I cannot add it without going about it with this different approach.
SELECT a.[RFIDTAGID], a.[JOB_NUMBER], d.[PROJECT_NUMBER], a.[PART_NUMBER], a.[QUANTITY], b.[DESIGNATION] as LOCATION,
c.[DESIGNATION] as CONTAINER, a.[LAST_SEEN_TIME], b.[TYPE], b.[BLDG], d.[PBG], d.[PLANNED_MFG_DELIVERY_DATE], d.[EXTENSION_DATE], a.[ORG_ID]
FROM [LTS].[dbo].[LTS_PACKAGE] as a
LEFT OUTER JOIN (
SELECT [DESIGNATION], [CONTAINER_ID], [LOCATION_ID]
FROM [LTS].[dbo].[LTS_CONTAINER]
) c ON a.[CONTAINER_ID] = c.[CONTAINER_ID]
LEFT OUTER JOIN (
SELECT [DESIGNATION], [TYPE], [BLDG], [LOCATION_ID]
FROM [LTS].[dbo].[LTS_LOCATION]
) b ON a.[LAST_SEEN_LOC_ID] = b.[LOCATION_ID] OR b.[LOCATION_ID] = c.[LOCATION_ID]
INNER JOIN (
SELECT [PBG], [PLANNED_MFG_DELIVERY_DATE], [EXTENSION_DATE], [DISCRETE_JOB_NUMBER], [PROJECT_NUMBER]
FROM [LTS].[dbo].[LTS_DISCRETE_JOB_SUMMARY]
)d ON a.[JOB_NUMBER] = d.[DISCRETE_JOB_NUMBER]
WHERE
d.[PLANNED_MFG_DELIVERY_DATE] <= GETDATE()
AND b.[TYPE] NOT IN('MFG', 'Manufacturing')
AND (b.[DESIGNATION] IS NOT NULL OR c.[DESIGNATION] IS NOT NULL)
ORDER BY [JOB_NUMBER], d.[PLANNED_MFG_DELIVERY_DATE] desc, [RFIDTAGID];
You can see below the usage, 100% is roughly 20,000, whereas my other query is about 900:
Is there something I can do to speed up my query, or where did I bog it down?
Remove inner selects and join directly to the tables:
SELECT a.[RFIDTAGID], a.[JOB_NUMBER], d.[PROJECT_NUMBER], a.[PART_NUMBER], a.[QUANTITY], b.[DESIGNATION] as LOCATION,
c.[DESIGNATION] as CONTAINER, a.[LAST_SEEN_TIME], b.[TYPE], b.[BLDG], d.[PBG], d.[PLANNED_MFG_DELIVERY_DATE], d.[EXTENSION_DATE], a.[ORG_ID]
FROM [LTS].[dbo].[LTS_PACKAGE] a
LEFT OUTER JOIN [LTS].[dbo].[LTS_CONTAINER]
c ON a.[CONTAINER_ID] = c.[CONTAINER_ID]
LEFT OUTER JOIN [dbo].[LTS_LOCATION]
b ON a.[LAST_SEEN_LOC_ID] = b.[LOCATION_ID] OR b.[LOCATION_ID] = c.[LOCATION_ID]
INNER JOIN
[LTS].[dbo].[LTS_DISCRETE_JOB_SUMMARY]
d ON a.[JOB_NUMBER] = d.[DISCRETE_JOB_NUMBER]
WHERE
d.[PLANNED_MFG_DELIVERY_DATE] <= GETDATE()
AND b.[TYPE] NOT IN('MFG', 'Manufacturing')
AND (b.[DESIGNATION] IS NOT NULL OR c.[DESIGNATION] IS NOT NULL)
ORDER BY [JOB_NUMBER], d.[PLANNED_MFG_DELIVERY_DATE] desc, [RFIDTAGID];
I have the below query which takes a while to run, since ir_sales_summary is ~ 2 billion rows:
select c.ChainIdentifier, s.SupplierIdentifier, s.SupplierName, we.Weekend,
sum(sales_units_cy) as TY_unitSales, sum(sales_cost_cy) as TY_costDollars, sum(sales_units_ret_cy) as TY_retailDollars,
sum(sales_units_ly) as LY_unitSales, sum(sales_cost_ly) as LY_costDollars, sum(sales_units_ret_ly) as LY_retailDollars
from ir_sales_summary i
left join Chains c
on c.ChainID = i.ChainID
inner join Suppliers s
on s.SupplierID = i.SupplierID
inner join tmpWeekend we
on we.SaleDate = i.saledate
where year(i.saledate) = '2017'
group by c.ChainIdentifier, s.SupplierIdentifier, s.SupplierName, we.Weekend
(Worth noting, it takes roughly 3 hours to run since it is using a view that brings in data from a legacy service)
I'm thinking there's a way to speed up the filtering, since I just need the data from 2017. Should I be filtering from the big table (i) or be filtering from the much smaller weekending table (which gives us just the week ending dates)?
Try this. This might help, joining a static table as first table in query onto a fact/dynamic table will impact query performance i believe.
SELECT c.ChainIdentifier
,s.SupplierIdentifier
,s.SupplierName
,i.Weekend
,sum(sales_units_cy) AS TY_unitSales
,sum(sales_cost_cy) AS TY_costDollars
,sum(sales_units_ret_cy) AS TY_retailDollars
,sum(sales_units_ly) AS LY_unitSales
,sum(sales_cost_ly) AS LY_costDollars
,sum(sales_units_ret_ly) AS LY_retailDollars
FROM Suppliers s
INNER JOIN (
SELECT we
,weeekend
,supplierid
,chainid
,sales_units_cy
,sales_cost_cy
,sales_units_ret_cy
,sales_units_ly
,sales_cost_ly
,sales_units_ret_ly
FROM ir_sales_summary i
INNER JOIN tmpWeekend we
ON we.SaleDate = i.saledate
WHERE year(i.saledate) = '2017'
) i
ON s.SupplierID = i.SupplierID
INNER JOIN Chains c
ON c.ChainID = i.ChainID
GROUP BY c.ChainIdentifier
,s.SupplierIdentifier
,s.SupplierName
,i.Weekend
Goal
I would like to simplify the following sql query in term of visual length (decrease amount nested sub-selects) and/or performance and/or readability. This query is dedicated for MS Access, that's why there are parenthesis around INNER JOIN.
Explanation
To not lose global view all table relations are shortened after ON clauses.
Table S1 and S2 is the same table S. To work, the query (MS Access to be precise) needed to separate names of S in different sub-selects.
Level 1 query (T1) calculates: qty of defects, qty of production and defect rate in PPM for all production areas and subareas in specific period of time.
Level 2 query (T2) calculates sum of defect rates from in T1 per area and adds some additional information like specific one main user from many attached to areas, area name itself and target taken for specific year and month.
Level 3 query takes all from level 2 and adds some comment defined for specific year and week and also filter all the list down to areas where sum of defect rates exceeds target.
This query works as expected and all the columns in the result table show properlly calculated values and fields.
Is it possible to simplify this query?
Note
I added sql-server tag to reach more pro-users. Sorry if it doesn't fit.
Query
SELECT s1id, s2name, user, target, ratio, C.msg AS msg
FROM
(SELECT s1id, S2.nam AS s2name, (U.lastname & ' ' & U.firstname) AS user, T.m1 AS target, SUM(ratio1) as ratio
FROM ((((
(SELECT S1.id AS s1id, WG.id AS wgid, SUM(IIF(D.ok=False,F.qty,0)) AS nok, SUM(F.qty) AS production, IIF(production > 0, CLNG(nok/production * 1000000), 0) AS ratio1
FROM (((F
INNER JOIN D ON D.x = F.x)
INNER JOIN W ON W.x = D.x)
INNER JOIN WG ON WG.x = W.x)
INNER JOIN S1 ON S1.x = WG.x
WHERE F.entrydate BETWEEN #2017-01-23# And #2017-01-29#
GROUP BY S1.id, WG.id) AS T1
INNER JOIN S2 ON S2.x = T1.x)
INNER JOIN UPS ON UPS.x = S2.x)
INNER JOIN UP ON UP.x = UPS.x)
INNER JOIN U ON U.x = UP.x)
INNER JOIN T ON T.x = S2.x
WHERE UPS.main = true AND UP.positionid = 3 AND T.y = 2017
GROUP BY sectorid, S2.nam, U.lastname, U.firstname, T.m1) AS T2
INNER JOIN C ON C.x = T2.x
WHERE C.yearnum = 2017 AND C.weeknum = 4 AND ratio > target
I have written the following SQL Stored Procedure and because of all the select commands (I think) it's really slow running now the database has been populated with lots of data. Is there a way to optimise it so that it runs much quicker? Currently in an Azure S0 DB it takes around 1:40 min to process. Here's the stored procedure:
USE [cmb2SQL]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[spStockReport] #StockReportId as INT
AS
select
ProductId,
QtyCounted,
LastStockCount,
Purchases,
UnitRetailPrice,
CostPrice,
GrossProfit,
Consumed,
(Consumed * UnitRetailPrice) as ValueOfSales,
(QtyCounted * CostPrice) as StockOnHand,
StockCountId
from (
select
ProductId,
QtyCounted,
LastStockCount,
Purchases,
UnitRetailPrice,
CostPrice,
GrossProfit,
(LastStockCount + Purchases) - QtyCounted as Consumed,
StockCountId
from (
select
distinct
sci.StockCountItem_Product as ProductId,
(Select ISNULL(SUM(Qty), 0) as tmpQty from
(Select Qty from stockcountitems
join stockcounts on stockcountitems.stockcountitem_stockcount = stockcounts.id
where stockcountitem_Product = p.Id and stockcountitem_stockcount = sc.id and stockcounts.stockcount_pub = sc.StockCount_Pub
) as data
) as QtyCounted,
(Select ISNULL(SUM(Qty), 0) as LastStockCount from
(Select Qty from StockCountItems as sci
join StockCounts on sci.StockCountItem_StockCount = StockCounts.Id
join Products on sci.StockCountItem_Product = Products.id
where sci.StockCountItem_Product = p.id and sci.stockcountitem_stockcount =
(select top 1 stockcounts.id from stockcounts
join stockcountitems on stockcounts.id = stockcountitem_stockcount
where stockcountitems.stockcountitem_product = p.id and stockcounts.id < sc.id and StockCounts.StockCount_Pub = sc.StockCount_Pub
order by stockcounts.id desc)
) as data
) as LastStockCount,
(Select ISNULL(SUM(Qty * CaseSize), 0) as Purchased from
(select Qty, Products.CaseSize from StockPurchaseItems
join Products on stockpurchaseitems.stockpurchaseitem_product = products.id
join StockPurchases on stockpurchaseitem_stockpurchase = stockpurchases.id
join Periods on stockpurchases.stockpurchase_period = periods.id
where Products.id = p.Id and StockPurchases.StockPurchase_Period = sc.StockCount_Period and StockPurchases.StockPurchase_Pub = sc.StockCount_Pub) as data
) as Purchases,
sci.RetailPrice as UnitRetailPrice,
sci.CostPrice,
(select top 1 GrossProfit from Pub_Products where Pub_Products.Pub_Product_Product = p.id and Pub_Products.Pub_Product_Pub = sc.StockCount_Pub) as GrossProfit,
sc.Id as StockCountId
from StockCountItems as sci
join StockCounts as sc on sci.StockCountItem_StockCount = sc.Id
join Pubs on sc.StockCount_Pub = pubs.Id
join Periods as pd on sc.StockCount_Period = pd.Id
join Products as p on sci.StockCountItem_Product = p.Id
join Pub_Products as pp on p.Id = pp.Pub_Product_Product
Where StockCountItem_StockCount = #StockReportId and pp.Pub_Product_Pub = sc.StockCount_Pub
Group By sci.CostPrice, sci.StockCountItem_Product, sci.Qty, sc.Id, p.Id, sc.StockCount_Period, pd.Id, sci.RetailPrice, pp.CountPrice, sc.StockCount_Pub
) as data
) as final
GO
As requested here is the execution plan in XML (had to upload it to tinyupload as it exceeds the message character length):
execusionplan.xml
Schema:
Row Counts:
Table row_count
StockPurchaseItems 57511
Products 3116
StockCountItems 60949
StockPurchases 6494
StockCounts 240
Periods 30
Pub_Products 5694
Pubs 7
Without getting into the query rewrite, it's the most expensive and the last thing you should do probably. Try these steps first, one by one, and measure the impact - time, execution plan, and SET STATISTICS IO ON output. Create the baseline first for these metrics. Stop when you achieve the acceptable performance.
First of all, update statistics on relevant tables, I see some of the estimates are way off. Check the exec plan for estimated vs actual rows - any better now?
Create indexes on StockPurchaseItems(StockPurchaseItem_Product) and on StockCountItems(StockCountItem_Product, StockCountItem_StockCount). Check the execution plan, did optimizer consider using the indexes at all?
Add (include) other referenced columns to those two indexes in order to cover the query. Are they used now?
If nothing of above helped, consider breaking the query into smaller ones. Would be nice to have some real data to experiment with to be more specific.
** That "select distinct" smells real bad, are you sure the joins are all ok?
I am trying to join 14 tables in which few tables I need to join using left join.
With the existing data which is around 7000 records,its taking around 10 seconds to execute the below query.I am afraid what if the records are more than million.Please help me improve the performance of the below query.
CREATE proc [dbo].[GetTodaysActualInvoiceItemSoldHistory]
#fromdate datetime,
#todate datetime
as
Begin
select SDID.InvoiceDate as [Sold Date],Cust.custCompanyName as [Sold To] ,
case SQBD.TransferNo when '0' then IVM.VendorName else SQBD.TransferNo end as [Purchase From],
SQBD.BatchSellQty as SoldQty,SQID.SellPrice,
SDID.InvoiceNo as [Sales Invoice No],INV.PRInvoiceNo as [PO Invoice No],INV.PRInvoiceDate as [PO Invoice Date],
SQID.ItemDesc as [Item Description],SQID.NetPrice,SDHM.DeliveryHeaderMasterName as DeliveryHeaderName,
SQID.ItemCode as [Item Code],
SQBD.BatchNo,SQBD.ExpiryDate,SQID.Amount,
SQID.Dept_ID as Dept_ID,
Dept_Name as [Department],SQID.Catg_ID as Catg_ID,
Category_Name as [Category],SQID.Brand_ID as Brand_ID,
BrandName as BrandName, SQID.Manf_Id as Manf_Id,
Manf.ManfName as [Manufacturer],
STM.TaxName, SQID.Tax_ID as Tax_ID,
INV.VendorID as VendorID,
SQBD.ItemID,SQM.Isdeleted,
SDHM.DeliveryHeaderMasterID,Cust.CustomerMasterID
from SD_QuotationMaster SQM
inner join SD_InvoiceDetails SDID on SQM.QuoteID = SDID.QuoteID
inner join SD_QuoteItemDetails SQID on SDID.QuoteID = SQID.QuoteID
inner join SD_QuoteBatchDetails SQBD on SDID.QuoteID = SQBD.QuoteID and SQID.ItemID=SQBD.ItemID
inner join INV_ProductInvoice INV on SQBD.InvoiceID=INV.ProductInvoiceID
inner jOIN INV_VendorMaster IVM ON INV.VendorID = IVM.VendorID
inner jOIN Sys_TaxMaster STM ON SQID.Tax_ID = STM.Tax_ID
inner join Cust_CustomerMaster Cust on SQM.CustomerMasterID = Cust.CustomerMasterID
left jOIN INV_DeptartmentMaster Dept ON SQID.Dept_ID = Dept.Dept_ID
left jOIN INV_BrandMaster BRD ON SQID.Brand_ID = BRD.Brand_ID
left jOIN INV_ManufacturerMaster Manf ON SQID.Manf_Id = Manf.Manf_Id
left join INV_CategoryMaster CAT ON SQID.Catg_ID = CAT.Catg_ID
left join SLRB_DeliveryCustomerMaster SDCM on SQM.CustomerMasterID=SDCM.CustomerMasterID and SQM.DeliveryHeaderMasterID=SDCM.DeliveryHeaderMasterID
left join SLRB_DeliveryHeaderMaster SDHM on SDCM.DeliveryHeaderMasterID=SDHM.DeliveryHeaderMasterID
where (SQM.IsDeleted=0) and SQBD.BatchSellQty > 0
and SDID.InvoiceDate between #fromdate and #todate
order by ItemDesc
End
Only the below tables contain more data while other tables have records less than 20
InvoiceDetails, QuoteMaster, QuoteItemDetails, QuoteBatchDetails ProductInvoice
Below is the link for execution plan
http://jmp.sh/CSZc2x2
Thanks.
Let's start with an obvious error:
(isnull(SQBD.BatchSellQty,0) > 0)
That one is not indexable, so it should not happen. Seriously, BatchSellQty should not be unknown (nullable) in most cases, or you better handle null properly. That field should be indexed and I am not sure I would like that with an isNull - there are likely tons of batches. Also note that a filtered index (condition >0) may work here.
Second, check that you have all proper indices and the execution plan makes sense.
Thids, you have to test with a ton of data. Index statistics may make a difference. Check where the time is spent - it may be tempdb in which case you really need a good tempdb IO speed.... and it is not realted to the input side.
You can try to use query hints to help SQL Server optimizer build a optimal query execution plan. For example, you can force the order of tables will be joined, using FORCE ORDER statement. If you order your tables in order that joins with minimum result size at each step, query will execute faster (may be, needs to try). Example:
We need to A join B join C
If A join B = 2000 records x 1000 records = ~400 records (we suspect this result)
And A join C = 2000 records x 10 records = ~3 records (and this)
And B join C = 1000 records x 10 records = 10 000 records (and this)
In this case optimal order will be
A join C join B = ~3 records x 1000 records = ~3000 records