Performance issue using IsNull function in the Select statement - sql

I have a financial application. I have ViewHistoricInstrumentValue which has rows like this
instrument1, date1, price, grossValue, netValue
instrument2, date1, price, grossValue, netValue
...
instrument1, date2, price, grossValue, netValue
...
My views are complicated but the db itself is small (4000 transactions). ViewHistoricInstrumentValue was executed in less than 1 second before I added the next CTE to the view. After that it takes 26s. ActualEvaluationPrice is the price for instrumentX at dateY. If this value is missing from HistoricPrice table then I find the previous price for instrumentX.
, UsedEvaluationPriceCte AS (
SELECT *
, isnull(ActualEvaluationPrice,
(select top 1 HistoricPrice.Price -- PreviousPrice
from HistoricPrice JOIN ValidDate
on HistoricPrice.DateId = ValidDate.Id
and HistoricPrice.InstrumentId = StartingCte.InstrumentId
and ValidDate.[Date] < StartingCte.DateValue
order by ValidDate.[Date]))
as UsedEvaluationPrice
FROM StartingCte
)
My problem is that the execution time increased needlessly. Right now the HistoricPrice table has no missing value so ActualEvaluationPrice is never null, so the previous price should be never determined.
ViewHistoricInstrumentValue returns 1815 rows. One other mystery is that the first query takes 26s, but the second only 2s.
SELECT * FROM [ViewHistoricInstrumentValue]
SELECT top(2000) * FROM [ViewHistoricInstrumentValue]
Appendix
The execution plan: https://www.dropbox.com/s/5st69uhjkpd3b5y/IsNull.sqlplan?dl=0
The same plan: https://www.brentozar.com/pastetheplan/?id=rk9bK1Wiv
The view:
ALTER VIEW [dbo].[ViewHistoricInstrumentValue] AS
WITH StartingCte AS (
SELECT
HistoricInstrumentValue.DateId
, ValidDate.Date as DateValue
, TransactionId
, TransactionId AS [Row]
, AccountId
, AccountName
, ViewTransaction.InstrumentId
, ViewTransaction.InstrumentName
, OpeningDate
, OpeningPrice
, Price AS ActualEvaluationPrice
, ClosingDate
, Amount
, isnull(ViewTransaction.FeeValue, 0) as FeeValue
, HistoricInstrumentValue.Id AS Id
FROM ViewBriefHistoricInstrumentValue as HistoricInstrumentValue
JOIN ValidDate on HistoricInstrumentValue.DateId = ValidDate.Id
JOIN ViewTransaction ON ViewTransaction.Id = HistoricInstrumentValue.TransactionId
left JOIN ViewHistoricPrice ON ViewHistoricPrice.DateId = HistoricInstrumentValue.DateId AND
ViewHistoricPrice.InstrumentId = ViewTransaction.InstrumentId
)
, UsedEvaluationPriceCte AS (
SELECT *
, isnull(ActualEvaluationPrice,
(select top 1 HistoricPrice.Price -- PreviousPrice
from HistoricPrice JOIN ValidDate
on HistoricPrice.DateId = ValidDate.Id
and HistoricPrice.InstrumentId = StartingCte.InstrumentId
and ValidDate.[Date] < StartingCte.DateValue
order by ValidDate.[Date]))
as UsedEvaluationPrice
FROM StartingCte
)
, GrossEvaluationValueCte AS (
SELECT *
, Amount * UsedEvaluationPrice AS GrossEvaluationValue
, (UsedEvaluationPrice - OpeningPrice) * Amount AS GrossCapitalGains
FROM UsedEvaluationPriceCte
)
, CapitalGainsTaxCte AS (
SELECT *
, dbo.MyMax(GrossCapitalGains * 0.15, 0) AS CapitalGainsTax
FROM GrossEvaluationValueCte
)
, IsOpenCte AS (
SELECT
DateId
, DateValue
, TransactionId
, [Row]
, AccountId
, AccountName
, InstrumentId
, InstrumentName
, OpeningDate
, OpeningPrice
, ActualEvaluationPrice
, UsedEvaluationPrice
, ClosingDate
, Amount
, GrossEvaluationValue
, GrossCapitalGains
, CapitalGainsTax
, FeeValue
, GrossEvaluationValue - CapitalGainsTax - FeeValue AS NetEvaluationValue
, GrossCapitalGains - CapitalGainsTax - FeeValue AS NetUnrealizedGains
, CASE WHEN ClosingDate IS NULL OR DateValue < ClosingDate
THEN CAST(1 AS BIT)
ELSE CAST(0 AS BIT)
END
AS IsOpen
, convert(NVARCHAR, DateValue, 20) + cast([Id] AS NVARCHAR(MAX)) AS Temp
, Id
FROM CapitalGainsTaxCte
)
Select * from IsOpenCte

I have no idea what your query is supposed to be doing. But this process:
ActualEvaluationPrice is the price for instrumentX at dateY. If this value is missing from HistoricPrice table then I find the previous price for instrumentX.
is handled easily with lag():
select vhiv.*
coalesce(vhiv.ActualEvaluationPrice,
lag(vhiv.ActualEvaluationPrice) over (partition by vhiv.InstrumentId order by DateValue)
) as UsedEvaluationPrice
from ViewHistoricInstrumentValue vhiv;
Note: If you need to filter out certain dates by joining to ValidDates, you can include the JOIN in the query. However, that is not part of the problem statement.

Related

Remove rows where one row offsets the next using accounting rules

I have a view of materials data which contains what was purchased and reversals of some of the purchases. I need a query that removes records that have reversals of purchase transactions. NOTE: The view does not have a primary key.
In the example I need to remove the first two rows as the second row offsets the first row because it reverses the purchase, but I need to keep the third row. Any ideas?
Here is the SQL for the view:
SELECT LEFT(mi.Plnt, 3) AS SBUID ,
oth.EQUIP AS PROJECTID ,
ms.Req_No AS GI ,
ms.Req_Item AS GI_LINE ,
CONVERT(VARCHAR(11), [Doc_Date], 100) + ' 12:00 AM' AS DOC_DATE ,
mi.[SLoc] AS SLOC ,
[Material] AS MATERIAL ,
mi.[Description] AS MATERIAL_DESCRIPTION ,
[Qty] AS QUANTITY ,
mi.[UoM] AS UOM ,
CASE WHEN mi.Mvt IN ( '101', '103', '105', '123', '261' ) THEN
mi.Amount
ELSE mi.Amount * -1
END AS Cost ,
mi.Amount AS EXT_ORG_COST ,
mi.PO AS [PO] ,
mi.Batch ,
mi.Vendor AS VENDOR ,
mi.VendorName AS VENDOR_NAME ,
at.AC_Group AS AC_TYPE ,
[Mvt] AS MVT
FROM [dbo].[MatIssued] mi
INNER JOIN dbo.OrderTableHistory oth ON oth.SUB_ORDER = mi.SubOrder
INNER JOIN dbo.Aircraft_Information2 ai ON ai.Equip = oth.EQUIP
INNER JOIN dbo.RFC_AcftTypeList at ON at.ID = ai.AC_TypeID
LEFT OUTER JOIN dbo.MatStatus ms ON ms.MPN = mi.Material
AND ms.SubOrder = mi.SubOrder
WHERE mi.Plnt IN ( '9131', '9132' )
AND mi.Mvt IN ( '101', '102', '103', '104', '105', '106', '122', '123' ,
'261' ,'262' )
AND mi.Doc_Date >= DATEADD(YEAR, -1, GETDATE())
ORDER BY mi.PO ,
mi.Batch ,
PROJECTID ,
mi.Mvt;
Some assumptions, based on your screenshot:
Reversals have same DOC_DATE as purchases
Reversals have same Batch as purchases
If the above assumptions are correct, try something like this:
DELETE FROM t
FROM MyTable t
WHERE EXISTS (
SELECT 1
FROM MyTable t2
WHERE
-- Join to outer table
t2.SLOC = t.SLOC
AND t2.MATERIAL = t.MATERIAL
AND t2.QUANTITY = t.QUANTITY
AND t2.PO = t.PO
AND t2.Batch = t.Batch
AND t2.VENDOR = t.VENDOR
GROUP BY SLOC, MATERIAL, QUANTITY, PO, Batch, VENDOR
HAVING COUNT(*) = 2 -- There are 2 matching rows
AND -MIN(QUANTITY) = MAX(QUANTITY) -- Minimum quantity negates Maximum quantity
AND MIN(COST) + MAX(COST) = 0 -- Costs cancel each other out
AND MIN(CASE WHEN Cost > 0 THEN DOC_DATE END) <= MIN(CASE WHEN Cost < 0 THEN DOC_DATE END) -- Purchase DOC_DATE less than or equal to reversal DOC_DATE
AND MIN(MVT) = MAX(MVT) + 1 -- Correlate purchase and reversal movement
AND (t.DOC_DATE = MIN(DOC_DATE) OR t.DOC_DATE = MAX(DOC_DATE)) -- Join to outer table
)

Calculate date in SQL Query

I am trying to calculate a date within an SQL query which is the date_required add the number of days in despatchdate - I am currently getting the error Invalid column name 'despatchdays'. If the despatchdays is null or "" then I want it to default to -14.
SELECT order_no, order_line_no, currency, product, address1,description, date_required,
(SELECT despatchdays
FROM scheme.PDBtblSuppliers
WHERE SupplierIDFK = dbo.qryArrears2.supplier) AS despatchdays,
DATEADD(d, despatchdays, date_required) AS despatchdate
FROM dbo.qryArrears2
How can I optimise this to get it working ?
I am using SQL 2000.
You can use:
SELECT order_no
, order_line_no
, currency
, product
, address1
,description
, date_required
, DATEADD(day, coalesce(pd.despatchdays, -14), date_required) AS despatchdate
FROM dbo.qryArrears2 qa
INNER JOIN scheme.PDBtblSuppliers pd On pd.SupplierIDFK = qa.supplier
But it may give you more rows if there are several rows in PDBtblSuppliers for a supplier in qryArrears2.
You can also move despatchdate query inside the DATEADD:
SELECT * FROM (
SELECT order_no
, order_line_no
, currency
, product
, address1
, description
, date_required
, DATEADD(day
, coalesce((SELECT despatchdays FROM scheme.PDBtblSuppliers WHERE SupplierIDFK = qa.supplier), -14)
, date_required
) AS despatchdate
FROM dbo.qryArrears2 qa
) as d
WHERE despatchdate = '20150101'
coalesce(despatchdays, -14) will replace despatchdays by -14 if despatchdays is NULL.
If date_required is a (var)char, you should replace it by a date in your table or cast is to data(time): CAST(date_required as date)
Can't you just JOIN the table and then use it like this:
SELECT
order_no,
order_line_no,
currency,
product,
address1,
description,
date_required,
despatchdays,
DATEADD(d, despatchdays, date_required) AS despatchdate
FROM
dbo.qryArrears2
JOIN scheme.PDBtblSuppliers
ON SupplierIDFK = dbo.qryArrears2.supplier
Update:
You can cast the date_required to a datetime like this:
CAST(date_required AS DATETIME)

recursive common table expression with a view

This is what Im trying to do:
Create view vDetailsCommunications as
WITH Tickets AS
(
SELECT CallLog.CallID
, CallLog.RecvdDate
, Detail.ReqEffDate
, Asgnmnt.DateAcknow
, Asgnmnt.DateResolv
, Asgnmnt.Assignee
, Asgnmnt.GroupName
, CallLog.CallType
, Detail.ActionReq
, Detail.action_type
, (SELECT [Days] FROM NonWorkingDays(Asgnmnt.DateAcknow, Asgnmnt.DateResolv) AS NonWorkingDays_1) AS [Working Days]
, DATEDIFF(day, Asgnmnt.DateAcknow, Asgnmnt.DateResolv) AS [Days]
, (ROW_NUMBER() OVER(PARTITION BY Asgnmnt.CallID ORDER BY Asgnmnt.DateAcknow)) AS [Row Number]
FROM CallLog
INNER JOIN Detail ON CallLog.CallID = Detail.CallID
INNER JOIN Asgnmnt ON CallLog.CallID = Asgnmnt.CallID
WHERE (CallLog.CallType = 'ID Request-PAF')
AND (Detail.ActionReq = 'Hiring Action')
AND (DATEDIFF (DAY, Asgnmnt.DateAcknow, Asgnmnt.DateResolv) BETWEEN 0 AND 99)
AND (Asgnmnt.GroupName IN ('ID Admin', 'Systems Admin'))
AND (Detail.action_type IN ('Applicant Hire', 'Re-Hire'))
)
SELECT *
FROM Tickets
WHERE [Row Number] = 1
you need to define a schema in this function: NonWorkingDays.
ex:
SELECT [Days] FROM dbo.NonWorkingDays(Asgnmnt.DateAcknow, Asgnmnt.DateResolv)

Adding total column and row to sql server pivot

I have a pivot table that I believe to be working. I want to add a total column and a total row to this pivot. Here is the code for the pivot table...
SELECT Month,
[N] AS Expected,
[R] AS Requested,
[T] AS Tires,
[U] AS Unexpected,
[D] AS Damage
FROM (
SELECT CustomerNo,
DATEPART(mm,InvoiceDate) AS Month,
[Type],
SUM(Total) AS Cost
FROM tbl_PM_History
WHERE (InvoiceDate >= #Start)
AND (InvoiceDate <= #End)
AND (CustomerNo = #Cust)
GROUP BY CustomerNo,
DATEPART(mm,InvoiceDate),
TYPE
) p
PIVOT (SUM(Cost) FOR [Type] IN ([N],[R],[T],[U],[D]))AS pvt
ORDER BY Month
Here's a fairly quick way (in programming time) to achieve what you want:
;
WITH details -- Wrap your original query in a CTE so as to encapsulate all your calculations
AS (
SELECT Month,
[N] AS Expected,
[R] AS Requested,
[T] AS Tires,
[U] AS Unexpected,
[D] AS Damage,
ISNULL([N],0) + ISNULL([R],0) + ISNULL([T],0) + ISNULL([U],0) + ISNULL([D], 0) as Total
FROM (
SELECT CustomerNo,
DATEPART(mm,InvoiceDate) AS Month,
[Type],
SUM(Total) AS Cost
FROM tbl_PM_History
WHERE (InvoiceDate >= #Start)
AND (InvoiceDate <= #End)
AND (CustomerNo = #Cust)
GROUP BY CustomerNo,
DATEPART(mm,InvoiceDate),
TYPE
) p
PIVOT (SUM(Cost) FOR [Type] IN ([N],[R],[T],[U],[D])) AS pvt
)
, summary // Use another CTE to add a total line, using UNION ALL
AS (
SELECT Month
, Expected,
, Requested
, Tires
, Unexpected
, Damage
, Total
, 0 as RecordCode
FROM details
UNION ALL
SELECT Null
, SUM(Expected)
, SUM(Requested)
, SUM(Tires)
, SUM(Unexpected)
, SUM(Damage)
, SUM(Total)
, 1
FROM details
)
SELECT Month -- Do your actual sorting.
, Expected,
, Requested
, Tires
, Unexpected
, Damage
, Total
FROM summary
ORDER by RecordCode, Month

SQL query help to generate data

Below the query I created to get certain itemnumbers, qty ordered and price and others from the database. The problem is that sometimes an order doesn't contain 20 itemsnumbers but only 2. Now my question is if it's possible to fill the spaces with other itemnumbers random from the DB. It doesn't need to be correct because it's just for testing.
So can anybody help?
select
t.*,
-- THE THREE SUMVAT VALUES BELOW ARE VERY IMPORTANT. THEY ARE ONLY CORRECT HOWEVER WHEN THERE ARE NO NULL VALUES INVOLVED IN THE MATH,
-- I.E. WHEN THERE ARE 20 ITEMS/QTYS/PRICES INVOLVED WITH A CERTAIN ORDER_NO
((t.QTY1*t.PRICE1)+(t.QTY2*t.PRICE2)+(t.QTY3*t.PRICE3)+(t.QTY4*t.PRICE4)+(t.QTY5*t.PRICE5)) SUMVAT0, -- example: 5123.45 <- lines 1-5: Q*P
((t.QTY6*t.PRICE6)+(t.QTY7*t.PRICE7)+(t.QTY8*t.PRICE8)+(t.QTY9*t.PRICE9)+(t.QTY10*t.PRICE10)+(t.QTY11*t.PRICE11)+(t.QTY12*t.PRICE12)+(t.QTY13*t.PRICE13)+(t.QTY14*t.PRICE14)+(t.QTY15*t.PRICE15))
SUMVAT6, -- example: 1234.56 <- lines 6-15: Q*P
((t.QTY16*t.PRICE16)+(t.QTY17*t.PRICE17)+(t.QTY18*t.PRICE18)+(t.QTY19*t.PRICE19)+(t.QTY20*t.PRICE20)) SUMVAT19 -- example: 4567.89 <- lines 16-20: Q*P
from (
select
(to_char(p.vdate, 'YYYYMMDD') || to_char(sysdate, 'HH24MISS')) DT,
(to_char(p.vdate, 'YYYY-MM-DD') ||'T' || to_char(sysdate, 'HH24:MI:') || '00') DATETIME,
(to_char(orh.written_date, 'YYYY-MM-DD') ||'T00:00:00') DATETIME2,
orh.supplier FAKE_GLN,
y.*
from (
select
x.order_no ORDNO
, max(decode(r,1 ,x.item,null)) FAKE_GTIN1
, max(decode(r,2 ,x.item,null)) FAKE_GTIN2
, max(decode(r,3 ,x.item,null)) FAKE_GTIN3
, max(decode(r,4 ,x.item,null)) FAKE_GTIN4
, max(decode(r,5 ,x.item,null)) FAKE_GTIN5
, max(decode(r,6 ,x.item,null)) FAKE_GTIN6
, max(decode(r,7 ,x.item,null)) FAKE_GTIN7
, max(decode(r,8 ,x.item,null)) FAKE_GTIN8
, max(decode(r,9 ,x.item,null)) FAKE_GTIN9
, max(decode(r,10,x.item,null)) FAKE_GTIN10
, max(decode(r,11,x.item,null)) FAKE_GTIN11
, max(decode(r,12,x.item,null)) FAKE_GTIN12
, max(decode(r,13,x.item,null)) FAKE_GTIN13
, max(decode(r,14,x.item,null)) FAKE_GTIN14
, max(decode(r,15,x.item,null)) FAKE_GTIN15
, max(decode(r,16,x.item,null)) FAKE_GTIN16
, max(decode(r,17,x.item,null)) FAKE_GTIN17
, max(decode(r,18,x.item,null)) FAKE_GTIN18
, max(decode(r,19,x.item,null)) FAKE_GTIN19
, max(decode(r,20,x.item,null)) FAKE_GTIN20
, max(decode(r,1 ,x.qty_ordered,null)) QTY1
, max(decode(r,2 ,x.qty_ordered,null)) QTY2
, max(decode(r,3 ,x.qty_ordered,null)) QTY3
, max(decode(r,4 ,x.qty_ordered,null)) QTY4
, max(decode(r,5 ,x.qty_ordered,null)) QTY5
, max(decode(r,6 ,x.qty_ordered,null)) QTY6
, max(decode(r,7 ,x.qty_ordered,null)) QTY7
, max(decode(r,8 ,x.qty_ordered,null)) QTY8
, max(decode(r,9 ,x.qty_ordered,null)) QTY9
, max(decode(r,10,x.qty_ordered,null)) QTY10
, max(decode(r,11,x.qty_ordered,null)) QTY11
, max(decode(r,12,x.qty_ordered,null)) QTY12
, max(decode(r,13,x.qty_ordered,null)) QTY13
, max(decode(r,14,x.qty_ordered,null)) QTY14
, max(decode(r,15,x.qty_ordered,null)) QTY15
, max(decode(r,16,x.qty_ordered,null)) QTY16
, max(decode(r,17,x.qty_ordered,null)) QTY17
, max(decode(r,18,x.qty_ordered,null)) QTY18
, max(decode(r,19,x.qty_ordered,null)) QTY19
, max(decode(r,20,x.qty_ordered,null)) QTY20
, max(decode(r,1 ,x.unit_cost,null)) PRICE1
, max(decode(r,2 ,x.unit_cost,null)) PRICE2
, max(decode(r,3 ,x.unit_cost,null)) PRICE3
, max(decode(r,4 ,x.unit_cost,null)) PRICE4
, max(decode(r,5 ,x.unit_cost,null)) PRICE5
, max(decode(r,6 ,x.unit_cost,null)) PRICE6
, max(decode(r,7 ,x.unit_cost,null)) PRICE7
, max(decode(r,8 ,x.unit_cost,null)) PRICE8
, max(decode(r,9 ,x.unit_cost,null)) PRICE9
, max(decode(r,10,x.unit_cost,null)) PRICE10
, max(decode(r,11,x.unit_cost,null)) PRICE11
, max(decode(r,12,x.unit_cost,null)) PRICE12
, max(decode(r,13,x.unit_cost,null)) PRICE13
, max(decode(r,14,x.unit_cost,null)) PRICE14
, max(decode(r,15,x.unit_cost,null)) PRICE15
, max(decode(r,16,x.unit_cost,null)) PRICE16
, max(decode(r,17,x.unit_cost,null)) PRICE17
, max(decode(r,18,x.unit_cost,null)) PRICE18
, max(decode(r,19,x.unit_cost,null)) PRICE19
, max(decode(r,20,x.unit_cost,null)) PRICE20
from (
select
rank() over (partition by oh.order_no order by ol.item asc) r,
oh.supplier,
oh.order_no,
oh.written_date,
ol.item,
ol.qty_ordered,
ol.unit_cost
from
ordhead oh
JOIN ordloc ol ON oh.order_no = ol.order_no
where
-- count(numrows) = 1500
not unit_cost is null
-- and ol.order_no in (6181,6121)
) x
group by x.order_no
) y
JOIN ordhead orh ON orh.order_no = y.ORDNO,
period p
) t
;
Without being able to really test this, you might try something like this. Replace the inline view 'x' with this:
FROM (
WITH q AS (
SELECT LEVEL r, TO_CHAR(TRUNC(dbms_random.value*1000,0)) item
, TRUNC(dbms_random.value*100,0) qty_ordered
, TRUNC(dbms_random.value*10,2) unit_cost
FROM dual CONNECT BY LEVEL <= 20
)
SELECT COALESCE(x1.r, q.r) r, supplier, order_no, written_date
, COALESCE(x1.item, q.item) item
, COALESCE(x1.qty_ordered, q.qty_ordered) qty_ordered
, COALESCE(x1.unit_cost, q.unit_cost) unit_cost
FROM (SELECT ROW_NUMBER() OVER (PARTITION BY oh.order_no ORDER BY ol.item ASC) r
, oh.supplier
, oh.order_no
, oh.written_date
, ol.item
, ol.qty_ordered
, ol.unit_cost
FROM ordhead oh JOIN ordloc ol ON oh.order_no = ol.order_no
WHERE NOT unit_cost IS NULL) x1 RIGHT JOIN q ON x1.r = q.r
) x
GROUP BY x.order_no
The WITH clause will give you a table with 20 rows of random data. Outer join that with your old 'x' data and you will be guaranteed 20 rows of data. You might not need to cast the item as a varchar2 depending on data. (N.B., I finally found a query that it makes sense to use a RIGHT JOIN with. See this SO question)
I'm not quite sure what you're trying to do with the GROUP BY and MAX stuff? In the future it would be helpful to condense your examples into something others can easily test, a minimal case that gets your point across.
I also incorporated #Kevin's good suggestion to use ROW_NUMBER instead of RANK.
very difficult to understand...
i think you might be ok if you put a 0 instead of null in the price values...
, max(decode(r,18,x.unit_cost,0)) PRICE18
and
, max(decode(r,20,x.qty_ordered,0)) QTY20
then at least the math should work.
Rank will not guarantee a sequential count of the items in the groups there, may be gaps when you have several rows with the same value.
for a decent explanation see:
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:2920665938600
I think you need to use row_number