I have a financial application. I have ViewHistoricInstrumentValue which has rows like this
instrument1, date1, price, grossValue, netValue
instrument2, date1, price, grossValue, netValue
...
instrument1, date2, price, grossValue, netValue
...
My views are complicated but the db itself is small (4000 transactions). ViewHistoricInstrumentValue was executed in less than 1 second before I added the next CTE to the view. After that it takes 26s. ActualEvaluationPrice is the price for instrumentX at dateY. If this value is missing from HistoricPrice table then I find the previous price for instrumentX.
, UsedEvaluationPriceCte AS (
SELECT *
, isnull(ActualEvaluationPrice,
(select top 1 HistoricPrice.Price -- PreviousPrice
from HistoricPrice JOIN ValidDate
on HistoricPrice.DateId = ValidDate.Id
and HistoricPrice.InstrumentId = StartingCte.InstrumentId
and ValidDate.[Date] < StartingCte.DateValue
order by ValidDate.[Date]))
as UsedEvaluationPrice
FROM StartingCte
)
My problem is that the execution time increased needlessly. Right now the HistoricPrice table has no missing value so ActualEvaluationPrice is never null, so the previous price should be never determined.
ViewHistoricInstrumentValue returns 1815 rows. One other mystery is that the first query takes 26s, but the second only 2s.
SELECT * FROM [ViewHistoricInstrumentValue]
SELECT top(2000) * FROM [ViewHistoricInstrumentValue]
Appendix
The execution plan: https://www.dropbox.com/s/5st69uhjkpd3b5y/IsNull.sqlplan?dl=0
The same plan: https://www.brentozar.com/pastetheplan/?id=rk9bK1Wiv
The view:
ALTER VIEW [dbo].[ViewHistoricInstrumentValue] AS
WITH StartingCte AS (
SELECT
HistoricInstrumentValue.DateId
, ValidDate.Date as DateValue
, TransactionId
, TransactionId AS [Row]
, AccountId
, AccountName
, ViewTransaction.InstrumentId
, ViewTransaction.InstrumentName
, OpeningDate
, OpeningPrice
, Price AS ActualEvaluationPrice
, ClosingDate
, Amount
, isnull(ViewTransaction.FeeValue, 0) as FeeValue
, HistoricInstrumentValue.Id AS Id
FROM ViewBriefHistoricInstrumentValue as HistoricInstrumentValue
JOIN ValidDate on HistoricInstrumentValue.DateId = ValidDate.Id
JOIN ViewTransaction ON ViewTransaction.Id = HistoricInstrumentValue.TransactionId
left JOIN ViewHistoricPrice ON ViewHistoricPrice.DateId = HistoricInstrumentValue.DateId AND
ViewHistoricPrice.InstrumentId = ViewTransaction.InstrumentId
)
, UsedEvaluationPriceCte AS (
SELECT *
, isnull(ActualEvaluationPrice,
(select top 1 HistoricPrice.Price -- PreviousPrice
from HistoricPrice JOIN ValidDate
on HistoricPrice.DateId = ValidDate.Id
and HistoricPrice.InstrumentId = StartingCte.InstrumentId
and ValidDate.[Date] < StartingCte.DateValue
order by ValidDate.[Date]))
as UsedEvaluationPrice
FROM StartingCte
)
, GrossEvaluationValueCte AS (
SELECT *
, Amount * UsedEvaluationPrice AS GrossEvaluationValue
, (UsedEvaluationPrice - OpeningPrice) * Amount AS GrossCapitalGains
FROM UsedEvaluationPriceCte
)
, CapitalGainsTaxCte AS (
SELECT *
, dbo.MyMax(GrossCapitalGains * 0.15, 0) AS CapitalGainsTax
FROM GrossEvaluationValueCte
)
, IsOpenCte AS (
SELECT
DateId
, DateValue
, TransactionId
, [Row]
, AccountId
, AccountName
, InstrumentId
, InstrumentName
, OpeningDate
, OpeningPrice
, ActualEvaluationPrice
, UsedEvaluationPrice
, ClosingDate
, Amount
, GrossEvaluationValue
, GrossCapitalGains
, CapitalGainsTax
, FeeValue
, GrossEvaluationValue - CapitalGainsTax - FeeValue AS NetEvaluationValue
, GrossCapitalGains - CapitalGainsTax - FeeValue AS NetUnrealizedGains
, CASE WHEN ClosingDate IS NULL OR DateValue < ClosingDate
THEN CAST(1 AS BIT)
ELSE CAST(0 AS BIT)
END
AS IsOpen
, convert(NVARCHAR, DateValue, 20) + cast([Id] AS NVARCHAR(MAX)) AS Temp
, Id
FROM CapitalGainsTaxCte
)
Select * from IsOpenCte
I have no idea what your query is supposed to be doing. But this process:
ActualEvaluationPrice is the price for instrumentX at dateY. If this value is missing from HistoricPrice table then I find the previous price for instrumentX.
is handled easily with lag():
select vhiv.*
coalesce(vhiv.ActualEvaluationPrice,
lag(vhiv.ActualEvaluationPrice) over (partition by vhiv.InstrumentId order by DateValue)
) as UsedEvaluationPrice
from ViewHistoricInstrumentValue vhiv;
Note: If you need to filter out certain dates by joining to ValidDates, you can include the JOIN in the query. However, that is not part of the problem statement.
,
I want to get the percentage increase in price by Country and City based on latest transaction date and date of Previous Transaction.
How can I Query this? I am not getting it. This is What I have tried:
SELECT Country,City, Price
From tbl
Group by Country,City
Percentage increase = [( Latest Price - Previous Price ) / Previous Price] * 100
Expected Outout:
Unique Country and City Name + Percentage increase in Price.
Country | City | Percentage
This might be overly complicated.
Set up some random data:
IF OBJECT_ID('tempdb..#Cities') IS NOT NULL
BEGIN
DROP TABLE #Cities;
END;
CREATE TABLE #Cities
(
Country VARCHAR(20)
, City VARCHAR(20)
);
IF OBJECT_ID('tempdb..#Data') IS NOT NULL
BEGIN
DROP TABLE #Data;
END;
CREATE TABLE #Data
(
Country VARCHAR(20)
, City VARCHAR(20)
, Price DECIMAL(13, 4)
, Date DATETIME
);
INSERT INTO #Cities
VALUES ('Country 1', 'City 1'), ('Country 1', 'City 2'), ('Country 1', 'City 3'), ('Country 2', 'City 4'), ('Country 2', 'City 5');
INSERT INTO #Data
SELECT Country
, City
, ROUND(RAND(CHECKSUM(NEWID())) * 100, 4) AS Price
, DATEADD(DAY, ROUND(RAND(CHECKSUM(NEWID())) * 10, 0), GETDATE()) AS Date
FROM #Cities
UNION
SELECT Country
, City
, ROUND(RAND(CHECKSUM(NEWID())) * 100, 4)
, DATEADD(DAY, ROUND(RAND(CHECKSUM(NEWID())) * 10, 0), GETDATE())
FROM #Cities;
--Delete duplicate dates
WITH data3 AS
(
SELECT *,ROW_NUMBER() OVER (PARTITION BY Country,City,Date ORDER BY Country,City,Date) AS RN
FROM #Data
)
DELETE FROM data3 WHERE RN<>1
Query the data to get the most recent price, date and percentage increase:
SELECT Dates.*
, Latest.Price AS Latestprice
, Previous.Price AS Previousprice
, ((Latest.Price - Previous.Price) / Previous.Price) * 100 AS Percentageincrease
FROM
(
SELECT C.*
, Latestdate.Latestdate
, Previousdate.Previousdate
FROM #Cities AS C
LEFT JOIN
(
--Latest Date for each county, city
SELECT Country
, City
, MAX(Date) AS Latestdate
FROM #Data
GROUP BY Country
, City
) AS Latestdate ON Latestdate.Country = C.Country
AND Latestdate.City = C.City
LEFT JOIN
(
--Previous Date for each county, city
SELECT Country
, City
, Date AS Previousdate
FROM
(
SELECT Country
, City
, Date
, RANK() OVER(PARTITION BY Country
, City ORDER BY Date DESC) AS Rank
FROM #Data
) AS A
WHERE Rank = 2
) AS Previousdate ON Previousdate.Country = C.Country
AND Previousdate.City = C.City
) AS Dates
JOIN #Data AS Latest ON Latest.Country = Dates.Country
AND Latest.City = Dates.City
AND Latest.Date = Dates.Latestdate
JOIN #Data AS Previous ON Previous.Country = Dates.Country
AND Previous.City = Dates.City
AND Previous.Date = Dates.Previousdate
And for comparison, using lag() to get the percentage increase for each date. Similar to Gordon's answer:
SELECT D.Country
, D.City
, D.Date
, Lag(Date) OVER(PARTITION BY Country
, City ORDER BY Date) AS Previousdate
, D.Price
, Lag(Price) OVER(PARTITION BY Country
, City ORDER BY Date) AS Previousprice
, 100 * (Price / Lag(Price) OVER(PARTITION BY Country
, City ORDER BY Date) - 1) AS PercentageIncrease
FROM #Data AS D;
Using lag to get the same results (latest info per city) as my first query:
SELECT *
FROM
(
SELECT D.Country
, D.City
, D.Date
, Lag(Date) OVER(PARTITION BY Country
, City ORDER BY Date) AS Previousdate
, D.Price
, Lag(Price) OVER(PARTITION BY Country
, City ORDER BY Date) AS Previousprice
, 100 * (Price / Lag(Price) OVER(PARTITION BY Country
, City ORDER BY Date) - 1) AS Percentageincrease
, ROW_NUMBER() OVER(PARTITION BY Country
, City ORDER BY Date DESC) AS Rn
FROM #Data AS D
) AS A
WHERE Rn = 1
ORDER BY Country
, City;
Use lag():
select t.*,
100 * ((price / lag(price) over (partition by country, city order by t_date) - 1) as increase
from t;
I have the following problem. I need to fetch the row for each distinct user_id with the min rank, max end date, max begin date, and max sequence number in that order. Essentially, I want a user's highest ranked, most recent MAJOR_ID. I am trying to avoid making separate temp tables and joining on those aggregate functions like the following:
select USER_ID
, SEQ_NBR
, BEGIN_DATE
, END_DATE
, MAJOR_RANK
, MAJOR_ID
, DEGREE_CODE
into #major0
from majors
select USER_ID
, MIN(MAJOR_RANK) as MAJOR_RANK
into #major1
from #major0
group by USER_ID
select #major0.USER_ID
, #major0.MAJOR_RANK
, MAX(#major0.END_DATE) as END_DATE
into #major2
from #major0
inner join #major1 on #major0.USER_ID = #major1.USER_ID and #major0.MAJOR_RANK = #major1.MAJOR_RANK
group by #major0.USER_ID
, #major0.MAJOR_RANK
etc...
until I get to that row that satisfies all the criteria, and I join back on all the fields from the original query. Does that make sense? It's a lot of work to write this out, and I can't create a view of it unless I made a absurdly long set of subqueries, I don't think I can utilize MIN(MAJOR_RANK) OVER (PARTITION BY USER_ID) in a subquery for all these fields because I will lose records that don't satisfy all of them.
Any suggestions would help! Thank you!
I do not see what the sequence_number is, but you can most likely solve this using a common table expression with row_number()
;with cte as (
select
user_id
, begin_date
, end_date
, major_rank
, major_id
, degree_code
, rn = row_number() over (
partition by user_id
order by major_rank asc, end_date desc, begin_date desc /*, sequence_number? desc*/
)
from majors
)
select
user_id
, begin_date
, end_date
, major_rank
, major_id
, degree_code
from cte
where rn = 1
without the cte
select
user_id
, begin_date
, end_date
, major_rank
, major_id
, degree_code
from (
select
user_id
, begin_date
, end_date
, major_rank
, major_id
, degree_code
, rn = row_number() over (
partition by user_id
order by major_rank asc, end_date desc, begin_date desc /*, sequence_number? desc*/
)
from majors
) sub
where rn = 1
I am converting an MS Access query to a SQL Server stored procedure. I get to this point:
SELECT
AuthNum, AuthStatus, DateCreated,
MIN(DateInitiated) AS DateInitiated,
EventClassification,
FIRST(PlaceOfService) AS PlaceOfService,
Lob, MemId,
MAX(NoticeDate) AS NoticeDate,
MAX(Tat) AS Tat,
FIRST(StaffId) AS StaffId
FROM
PA_TAT_Detailed
GROUP BY
AuthNum, AuthStatus, DateCreated, EventClassification, Lob, MemId
HAVING
((FIRST(PlaceOfService) <> 'Inpatient Hospital')
AND (FIRST(PlaceOfService) <> 'Office - Dental')
AND (FIRST(PlaceOfService) <> 'Dialysis Center'))
AND
((MAX(Tat) Is Null) OR ((MAX(Tat) >= 0) AND (MAX(Tat) <= 28)))
ORDER BY
AuthNum;
But I don't know how to converted the FIRST operator. Any thoughts? Do I need to add an ORDER BY associated with the GROUP BY so I can TAKE 1 perhaps?
Does MIN give the same result in this case?
BTW PlaceOfService, StaffId are strings.
Something like this should be pretty close. Double check that this returns the right info.
SELECT top 1 AuthNum
, AuthStatus
, DateCreated
, MIN(DateInitiated) AS DateInitiated
, EventClassification
, PlaceOfService
, Lob
, MemId
, MAX(NoticeDate) AS NoticeDate
, MAX(Tat) AS Tat
, StaffId
FROM PA_TAT_Detailed
where PlaceOfService not in ('Inpatient Hospital', 'Office - Dental', 'Dialysis Center')
GROUP BY AuthNum
, AuthStatus
, DateCreated
, EventClassification
, PlaceOfService
, Lob
, MemId
, StaffId
HAVING MAX(Tat) Is Null
OR (MAX(Tat) >= 0 AND MAX(Tat) <= 28)
ORDER BY AuthNum;
Below the query I created to get certain itemnumbers, qty ordered and price and others from the database. The problem is that sometimes an order doesn't contain 20 itemsnumbers but only 2. Now my question is if it's possible to fill the spaces with other itemnumbers random from the DB. It doesn't need to be correct because it's just for testing.
So can anybody help?
select
t.*,
-- THE THREE SUMVAT VALUES BELOW ARE VERY IMPORTANT. THEY ARE ONLY CORRECT HOWEVER WHEN THERE ARE NO NULL VALUES INVOLVED IN THE MATH,
-- I.E. WHEN THERE ARE 20 ITEMS/QTYS/PRICES INVOLVED WITH A CERTAIN ORDER_NO
((t.QTY1*t.PRICE1)+(t.QTY2*t.PRICE2)+(t.QTY3*t.PRICE3)+(t.QTY4*t.PRICE4)+(t.QTY5*t.PRICE5)) SUMVAT0, -- example: 5123.45 <- lines 1-5: Q*P
((t.QTY6*t.PRICE6)+(t.QTY7*t.PRICE7)+(t.QTY8*t.PRICE8)+(t.QTY9*t.PRICE9)+(t.QTY10*t.PRICE10)+(t.QTY11*t.PRICE11)+(t.QTY12*t.PRICE12)+(t.QTY13*t.PRICE13)+(t.QTY14*t.PRICE14)+(t.QTY15*t.PRICE15))
SUMVAT6, -- example: 1234.56 <- lines 6-15: Q*P
((t.QTY16*t.PRICE16)+(t.QTY17*t.PRICE17)+(t.QTY18*t.PRICE18)+(t.QTY19*t.PRICE19)+(t.QTY20*t.PRICE20)) SUMVAT19 -- example: 4567.89 <- lines 16-20: Q*P
from (
select
(to_char(p.vdate, 'YYYYMMDD') || to_char(sysdate, 'HH24MISS')) DT,
(to_char(p.vdate, 'YYYY-MM-DD') ||'T' || to_char(sysdate, 'HH24:MI:') || '00') DATETIME,
(to_char(orh.written_date, 'YYYY-MM-DD') ||'T00:00:00') DATETIME2,
orh.supplier FAKE_GLN,
y.*
from (
select
x.order_no ORDNO
, max(decode(r,1 ,x.item,null)) FAKE_GTIN1
, max(decode(r,2 ,x.item,null)) FAKE_GTIN2
, max(decode(r,3 ,x.item,null)) FAKE_GTIN3
, max(decode(r,4 ,x.item,null)) FAKE_GTIN4
, max(decode(r,5 ,x.item,null)) FAKE_GTIN5
, max(decode(r,6 ,x.item,null)) FAKE_GTIN6
, max(decode(r,7 ,x.item,null)) FAKE_GTIN7
, max(decode(r,8 ,x.item,null)) FAKE_GTIN8
, max(decode(r,9 ,x.item,null)) FAKE_GTIN9
, max(decode(r,10,x.item,null)) FAKE_GTIN10
, max(decode(r,11,x.item,null)) FAKE_GTIN11
, max(decode(r,12,x.item,null)) FAKE_GTIN12
, max(decode(r,13,x.item,null)) FAKE_GTIN13
, max(decode(r,14,x.item,null)) FAKE_GTIN14
, max(decode(r,15,x.item,null)) FAKE_GTIN15
, max(decode(r,16,x.item,null)) FAKE_GTIN16
, max(decode(r,17,x.item,null)) FAKE_GTIN17
, max(decode(r,18,x.item,null)) FAKE_GTIN18
, max(decode(r,19,x.item,null)) FAKE_GTIN19
, max(decode(r,20,x.item,null)) FAKE_GTIN20
, max(decode(r,1 ,x.qty_ordered,null)) QTY1
, max(decode(r,2 ,x.qty_ordered,null)) QTY2
, max(decode(r,3 ,x.qty_ordered,null)) QTY3
, max(decode(r,4 ,x.qty_ordered,null)) QTY4
, max(decode(r,5 ,x.qty_ordered,null)) QTY5
, max(decode(r,6 ,x.qty_ordered,null)) QTY6
, max(decode(r,7 ,x.qty_ordered,null)) QTY7
, max(decode(r,8 ,x.qty_ordered,null)) QTY8
, max(decode(r,9 ,x.qty_ordered,null)) QTY9
, max(decode(r,10,x.qty_ordered,null)) QTY10
, max(decode(r,11,x.qty_ordered,null)) QTY11
, max(decode(r,12,x.qty_ordered,null)) QTY12
, max(decode(r,13,x.qty_ordered,null)) QTY13
, max(decode(r,14,x.qty_ordered,null)) QTY14
, max(decode(r,15,x.qty_ordered,null)) QTY15
, max(decode(r,16,x.qty_ordered,null)) QTY16
, max(decode(r,17,x.qty_ordered,null)) QTY17
, max(decode(r,18,x.qty_ordered,null)) QTY18
, max(decode(r,19,x.qty_ordered,null)) QTY19
, max(decode(r,20,x.qty_ordered,null)) QTY20
, max(decode(r,1 ,x.unit_cost,null)) PRICE1
, max(decode(r,2 ,x.unit_cost,null)) PRICE2
, max(decode(r,3 ,x.unit_cost,null)) PRICE3
, max(decode(r,4 ,x.unit_cost,null)) PRICE4
, max(decode(r,5 ,x.unit_cost,null)) PRICE5
, max(decode(r,6 ,x.unit_cost,null)) PRICE6
, max(decode(r,7 ,x.unit_cost,null)) PRICE7
, max(decode(r,8 ,x.unit_cost,null)) PRICE8
, max(decode(r,9 ,x.unit_cost,null)) PRICE9
, max(decode(r,10,x.unit_cost,null)) PRICE10
, max(decode(r,11,x.unit_cost,null)) PRICE11
, max(decode(r,12,x.unit_cost,null)) PRICE12
, max(decode(r,13,x.unit_cost,null)) PRICE13
, max(decode(r,14,x.unit_cost,null)) PRICE14
, max(decode(r,15,x.unit_cost,null)) PRICE15
, max(decode(r,16,x.unit_cost,null)) PRICE16
, max(decode(r,17,x.unit_cost,null)) PRICE17
, max(decode(r,18,x.unit_cost,null)) PRICE18
, max(decode(r,19,x.unit_cost,null)) PRICE19
, max(decode(r,20,x.unit_cost,null)) PRICE20
from (
select
rank() over (partition by oh.order_no order by ol.item asc) r,
oh.supplier,
oh.order_no,
oh.written_date,
ol.item,
ol.qty_ordered,
ol.unit_cost
from
ordhead oh
JOIN ordloc ol ON oh.order_no = ol.order_no
where
-- count(numrows) = 1500
not unit_cost is null
-- and ol.order_no in (6181,6121)
) x
group by x.order_no
) y
JOIN ordhead orh ON orh.order_no = y.ORDNO,
period p
) t
;
Without being able to really test this, you might try something like this. Replace the inline view 'x' with this:
FROM (
WITH q AS (
SELECT LEVEL r, TO_CHAR(TRUNC(dbms_random.value*1000,0)) item
, TRUNC(dbms_random.value*100,0) qty_ordered
, TRUNC(dbms_random.value*10,2) unit_cost
FROM dual CONNECT BY LEVEL <= 20
)
SELECT COALESCE(x1.r, q.r) r, supplier, order_no, written_date
, COALESCE(x1.item, q.item) item
, COALESCE(x1.qty_ordered, q.qty_ordered) qty_ordered
, COALESCE(x1.unit_cost, q.unit_cost) unit_cost
FROM (SELECT ROW_NUMBER() OVER (PARTITION BY oh.order_no ORDER BY ol.item ASC) r
, oh.supplier
, oh.order_no
, oh.written_date
, ol.item
, ol.qty_ordered
, ol.unit_cost
FROM ordhead oh JOIN ordloc ol ON oh.order_no = ol.order_no
WHERE NOT unit_cost IS NULL) x1 RIGHT JOIN q ON x1.r = q.r
) x
GROUP BY x.order_no
The WITH clause will give you a table with 20 rows of random data. Outer join that with your old 'x' data and you will be guaranteed 20 rows of data. You might not need to cast the item as a varchar2 depending on data. (N.B., I finally found a query that it makes sense to use a RIGHT JOIN with. See this SO question)
I'm not quite sure what you're trying to do with the GROUP BY and MAX stuff? In the future it would be helpful to condense your examples into something others can easily test, a minimal case that gets your point across.
I also incorporated #Kevin's good suggestion to use ROW_NUMBER instead of RANK.
very difficult to understand...
i think you might be ok if you put a 0 instead of null in the price values...
, max(decode(r,18,x.unit_cost,0)) PRICE18
and
, max(decode(r,20,x.qty_ordered,0)) QTY20
then at least the math should work.
Rank will not guarantee a sequential count of the items in the groups there, may be gaps when you have several rows with the same value.
for a decent explanation see:
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:2920665938600
I think you need to use row_number