How/Can I specify this window frame:
sum(Quantity) over (partition by AccountId, SymbolId order by Time rows between unbounded preceding and current row -1) PositionAmount?
I tried to full it by
sum(Quantity) over (partition by AccountId, SymbolId order by Time rows between unbounded preceding and -1 following)
but -1 is not allowed.
I can of course make a second select over it and find prev value of PositionAmount with lag or something.
The documention specifies, in case you use between, that both parts are window frame bound, without forcing you to use following for the second part. Try this:
rows between unbounded preceding and 1 preceding
Related
I need the lowest/highest price of stocks for the past n days. The following query works really slow. I would appreciate faster alternative:
SELECT
*,
MIN(Close) OVER (PARTITION BY Ticker ORDER BY PriceDate ROWS BETWEEN 14 PRECEDING AND 1 PRECEDING) AS MinPrice14d,
MAX(Close) OVER (PARTITION BY Ticker ORDER BY PriceDate ROWS BETWEEN 14 PRECEDING AND 1 PRECEDING) AS MaxPrice14d
FROM
(SELECT CompanyID, Ticker, PriceDate, Close
FROM price.PriceHistoryDaily) a
I need the columns specified.
It is trailing, so I need it day by day.
As for period, I will limit it to one year.
Although it doesn't affect the performance, no subquery is needed. So start with the simpler version:
SELECT phd.CompanyID, phd.Ticker, phd.PriceDate, phd.Close,
min(Close) over (partition by Ticker
order by PriceDate
rows between 14 preceding and 1 preceding
) as MinPrice14d,
max(Close) over (partition by Ticker
order by PriceDate
rows between 14 preceding and 1 preceding
) as MaxPrice14d
FROM price.PriceHistoryDaily phd;
Then try adding an index: PriceHistoryDaily(Ticker, PriceDate).
Note: That this returns all rows from PriceHistoryDaily and -- depending on the size of the table -- that might be what is driving the performance.
I am not able to get the last value, rather it is just returning the same value with my code below in snowflake - does anyone have any idea? Is there something glaring wrong?
select MNTH,
sum_cust,
last_value(sum_cust) over (partition by MNTH order by sum_cust desc ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) as sum_cust_last
from block_2;
I think what you actually want is to LAG the value from the previous MNTH:
SELECT MNTH,
sum_cust,
LAG(sum_cust) OVER (ORDER BY MNTH) AS sum_cust_last
FROM block_2;
I actually recommend first_value() rather than last_value() for some technical reasons involving window frames. If you want the last value, order by the month desc and choose the first row:
select MNTH, sum_cust,
first_value(sum_cust) over (order by MNTH desc
rows between current_row AND UNBOUNDED FOLLOWING
) as sum_cust_last
from block_2;
with cte
as
(
SELECT
year(h.orderdate)*100+month(h.orderdate) as yearmonth,
YEAR(h.orderdate) as orderyear,
sum(d.OrderQty*d.UnitPrice) as amount
FROM [AdventureWorks].[Sales].[SalesOrderDetail] d
inner join sales.SalesOrderHeader h
on d.SalesOrderID=h.SalesOrderID
group by
year(h.orderdate)*100+month(h.orderdate),
year(h.orderdate)
)
select
c.*,
last_value(c.amount) over (partition by c.orderyear order by c.yearmonth) as lastvalue,
first_value(c.amount) over (partition by c.orderyear order by c.yearmonth) as firstvalue
from cte c
order by c.yearmonth
I am expecting to see the lastvalue of each year (say december value), similar to the firstvalue of each year (jan value). however, last_value is not working at all. It just returns the same value of that month. What did I do wrong?
Thanks for the help.
Your problem is that the default row range for LAST_VALUE is RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, so the value you are getting is the current month's value (that being the last value in that range). To get LAST_VALUE to look at all values in the partition you need to expand the range to include the rows after the current row as well. So you need to change your query to:
last_value(c.amount) over (partition by c.orderyear order by c.yearmonth
rows between unbounded preceding and unbounded following) as lastvalue,
I'm running code like this:
SELECT ID, Date, Price,
STDEV(Price) OVER (ORDER BY ID, Date ROWS BETWEEN 30 PRECEDING AND CURRENT ROW) As OneMonths,
STDEV(Price) OVER (ORDER BY ID, Date ROWS BETWEEN 60 PRECEDING AND CURRENT ROW) As TwoMonths,
STDEV(Price) OVER (ORDER BY ID, Date ROWS BETWEEN 90 PRECEDING AND CURRENT ROW) As ThreeMonths
FROM Price_Table
That gives me this result.
In the fiver first row I always have three nulls for the three variances. This makes sense. However, every time the ID changes, I must be getting the preceding ID's prices, because each time the ID changes, I would expect the standard deviation to get reset. So, the first line in orange should be null, I think, and the next one should be zero, because there is no change in price the second day. I tried wrapping the LAG function around the STDEV function and I got an error. I tried the opposite and also got an error.
If you want the value per id, then you need partition by:
SELECT ID, Date, Price,
STDEV(Price) OVER (PARTITION BY ID ORDER BY Date ROWS BETWEEN 30 PRECEDING AND CURRENT ROW) As OneMonths,
STDEV(Price) OVER (PARTITION BY ID ORDER BY Date ROWS BETWEEN 60 PRECEDING AND CURRENT ROW) As TwoMonths,
STDEV(Price) OVER (PARTITION BY ID ORDER BY Date ROWS BETWEEN 90 PRECEDING AND CURRENT ROW) As ThreeMonths
FROM Price_Table;
As part of some solution building, I had to implement a view which is performing a running total (cumulative sum calculation). I took the most simple and basic approach of joining table on table with list of dates but it seems that the view is still fairly slow. Addition of indexes on the table didn't help much, even though the table itself have just 15K rows or so. I was wondering if someone could advice on what would be the right approach to speed it up?
There are several considerations:
I need to calculate cumulative sum up to a date for specific ProjectID and ContractorID. So for same date I may have a lot of ProjectIDs and ContractorIds combinations, but combination of Date, ProjectID and ContractorID is always unique
There is a master table with dates, projectids (but no contractorids) and I need a cumulative sum for each date, projectid in this master dates table
I need to calculate a cumulative sum of several columns at the same time, not just of one
To walk you through the situation slightly more, the tables I have are:
dbo.Project_Reporting_Schedule which holds a master list of projectid, dates. For each of this combinations I need to calculate a cumulative sum based on another table. Please note it has no contractorid!
Project_value_delivery is a table where I have actual value columns to perform a cumulative sum calculation. It has its own set of dates which may or may not match dates in Project_Reporting_Schedule, hence we can't just join the table on itself. Please also note it has contractorid!
Currently I have the following select which is rather self-explanatory and just joins table with values on table with master date list and does the summation. Select works fine, but even with just 15K records it takes almost 5 seconds to run which is fairly slow.
select
pv2.ProjectID,
pv2.ContractorID,
pv1.Date,
sum(pv2.ValuePlanned) as PlannedCumulative,
sum(pv2.ValueActual) as ActualCumulative,
sum(pv2.MobilizationPlanned) as MobilizationPlanned,
sum(pv2.MobilizationActual) as MobilizationActual,
sum(pv2.EngineeringPlanned) as EngineeringPlanned,
sum(pv2.EngineeringActual) as EngineeringActual,
sum(pv2.ProcurementPlanned) as ProcurementPlanned,
sum(pv2.ProcurementActual) as ProcurementActual,
sum(pv2.ConstructionPlanned) as ConstructionPlanned,
sum(pv2.ConstructionActual) as ConstructionActual,
sum(pv2.CommisioningTestingPlanned) as CommisioningTestingPlanned,
sum(pv2.CommisioningTestingActual) as CommisioningTestingActual
from
dbo.Project_Reporting_Schedule as pv1
join
dbo.Project_value_delivery as pv2 on pv1.Date >= pv2.Date and pv1.ProjectID = pv2.ProjectID
group by
pv2.ProjectID, pv2.ContractorID, pv1.Date
UPDATE
For further clarifications, I put execution plan here:
https://www.brentozar.com/pastetheplan/?id=H12t-O1PS
Indexes created are same and on both tables I have them for Projectid, Date combination as well as standalone indexes on ProjectID and Date columns.
All indexes are Unique Nonclustered where applicable or just Nonclustered where applicable.
We can see it does 'non-clustered index seek' which costs most of the execution. Maybe index needs to be adjusted?
OK, so per suggestion from #Alex in the comments windowed functions are a way to go. The below code works lightning-fast compared to original code:
select
pv2.ProjectID,
pv2.ContractorID,
pv1.Date,
sum(pv2.ValuePlanned) over (partition by pv2.ProjectID, pv2.ContractorID order by pv1.Date ROWS between unbounded preceding and current row) as PlannedCumulative,
sum(pv2.ValueActual) over (partition by pv2.ProjectID, pv2.ContractorID order by pv1.Date ROWS between unbounded preceding and current row) as ActualCumulative,
sum(pv2.MobilizationPlanned) over (partition by pv2.ProjectID, pv2.ContractorID order by pv1.Date ROWS between unbounded preceding and current row) as MobilizationPlanned,
sum(pv2.MobilizationActual) over (partition by pv2.ProjectID, pv2.ContractorID order by pv1.Date ROWS between unbounded preceding and current row) as MobilizationActual,
sum(pv2.EngineeringPlanned) over (partition by pv2.ProjectID, pv2.ContractorID order by pv1.Date ROWS between unbounded preceding and current row) as EngineeringPlanned,
sum(pv2.EngineeringActual) over (partition by pv2.ProjectID, pv2.ContractorID order by pv1.Date ROWS between unbounded preceding and current row) as EngineeringActual,
sum(pv2.ProcurementPlanned) over (partition by pv2.ProjectID, pv2.ContractorID order by pv1.Date ROWS between unbounded preceding and current row) as ProcurementPlanned,
sum(pv2.ProcurementActual) over (partition by pv2.ProjectID, pv2.ContractorID order by pv1.Date ROWS between unbounded preceding and current row) as ProcurementActual,
sum(pv2.ConstructionPlanned) over (partition by pv2.ProjectID, pv2.ContractorID order by pv1.Date ROWS between unbounded preceding and current row) as ConstructionPlanned,
sum(pv2.ConstructionActual) over (partition by pv2.ProjectID, pv2.ContractorID order by pv1.Date ROWS between unbounded preceding and current row) as ConstructionActual,
sum(pv2.CommisioningTestingPlanned) over (partition by pv2.ProjectID, pv2.ContractorID order by pv1.Date ROWS between unbounded preceding and current row) as CommisioningTestingPlanned,
sum(pv2.CommisioningTestingActual) over (partition by pv2.ProjectID, pv2.ContractorID order by pv1.Date ROWS between unbounded preceding and current row) as CommisioningTestingActual
from
dbo.Project_Reporting_Schedule as pv1
join dbo.Project_value_delivery as pv2 on pv1.Date = pv2.Date and pv1.ProjectID = pv2.ProjectID
Take the comparison out of the JOIN clause and move it to a WHERE clause:
select
pv2.ProjectID,
pv2.ContractorID,
pv1.Date,
sum(pv2.ValuePlanned) as PlannedCumulative,
sum(pv2.ValueActual) as ActualCumulative,
sum(pv2.MobilizationPlanned) as MobilizationPlanned,
sum(pv2.MobilizationActual) as MobilizationActual,
sum(pv2.EngineeringPlanned) as EngineeringPlanned,
sum(pv2.EngineeringActual) as EngineeringActual,
sum(pv2.ProcurementPlanned) as ProcurementPlanned,
sum(pv2.ProcurementActual) as ProcurementActual,
sum(pv2.ConstructionPlanned) as ConstructionPlanned,
sum(pv2.ConstructionActual) as ConstructionActual,
sum(pv2.CommisioningTestingPlanned) as CommisioningTestingPlanned,
sum(pv2.CommisioningTestingActual) as CommisioningTestingActual
FROM
dbo.Project_Reporting_Schedule as pv1
join dbo.Project_value_delivery as pv2 on pv1.ProjectID = pv2.ProjectID
WHERE pv1.Date >= pv2.Date
GROUP BY pv2.ProjectID, pv2.ContractorID, pv1.Date