This question already exists:
Multilevel CTE Expression slows the execution [closed]
Closed 1 year ago.
The community reviewed whether to reopen this question 1 year ago and left it closed:
Duplicate This question has been answered, is not unique, and doesn’t differentiate itself from another question.
In SQL Server, I have a table-valued function which contains a multi-level CTE as shown in the code here:
CREATE FUNCTION [dbo].[GetDataset_Test]
(#XMLBlock XML, #Id INT)
RETURNS
#tempTable TABLE
(
methodID INT,
[Id] INT,
SavingMessage varchar(50)
)
AS
BEGIN
DECLARE #ObjectID INT = 2;
DECLARE #ExchangeRate INT = 2;
DECLARE #Cond INT = 1;
DECLARE #Param1 varchar(50) = 'name1';
DECLARE #Param2 varchar(50) = 'name2';
WITH CTE AS
(
SELECT
Col1,
con,
DiscountLine,
tempNumber,
ItemCostExVAT,
Quantity,
SomeValue,
methodID,
VATAmount,
SubTypeID,
Line,
VATMultiplier,
ROUND(dbo.GetAmountCustomFunction(COALESCE(Id, AnotherId), COALESCE(Price, PriceValue)), 2) As [Amount]
FROM
dbo.GetObject(#Id, #ObjectID, #ParameterBlock) AS BC
INNER JOIN
dbo.fnPrices(#ID, #CurrencyID) BPD ON BPD.Id = BC.productid
),
CTE1 AS
(
SELECT
*,
CASE WHEN con = 0 THEN Quantity ELSE 1 END AS Quantity
FROM
CTE
WHERE
1 = SomeValue
),
CTE2 AS
(
Select *,
MIN(CASE WHEN DiscountLine=1 THEN 1 ELSE 20 END) over (PARTITION by tempNumber) As StockControlled,
SUM(ItemCostExVAT * Quantity) OVER ( PARTITION BY tempNumber ) AS tempCost ,
ROUND(CASE
WHEN methodID = 8 THEN DiscValue
WHEN methodID = 2 THEN END ,
DicsValue,VATAmount),2) AS AmountExVAT
From CTE1
),
CTE3
(
SELECT
*,
ROUND(CASE
WHEN SubTypeID = 1 AND Line = 1 THEN -1 * AmountExVAT
ELSE 20
END, 2) PriceExVAT
FROM
CTE2
),
CTE4
(
SELECT
*,
ROUND(#ExchangeRate * CASE WHEN #Cond = 1 THEN AmountExVAT * VATMultiplier ELSE 20 END, 2) CashBack
FROM
CTE3
),
CTE5
(
SELECT
*,
dbo.FormatMessage(#Param1, #Param2) AS SavingMessage
FROM
CTE4
)
INSERT INTO #tempTable
SELECT
methodID, [Id], SavingMessage
FROM
CTE5
RETURN
END
In above query because of multi-level CTE and table value parameter I can think that its trying to query recursively and taking more execution time.
I know that we cannot use temporary table as function parameter, is there any alternative of this or can I use temporary table by any way in function?
Or can I make some changes in CTE to improve my T-SQL function query execution time?
Related
Sample data:
create table #temp (id int, qty int, checkvalue int)
insert into #temp values (1,1,3)
insert into #temp values (2,2,3)
insert into #temp values (3,1,3)
insert into #temp values (4,1,3)
According to data above, I would like to show exact number of lines from top to bottom where sum(qty) = checkvalue. Note that checkvalue is same for all the records all the time. Regarding the sample data above, the desired output is:
Id Qty checkValue
1 1 3
2 2 3
Because 1+2=3 and no more data is needed to show. If checkvalue was 4, we would show the third record: Id:3 Qty:1 checkValue:4 as well.
This is the code I am handling this problem. The code is working very well.
declare #checkValue int = (select top 1 checkvalue from #temp);
declare #counter int = 0, #sumValue int = 0;
while #sumValue < #checkValue
begin
set #counter = #counter + 1;
set #sumValue = #sumValue + (
select t.qty from
(
SELECT * FROM (
SELECT
ROW_NUMBER() OVER (ORDER BY id ASC) AS rownumber,
id,qty,checkvalue
FROM #temp
) AS foo
WHERE rownumber = #counter
) t
)
end
declare #sql nvarchar(255) = 'select top '+cast(#counter as varchar(5))+' * from #temp'
EXECUTE sp_executesql #sql, N'#counter int', #counter = #counter;
However, I am not sure if this is the best way to deal with it and wonder if there is a better approach. There are many professionals here and I'd like to hear from them about what they think about my approach and how we can improve it. Any advice would be appreciated!
Try this:
select id, qty, checkvalue from (
select t1.*,
sum(t1.qty) over (partition by t2.id) [sum]
from #temp [t1] join #temp [t2] on t1.id <= t2.id
) a where checkvalue = [sum]
Smart self-join is all you need :)
For SQL Server 2012, and onwards, you can easily achieve this using ROWS BETWEEN in your OVER clause and the use of a CTE:
WITH Running AS(
SELECT *,
SUM(qty) OVER (ORDER BY id
ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS RunningQty
FROM #temp t)
SELECT id, qty, checkvalue
FROM Running
WHERE RunningQty <= checkvalue;
One basic improvement is to try & reduce the no. of iterations. You're incrementing by 1, but if you repurpose the logic behind binary searching, you'd get something close to this:
DECLARE #RoughAverage int = 1 -- Some arbitrary value. The closer it is to the real average, the faster things should be.
DECLARE #CheckValue int = (SELECT TOP 1 checkvalue FROM #temp)
DECLARE #Sum int = 0
WHILE 1 = 1 -- Refer to BREAK below.
BEGIN
SELECT TOP (#RoughAverage) #Sum = SUM(qty) OVER(ORDER BY id)
FROM #temp
ORDER BY id
IF #Sum = #CheckValue
BREAK -- Indicating you reached your objective.
ELSE
SET #RoughAverage = #CheckValue - #Sum -- Most likely incomplete like this.
END
For SQL 2008 you can use recursive cte. Top 1 with ties limits result with first combination. Remove it to see all combinations
with cte as (
select
*, rn = row_number() over (order by id)
from
#temp
)
, rcte as (
select
i = id, id, qty, sumV = qty, checkvalue, rn
from
cte
union all
select
a.id, b.id, b.qty, a.sumV + b.qty, a.checkvalue, b.rn
from
rcte a
join cte b on a.rn + 1 = b.rn
where
a.sumV < b.checkvalue
)
select
top 1 with ties id, qty, checkvalue
from (
select
*, needed = max(case when sumV = checkvalue then 1 else 0 end) over (partition by i)
from
rcte
) t
where
needed = 1
order by dense_rank() over (order by i)
I have a table of longitudinal data that looks like this:
where id is the partition variable, period is the time dimension, and val is the observation value.
I want to build up a history of val for each panel of id, like this:
I'm trying to do this with SQL window functions and not a cursor, but the issue I keep running into is the self-referential nature of the hist column definition. It almost seems like I'd have to create one row/column per period. For example, the closest I could come was this:
IF OBJECT_ID('dbo.my_try', 'U') IS NOT NULL
DROP TABLE dbo.my_try;
GO
SELECT
id, period, val,
CASE
WHEN (
period = MIN(period)
OVER (PARTITION by id order by period ROWS
BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)
) THEN CAST (val AS VARCHAR(60))
ELSE NULL
END AS hist
INTO my_try
FROM my_test
SELECT
id, period, val,
CASE
WHEN (
period = MIN(period) OVER
(PARTITION by id order by period ROWS
BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING)
) THEN hist
ELSE (
CONCAT(
val, ' | ', LAG(hist, 1) OVER (PARTITION by id order by period)
)
)
END AS hist2
FROM my_try
I would have to spool out the iteration and do a hist3, etc. for it to finally work.
Is it possible to accomplish this with SQL window functions, or is cursor the only route?
Sample Data
Here is some code to generate the original table:
CREATE TABLE my_test (
id INT,
period INT,
val INT
)
BEGIN
DECLARE #id INT = 1;
DECLARE #period INT = 1;
WHILE #id <= 3
BEGIN
SET #period = 1
WHILE #period <= 3
BEGIN
INSERT INTO my_test VALUES (#id, #period, #period * POWER(10, #id))
SET #period = #period + 1
END
SET #id = #id + 1
END
END
Actually you don't need recursion here. You can leverage STUFF pretty easily. Of course if you are on 2017 you can use string_agg as suggested above. But if you are like me and your company is not the fastest to adopt the latest and greatest you can use this.
select t1.id
, t1.period
, t1.val
, STUFF((select ' | ' + convert(varchar(10), val)
from my_test t2
where t2.id = t1.id
and t2.period <= t1.period
order by t1.period
FOR XML PATH('')), 1, 3,'')
from my_test t1
order by t1.id
, t1.period
As discussed in the comments try using recursive query
with cte as(
select id, [period], val, convert(varchar(max), val) as agg from my_try where [period] = 1
union all
select t.id, t.[period], t.val, CONCAT(c.agg, ' | ', t.val) from my_try t join cte c on c.[period] +1 = t.[period] and c.id = t.id
)
select * from cte order by id, [period]
How I can achieve incremental grouping in query ?
I need to group by all the non-zero values into different named groups.
Please help me write a query based on columns date and subscribers.
If you have SQL Server 2012 or newer, you can use few tricks with windows functions to get this kind of grouping without cursors, with something like this:
select
Date, Subscribers,
case when Subscribers = 0 then 'No group'
else 'Group' + convert(varchar, GRP) end as GRP
from (
select
Date, Subscribers,
sum (GRP) over (order by Date asc) as GRP
from (
select
*,
case when Subscribers > 0 and
isnull(lag(Subscribers) over (order by Date asc),0) = 0 then 1 else 0 end as GRP
from SubscribersCountByDay S
) X
) Y
Example in SQL Fiddle
In general I advocate AGAINST cursors but in this case it ill not hurt since it ill iterate, sum up and do the conditional all in one pass.
Also note I hinted it with FAST_FORWARD to not degrade performance.
I'm guessing you do want what #HABO commented.
See the working example below, it just sums up until find a ZERO, reset and starts again. Note the and #Sum > 0 handles the case where the first row is ZERO.
create table dbo.SubscribersCountByDay
(
[Date] date not null
,Subscribers int not null
)
GO
insert into dbo.SubscribersCountByDay
([Date], Subscribers)
values
('2015-10-01', 1)
,('2015-10-02', 2)
,('2015-10-03', 0)
,('2015-10-04', 4)
,('2015-10-05', 5)
,('2015-10-06', 0)
,('2015-10-07', 7)
GO
declare
#Date date
,#Subscribers int
,#Sum int = 0
,#GroupId int = 1
declare #Result as Table
(
GroupName varchar(10) not null
,[Sum] int not null
)
declare ScanIt cursor fast_forward
for
(
select [Date], Subscribers
from dbo.SubscribersCountByDay
union
select '2030-12-31', 0
) order by [Date]
open ScanIt
fetch next from ScanIt into #Date, #Subscribers
while ##FETCH_STATUS = 0
begin
if (#Subscribers = 0 and #Sum > 0)
begin
insert into #Result (GroupName, [Sum]) values ('Group ' + cast(#GroupId as varchar(6)), #Sum)
set #GroupId = #GroupId + 1
set #Sum = 0
end
else begin
set #Sum = #Sum + #Subscribers
end
fetch next from ScanIt into #Date, #Subscribers
end
close ScanIt
deallocate ScanIt
select * from #Result
GO
For the OP: Please next time write the table, just posting an image is lazy
In a version of SQL Server modern enough to support CTEs you can use the following cursorless query:
-- Sample data.
declare #SampleData as Table ( Id Int Identity, Subscribers Int );
insert into #SampleData ( Subscribers ) values
-- ( 0 ), -- Test edge case when we have a zero first row.
( 200 ), ( 100 ), ( 200 ),
( 0 ), ( 0 ), ( 0 ),
( 50 ), ( 50 ), ( 12 ),
( 0 ), ( 0 ),
( 43 ), ( 34 ), ( 34 );
select * from #SampleData;
-- Run the query.
with ZerosAndRows as (
-- Add IsZero to indicate zero/non-zero and a row number to each row.
select Id, Subscribers,
case when Subscribers = 0 then 0 else 1 end as IsZero,
Row_Number() over ( order by Id ) as RowNumber
from #SampleData ),
Groups as (
-- Add a group number to every row.
select Id, Subscribers, IsZero, RowNumber, 1 as GroupNumber
from ZerosAndRows
where RowNumber = 1
union all
select FAR.Id, FAR.Subscribers, FAR.IsZero, FAR.RowNumber,
-- Increment GroupNumber only when we move from a non-zero row to a zero row.
case when Groups.IsZero = 1 and FAR.IsZero = 0 then Groups.GroupNumber + 1 else Groups.GroupNumber end
from ZerosAndRows as FAR inner join Groups on Groups.RowNumber + 1 = FAR.RowNumber
)
-- Display the results.
select Id, Subscribers,
case when IsZero = 0 then 'no group' else 'Group' + Cast( GroupNumber as VarChar(10) ) end as Grouped
from Groups
order by Id;
To see the intermediate results just replace the final select with select * from FlagsAndRows or select * from Groups.
I got a table with some measurements, containing basically records. And now, I need to determine, if the values are monotonically increasing from time to time, decreasing or none of the above.
I achieved the desired result using CTE expression (the code is below), but the solution seems quite reduntant to me.
Is there a better way to determine, if the field value sequence is monotone, or not?
CREATE TABLE [dbo].[Measurements](
[ObjectID] [int] IDENTITY(1,1) NOT NULL,
[measDate] [datetime] NULL,
[measValue] [float] NULL
) ON [PRIMARY];
DECLARE
#ObjectID INT = 1;
with measSet as (
select row_number() over(order by measDate ) rownum, measValue, measDate
from dbo.Measurements M
where M.measDate > convert( datetime, '2013-10-02 08:13:00', 120)
and M.ObjectID = #ObjectID
)
select case when count(b.DiffSign) = 1 then 1 else 0 end as IsMonotone
from (
select DiffSign from
(
select MSS.measDate , MSS.measValue, MSS.measValue- MSSD.measValue as Diff,
case
when MSS.measValue- MSSD.measValue is null then NULL
when MSS.measValue- MSSD.measValue= 0 then NULL
when MSS.measValue- MSSD.measValue< 0
then -1
else 1
end as DiffSign
from measSet MSS
left join measSet MSSD
on MSSD .rownum = MSS.rownum - 1
) a
where a.DiffSign is not null
group by a.DiffSign
) b
If you don't care about knowing what particular records are breaking the monotony, then you could use something like this, which is a little more compact:
SELECT CASE WHEN COUNT(*) = 0 THEN 1 ELSE 0 END AS IsMonotone
FROM (
SELECT ROW_NUMBER() OVER (ORDER BY measDate) AS RowNum, measValue
FROM Measurements
) T1 INNER JOIN (
SELECT ROW_NUMBER() OVER (ORDER BY measValue) AS RowNum, measValue
FROM Measurements
) T2 ON T1.RowNum = T2.RowNum
WHERE T1.measValue <> T2.measValue
Say I have these rows,
InstrumentID
547
698
708
InstrumentID is not autogenerated column.
Say if I pass the parameter in procedure as 698, I should get previous value as 547 and next value as 708. How do I do this efficiently in SQL?
I have this procedure but it is not efficient (and not correct).
Alter PROCEDURE GetNextAndPreviousInsturmentID
(
#InstrumentID varchar(14),
#PreviousInstrumentID varchar(14) OUT,
#NextInstrumentID varchar(14) OUT
)
AS
BEGIN
Declare #RowNum int = 0
Select #RowNum = ROW_NUMBER() Over (Order by Cast(#InstrumentID as decimal(18))) From Documents Where InstrumentID = #InstrumentID
;With normal As
(
Select ROW_NUMBER() Over (Order by Cast(#InstrumentID as decimal(18))) as RowNum, Cast(InstrumentID as decimal(18)) as InstrumentID
From Documents
)
Select #PreviousInstrumentID = InstrumentID From normal
Where RowNum = #RowNum - 1
Select #NextInstrumentID = InstrumentID From normal
Where RowNum = #RowNum + 1
END
GO
Here is a simpler solution, still it's more efficient
SELECT P.PreviousID, N.NextID
FROM
(SELECT MAX(D.InstrumentID) PreviousID
FROM Documents D
WHERE InstrumentID < #InstrumentID) P
CROSS JOIN
(SELECT MIN(D.InstrumentID) NextID
FROM Documents D
WHERE InstrumentID > #InstrumentID) N
Try this:
Alter PROCEDURE GetNextAndPreviousInsturmentID
(
#InstrumentID varchar(14),
#PreviousInstrumentID varchar(14) OUT,
#NextInstrumentID varchar(14) OUT
)
AS
BEGIN
Declare #Ids TABLE(Id varchar(14))
;With normal As
(
--Numerate our rows
Select ROW_NUMBER() Over (Order by Cast(Documents.InstrumentID as decimal(18)) as RowNumber,
Documents.InstrumentID
From Documents
)
--Insert three rows from our table with our id and previos/next id
INSERT INTO #Ids(Id)
SELECT TOP(3) normal.InstrumentID
FROM normal
WHERE RowNumber >=
(
SELECT RowNumber - 1
FROM normal
WHERE normal.InstrumentID = #InstrumentID
)
ORDER BY normal.RowNumber
--select next and previos ids
SELECT #PreviousInstrumentID = Min(CAST(Id as decimal(18))),
#NextInstrumentID = MAX(CAST(Id as decimal(18)))
FROM #Ids
END
GO
In MS SQL 2012 we have new window functions like FIRST_VALUE and LAST_VALUE, unfortunately in sql 2008 these functions are missing.
WITH CTE AS (
SELECT rownum = ROW_NUMBER() OVER (ORDER BY p.LogDate), p.LogDate
FROM DeviceLogs p
)
SELECT prev.logdate PreviousValue, CTE.logdate, nex.logdate NextValue
FROM CTE
LEFT JOIN CTE prev ON prev.rownum = CTE.rownum - 1
LEFT JOIN CTE nex ON nex.rownum = CTE.rownum + 1
GO
select
LAG(InstrumentID) OVER (ORDER BY InstrumentID) PreviousValue,
InstrumentID,
LEAD(InstrumentID) OVER (ORDER BY InstrumentID) NextValue
from documents
Hi I think this will be much more efficient:
Select Next :
select top 1 ID from mytable
where ID >'698'
order by ID asc
Select Prev:
select top 1 ID from mytable
where ID <'698'
order by ID desc