Related
I have a table like this:
ItemID ItemFormula
100 'ID_3+ID_5'
110 'ID_2+ID_6'
120 'ID_100+ID_110'
130 'ID_120+ID_4'
This is the simplified version of a formula table with nearly 1000 records and up to 40 levels of reference (items used in other items). The task is breaking down the formulas to just one level reference where no other items are in one item. For example in the table above for id=130 I should have '((ID_3+ID_5)+(ID_2+ID_6))+ID_4'
EDIT: The operations do not limit to "+" and items have a character between them to be recognizable. For the sake of simplicity, I removed that character.
I can use recursive CTE for that. but my problem is that due to high levels of reference, my recursive select has lots of records joining so it takes a lot to complete.
My question is that: Can I keep the previous recursion only each time the recursion happens?
Here is my CTE Code
WITH Formula
AS (SELECT A.ItemID
,'ID_' + CONVERT(VARCHAR(20), A.ItemID) AS ItemText
,CONVERT(VARCHAR(MAX), A.ItemFormula) AS ItemFormula
FROM (VALUES (100,'ID_3+ID_5'),
(110,'ID_2+ID_6'),
(120,'ID_100+ID_110'),
(130,'ID_120+ID_4')
) A (ItemID,ItemFormula)
)
,REC
AS
(
SELECT A.ItemID
,A.ItemText
,A.ItemFormula
,1 AS LevelID
FROM Formula A
UNION ALL
SELECT A.ItemID
,A.ItemText
,' '
+ TRIM (REPLACE (REPLACE (A.ItemFormula, B.ItemText, ' ( ' + B.ItemFormula + ' ) '), ' ', ' '))
+ ' ' AS ItemFormula
,A.LevelID + 1 AS LevelID
FROM REC A
CROSS APPLY
(
SELECT *
FROM
(
SELECT *
,ROW_NUMBER () OVER (ORDER BY GETDATE ()) AS RowNum
FROM Formula B2
WHERE CHARINDEX (B2.ItemText, A.ItemFormula) > 0
) B3
WHERE B3.RowNum = 1
) B
)
,FinalQ
AS
(
SELECT A2.ItemID
,A2.ItemFormula
,A2.LevelID
FROM
(
SELECT A.ItemID
,REPLACE (TRIM (A.ItemFormula), ' ', '') AS ItemFormula
,A.LevelID
,ROW_NUMBER () OVER (PARTITION BY A.ItemID ORDER BY A.LevelID DESC) AS RowNum
FROM REC A
) A2
WHERE A2.RowNum = 1
)
SELECT * FROM FinalQ A2 ORDER BY A2.ItemID;
Thanks in advance.
My question is that: Can I keep the previous recursion only each time the recursion happens?
No. The recursive CTE will keep adding rows to the ones found in previous iterations. You don't have some kind of control that would allow you to remove rows of the recursive CTE during its iterations.
You can, however, filter them out after the recursive CTE is complete, maybe on a secondary CTE that takes into account only the last meaninful rows (by some kind of rule to be defined).
The only vaguely similar idea is found in PostgreSQL where you can use the UNION clause in addition to UNION ALL, to avoid producing more identical rows. But this is different to what you need, anyway.
This is an enormously complicated problem. Here are the ideas:
Find which items do not need any insertions. These are the ones that have no references to any others.
Build an ordering for item insertion. An insertion can go into an item, assuming that the item is already defined. A recursive CTE can be used for this.
Enumerate the insertions. Everything from (1) gets a "1". The rest are in order.
Process the insertions in the insertion order.
Here is my solution:
with ordering as (
select itemid, itemtext, itemformula, convert(varchar(max), null) as otheritemtext, 1 as lev
from formula f
where not exists (select 1
from formula f2 join
string_split(f.itemformula, '+') s
on f2.itemtext = s.value
where f2.itemid <> f.itemid
)
union all
select f.itemid, f.itemtext, f.itemformula, convert(varchar(max), s.value), lev + 1
from formula f cross apply
string_split(f.itemformula, '+') s join
ordering o
on o.itemtext = s.value
-- where lev <= 2
),
ordered as (
select distinct o.*,
dense_rank() over (order by (case when lev = 1 then -1 else lev end), (case when lev = 1 then '' else otheritemtext end)) as seqnum
from ordering o
),
cte as (
select o.itemid, o.itemtext, o.itemformula, convert(varchar(max), o.otheritemtext) as otheritemtext,
o.itemformula as newformula, o.seqnum, 1 as lev
from ordered o
where seqnum = 1
union all
select cte.itemid, o.itemtext, o.itemformula, convert(varchar(max), cte.itemtext),
replace(o.itemformula, o.otheritemtext, concat('(', cte.newformula, ')')), o.seqnum, cte.lev + 1
from cte join
ordered o
on cte.itemtext = o.otheritemtext and cte.seqnum < o.seqnum
)
select *
from cte;
And the db<>fiddle.
You could take advantage of the logical order of the formulas if any (Item_100 can not reference Item_150) and process items in a descending order.
The following uses LIKE and it will not work for formulas which have overlapping patterns (eg ID_10 & ID_100) you could fix that by some string manipulation or by keeping ItemIDs of fixed length (eg. ID_10010 & ID_10100: start numbering of items from a high number like 10000)
declare #f table
(
ItemId int,
ItemFormula varchar(1000)
);
insert into #f(ItemId, ItemFormula)
values
(100, 'ID_3+ID_5'),
(110, 'ID_2+ID_6'),
(120, 'ID_100+ID_110'),
(130, 'ID_120+ID_4'),
(140, '(ID_130+ID_110)/ID_100'),
(150, 'sqrt(ID_140, ID_130)'),
(160, 'ID_150-ID_120+ID_140');
;with cte
as
(
select f.ItemId, replace(cast(f.ItemFormula as varchar(max)), isnull('ID_' + cast(r.ItemId as varchar(max)), ''), isnull('(' + r.ItemFormula+ ')', '')) as therepl, 1 as lvl
from #f as f
outer apply (
select *
from
(
select rr.*, row_number() over(order by rr.ItemId desc) as rownum
from #f as rr
where f.ItemFormula like '%ID_' + cast(rr.ItemId as varchar(1000)) + '%'
) as src
where rownum = 1
) as r
union all
select c.ItemId, replace(c.therepl, 'ID_' + cast(r.ItemId as varchar(max)), '(' + r.ItemFormula+ ')'), c.lvl+1
from cte as c
cross apply (
select *
from
(
select rr.*, row_number() over(order by rr.ItemId desc) as rownum
from #f as rr
where c.therepl like '%ID_' + cast(rr.ItemId as varchar(1000)) + '%'
) as src
where rownum = 1
) as r
),
rown
as
(
select *, row_number() over (partition by itemid order by lvl desc) as rownum
from cte
)
select *
from rown
where rownum = 1;
I have first table (the count of rows are varibale) and I want to create the second table. what is the efficient way?
First you have to bring your data to a more 'friendly' format:
;with
data as
(
-- replace this with your select
select * from
(
VALUES ('1', 'a', 'b'),
('2', 'c', 'd'),
('3', 'e', 'f')
) as data(aa,bb,cc)
--------------------------------
),
dataAsXml as
(
select CAST(STUFF((SELECT '<i>' + d.[aa] + '</i><i>' + d.[bb] + '</i><i>' + d.[cc] + '</i>' FROM data d FOR XML PATH(''), TYPE ).value('.', 'NVARCHAR(MAX)'),1,0,'') as XML) as data
),
dataAsList as
(
select x.i.value('for $i in . return count(../*[. << $i]) + 1', 'int') as 'Ord',
x.i.value('.', 'NVARCHAR(100)') AS 'Value'
from dataAsXml
CROSS APPLY [data].nodes('//i') x(i)
),
normalized AS
(
select
case (Ord - 1) % 3 + 1
when 1 then 'aa'
when 2 then 'bb'
when 3 then 'cc'
end + cast((Ord - 1) / 3 + 1 as varchar(10)) as columnName, --fix here
value
from dataAsList
)
select * from normalized
In the query above you can plug in your data in data CTE to see the result.
The output will have two columns one that stores your column names and one with values.
SQL Fiddle
From here you have to use a dynamic query where you pivot the obtained table for columnName in the list of all the column names. I won't describe this process because it has been done many times. Take a look at this answer:
Convert Rows to columns using 'Pivot' in SQL Server
Note:
I didn't tested the performance of this method with large sets of data but
from some points of view it's efficient.
Try this one. I pivoted each of the columns then join them together in one row.
SELECT aa1,bb1,cc1,aa2,bb2,cc2,aa3,bb3,cc3 FROM
(SELECT 1 id,[2]aa1,[3]aa2,[4]aa3 FROM(SELECT aa FROM tablea) AS A
PIVOT(SUM(aa) FOR aa in([2],[3],[4])) AS pvt) A
INNER JOIN
(SELECT 1 id,[400]bb1,[200]bb2,[500]bb3 FROM(SELECT bb FROM tablea) AS A
PIVOT(SUM(bb) FOR bb in([400],[200],[500])) AS pvt) B ON A.id=B.id
INNER JOIN
(SELECT 1 id,[20]cc1,[25]cc2,[20]cc3 FROM(SELECT cc FROM tablea) AS A
PIVOT(MIN(cc) FOR cc in([20],[25])) AS pvt) C ON B.id=C.id
If I have a string with numbers separated by commas, like this:
Declare #string varchar(20) = '123,456,789'
And would like to return every possible combination + sort order of the values by doing this:
Select Combination FROM dbo.GetAllCombinations(#string)
Which would in result return this:
123
456
789
123,456
456,123
123,789
789,123
456,789
789,456
123,456,789
123,789,456
456,789,123
456,123,789
789,456,123
789,123,456
As you can see not only is every combination returned, but also each combination+sort order as well. The example shows only 3 values separated by commas, but should parse any amount--Recursive.
The logic needed would be somewhere in the realm of using a WITH CUBE statement, but the problem with using WITH CUBE (in a table structure instead of CSV of course), is that it won't shuffle the order of the values 123,456 456,123 etc., and will only provide each combination, which is only half of the battle.
Currently I have no idea what to try. If someone can provide some assistance it would be appreciated.
I use a User Defined Table-valued Function called split_delimiter that takes 2 values: the #delimited_string and the #delimiter_type.
CREATE FUNCTION [dbo].[split_delimiter](#delimited_string VARCHAR(8000), #delimiter_type CHAR(1))
RETURNS TABLE AS
RETURN
WITH cte10(num) AS
(
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL
SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1 UNION ALL SELECT 1
)
,cte100(num) AS
(
SELECT 1
FROM cte10 t1, cte10 t2
)
,cte10000(num) AS
(
SELECT 1
FROM cte100 t1, cte100 t2
)
,cte1(num) AS
(
SELECT TOP (ISNULL(DATALENGTH(#delimited_string),0)) ROW_NUMBER() OVER (ORDER BY (SELECT NULL))
FROM cte10000
)
,cte2(num) AS
(
SELECT 1
UNION ALL
SELECT t.num+1
FROM cte1 t
WHERE SUBSTRING(#delimited_string,t.num,1) = #delimiter_type
)
,cte3(num,[len]) AS
(
SELECT t.num
,ISNULL(NULLIF(CHARINDEX(#delimiter_type,#delimited_string,t.num),0)-t.num,8000)
FROM cte2 t
)
SELECT delimited_item_num = ROW_NUMBER() OVER(ORDER BY t.num)
,delimited_value = SUBSTRING(#delimited_string, t.num, t.[len])
FROM cte3 t;
Using that I was able to parse the CSV to a table and join it back to itself multiple times and use WITH ROLLUP to get the permutations you are looking for.
WITH Numbers as
(
SELECT delimited_value
FROM dbo.split_delimiter('123,456,789',',')
)
SELECT CAST(Nums1.delimited_value AS VARCHAR)
,ISNULL(CAST(Nums2.delimited_value AS VARCHAR),'')
,ISNULL(CAST(Nums3.delimited_value AS VARCHAR),'')
,CAST(Nums4.delimited_value AS VARCHAR)
FROM Numbers as Nums1
LEFT JOIN Numbers as Nums2
ON Nums2.delimited_value not in (Nums1.delimited_value)
LEFT JOIN Numbers as Nums3
ON Nums3.delimited_value not in (Nums1.delimited_value, Nums2.delimited_value)
LEFT JOIN Numbers as Nums4
ON Nums4.delimited_value not in (Nums1.delimited_value, Nums2.delimited_value, Nums3.delimited_value)
GROUP BY CAST(Nums1.delimited_value AS VARCHAR)
,ISNULL(CAST(Nums2.delimited_value AS VARCHAR),'')
,ISNULL(CAST(Nums3.delimited_value AS VARCHAR),'')
,CAST(Nums4.delimited_value AS VARCHAR) WITH ROLLUP
If you will potentially have more than 3 or 4, you'll want to expand your code accordingly.
I have ONE table like this
in this table orderID column is repeat but its CollectDate is different
So i want to SET MY table AS means IF OrderID Same then concatenate Collect Date
MY query is following and its work
SELECT distinct df.OrderId as OrderId,df.FileName as FileName, df.RandomKey as RandomKey,
df.ClientId as ClientId, df.ProjectId as ProjectId, df.Status as Status,df.UserId as UserId,
df.emailTo as emailTo,df.PackageType as PackageType,df.RequestedDatetime as RequestedDatetime,
STUFF(
(SELECT ' | ' + convert(varchar(10),ord.CollectDate, 101)
FROM SMXPSU.OrderDetails ord
WHERE df.OrderId =ord.OrderId --and ord.OrderId in(getdate()-30)
FOR XML PATH('')),1,1,'') AS CollectDate
FROM SMXPSU.downloadfiles AS df
INNER Join
SMXPSU.OrderDetails ord (NOLOCK)
ON df.OrderId=ord.OrderId
WHERE df.ClientId='shaw22' and df.PackageType='Hard Copy' and df.RequestedDatetime>=getdate()-30
GROUP BY df.OrderId,FileName,PackageType,RequestedDatetime,RandomKey,ClientId,ProjectId,Status,UserId,emailTo,PackageType,RequestedDatetime,CollectDate
order by df.OrderId desc ![enter image description here][3]
But Output give me like this
Means if CollectDate same then cumming multiple time but i want only one time if same
You can try the following query
;WITH order_cte
AS
(
SELECT OrderId,RandomKey,PackageType,RequestedDate,
(
SELECT CAST(CAST(orders.CollectedData AS DATE) AS VARCHAR) + '-' AS 'text()'
FROM Orders WHERE Orders.OrderId=o.OrderId
FOR XML Path('')
) [CollectDate]
FROM Orders o
GROUP BY OrderId,RandomKey,PackageType,RequestedDate
)
SELECT OrderId,RandomKey,PackageType,RequestedDate,LEFT([CollectDate],LEN([CollectDate])-1) AS [CollectDate]
FROM order_cte;
SELECT OrderId, RandomKey, PackageType, RequestedDate,
CASE
WHEN COUNT(*)=1 THEN CAST(MIN(CollectDate) AS VARCHAR(20))
ELSE CAST(MIN(CollectDate) AS VARCHAR(20)) + '-' + CAST(MAX(CollectDate) AS VARCHAR (20)) AS CollectDate
FROM
Table
GROUP BY OrderId, RandomKey, PackageType, RequestedDate
I have a table with following columns
defect_id, developer_name, status, summary, root_cause,
Secondary_RC, description, Comments, environment_name
The column root_cause has Enviro, Requi, Dev, TSc, TD, Unkn as its values and
column environment_name has QA1, QA2, QA3
I need to prepare a report in the below format
Enviro Requi Dev TSc TD Unkn Total
QA1 9 1 14 17 2 3 46
QA2 8 1 14 0 5 1 29
QA3 1 1 7 0 0 1 10
Total 18 3 35 17 7 5 85
I have prepare the report till
Enviro Requi Dev TSc TD Unkn
QA1 9 1 14 17 2 3
QA2 8 1 14 0 5 1
QA3 1 1 7 0 0 1
I used the below query to get the above result
select *
from
(
select environment_name as " ", value
from test1
unpivot
(
value
for col in (root_cause)
) unp
) src
pivot
(
count(value)
for value in ([Enviro] , [Requi] , [Dev] , [Tsc], [TD] , [Unkn])
) piv
Can anyone help to get the totals for columns and rows?
There may be various approaches to this. You can calculate all the totals after the pivot, or you can get the totals first, then pivot all the results. It is also possible to have kind of middle ground: get one kind of the totals (e.g. the row-wise ones), pivot, then get the other kind, although that might be overdoing it.
The first of the mentioned approaches, getting all the totals after the pivot, could be done in a very straightforward way, and the only thing potentially new to you in the below implementation might be GROUP BY ROLLUP():
SELECT
[ ] = ISNULL(environment_name, 'Total'),
[Enviro] = SUM([Enviro]),
[Requi] = SUM([Requi]),
[Dev] = SUM([Dev]),
[Tsc] = SUM([Tsc]),
[TD] = SUM([TD]),
[Unkn] = SUM([Unkn]),
Total = SUM([Enviro] + [Requi] + [Dev] + [Tsc] + [TD] + [Unkn])
FROM (
SELECT environment_name, root_cause
FROM test1
) s
PIVOT (
COUNT(root_cause)
FOR root_cause IN ([Enviro], [Requi], [Dev], [Tsc], [TD], [Unkn])
) p
GROUP BY
ROLLUP(environment_name)
;
Basically, the GROUP BY ROLLUP() part produces the Total row for you. The grouping is first done by environment_name, then the grand total row is added.
To do just the opposite, i.e. get the totals prior to pivoting, you could employ GROUP BY CUBE() like this:
SELECT
[ ] = environment_name,
[Enviro] = ISNULL([Enviro], 0),
[Requi] = ISNULL([Requi] , 0),
[Dev] = ISNULL([Dev] , 0),
[Tsc] = ISNULL([Tsc] , 0),
[TD] = ISNULL([TD] , 0),
[Unkn] = ISNULL([Unkn] , 0),
Total = ISNULL(Total , 0)
FROM (
SELECT
environment_name = ISNULL(environment_name, 'Total'),
root_cause = ISNULL(root_cause, 'Total'),
cnt = COUNT(*)
FROM test1
WHERE root_cause IS NOT NULL
GROUP BY
CUBE(environment_name, root_cause)
) s
PIVOT (
SUM(cnt)
FOR root_cause IN ([Enviro], [Requi], [Dev], [Tsc], [TD], [Unkn], Total)
) p
;
Both methods can be tested and played with at SQL Fiddle:
Method 1
Method 2
Note. I've omitted the unpivoting step in both suggestions because unpivoting a single column seemed clearly redundant. If there's more to it, though, adjusting either of the queries should be easy.
You can find Total for root_cause and environment_name using ROLLUP.
RNO_COLTOTAL - Logic to place Total in last column, since the columns Tsc,Unkn will overlap the column Total when pivoting, since its ordering alphabetically.
RNO_ROWTOTAL - Logic to place Total in last row since a value that is starting with U,W,X,Y,Z can overlap the value Total, since its ordering alphabetically.
SUM(VALUE) - Can define on what aggregate function we can use with ROLLUP.
QUERY 1
SELECT CASE WHEN root_cause IS NULL THEN 1 ELSE 0 END RNO_COLTOTAL,
CASE WHEN environment_name IS NULL THEN 1 ELSE 0 END RNO_ROWTOTAL,
ISNULL(environment_name,'Total')environment_name,
ISNULL(root_cause,'Total')root_cause,
SUM(VALUE) VALUE
INTO #NEWTABLE
FROM
(
-- Find the count for environment_name,root_cause
SELECT DISTINCT *,COUNT(*) OVER(PARTITION BY environment_name,root_cause)VALUE
FROM #TEMP
)TAB
GROUP BY root_cause,environment_name
WITH CUBE
We will get the following logic when CUBE is used
We declare variables for pivoting.
#cols - Column values for pivoting.
#NulltoZeroCols - Replace null values with zero.
QUERY 2
DECLARE #cols NVARCHAR (MAX)
SELECT #cols = COALESCE (#cols + ',[' + root_cause + ']',
'[' + root_cause + ']')
FROM (SELECT DISTINCT RNO_COLTOTAL,root_cause FROM #NEWTABLE) PV
ORDER BY RNO_COLTOTAL,root_cause
DECLARE #NulltoZeroCols NVARCHAR (MAX)
SET #NullToZeroCols = SUBSTRING((SELECT ',ISNULL(['+root_cause+'],0) AS ['+root_cause+']'
FROM(SELECT DISTINCT RNO_COLTOTAL,root_cause FROM #NEWTABLE GROUP BY RNO_COLTOTAL,root_cause)TAB
ORDER BY RNO_COLTOTAL FOR XML PATH('')),2,8000)
Now pivot it dynamically
DECLARE #query NVARCHAR(MAX)
SET #query = 'SELECT environment_name,'+ #NulltoZeroCols +' FROM
(
SELECT RNO_ROWTOTAL,environment_name,root_cause,VALUE
FROM #NEWTABLE
) x
PIVOT
(
MIN(VALUE)
FOR [root_cause] IN (' + #cols + ')
) p
ORDER BY RNO_ROWTOTAL,environment_name;'
EXEC SP_EXECUTESQL #query
RESULT
I think you need to calculate the Total separately. Using this simple query for the total (sorry, had to give the alias name for your " " column):
select environment_name as en,
count (*) AS Total
FROM test1
WHERE value in ('Enviro', 'Requi', 'Dev', 'Tsc', 'TD', 'Unkn')
GROUP BY environment_name
you can easily join both queries together to get the required report:
SELECT * FROM
(select *
from
(
select environment_name as en, value
from test1
unpivot
(
value
for col in (root_cause)
) unp
) src
pivot
(
count(value)
for value in ([Enviro] , [Requi] , [Dev] , [Tsc], [TD] , [Unkn])
) piv
) AS a
INNER JOIN
( select environment_name as en,
count (*) AS Total
FROM test1
WHERE value in ('Enviro', 'Requi', 'Dev', 'Tsc', 'TD', 'Unkn')
GROUP BY environment_name
) AS b ON a.en = b.en
UNION ALL
SELECT * FROM
(select *
from
(
select 'Total' as en, value
from test1
unpivot
(
value
for col in (root_cause)
) unp
) src
pivot
(
count(value)
for value in ([Enviro] , [Requi] , [Dev] , [Tsc], [TD] , [Unkn])
) piv
) AS a
INNER JOIN
( select 'Total' as en,
count (*) AS Total
FROM test1
WHERE value in ('Enviro', 'Requi', 'Dev', 'Tsc', 'TD', 'Unkn')
) AS b
I have not tested it, but believe it will work
You can also put you select pivot, without totals, in a temp table and then add the totalS with a new select:
SELECT environment_name=ISNULL(environment_name, ‘Total’) , Enviro=SUM(Enviro), Requi=SUM(Requi), Dev=SUM(Dev), TSc=SUM(TSc), TD=SUM(TD), Unkn =SUM(Unkn),
Total = sum(Enviro+Requi+Dev+TSc+TD+Unkn)
FROM #temp
GROUP BY ROLLUP(environment_name)