Frequency of all combinations of values for certain column - sql

I have a dataset in SQL Server 2012 with a column for id and value, like this:
[id] [value]
--------------
A 15
A 11
A 11
B 13
B 15
B 12
C 12
C 13
D 13
D 12
My goal is to get a frequency count of all combinations of [value], with two caveats:
Order doesn't matter, so [11,12,15] is not counted separately from [12,11,15]
Repeated values are counted separately, so [11,11,12,15] is counted separately from [11,12,15]
I'm interested in all combinations, of any length (not just pairs)
So the outcome would look like:
[combo] [frequency]
---------------------
11,11,15 1
12,13,15 1
12,13 2
I've seen answers here involving recursion that answer similar questions but where order counts, and answers here involving self-joins that yield pair-wise combinations. These come close but I'm not quite sure how to adapt for my specific needs.

You can use string_agg():
select vals, count(*) as frequency
from (select string_agg(value, ',') within group (order by value) as vals, id
from t
group by id
) i
group by vals;
SQL Server 2012 doesn't support string_agg() but you can use the XML hack:
select vals, count(*) as frequency
from (select id,
stuff( (select concat(',', value)
from t t2
where t2.id = i.id
for xml path ('')
), 1, 1, ''
) as vals
from (select distinct id from t) i
) i
group by vals;

Your number string is just all the values with the same id in increasing order. So I'm treating the lowest id as a canonical name for the full sequence and all its matches. This spares all the string manipulations though you can expand as necessary.
Just tag each duplicate value with a counter and then look for groups that pair up completely.
with data as (
select id, value,
row_number() over (partition by id, value) as rn
), matches as (
select l.id, r.id as match
from data l full outer join data r on
l.value = r.value and l.rn = r.rn and l.id <= r.id
group by l.id
having count(l.id) = count(*) and count(r.id) = count(*)
)
select id, count(match) as frequency
from matches
group by id;
The logic in the middle query is also easily adaptable for finding subset of values in common.

You can achieve this using CTEs and row_number functions.
DECLARE #table table(id CHAR(1), val int)
insert into #table VALUES
('A',15),
('A',11),
('A',11),
('B',13),
('B',15),
('B',12),
('C',12),
('C',13),
('D',13),
('D',12);
;WITH CTE_rnk as
(
SELECT id,val, row_number() over (partition by id order by val) as rnk
from #table
),
CTE_concat as
(
SELECT id, cast(val as varchar(100)) as val, rnk
from CTE_rnk
where rnk =1
union all
SELECT r.id, cast(concat(c.val,',',r.val) as varchar(100)) as val,r.rnk
from CTE_rnk as r
inner join CTE_concat as c
on r.rnk = c.rnk+1
and r.id = c.id
),
CTE_maxcombo as
(
SELECT id,val, row_number() over(partition by id order by rnk desc) as rnk
from CTE_concat
)
select val as combo, count(*) as frequency
from CTE_maxcombo where rnk = 1
group by val
+----------+-----------+
| combo | frequency |
+----------+-----------+
| 11,11,15 | 1 |
| 12,13 | 2 |
| 12,13,15 | 1 |
+----------+-----------+

Related

Swap two adjacent rows of a column in sql

I'm trying to solve this following problem:
Write a sql query to swap two adjacent rows in a column of a table.
Input table
Name Id
A 1
B 2
C 3
D 4
E 5
Output table
Name Id
A 2
B 1
C 4
D 3
E 5
Description:- 1 is associated with A and 2 with B, swap them, thus now 1 is associated with B and 2 with A, Similarly do for C and D, Since E doesn't has any pair, leave it as it is.
Note:- This may be solved using CASE Statements, but I am trying for a generalized solution, Say currently it is only 5 rows, it may be 10,20 etc..
Eg:
SELECT
*,CASE WHEN Name = A then 2 ELSEIF Name = B then 1 etc...
FROM YourTable
You can use window functions to solve this.
on MySQL (>= 8.0):
SELECT ID, IFNULL(CASE WHEN t.rn % 2 = 0 THEN LAG(Name) OVER (ORDER BY ID) ELSE LEAD(Name) OVER (ORDER BY ID) END, Name) AS Name
FROM (
SELECT ID, Name, ROW_NUMBER() OVER (ORDER BY ID) AS rn
FROM table_name
) t
demo on dbfiddle.uk
on SQL-Server:
SELECT ID, ISNULL(CASE WHEN t.rn % 2 = 0 THEN LAG(Name) OVER (ORDER BY ID) ELSE LEAD(Name) OVER (ORDER BY ID) END, Name) AS Name
FROM (
SELECT ID, Name, ROW_NUMBER() OVER (ORDER BY ID) AS rn
FROM table_name
) t
demo on dbfiddle.uk
If you have sql-server, you can try this.
DECLARE #YourTable TABLE (Name VARCHAR(10), Id INT)
INSERT INTO #YourTable VALUES
('A', 1),
('B', 2),
('C', 3),
('D', 4),
('E', 5)
;WITH CTE AS (
SELECT *, ROW_NUMBER()OVER(ORDER BY Name) AS RN FROM #YourTable
)
SELECT T1.Name, ISNULL(T2.Id, T1.Id) Id FROM CTE T1
LEFT JOIN CTE T2 ON T1.RN + CASE WHEN T1.RN%2 = 0 THEN - 1 ELSE 1 END = T2.RN
Result:
Name Id
---------- -----------
A 2
B 1
C 4
D 3
E 5
You didn't specify your DBMS, but the following is standard ANSI SQL.
You can use a values() clause to provide the mapping of the IDs and then join against that:
with id_map (source_id, target_id) as (
values
(1, 2),
(2, 1)
)
select t.name, coalesce(m.target_id, t.id) as mapped_id
from the_table t
left join id_map m on m.source_id = t.id
order by name;
Alternatively if you only want to specify the mapping once for one direction, you can use this:
with id_map (source_id, target_id) as (
values
(1, 2)
)
select t.name,
case id
when m.source_id then m.target_id
when m.target_id then m.source_id
else id
end as mapped_id
from the_table t
left join id_map m on t.id in (m.source_id, m.target_id)
order by name;
Online example: https://rextester.com/FBFH52231

concatenate recursive cross join

I need to concatenate the name in a recursive cross join way. I don't know how to do this, I have tried a CTE using WITH RECURSIVE but no success.
I have a table like this:
group_id | name
---------------
13 | A
13 | B
19 | C
19 | D
31 | E
31 | F
31 | G
Desired output:
combinations
------------
ACE
ACF
ACG
ADE
ADF
ADG
BCE
BCF
BCG
BDE
BDF
BDG
Of course, the results should multiply if I add a 4th (or more) group.
Native Postgresql Syntax:
SqlFiddleDemo
WITH RECURSIVE cte1 AS
(
SELECT *, DENSE_RANK() OVER (ORDER BY group_id) AS rn
FROM mytable
),cte2 AS
(
SELECT
CAST(name AS VARCHAR(4000)) AS name,
rn
FROM cte1
WHERE rn = 1
UNION ALL
SELECT
CAST(CONCAT(c2.name,c1.name) AS VARCHAR(4000)) AS name
,c1.rn
FROM cte1 c1
JOIN cte2 c2
ON c1.rn = c2.rn + 1
)
SELECT name as combinations
FROM cte2
WHERE LENGTH(name) = (SELECT MAX(rn) FROM cte1)
ORDER BY name;
Before:
I hope if you don't mind that I use SQL Server Syntax:
Sample:
CREATE TABLE #mytable(
ID INTEGER NOT NULL
,TYPE VARCHAR(MAX) NOT NULL
);
INSERT INTO #mytable(ID,TYPE) VALUES (13,'A');
INSERT INTO #mytable(ID,TYPE) VALUES (13,'B');
INSERT INTO #mytable(ID,TYPE) VALUES (19,'C');
INSERT INTO #mytable(ID,TYPE) VALUES (19,'D');
INSERT INTO #mytable(ID,TYPE) VALUES (31,'E');
INSERT INTO #mytable(ID,TYPE) VALUES (31,'F');
INSERT INTO #mytable(ID,TYPE) VALUES (31,'G');
Main query:
WITH cte1 AS
(
SELECT *, rn = DENSE_RANK() OVER (ORDER BY ID)
FROM #mytable
),cte2 AS
(
SELECT
TYPE = CAST(TYPE AS VARCHAR(MAX)),
rn
FROM cte1
WHERE rn = 1
UNION ALL
SELECT
[Type] = CAST(CONCAT(c2.TYPE,c1.TYPE) AS VARCHAR(MAX))
,c1.rn
FROM cte1 c1
JOIN cte2 c2
ON c1.rn = c2.rn + 1
)
SELECT *
FROM cte2
WHERE LEN(Type) = (SELECT MAX(rn) FROM cte1)
ORDER BY Type;
LiveDemo
I've assumed that the order of "cross join" is dependent on ascending ID.
cte1 generate DENSE_RANK() because your IDs contain gaps
cte2 recursive part with CONCAT
main query just filter out required length and sort string
The recursive query is a bit simpler in Postgres:
WITH RECURSIVE t AS ( -- to produce gapless group numbers
SELECT dense_rank() OVER (ORDER BY group_id) AS grp, name
FROM tbl
)
, cte AS (
SELECT grp, name
FROM t
WHERE grp = 1
UNION ALL
SELECT t.grp, c.name || t.name
FROM cte c
JOIN t ON t.grp = c.grp + 1
)
SELECT name AS combi
FROM cte
WHERE grp = (SELECT max(grp) FROM t)
ORDER BY 1;
The basic logic is the same as in the SQL Server version provided by #lad2025, I added a couple of minor improvements.
Or you can use a simple version if your maximum number of groups is not too big (can't be very big, really, since the result set grows exponentially). For a maximum of 5 groups:
WITH t AS ( -- to produce gapless group numbers
SELECT dense_rank() OVER (ORDER BY group_id) AS grp, name AS n
FROM tbl
)
SELECT concat(t1.n, t2.n, t3.n, t4.n, t5.n) AS combi
FROM (SELECT n FROM t WHERE grp = 1) t1
LEFT JOIN (SELECT n FROM t WHERE grp = 2) t2 ON true
LEFT JOIN (SELECT n FROM t WHERE grp = 3) t3 ON true
LEFT JOIN (SELECT n FROM t WHERE grp = 4) t4 ON true
LEFT JOIN (SELECT n FROM t WHERE grp = 5) t5 ON true
ORDER BY 1;
Probably faster for few groups. LEFT JOIN .. ON true makes this work even if higher levels are missing. concat() ignores NULL values. Test with EXPLAIN ANALYZE to be sure.
SQL Fiddle showing both.

How to create "subsets" as a result from a column in SQL

Let's suppose that I've got as a result from one query the next set of values of one column:
Value
1 A
2 B
3 C
4 D
5 E
6 F
7 G
8 H
9 I
10 J
Now, I would like to see this information with another order, establishing a limit to the number of values of every single subset. Now suppose that I choose 3 as a limit,the information will be given like this (one column for all the subsets):
Values
1 A, B, C
2 D, E, F
3 G, H, I
4 J,
Obviously, the last row will contain the remaining values when their number is smaller than the limit established.
Is it possible to perform a query like this in SQL?
What about if the limit is dynamic?. It can be chosen randomly.
create table dee_t (id int identity(1,1),name varchar(10))
insert into dee_t values ('A'),('B'),('c'),('D'),('E'),('F'),('g'),('H'),('I'),('J')
;with cte as
(
select (id-1)/3 +1 rno ,* from dee_t
) select rno ,
(select name+',' from cte where rno = c.rno for xml path('') )
from cte c group by rno
You can do this by using few calculations with row_number, like this:
select
GRP,
max(case when RN = 1 then Value end),
max(case when RN = 2 then Value end),
max(case when RN = 0 then Value end)
from (
select
row_number() over (order by Value) % 3 as RN,
(row_number() over (order by Value)+2) / 3 as GRP,
Value
from Table1
) X
group by GRP
The first row_number creates numbers for the columns (1,2,0,1,2,0...) and the second one creates numbers for the rows (1,1,1,2...). Those are then used to group the values into correct place using case, but you can also use pivot instead of it if you like it more.
If you want them into same column, of course just concatenate the cases instead of selecting them on different columns, but beware of nulls.
Example in SQL Fiddle
Thanks a lot for all your reply. Finally I've got a Solution with the help of Rajen Singh
This is the code than can be used:
WITH CTE_0 AS
(
SELECT DISTINCT column_A_VALUE AS id
FROM Table
WHERE column_A_VALUE IS NOT NULL
), CTE_1 AS
(
SELECT ROW_NUMBER() OVER (ORDER BY id) RN, id
FROM CTE_0
), CTE_2 AS
(
SELECT RN%30 GROUP, ID
FROM CTE_1
)SELECT STUFF(( SELECT ','''+CAST(ID AS NVARCHAR(20))+''''
FROM CTE_2
WHERE GROUP = A.GROUP
FOR XML PATH('')),1,1,'') IDS
FROM CTE_2 A
GROUP BY GROUP

SUM Column in SQL

I have a table in SQL Server, and I need to sum a column, like the example below:
CREATE TABLE B
(
ID int,
Qty int,
)
INSERT INTO B VALUES (1,2)
INSERT INTO B VALUES (2,7)
INSERT INTO B VALUES (3,2)
INSERT INTO B VALUES (4,11)
SELECT *, '' AS TotalQty FROM B
ORDER BY ID
In this example what I need is the column TotalQty give me the values like:
2
9
11
22
How can it be achieved?
You can use SUM in a co-related subquery or CROSS APPLY like this
Co-related Subquery
SELECT ID,(SELECT SUM(Qty) FROM B WHERE B.id <= C.id) FROM B as C
ORDER BY ID
Using CROSS APPLY
SELECT ID,D.Qty FROM B as C
CROSS APPLY
(
SELECT SUM(Qty) Qty
FROM B WHERE B.id <= C.id
)AS D
ORDER BY ID
Output
1 2
2 9
3 11
4 22
If you were using SQL Server 2012 or above, SUM() with Over() clause could have been used like this.
SELECT ID, SUM(Qty) OVER(ORDER BY ID ASC) FROM B as C
ORDER BY ID
Edit
Another way to do this in SQL Server 2008 is using Recursive CTE. Something like this.
Note: This method is based on the answer by Roman Pekar on this thread Calculate a Running Total in SQL Server. Based on his observation this would perform better than co related subquery and CROSS APPLY both
;WITH CTE as
(
SELECT ID,Qty,ROW_NUMBER()OVER(ORDER BY ID ASC) as rn
FROM B
), CTE_Running_Total as
(
SELECT Id,rn,Qty,Qty as Running_Total
FROM CTE
WHERE rn = 1
UNION ALL
SELECT C1.Id,C1.rn,C1.Qty,C1.Qty + C2.Running_Total as Running_Total
FROM CTE C1
INNER JOIN CTE_Running_Total C2
ON C1.rn = C2.rn + 1
)
SELECT *
FROM CTE_Running_Total
ORDER BY Id
OPTION (maxrecursion 0)

Find all integer gaps in SQL

I have a database which is used to store information about different matches for a game that I pull in from an external source. Due to a few issues, there are occasional gaps (which could be anywhere from 1 missing ID to a few hundred) in the database. I want to have the program pull in the data for the missing games, but I need to get that list first.
Here is the format of the table:
id (pk-identity) | GameID (int) | etc. | etc.
I had thought of writing a program to run through a loop and query for each GameID starting at 1, but it seems like there should be a more efficient way to get the missing numbers.
Is there an easy and efficient way, using SQL Server, to find all the missing numbers from the range?
The idea is to look at where the gaps start. Let me assume you are using SQL Server 2012, and so have the lag() and lead() functions. The following gets the next id:
select t.*, lead(id) over (order by id) as nextid
from t;
If there is a gap, then nextid <> id+1. You can now characterize the gaps using where:
select id+1 as FirstMissingId, nextid - 1 as LastMissingId
from (select t.*, lead(id) over (order by id) as nextid
from t
) t
where nextid <> id+1;
EDIT:
Without the lead(), I would do the same thing with a correlated subquery:
select id+1 as FirstMissingId, nextid - 1 as LastMissingId
from (select t.*,
(select top 1 id
from t t2
where t2.id > t.id
order by t2.id
) as nextid
from t
) t
where nextid <> id+1;
Assuming the id is a primary key on the table (or even that it just has an index), both methods should have reasonable performance.
Numbers table!
CREATE TABLE dbo.numbers (
number int NOT NULL
)
ALTER TABLE dbo.numbers
ADD
CONSTRAINT pk_numbers PRIMARY KEY CLUSTERED (number)
WITH FILLFACTOR = 100
GO
INSERT INTO dbo.numbers (number)
SELECT (a.number * 256) + b.number As number
FROM (
SELECT number
FROM master..spt_values
WHERE type = 'P'
AND number <= 255
) As a
CROSS
JOIN (
SELECT number
FROM master..spt_values
WHERE type = 'P'
AND number <= 255
) As b
GO
Then you can perform an OUTER JOIN or EXISTS` between your two tables and find the gaps...
SELECT *
FROM dbo.numbers
WHERE NOT EXISTS (
SELECT *
FROM your_table
WHERE id = numbers.number
)
-- OR
SELECT *
FROM dbo.numbers
LEFT
JOIN your_table
ON your_table.id = numbers.number
WHERE your_table.id IS NULL
I like the "gaps and islands" approach. It goes a little something like this:
WITH Islands AS (
SELECT GameId, GameID - ROW_NUMBER() OVER (ORDER BY GameID) AS [IslandID]
FROM dbo.yourTable
)
SELECT MIN(GameID), MAX(Game_id)
FROM Islands
GROUP BY IslandID
That query will get you the list of contiguous ranges. From there, you can self-join that result set (on successive IslandIDs) to get the gaps. There is a bit of work in getting the IslandIDs themselves to be contiguous though. So, extending the above query:
WITH
cte1 AS (
SELECT GameId, GameId - ROW_NUMBER() OVER (ORDER BY GameId) AS [rn]
FROM dbo.yourTable
)
, cte2 AS (
SELECT [rn], MIN(GameId) AS [Start], MAX(GameId) AS [End]
FROM cte1
GROUP BY [rn]
)
,Islands AS (
SELECT ROW_NUMBER() OVER (ORDER BY [rn]) AS IslandId, [Start], [End]
from cte2
)
SELECT a.[End] + 1 AS [GapStart], b.[Start] - 1 AS [GapEnd]
FROM Islands AS a
LEFT JOIN Islands AS b
ON a.IslandID + 1 = b.IslandID
SELECT * FROM #tab1
id col1
----------- --------------------
1 a
2 a
3 a
8 a
9 a
10 a
11 a
15 a
16 a
17 a
18 a
WITH cte (id,nextId) as
(SELECT t.id, (SELECT TOP 1 t1.id FROM #tab1 t1 WHERE t1.id > t.id) AS nextId FROM #tab1 t)
SELECT id AS 'GapStart', nextId AS 'GapEnd' FROM cte
WHERE id + 1 <> nextId
GapStart GapEnd
----------- -----------
3 8
11 15
Try this (This covers upto 10000 Ids starting from 1, if you need more you can add more to Numbers table below):
;WITH Digits AS (
select Digit
from ( values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9)) as t(Digit))
,Numbers AS (
select u.Digit
+ t.Digit*10
+ h.Digit*100
+ th.Digit*1000
+ tth.Digit*10000
--Add 10000, 100000 multipliers if required here.
as myId
from Digits u
cross join Digits t
cross join Digits h
cross join Digits th
cross join Digits tth
--Add the cross join for higher numbers
)
SELECT myId
FROM Numbers
WHERE myId NOT IN (SELECT GameId FROM YourTable)
Problem: we need to find the gap range in id field
SELECT * FROM #tab1
id col1
----------- --------------------
1 a
2 a
3 a
8 a
9 a
10 a
11 a
15 a
16 a
17 a
18 a
Solution
WITH cte (id,nextId) as
(SELECT t.id, (SELECT TOP 1 t1.id FROM #tab1 t1 WHERE t1.id > t.id) AS nextId FROM #tab1 t)
SELECT id + 1, nextId - 1 FROM cte
WHERE id + 1 <> nextId
Output
GapStart GapEnd
----------- -----------
4 7
12 14