Using MS-SQL, I have the following table excerpt:
-----------------------------------
Market | Cycle | Milestone | Sale |
A | NULL | NULL | NULL |
A | 1 | NULL | NULL |
B | NULL | NULL | NULL |
B | 3 | 4 | NULL |
B | 3 | 4 | 5 |
A | 1 | 2 | NULL |
A | 1 | 2 | 1 |
NULL | C | 6 | 7 |
NULL | C | NULL | NULL |
D | 8 | NULL | NULL |
D | 8 | 9 | NULL |
Each row represents a new stage in the product life-cycle.
If the first stage of product C was Cycle, the next row for it will have values in Cycle and Milestone, and so forth.
I need to add an identifier for each group, based on the first not-null column for each value.
The required output for the above table would be as follows:
-------------------------------------------
Market | Cycle | Milestone | Sale | Group
A | NULL | NULL | NULL | 1
A | 1 | NULL | NULL | 1
B | NULL | NULL | NULL | 2
B | 3 | 4 | NULL | 2
B | 3 | 4 | 5 | 2
A | 1 | 2 | NULL | 1
A | 1 | 2 | 1 | 1
NULL | C | 6 | 7 | 3
NULL | C | NULL | NULL | 3
D | 8 | NULL | NULL | 4
D | 8 | 9 | NULL | 4
If a new row will be added with Market "D", it will receive Group 1.
If a new row will be added with Market Null and Cycle which has no appeared yet, it will start a new group 5. Future rows with the same cycle will also receive 5.
Hopefully this is clear enough...
Any assistance with SQL-Server code for this will be helpful.
Thank you!
This will set the Group value in the way you want, although it seems inelegant:
UPDATE tblGroup
SET Group = ASCII(COALESCE(Market, Cycle, Milestone, Sale)) - 64
...or select it via:
SELECT *, ASCII(COALESCE(Market, Cycle, Milestone, Sale)) - 64 AS Group
FROM tblGroup
You could use a window function as
SELECT *, DENSE_RANK() OVER(ORDER BY Market) [Group]
-- Or DENSE_RANK() OVER(ORDER BY COALESCE(Market, Cycle)) [Group] to get the exact results
FROM
(
VALUES
('A', NULL, NULL, NULL),
('A', '1', NULL, NULL),
('B', NULL, NULL, NULL),
('B', '3', 4, NULL),
('B', '3', 4, 5 ),
('A', '1', 2, NULL),
('A', '1', 2, 1 ),
(NULL, 'C', 6, 7 ),
(NULL, 'C', NULL, NULL),
('D', '8', NULL, NULL),
('D', '8', 9, NULL)
)T(Market, Cycle, Milestone, Sale)
Online Demo
Related
Suppose I have the following table:
table
| a | b | c |
|:-----|:----|:-----|
| 1 | a | NULL |
| NULL | b | NULL |
| 3 | c | NULL |
| 4 | d | 23 |
| NULL | e | 231 |
How can I count the number of NULL values by each column?
My final result would be:
| column_name | n_nulls |
|:---------------|:----------|
| a | 2 |
| b | 0 |
| c | 3 |
You can use union all:
select 'a', count(*) - count(a) as n_nulls from t
union all
select 'b', count(*) - count(b) as n_nulls from t
union all
select 'c', count(*) - count(c) as n_nulls from t;
Redshift is a column-store database, so there probably is not a more efficient method.
I need help with ranking of rows in one table.
+-----+-------+-------------+------------+-------+------+
| ID | group | typeInGroup | rankOfType | score | Rank |
+-----+-------+-------------+------------+-------+------+
| 1 | a | type1 | 1 | 40 | |
| 2 | a | type2 | 2 | 55 | |
| 3 | a | type1 | 1 | 20 | |
| 4 | b | type3 | 3 | 80 | |
| 5 | b | type2 | 2 | 60 | |
| 6 | b | type1 | 1 | 70 | |
| 7 | b | type1 | 1 | 70 | |
+-----+-------+-------------+------------+-------+------+
I am basically looking for solution which would give me order for last column "Rank".
Each "group" has up to 9 "typeInGroup" which are ranked by 1-9 in column "rankOfTypes". Each "typeInGroup" has "score". When i am calculating last column "Rank" i look at the "score" and "rankOfType" column.
The row with the higgest score should be ranked first unless there is row with "rankOfType" column that has lower value and score that is <= 15 than the score we have been looking at. Order of rows with same "score" and "rankOfType" is not important.
I should do this check for every single row in group and in the end end with something like this:
+-----+-------+-------------+------------+-------+------+
| ID | group | typeInGroup | rankOfType | score | Rank |
+-----+-------+-------------+------------+-------+------+
| 1 | a | type1 | 1 | 40 | 1 |
| 2 | a | type2 | 2 | 55 | 2 |
| 3 | a | type1 | 1 | 20 | 3 |
| 4 | b | type3 | 3 | 80 | 3 |
| 5 | b | type2 | 2 | 60 | 4 |
| 6 | b | type1 | 1 | 70 | 1 |
| 7 | b | type1 | 1 | 70 | 2 |
+-----+-------+-------------+------------+-------+------+
Any idea how to do this?
the CROSS APPLY query, checks for any rows that meet your special requirement, if exists, than that row will have higher priority
try it out with larger data set and verify the result
declare #tbl table
(
ID int,
Grp char,
typeInGrp varchar(5),
rankOfType int,
score int
)
insert into #tbl select 1, 'a', 'type1', 1, 40
insert into #tbl select 2, 'a', 'type2', 2, 55
insert into #tbl select 3, 'a', 'type1', 1, 20
insert into #tbl select 4, 'b', 'type3', 3, 80
insert into #tbl select 5, 'b', 'type2', 2, 60
insert into #tbl select 6, 'b', 'type1', 1, 70
insert into #tbl select 7, 'b', 'type1', 1, 70
select *,
[rank] = row_number() over (partition by Grp
order by case when cnt > 0 then 1 else 2 end,
score desc)
from #tbl t
cross apply
(
select cnt = count(*)
from #tbl x
where x.Grp = t.Grp
and x.ID <> t.ID
and x.rankOfType > t.rankOfType
and x.score - t.score <= 15
) s
order by ID
I am trying to find a way to count based on groups and I was not able to figure out a way without having to use a Cursor. Since using a Cursor will be relatively slow I was hoping there might be a better way.
Simplified the data is structured as follows:
+----+--------+-------+--------+
| ID | NEXTID | RowNo | Status |
+----+--------+-------+--------+
| 1 | 2 | 1 | 1 |
| 2 | 3 | 1 | 1 |
| 3 | 4 | 1 | 0 |
| 4 | | 1 | 1 |
| 1 | 2 | 2 | 0 |
| 2 | 3 | 2 | 1 |
| 3 | 4 | 2 | 1 |
| 4 | | 2 | 1 |
| 1 | 2 | 3 | 1 |
| 2 | 3 | 3 | 1 |
| 3 | 4 | 3 | 1 |
| 4 | | 3 | 1 |
+----+--------+-------+--------+
I now want to COUNT the Status column in groups resulting in:
+-----+-------------+
| Row | StatusCount |
+-----+-------------+
| 1 | 2 |
| 1 | 1 |
| 2 | 3 |
| 3 | 4 |
+-----+-------------+
For Testing purposes I creating the following code:
SELECT
ID,
NEXTID,
RowNo,
Status,
LEAD(ID,1,0)
OVER (ORDER BY RowNo,ID) AS LEADER
INTO #TestTable
FROM
(
VALUES
(1, 2, 1, 1),
(2, 3, 1, 1),
(3, 4, 1, 0),
(4, '', 1, 1),
(1, 2, 2, 0),
(2, 3, 2, 1),
(3, 4, 2, 1),
(4, '', 2, 1),
(1, 2, 3, 1),
(2, 3, 3, 1),
(3, 4, 3, 1),
(4, '', 3, 1)
)
AS TestTable(
ID,
NEXTID,
RowNo,
Status);
GO
SELECT
RowNo,
Count(Status) AS StatusCount
FROM #TestTable
WHERE
Status = 1
GROUP BY
RowNo
This results in
+-----+-------------+
| Row | StatusCount |
+-----+-------------+
| 1 | 3 |
| 2 | 3 |
| 3 | 4 |
+-----+-------------+
Not separating the first row. I do realise that I need another GROUP BY condition but I can not figure out the appropriate condition.
Thank you very much for your help. If this has already been answered I was unable to find the topic and hints will also be appreciated.
With kind regards
freubau
You can identify the groups by doing a cumulative sum of the zeros up to each number. Then, the rest is just aggregation:
select rowno, count(*)
from (select t.*,
sum(case when status = 0 then 1 else 0 end) over (partition by rowno order by id) as grp
from #TestTable t
) t
where status = 1
group by rowno, grp
order by rowno, grp;
Here is a rex tester for it.
I have a table of actions within a session and duration (milliseconds) between each step:
+-----------------------------------------------------------------------+
| | userid | sessionid | action sequence | action | milliseconds | |
| +--------+-----------+-----------------+-------------+--------------+ |
| | 1 | 1 | 1 | event start | 0 | |
| | 1 | 1 | 2 | other | 188114 | |
| | 1 | 1 | 3 | event end | 248641 | |
| | 1 | 1 | 4 | other | 398215 | |
| | 1 | 1 | 5 | event start | 488284 | |
| | 1 | 1 | 6 | other | 528445 | |
| | 1 | 1 | 7 | other | 572711 | |
| | 1 | 1 | 8 | event end | 598123 | |
| | 1 | 2 | 1 | event start | 0 | |
| | 1 | 2 | 2 | event end | 54363 | |
| | 2 | 1 | 1 | other | 0 | |
| | 2 | 1 | 2 | other | 2345 | |
| | 2 | 1 | 1 | other | 75647 | |
| | 3 | 1 | 2 | other | 0 | |
| | 3 | 1 | 3 | event start | 34678 | |
| | 3 | 1 | 4 | other | 46784 | |
| | 3 | 1 | 5 | other | 78905 | |
| | 4 | 1 | 1 | event start | 0 | |
| | 4 | 1 | 2 | other | 7454 | |
| | 4 | 1 | 3 | other | 11245 | |
| | 4 | 1 | 4 | event end | 24567 | |
| | 4 | 1 | 5 | other | 29562 | |
| | 4 | 1 | 6 | other | 43015 | |
| +--------+-----------+-----------------+-------------+--------------+ |
I would like to capture complete events -- sessions containing both an event start and end (some may have a start but no end, an end but no start, or neither -- I don't want those), and their start and end times. Ultimately I want to track duration by transposing the sequential rows of times into columns so I can calculate a difference. The above data table would ideally be transposed into:
+--------+-----------+---------------+--------+--------+
| userid | sessionid | full event id | start | end |
+--------+-----------+---------------+--------+--------+
| 1 | 1 | 1 | 0 | 248641 |
| 1 | 1 | 2 | 488284 | 598123 |
| 1 | 2 | 1 | 0 | 54363 |
| 4 | 1 | 1 | 0 | 24567 |
+--------+-----------+---------------+--------+--------+
I attempted something like:
select a.userid, a.sessionid, a.milliseconds as start, b.milliseconds as end
from #table a
inner join #table b
on a.userid=b.userid
and a.sessionid=b.sessionid
and a.action='event start'
and b.action='event end'
However, that doesn't work since some users may have multiple event start and ends in on session (like userid 1). I am stuck on how to best transpose the times data for each event. Thanks for you help!
So, given your above data:
CREATE TABLE test_table (
`userid` int,
`sessionid` int,
`actionSequence` int,
`action` varchar(11),
`milliseconds` int
);
INSERT INTO test_table
(`userid`, `sessionid`, `actionSequence`, `action`, `milliseconds`)
VALUES
(1, 1, 1, 'event start', 0),
(1, 1, 2, 'other', 188114),
(1, 1, 3, 'event end', 248641),
(1, 1, 4, 'other', 398215),
(1, 1, 5, 'event start', 488284),
(1, 1, 6, 'other', 528445),
(1, 1, 7, 'other', 572711),
(1, 1, 8, 'event end', 598123),
(1, 2, 1, 'event start', 0),
(1, 2, 2, 'event end', 54363),
(2, 1, 1, 'other', 0),
(2, 1, 2, 'other', 2345),
(2, 1, 1, 'other', 75647),
(3, 1, 2, 'other', 0),
(3, 1, 3, 'event start', 34678),
(3, 1, 4, 'other', 46784),
(3, 1, 5, 'other', 78905),
(4, 1, 1, 'event start', 0),
(4, 1, 2, 'other', 7454),
(4, 1, 3, 'other', 11245),
(4, 1, 4, 'event end', 24567),
(4, 1, 5, 'other', 29562),
(4, 1, 6, 'other', 43015);
The following query should get you where you want to be (you were on the right track):
SELECT
tt1.userid,
tt1.sessionid,
tt1.actionSequence,
tt1.milliseconds AS startMS,
MIN(tt2.milliseconds) AS endMS,
MIN(tt2.milliseconds) - tt1.milliseconds AS totalMS
FROM test_table tt1
INNER JOIN test_table tt2
ON tt2.userid = tt1.userid
AND tt2.sessionid = tt1.sessionid
AND tt2.actionSequence > tt1.actionSequence
AND tt2.action = 'event end'
WHERE tt1.action = 'event start'
GROUP BY tt1.userid, tt1.sessionid, tt1.actionSequence, startMS
Giving you this result set:
userid sessionid actionSequence startMS endMS totalMS
1 1 1 0 248641 248641
1 1 5 488284 598123 109839
1 2 1 0 54363 54363
4 1 1 0 24567 24567
The GROUP BY is important, because there are two rows with action = 'event end' and sequence > 1 for sessionid = 1 and userid = 1, so (I assume) we want the one closest to the current sequence, i.e. the MIN(milliseconds). As you can see, it also allows you to go ahead and take the difference of the two columns in this result set, saving you the extra step you were planning :]
Here is a SQLFiddle of this query in action on MySQL 5.6. You did not specify an RDBMS, but I believe the language used by this query should be simple enough to work in any sql engine.
I've got a table of temperature samples over time from several sources and I want to find the minimum, maximum, and average temperatures across all sources at set time intervals. At first glance this is easily done like so:
SELECT MIN(temp), MAX(temp), AVG(temp) FROM samples GROUP BY time;
However, things become much more complicated (to the point of where I'm stumped!) if sources drop in and out and rather than ignoring the missing sources during the intervals in question I want to use the sources' last know temperatures for the missing samples. Using datetimes and constructing intervals (say every minute) across samples unevenly distributed over time further complicates things.
I think it should be possible to create the results I want by doing a self-join on the samples table where the time from the first table is greater than or equal to the time of the second table and then calculating aggregate values for rows grouped by source. However, I'm stumped about how to actually do this.
Here's my test table:
+------+------+------+
| time | source | temp |
+------+------+------+
| 1 | a | 20 |
| 1 | b | 18 |
| 1 | c | 23 |
| 2 | b | 21 |
| 2 | c | 20 |
| 2 | a | 18 |
| 3 | a | 16 |
| 3 | c | 13 |
| 4 | c | 15 |
| 4 | a | 4 |
| 4 | b | 31 |
| 5 | b | 10 |
| 5 | c | 16 |
| 5 | a | 22 |
| 6 | a | 18 |
| 6 | b | 17 |
| 7 | a | 20 |
| 7 | b | 19 |
+------+------+------+
INSERT INTO samples (time, source, temp) VALUES (1, 'a', 20), (1, 'b', 18), (1, 'c', 23), (2, 'b', 21), (2, 'c', 20), (2, 'a', 18), (3, 'a', 16), (3, 'c', 13), (4, 'c', 15), (4, 'a', 4), (4, 'b', 31), (5, 'b', 10), (5, 'c', 16), (5, 'a', 22), (6, 'a', 18), (6, 'b', 17), (7, 'a', 20), (7, 'b', 19);
To do my min, max and avg calculations, I want an intermediate table that looks like this:
+------+------+------+
| time | source | temp |
+------+------+------+
| 1 | a | 20 |
| 1 | b | 18 |
| 1 | c | 23 |
| 2 | b | 21 |
| 2 | c | 20 |
| 2 | a | 18 |
| 3 | a | 16 |
| 3 | b | 21 |
| 3 | c | 13 |
| 4 | c | 15 |
| 4 | a | 4 |
| 4 | b | 31 |
| 5 | b | 10 |
| 5 | c | 16 |
| 5 | a | 22 |
| 6 | a | 18 |
| 6 | b | 17 |
| 6 | c | 16 |
| 7 | a | 20 |
| 7 | b | 19 |
| 7 | c | 16 |
+------+------+------+
The following query is getting me close to what I want but it takes the temperature value of the source's first result, rather than the most recent one at the given time interval:
SELECT s.dt as sdt, s.mac, ss.temp, MAX(ss.dt) as maxdt FROM (SELECT DISTINCT dt FROM samples) AS s CROSS JOIN samples AS ss WHERE s.dt >= ss.dt GROUP BY sdt, mac HAVING maxdt <= s.dt ORDER BY sdt ASC, maxdt ASC;
+------+------+------+-------+
| sdt | mac | temp | maxdt |
+------+------+------+-------+
| 1 | a | 20 | 1 |
| 1 | c | 23 | 1 |
| 1 | b | 18 | 1 |
| 2 | a | 20 | 2 |
| 2 | c | 23 | 2 |
| 2 | b | 18 | 2 |
| 3 | b | 18 | 2 |
| 3 | a | 20 | 3 |
| 3 | c | 23 | 3 |
| 4 | a | 20 | 4 |
| 4 | c | 23 | 4 |
| 4 | b | 18 | 4 |
| 5 | a | 20 | 5 |
| 5 | c | 23 | 5 |
| 5 | b | 18 | 5 |
| 6 | c | 23 | 5 |
| 6 | a | 20 | 6 |
| 6 | b | 18 | 6 |
| 7 | c | 23 | 5 |
| 7 | b | 18 | 7 |
| 7 | a | 20 | 7 |
+------+------+------+-------+
Update: chadhoc (great name, by the way!) gives a nice solution that unfortunately does not work in MySQL, since it does not support the FULL JOIN he uses. Luckily, I believe a simple UNION is an effective replacement:
-- Unify the original samples with the missing values that we've calculated
(
SELECT time, source, temp
FROM samples
)
UNION
( -- Pull all the time/source combinations that we are missing from the sample set, along with the temp
-- from the last sampled interval for the same time/source combination if we do not have one
SELECT a.time, a.source, (SELECT t2.temp FROM samples AS t2 WHERE t2.time < a.time AND t2.source = a.source ORDER BY t2.time DESC LIMIT 1) AS temp
FROM
( -- All values we want to get should be a cross of time/temp
SELECT t1.time, s1.source
FROM
(SELECT DISTINCT time FROM samples) AS t1
CROSS JOIN
(SELECT DISTINCT source FROM samples) AS s1
) AS a
LEFT JOIN samples s
ON a.time = s.time
AND a.source = s.source
WHERE s.source IS NULL
)
ORDER BY time, source;
Update 2: MySQL gives the following EXPLAIN output for chadhoc's code:
+----+--------------------+------------+------+---------------+------+---------+------+------+-----------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+------------+------+---------------+------+---------+------+------+-----------------------------+
| 1 | PRIMARY | temp | ALL | NULL | NULL | NULL | NULL | 18 | |
| 2 | UNION | <derived4> | ALL | NULL | NULL | NULL | NULL | 21 | |
| 2 | UNION | s | ALL | NULL | NULL | NULL | NULL | 18 | Using where |
| 4 | DERIVED | <derived6> | ALL | NULL | NULL | NULL | NULL | 3 | |
| 4 | DERIVED | <derived5> | ALL | NULL | NULL | NULL | NULL | 7 | |
| 6 | DERIVED | temp | ALL | NULL | NULL | NULL | NULL | 18 | Using temporary |
| 5 | DERIVED | temp | ALL | NULL | NULL | NULL | NULL | 18 | Using temporary |
| 3 | DEPENDENT SUBQUERY | t2 | ALL | NULL | NULL | NULL | NULL | 18 | Using where; Using filesort |
| NULL | UNION RESULT | <union1,2> | ALL | NULL | NULL | NULL | NULL | NULL | Using filesort |
+----+--------------------+------------+------+---------------+------+---------+------+------+-----------------------------+
I was able to get Charles' code working like so:
SELECT T.time, S.source,
COALESCE(
D.temp,
(
SELECT temp FROM samples
WHERE source = S.source AND time = (
SELECT MAX(time)
FROM samples
WHERE
source = S.source
AND time < T.time
)
)
) AS temp
FROM (SELECT DISTINCT time FROM samples) AS T
CROSS JOIN (SELECT DISTINCT source FROM samples) AS S
LEFT JOIN samples AS D
ON D.source = S.source AND D.time = T.time
Its explanation is:
+----+--------------------+------------+------+---------------+------+---------+------+------+-----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+------------+------+---------------+------+---------+------+------+-----------------+
| 1 | PRIMARY | <derived5> | ALL | NULL | NULL | NULL | NULL | 3 | |
| 1 | PRIMARY | <derived4> | ALL | NULL | NULL | NULL | NULL | 7 | |
| 1 | PRIMARY | D | ALL | NULL | NULL | NULL | NULL | 18 | |
| 5 | DERIVED | temp | ALL | NULL | NULL | NULL | NULL | 18 | Using temporary |
| 4 | DERIVED | temp | ALL | NULL | NULL | NULL | NULL | 18 | Using temporary |
| 2 | DEPENDENT SUBQUERY | temp | ALL | NULL | NULL | NULL | NULL | 18 | Using where |
| 3 | DEPENDENT SUBQUERY | temp | ALL | NULL | NULL | NULL | NULL | 18 | Using where |
+----+--------------------+------------+------+---------------+------+---------+------+------+-----------------+
I think you'll get better performance making use of the ranking/windowing functions in mySql, but unfortunately I do not know those as well as the TSQL implementation. Here is an ANSI compliant solution that will work though:
-- Full join across the sample set and anything missing from the sample set, pulling the missing temp first if we do not have one
select coalesce(c1.[time], c2.[time]) as dt, coalesce(c1.source, c2.source) as source, coalesce(c2.temp, c1.temp) as temp
from samples c1
full join ( -- Pull all the time/source combinations that we are missing from the sample set, along with the temp
-- from the last sampled interval for the same time/source combination if we do not have one
select a.time, a.source,
(select top 1 t2.temp from samples t2 where t2.time < a.time and t2.source = a.source order by t2.time desc) as temp
from
( -- All values we want to get should be a cross of time/samples
select t1.[time], s1.source
from
(select distinct [time] from samples) as t1
cross join
(select distinct source from samples) as s1
) a
left join samples s
on a.[time] = s.time
and a.source = s.source
where s.source is null
) c2
on c1.time = c2.time
and c1.source = c2.source
order by dt, source
I know this looks complicated, but it's formatted to explain itself...
It should work... Hope you only have three sources... If you have an arbitrary number of sources than this won't work... In that case see the second query...
EDIT: Removed first attempt
EDIT: If you don't know the sources ahead of time, you'll have to do something where you create an intermediate result set that "Fills in" the missing values..
something like this:
2nd EDIT: Removed need for Coalesce by moving logic to retrieve most recent temp reading for each source from Select clause into the Join condition.
Select T.Time, Max(Temp) MaxTemp,
Min(Temp) MinTemp, Avg(Temp) AvgTemp
From
(Select T.TIme, S.Source, D.Temp
From (Select Distinct Time From Samples) T
Cross Join
(Select Distinct Source From Samples) S
Left Join Samples D
On D.Source = S.Source
And D.Time =
(Select Max(Time)
From Samples
Where Source = S.Source
And Time <= T.Time)) Z
Group By T.Time