Hive: window function - how to exclude the CURRENT ROW - hive

I wish to calculate the minimum of a value over a partition, but the current row should not be taken into account.
SELECT *,
MIN(val) OVER(PARTITION BY col1)
FROM table
outputs the minimum over all rows in the partition.
The documentation shows ways to use CURRENT ROW, but not how to exclude it while performing the windowing operation.
I am looking for something like this:
SELECT *,
MIN(val) OVER(PARTITION BY col1 ROWS NOT CURRENT ROW)
FROM table
but this does not work.

I can think of a way to do this. The min over a window excluding the current row will always be the min over the window except when the row you are at is the min; then then min will be the 2nd min over the window. Example:
Data:
-----------
key | val
-----------
1 8
1 2
1 4
1 6
1 11
2 3
2 5
2 7
2 9
Query:
select key, val, act_min, val_arr
, case when act_min=val then val_arr[1] else act_min
end as min_except_for_c_row
from (
select key, val, act_min, sort_array(val_arr) val_arr
from (
select key, val
, min(val) over (partition by key) act_min
, collect_set(val) over (partition by key) val_arr
from db.table ) A
) B
I left all the columns in for illustration. You can modify the query as needed.
Output:
key val act_min val_arr min_except_for_c_row
1 8 2 [2,4,6,8,11] 2
1 2 2 [2,4,6,8,11] 4
1 4 2 [2,4,6,8,11] 2
1 6 2 [2,4,6,8,11] 2
1 11 2 [2,4,6,8,11] 2
2 3 3 [3,5,7,9] 5
2 5 3 [3,5,7,9] 3
2 7 3 [3,5,7,9] 3
2 9 3 [3,5,7,9] 3

Related

calculate avg(value) for last 10 records postgresql

i have a tricky task,
lets assume we have table "Racings", and there we have columns TRACK, CAR, CIRCLE_TIME
here is an example how data could be look like:
id
track
car
circle_time
10
1
10
15
9
1
10
14
8
1
10
16
7
1
10
15
6
1
10
13
5
2
10
7
4
2
10
4
3
2
10
5
2
3
10
8
1
3
10
10
what i need, i to add one more coumn like avg3_circle_time which will show me an average time from last 3 circle_time from each track, example:
id
track
car
circle_time
avg3_circle_time
10
1
10
15
15
9
1
10
14
15
8
1
10
16
14.6
7
1
10
15
null
6
1
10
13
null
5
2
10
7
5.3
4
2
10
4
null
3
2
10
5
null
2
3
10
8
null
1
3
10
10
null
I know how it could works in oracle, you could use something like rowid, but in case of postgresql i don't know, i have a draft like .....avg(circle_time) OVER(PARTITION BY track,car.....) as avg3_circle_time..... help me to solve that task please
You can use window functions to calculate moving averages:
SELECT track, id, car, circle_time, AVG(circle_time) OVER (
PARTITION BY track
ORDER BY id
ROWS BETWEEN 2 PRECEDING AND CURRENT ROW
)
FROM t
ORDER BY track, id
Depending on your definition of previous three, the window could be ROWS BETWEEN 3 PRECEDING AND 1 PRECEDING.
If you want only values when at least 3 circles available
select *
, case when lag(id, 2) over(partition by TRACK, CAR order by id) is not null then
avg(CIRCLE_TIME) over(partition by TRACK, CAR order by id rows between 2 preceding and current row) end a
from Racing
order by id desc;
db<>fiddle
Output
id track car circle_time a
10 1 10 15 15.0000000000000000
9 1 10 14 15.0000000000000000
8 1 10 16 14.6666666666666667
7 1 10 15 null
6 1 10 13 null
5 2 10 7 5.3333333333333333
4 2 10 4 null
3 2 10 5 null
2 3 10 8 null
1 3 10 10 null
Use LAED() then checking one of the next 2 rows is NULL or not. THEN sum of three values for calculating average.
-- PostgreSQL
SELECT *
, CASE WHEN next_circle_time IS NULL OR next_next_circle_time IS NULL
THEN NULL
ELSE ((t.circle_time + COALESCE(next_circle_time, 0) + COALESCE(next_next_circle_time, 0)) / 3 :: DECIMAL) :: DECIMAL(10, 1)
END avg_circle_time
FROM (SELECT *
, LEAD(circle_time, 1) OVER (PARTITION BY track ORDER BY id DESC) next_circle_time
, LEAD(circle_time, 2) OVER (PARTITION BY track ORDER BY id DESC) next_next_circle_time
FROM Racings) t
Another way Use AVG()
SELECT *
, CASE WHEN LEAD(circle_time, 2) OVER (PARTITION BY track ORDER BY id DESC) IS NULL
OR LEAD(circle_time, 1) OVER (PARTITION BY track ORDER BY id DESC) IS NULL
THEN NULL
ELSE AVG(circle_time) OVER (PARTITION BY track ORDER BY id DESC ROWS BETWEEN CURRENT ROW AND 2 FOLLOWING)
END :: DECIMAL(10, 2) avg_circle_time
FROM Racings
Please check from url where both query exists https://dbfiddle.uk/?rdbms=postgres_11&fiddle=f0cd868623725a1b92bf988cfb2deba3
Several of the posted answers end up repeating the window definition. You can avoid this with the window clause:
select *,
case when row_number() over(track_window) > 2
then trunc(avg(CIRCLE_TIME) over(track_window rows 2 preceding), 1)
end a
from Racing
window track_window as (partition by track order by id)
order by id desc
Note how, in this sample, track_window is defined once, then reused for both row_number and avg. In the latter case, the window clause is embellished with a frame as well (rows 2 preceding).

SQL: how to use row_number() function to assign the same number for rows with duplicate ids in a repeating format

I have a table with two columns personid and taskid and want to use the ROW_NUMBER function to add a row that counts up to 3 but will duplicate the number as it counts if there are multiple rows for a personid.
The code below is only ordering by personid and repeating after the number 3, but I need it to order by personid and only go to the next number after all the taskid's for the personid are assigned to one number, or essentially any duplicate personid's I want to make sure they all only get one number assigned to it.
Select
personid,
taskid,
1 + ( (row_number() over (order by personid) - 1) % 3) as numberCount
from taskTable
Current Table Being Queried From:
PersonId Taskid
1 1
1 2
1 6
2 3
3 8
3 10
4 9
4 4
4 5
5 7
5 11
5 12
Expected Results After Query:
PersonId Taskid numberCount
1 1 1
1 2 1
1 6 1
2 3 2
3 8 3
3 10 3
4 9 1
4 4 1
4 5 1
5 7 2
5 11 2
5 12 2
Try this below script using DENSE_RANK -
SELECT *,
(DENSE_RANK() OVER(ORDER BY PersonId)-1)%3 + 1 AS numberCount
FROM your_table
I think you want dense_rank() and modulo arithmetic:
select t.*,
(dense_rank() over (order by personId) - 1) % 3) + 1 as numberCount
from t;
Note: The syntax for modulo arithmetic may vary in your database. Typically it is one of mod(), the % operator, or using mod as an operator.

SQL Server GROUP BY COUNT Consecutive Rows Only

I have a table called DATA on Microsoft SQL Server 2008 R2 with three non-nullable integer fields: ID, Sequence, and Value. Sequence values with the same ID will be consecutive, but can start with any value. I need a query that will return a count of consecutive rows with the same ID and Value.
For example, let's say I have the following data:
ID Sequence Value
-- -------- -----
1 1 1
5 1 100
5 2 200
5 3 200
5 4 100
10 10 10
I want the following result:
ID Start Value Count
-- ----- ----- -----
1 1 1 1
5 1 100 1
5 2 200 2
5 4 100 1
10 10 10 1
I tried
SELECT ID, MIN([Sequence]) AS Start, Value, COUNT(*) AS [Count]
FROM DATA
GROUP BY ID, Value
ORDER BY ID, Start
but that gives
ID Start Value Count
-- ----- ----- -----
1 1 1 1
5 1 100 2
5 2 200 2
10 10 10 1
which groups all rows with the same values, not just consecutive rows.
Any ideas? From what I've seen, I believe I have to left join the table with itself on consecutive rows using ROW_NUMBER(), but I am not sure exactly how to get counts from that.
Thanks in advance.
You can use Sequence - ROW_NUMBER() OVER (ORDER BY ID, Val, Sequence) AS g to create a group:
SELECT
ID,
MIN(Sequence) AS Sequence,
Val,
COUNT(*) AS cnt
FROM
(
SELECT
ID,
Sequence,
Sequence - ROW_NUMBER() OVER (ORDER BY ID, Val, Sequence) AS g,
Val
FROM
yourtable
) AS s
GROUP BY
ID, Val, g
Please see a fiddle here.

Select Query to Get Unique Cells in Two Columns

I have an SQL Server database, that logs weather device sensor data.
The table looks like this:
Id DeviceId SensorId Value
1 1 1 42
2 1 1 3
3 1 2 30
4 2 2 0
5 2 1 1
6 3 1 26
7 3 1 23
8 3 2 1
In return the query should return the following:
Id DeviceId SensorId Value
2 1 1 3
3 1 2 30
4 2 2 0
5 2 1 1
7 3 1 23
8 3 2 1
For each device the sensor should be unique. i.e. Values in Columns DeviceId and SensorId should be unique (row-wise).
Apologies if I'm not clear enough.
If you don't want to sum Value as your desired result suggest, so you just want to take an "arbitrary" row of each "DeviceId + SensorId"-group:
WITH CTE AS
(
SELECT Id, DeviceId, SensorId, Value,
RN = ROW_NUMBER() OVER (PARTITION BY DeviceId, SensorId ORDER BY ID DESC)
FROM dbo.TableName
)
SELECT Id, DeviceId, SensorId, Value
FROM CTE
WHERE RN = 1
ORDER BY ID
This returns the row with the highest ID per group. You need to change ORDER BY ID DESC if you want a different result. Demo: http://sqlfiddle.com/#!6/8e31b/2/0 (your result)

Increment Row Number on Group

I am working on a query for SQL Server 2005 that needs to return data with two 'index' fields. The first index 't_index' should increment every time the 'shade' column changes, whilst the second index increments within the partition of the values in the 'shade' column:
t_index s_index shade
1 1 A
1 2 A
1 3 A
1 4 A
1 5 A
2 1 B
2 2 B
2 3 B
2 4 B
2 5 B
To get the s_index column I am using the following:
Select ROW_NUMBER() OVER(PARTITION BY [shade] ORDER BY [shade]) as s_index
My question is how to get the first index to only increment when the value in the 'shade' column changes?
That can be accomplished with the DENSE_RANK() function:
DENSE_RANK() OVER(Order By [shade]) as t_index
You can try to use DENSE_RANK() for that:
SELECT
shade,
s_index = ROW_NUMBER() OVER(PARTITION BY [shade] ORDER BY [shade]),
t_index = DENSE_RANK() OVER (ORDER BY [shade])
FROM dbo.YourTableNameHEre
Gives output:
shade s_index t_index
A 1 1
A 2 1
A 3 1
A 4 1
A 5 1
B 1 2
B 2 2
B 3 2
B 4 2
B 5 2