I've got a long table that tracks a numerical 'state' value (0=new, 1=setup mode, 2=retired, 3=active, 4=inactive) of a collection of 'devices' historically. These devices may be activated/deactivated throughout the year, so the table is continuous collection of state changes - mostly state 3 and 4, ordered by id, with a timestamp on the end, for example:
id | device_id | new_state | when
----------+-----------+-----------+----------------------------
218010581 | 2505 | 0 | 2022-06-06 16:28:11.174084
218010580 | 2505 | 1 | 2022-06-06 16:28:11.174084
218010634 | 2505 | 3 | 2022-06-06 16:29:25.129019
218087737 | 659 | 3 | 2022-06-07 22:55:48.705208
218087744 | 1392 | 3 | 2022-06-07 22:55:59.016974
218087757 | 1556 | 3 | 2022-06-07 22:56:09.811876
218087758 | 2071 | 1 | 2022-06-07 22:56:20.850095
218087765 | 2071 | 3 | 2022-06-07 22:56:29.122074
When I want to look for a list of devices and see their 'history', I know I can use something like:
select *
from devstatechange
where device_id = 2345
order by "when";
id | device_id | new_state | when
-----------+-----------+-----------+----------------------------
184682659 | 2345 | 0 | 2021-05-27 17:03:36.894429
184682658 | 2345 | 1 | 2021-05-27 17:03:36.894429
184684721 | 2345 | 3 | 2021-05-27 17:31:01.968314
194933399 | 2345 | 4 | 2021-08-31 23:30:05.555407
195213746 | 2345 | 3 | 2021-09-03 16:53:39.043005
206278232 | 2345 | 4 | 2021-12-31 22:30:08.820068
206515355 | 2345 | 3 | 2022-01-03 16:06:01.223759
215709888 | 2345 | 4 | 2022-04-30 23:30:30.309389
215846807 | 2345 | 3 | 2022-05-02 19:40:31.525514
select *
from devstatechange
where device_id = 2351
order by "when";
id | device_id | new_state | when
-----------+-----------+-----------+----------------------------
186091252 | 2351 | 0 | 2021-06-09 15:36:02.775035
186091253 | 2351 | 1 | 2021-06-09 15:36:02.775035
186091349 | 2351 | 3 | 2021-06-09 15:37:56.965599
197880878 | 2351 | 4 | 2021-09-30 23:30:06.691835
197945073 | 2351 | 3 | 2021-10-01 15:32:35.907913
208981857 | 2351 | 4 | 2022-01-31 22:30:09.521694
209722639 | 2351 | 3 | 2022-02-09 15:20:12.412816
217666572 | 2351 | 4 | 2022-05-31 23:30:30.881928
What I am really looking for is a query that returns a unique list of devices where the latest dated entry for each device only contains a state of '4' ('inactive state'), and not include records that do not match.
So in using the above data samples, even though both devices 2345 and 2351 have states of 3 and 4 throughout their history, only device 2351 has it's last dated entry with a state of 4 - meaning it is currently in an 'inactive' state. Device 2345's would not appear in the result set since its last dated entry has a state of 3 - it's still active.
Stabbing in the dark, I've tried variants of:
SELECT DISTINCT *
FROM devstatechange
WHERE MAX("when") AND new_state = 4
ORDER BY "when";
SELECT DISTINCT device_id, new_state, MAX("when")
FROM devstatechange
WHERE new_state = 4
ORDER BY "when";
with obviously no success.
I'm thinking I might need to 'group' the entries together, but I don't know how to specify 'return last entry only if new_state = 4' in SQL, or rather PostgreSQL.
Any tidbits or pokes in the right direction would be appreciated.
SELECT * FROM (
SELECT DISTINCT ON (device_id)
*
FROM devstatechange
ORDER BY device_id, "when" DESC
) AS latest
WHERE new_state = 4;
The DISTINCT ON keyword together with the ORDER BY will pull the newest row for each device. The outer query then filters these by your condition.
You may use Row_Number() function with a partition by device_id and order by when.
Try the following CTE:
with cte as
(
Select id ,device_id ,new_state ,when_ ,
row_number() over (partition by device_id order by when_ desc) as rn
from devstatechange
)
select * from cte where rn=1 and new_state=4
See a demo from db-fiddle.
The problem with:
SELECT DISTINCT * FROM devstatechange WHERE MAX("when") AND new_state=4 ORDER BY "when";
is that MAX("when") refers to all the entrys on the table.
you should change it to:
when = (select max(when) from devstatechange dev2 where dev2.device_id = dev1.device_id )
You can use CTE to obtain a last state of each device and then select only those, whose last state is 4, like this
WITH device_last_state AS (
SELECT DISTINCT ON (device_id)
id,
device_id,
last_value (new_state) over (partition by device_id order by "when" desc) as new_state,
"when"
FROM devicestatechange
)
SELECT * FROM device_last_state
WHERE new_state = 4
Check a demo
Related
I have a table that has the data about user_ids, all their last log_in dates to the app
Table:
|----------|--------------|
| User_Id | log_in_dates |
|----------|--------------|
| 1 | 2021-09-01 |
| 1 | 2021-09-03 |
| 2 | 2021-09-02 |
| 2 | 2021-09-04 |
| 3 | 2021-09-01 |
| 3 | 2021-09-02 |
| 3 | 2021-09-03 |
| 3 | 2021-09-04 |
| 4 | 2021-09-03 |
| 4 | 2021-09-04 |
| 5 | 2021-09-01 |
| 6 | 2021-09-01 |
| 6 | 2021-09-09 |
|----------|--------------|
From the above table, I'm trying to understand the user's log in behavior from the present day to the past 90 days.
Num_users_no_log_in defines the count for the number of users who haven't logged in to the app from present_day to the previous days (last_log_in_date)
I want the table like below:
|---------------|------------------|--------------------|-------------------------|
| present_date | days_difference | last_log_in_date | Num_users_no_log_in |
|---------------|------------------|--------------------|-------------------------|
| 2021-09-01 | 0 | 2021-09-01 | 0 |
| 2021-09-02 | 1 | 2021-09-01 | 3 |->(Id = 1,5,6)
| 2021-09-02 | 0 | 2021-09-02 | 3 |->(Id = 1,5,6)
| 2021-09-03 | 2 | 2021-09-01 | 2 |->(Id = 5,6)
| 2021-09-03 | 1 | 2021-09-02 | 1 |->(Id = 2)
| 2021-09-03 | 0 | 2021-09-03 | 3 |->(Id = 2,5,6)
| 2021-09-04 | 3 | 2021-09-01 | 2 |->(Id = 5,6)
| 2021-09-04 | 2 | 2021-09-02 | 0 |
| 2021-09-04 | 1 | 2021-09-03 | 1 |->(Id= 1)
| 2021-09-04 | 0 | 2021-09-04 | 3 |->(Id = 1,5,6)
| .... | .... | .... | ....
|---------------|------------------|--------------------|-------------------------|
I was able to get the first three columns Present_date | days_difference | last_log_in_date using the following query:
with dts as
(
select distinct log_in from users_table
)
select x.log_in_dates as present_date,
DATEDIFF(DAY, y.log_in_dates ,x.log_in_dates ) as Days_since_last_log_in,
y.log_in_dates as log_in_dates
from dts x, dts y
where x.log_in_dates >= y.log_in_dates
I don't understand how I can get the fourth column Num_users_no_log_in
I do not really understand your need: are there values base on users or dates? It it's based on dates, as it looks like (elsewhere you would probably have user_id as first column), what does it mean to have multiple times the same date? I understand that you would like to have a recap for all dates since the beginning until the current date, but in my opinion in does not really make sens (imagine your dashboard in 1 year!!)
Once this is said, let's go to the approach.
In such cases, I develop step by step using common table extensions. For you example, it required 3 steps:
prepare the time series
integrate connections' dates and perform the first calculation (time difference)
Finally, calculate nb connection per day
Then, the final query will display the desired result.
Here is the query I proposed, developed with Postgresql (you did not precise your dbms, but converting should not be such a big deal here):
with init_calendar as (
-- Prepare date series and count total users
select generate_series(min(log_in_dates), now(), interval '1 day') as present_date,
count(distinct user_id) as nb_users
from users
),
calendar as (
-- Add connections' dates for each period from the beginning to current date in calendar
-- and calculate nb days difference for each of them
-- Syntax my vary depending dbms used
select distinct present_date, log_in_dates as last_date,
extract(day from present_date - log_in_dates) as days_difference,
nb_users
from init_calendar
join users on log_in_dates <= present_date
),
usr_con as (
-- Identify last user connection's dates according to running date
-- Tag the line to be counted as no connection
select c.present_date, c.last_date, c.days_difference, c.nb_users,
u.user_id, max(log_in_dates) as last_con,
case when max(log_in_dates) = present_date then 0 else 1 end as to_count
from calendar c
join users u on u.log_in_dates <= c.last_date
group by c.present_date, c.last_date, c.days_difference, c.nb_users, u.user_id
)
select present_date, last_date, days_difference,
nb_users - sum(to_count) as Num_users_no_log_in
from usr_con
group by present_date, last_date, days_difference, nb_users
order by present_date, last_date
Please note that there is a difference with your own expected result as you forgot user_id = 3 in your calculation.
If you want to play with the query, you can with dbfiddle
From the below table i want to write a select statement where i can select the price of the items for the
latest date.
Item | Price | Date
------|----------|--------
1001 | 10 | 26-5-2019
1001 | 11 | 15-02-2020
1001 | 9 | 28-08-2020
1002 | 5 | 1/7/2019
1002 | 3 | 8/11/2019
1002 | 4 | 5/5/2020
1003 | 6 | 26-05-2019
1003 | 7 | 1/2/2020
1003 | 5 | 15-09-2020
Result should be as below:
Item | Price | Date
------|----------|--------
1001 | 9 | 28-08-2020
1002 | 4 | 5/5/2020
1003 | 5 | 15-09-2020
Despite the fact that the table is unreadable and you haven't posted anything about what you have tried so far, I will try to help you...
You can get the price via Window Functions - in this case row_number. You should try something like the following:
SELECT x.*
FROM (SELECT Item, Price, [Date], ROW_NUMBER() OVER (PARTITION BY Item ORDER BY [Date] DESC) AS rn) x
WHERE x.rn = 1
Let's say I have a table with data...
| person_id | priority |
|------------|------------|
| 678 | 2 |
| 413 | 4 |
| 912 | 1 |
| 111 | 5 |
How can I update priority so that the values are contiguous? I.e....
| person_id | priority |
|------------|------------|
| 678 | 2 |
| 413 | 3 | -- updated from 4 to 3
| 912 | 1 |
| 111 | 4 | -- updated from 5 to 4
I know that I can use something like...
select
row_number() over (order by [priority]) as position
from
table_name
...to find a person's 'position', but how can I use this to update the same row?
The priority values should always start at 1.
You can use an updatable CTE or subquery:
with toupdate as (
select t.*, row_number() over (order by [priority]) as new_priority
from table_name
)
update toudpate
set priority = new_priority
where priority <> new_priority;
I have a table with multiple records for each patient.
My end goal is a table that is 1-to-1 between Patient_id and Value.
I would like to de-duplicate (in respect to patient_id) my rows based on "a hierarchical series of aggregate functions" (if someone has a better way to phrase this, I'd appreciate that as well.)
+----+------------+------------+------------+----------+-----------------+-------+
| ID | patient_id | Date | Date2 | Priority | Source | Value |
+----+------------+------------+------------+----------+-----------------+-------+
| 1 | 1 | 2017-09-09 | 2018-09-09 | 1 | 'verified' | 55 |
| 2 | 1 | 2017-09-09 | 2018-11-11 | 2 | 'verified' | 78 |
| 3 | 1 | 2017-11-11 | 2018-09-09 | 3 | 'verified' | 23 |
| 4 | 1 | 2017-11-11 | 2018-11-11 | 1 | 'self_reported' | 11 |
| 5 | 1 | 2017-09-09 | 2018-09-09 | 2 | 'self_reported' | 90 |
| 5 | 1 | 2017-09-09 | 2018-09-09 | 3 | 'self_reported' | 34 |
| 6 | 2 | 2017-11-11 | 2018-09-09 | 2 | 'self_reported' | 21 |
+----+------------+------------+------------+----------+-----------------+-------+
For each patient_id, I would like to get the row(s) that has/have the MAX(Date). In the case that there are still duplicated patient_id, I would like to get the row(s) with the MIN(Priority). In the case that there are still duplicated rows I would like to get the row(s) with the MIN(Date2).
The way I've approached this problem is using a series of queries like this to de-duplicate on the columns one at a time.
SELECT *
FROM #table t1
LEFT JOIN
(SELECT
patient_id,
MIN(priority) AS min_priority
FROM #table
GROUP BY patient_id) t2 ON t2.patient_id = t1.patient_id
WHERE t2.min_priority = t1.priority
Is there a way to do this that allows me to de-dup on multiple columns at once? Is there a more elegant way to do this?
I'm able to get my results, but my solution feels very inefficient, and I keep running into this. Thank you for any input.
You could use row_number(), if your RDBMS supports it:
select ID, patient_id, Date, Date2, Priority, Source, Value
from (
select
t.*,
row_number() over(partition by patient_id order by Date desc, Priority, Date2) rn
from mytable t
) where rn = 1
Another option is to filter with a correlated subquery that sorts the record according to your criteria, like so:
select t.*
from mytable t
where id = (
select id
from mytable t1
where t1.patient_id = t.patient_id
order by t1.Date desc, t1.Priority, t1.Date2
limit 1
)
The actual syntax for limit varies accross RDBMS.
I have a table that accommodates data that is logically groupable by multiple properties (foreign key for example). Data is sequential over continuous time interval; i.e. it is a time series data. What I am trying to achieve is to select only latest values for each group of groups.
Here is example data:
+-----------------------------------------+
| code | value | date | relation_id |
+-----------------------------------------+
| A | 1 | 01.01.2016 | 1 |
| A | 2 | 02.01.2016 | 1 |
| A | 3 | 03.01.2016 | 1 |
| A | 4 | 01.01.2016 | 2 |
| A | 5 | 02.01.2016 | 2 |
| A | 6 | 03.01.2016 | 2 |
| B | 1 | 01.01.2016 | 1 |
| B | 2 | 02.01.2016 | 1 |
| B | 3 | 03.01.2016 | 1 |
| B | 4 | 01.01.2016 | 2 |
| B | 5 | 02.01.2016 | 2 |
| B | 6 | 03.01.2016 | 2 |
+-----------------------------------------+
And here is example of desired output:
+-----------------------------------------+
| code | value | date | relation_id |
+-----------------------------------------+
| A | 3 | 03.01.2016 | 1 |
| A | 6 | 03.01.2016 | 2 |
| B | 3 | 03.01.2016 | 1 |
| B | 6 | 03.01.2016 | 2 |
+-----------------------------------------+
To put this in perspective — for every related object I want to select each code with latest date.
Here is a select I came with. I've used ROW_NUMBER OVER (PARTITION BY...) approach:
SELECT indicators.code, indicators.dimension, indicators.unit, x.value, x.date, x.ticker, x.name
FROM (
SELECT
ROW_NUMBER() OVER (PARTITION BY indicator_id ORDER BY date DESC) AS r,
t.indicator_id, t.value, t.date, t.company_id, companies.sic_id,
companies.ticker, companies.name
FROM fundamentals t
INNER JOIN companies on companies.id = t.company_id
WHERE companies.sic_id = 89
) x
INNER JOIN indicators on indicators.id = x.indicator_id
WHERE x.r <= (SELECT count(*) FROM companies where sic_id = 89)
It works but the problem is that it is painfully slow; when working with about 5% of production data which equals to roughly 3 million fundamentals records this select take about 10 seconds to finish. My guess is that happens due to subselect selecting huge amounts of records first.
Is there any way to speed this query up or am I digging in wrong direction trying to do it the way I do?
Postgres offers the convenient distinct on for this purpose:
select distinct on (relation_id, code) t.*
from t
order by relation_id, code, date desc;
So your query uses different column names than your sample data, so it's hard to tell, but it looks like you just want to group by everything except for date? Assuming you don't have multiple most recent dates, something like this should work. Basically don't use the window function, use a proper group by, and your engine should optimize the query better.
SELECT mytable.code,
mytable.value,
mytable.date,
mytable.relation_id
FROM mytable
JOIN (
SELECT code,
max(date) as date,
relation_id
FROM mytable
GROUP BY code, relation_id
) Q1
ON Q1.code = mytable.code
AND Q1.date = mytable.date
AND Q1.relation_id = mytable.relation_id
Other option:
SELECT DISTINCT Code,
Relation_ID,
FIRST_VALUE(Value) OVER (PARTITION BY Code, Relation_ID ORDER BY Date DESC) Value,
FIRST_VALUE(Date) OVER (PARTITION BY Code, Relation_ID ORDER BY Date DESC) Date
FROM mytable
This will return top value for what ever you partition by, and for whatever you order by.
I believe we can try something like this
SELECT CODE,Relation_ID,Date,MAX(value)value FROM mytable
GROUP BY CODE,Relation_ID,Date