I have a table with data and I want to fill the date gaps with generate_series with PostgreSQL but I can't solve it. How can I join my table to the generate_series result? My attempt is the following:
SELECT series AS time,
data
FROM generate_series('2012-11-11', '2012-11-12', '2 minutes'::interval) AS series
LEFT JOIN data_table ON data_table.time = series
In this date range the result is:
TIME DATA
2012.11.11. 13:00:06 | data 1
2012.11.11. 13:08:06 | data 2
My aim would be similar like this:
TIME DATA
2012.11.11. 13:00:06 | data 1
2012.11.11. 13:02:06 | NULL
2012.11.11. 13:06:06 | NULL
2012.11.11. 13:08:06 | data 2
2012.11.11. 13:10:06 | NULL
2012.11.11. 13:12:06 | NULL
...
Ergo the the whole table fill with time rows with 2 minutes interval. How can I achieve this?
I think your query should work. I would give the value a column name:
SELECT g.series AS time, t.data
FROM generate_series('2012-11-11', '2012-11-12', '2 minutes'::interval) AS g(series) LEFT JOIN
data_table t
ON t.time = g.series;
Related
I have three tables that all have a “date” column and another column with counts of different variables - let’s call the tables T1, T2, and T3 and each of their columns are counts of dogs, cats, and birds spotted that day.
Not every table has the same set of dates. Example:
T1: Dogs spotted by day
date | dogs
------------------
2020-08-26 | 1
2020-08-27 | 4
T2: Cats spotted by day
date | cats
---------------------
2020-08-25 | 2
2020-08-26 | 5
T3: Cats spotted by day
date | birds
---------------------
2020-08-26 | 8
2020-08-27 | 3
2020-08-28 | 5
I’m trying to join them together on date while keeping all column data, but I’m having trouble doing so without getting a table that has 3 date columns. There’s no table that has all of the dates, so if I just select one of the date columns (e.g. select t1.date, t1.dogs, t2.cats, t3.birds) then I lose some of the date data. What I’m seeking is a table like this:
Desired Output: All Animals Spotted by Day
date | dogs | cats | birds |
----------------------------------------------------------
2020-08-25 | 0 (or null) | 2 | 0 (or null) |
2020-08-26 | 1 | 5 | 8 |
2020-08-27 | 4 | 0 (or null) | 3 |
2020-08-28 | 0 (or null) | 0 (or null) | 5 |
I’ve read about every stack overflow post on this I could find but maybe I’m not putting in the correct keywords because I’m not finding this. I’m working specifically in Postgres. Thank you!!
Use generate_series to construct a table of dates and use outer joins with the other tables:
SELECT d.d::date,
t1.dogs,
t2.cats,
t3.birds
FROM generate_series ('2020-08-25'::timestamp, '2020-08-28'::timestamp, '1 day'::interval) AS d(d)
LEFT JOIN t1 ON t1.date = d.d::date
LEFT JOIN t2 ON t1.dat3 = d.d::date
LEFT JOIN t3 ON t3.date = d.d::date;
Regardless of knowing the design why you need or you could change it further,
Using union and aggregation could be one option,
select date
, max(dogs) as dogs
, max(cats) as cats
, max(birds) as birds
from
(
select date,dogs,0 cats,0 birds from t1
union all
select date,0,cats,0 from t2
union all
select date,0,0,birds from t3
) t
group by date
order by date;
Note: Don't know if multiple entry possible for a single date , in case yes you need to use sum instead max
You can also use full join. For your example select * does what you want:
select *
from cats c full join
dogs d
using (date) full join
birds b
using (date);
I might recommend, however, that you put all the counts into a single table, with an additional column specifying "cat", "dog" and so on. If you had that, then simple aggregation would work:
select date,
count(*) filter (where type = 'cat'),
count(*) filter (where type = 'dog'),
count(*) filter (where type = 'bird')
from t
group by date;
I am trying to retrieve the number of days between a random date and the next known date for a holiday. Let's say my first table looks like this :
date | is_holiday | zone
9/11/18 | 0 | A
22/12/18 | 1 | A
and my holidays table looks like this
start_date | end_date | zone
20/12/18 | 04/01/18 | A
21/12/18 | 04/01/18 | B
...
I want to be able to know how many days are between an entry that is not a holiday in the first table and the next holiday date.
I have tried to get the next row with a later date in a join clause but the join isn't the tool for this task. I also have tried grouping by date and comparing the date with the next row but I can have multiple entries with the same date in the first table so it doesn't work.
This is the join clause I have tried :
SELECT mai.*, vac.start_date, datediff(vac.start_date, mai.date)
FROM (SELECT *
FROM MAIN
WHERE is_holiday = 0
) mai LEFT JOIN
(SELECT start_date, zone
FROM VACATIONS_UPDATED
ORDER BY start_date
) vac
ON mai.date < vac.start_date AND mai.zone = vac.zone
I expect to get a table looking like this :
date | is_holiday | zone | next_holiday
9/11/18 | 0 | A | 11
22/12/18 | 1 | A | 0
Any lead on how to achieve this ?
It might get messy to do it in SQL but if in case you are open to doing it from code, here is what it should look like. You basically need a crossJoin
Dataset<Row> table1 = <readData>
Dataset<Row> holidays = <readData>
//then cache the small table to get the best performance
table1.crossJoin( holidays ).filter("table1.zone == holidays.zone AND table1.date < holidays.start_date").select( "table1.*", "holidays.start_date").withColumn("nextHoliday", *calc diff*)
In scenarios where one row from table1 matches multiple holidays, then you can add an id column to table1 and then group the crossJoin.
// add unique id to the rows
table1 = table1.withColumn("id", functions.monotonically_increasing_id() )
Some details on crossJoins:
http://kirillpavlov.com/blog/2016/04/23/beyond-traditional-join-with-apache-spark/
I have a table that has 3 columns- start, end and emp_num. I want to generate a new table which has all dates between these dates for every employee. Need to use Presto.
I refered this link - inserting dates into a table between a start and end date in Presto
Tried using unnest function by creating sequence but , I don't know how do I create sequence by pulling dates from two columns in another table.
select unnest(seq) as t(days)
from (select sequence(start, end, interval '1' day) as seq
from table1)
Here's table and expected format
Table 1:
start | end | emp_num
2018/01/01 | 2018/01/05 | 1
2019/02/01 | 2019/02/05 | 2
Expected:
start | emp_num
2018/01/01 | 1
2018/01/02 | 1
2018/01/03 | 1
2018/01/04 | 1
2018/01/05 | 1
2019/02/01 | 2
2019/01/02 | 2
2019/02/03 | 2
2019/02/04 | 2
2019/02/05 | 2
Here is a query that might get the job done for your use case.
The logic is to use Presto sequence() function to generate a wide date range (since year 2000 to end of 2018, you can adapt that as needed), that can be joined with the table to generate the output.
select dt.x, emp_num
from
( select x from unnest(sequence(date '2000-01-01', date '2018-01-31')) t(x) ) dt
inner join table1 ta on dt.x >= ta.start and dt.x <= ta.end
However, as commented JNevill, it would be more efficient to create a calendar table rather than generating it on the fly every time the query runs.
It should be a simple as :
create table calendar as
select x from unnest(sequence(date '1970-01-01', date '2099-01-01')) t(x);
And then your query would become :
select dt.x, emp_num
from
calendar dt
inner join table1 ta on dt.x >= ta.start and dt.x <= ta.end
PS : due to the lack of DB Fiddles for Presto in the wild, I could not test the queries (#PiotrFindeisen - if you happen to read this - a Presto fiddle would be nice to have !).
I am trying to fill in dates with empty data, so that my query returned has every date and does not skip any.
My application needs to count bookings for activities by date in a report, and I cannot have skipped dates in what is returned by my SQL
I am trying to use a date table (I have a table with every date from 1/1/2000 to 12/31/2030) to accomplish this by doing a RIGHT OUTER JOIN on this date table, which works when dealing with one set of activities. But I have multiple sets of activities, each needing their own full range of dates regardless if there were bookings on that date.
I also have a function (DateRange) I found that allows for this:
SELECT IndividualDate FROM DateRange('d', '11/01/2017', '11/10/2018')
Let me give an example of what I am getting and what I want to get:
BAD: Without empty date rows:
date | activity_id | bookings
-----------------------------
1/2 | 1 | 5
1/4 | 1 | 4
1/3 | 2 | 6
1/4 | 2 | 2
GOOD: With empty date rows:
date | activity_id | bookings
-----------------------------
1/2 | 1 | 5
1/3 | 1 | NULL
1/4 | 1 | 4
1/2 | 2 | NULL
1/3 | 2 | 6
1/4 | 2 | 2
I hope this makes sense. I get the whole point of joining to a table of just a list of dates OR using the DateRange table function. But neither get me the "GOOD" result above.
Use a cross join to generate the rows and then left join to fill in the values:
select d.date, a.activity_id, t.bookings
from DateRange('d', ''2017-11-01',''2018-11-10') d cross join
(select distinct activity_id from t) a left join
t
on t.date = d.date and t.activity_id = a.activity_id;
It is a bit hard to follow what your data is and what comes from the function. But the idea is the same, wherever the data comes from.
I figured it out:
SELECT TOP 100 PERCENT masterlist.dt, masterlist.activity_id, count(r_activity_sales_bymonth.bookings) AS totalbookings
FROM (SELECT c.activity_id, dateadd(d, b.incr, '2016-12-31') AS dt
FROM (SELECT TOP 365 incr = row_number() OVER (ORDER BY object_id, column_id), *
FROM (SELECT a.object_id, a.column_id
FROM sys.all_columns a CROSS JOIN
sys.all_columns b) AS a) AS b CROSS JOIN
(SELECT DISTINCT activity_id
FROM r_activity_sales_bymonth) AS c) AS masterlist LEFT OUTER JOIN
r_activity_sales_bymonth ON masterlist.dt = r_activity_sales_bymonth.purchase_date AND masterlist.activity_id = r_activity_sales_bymonth.activity_id
GROUP BY masterlist.dt, masterlist.activity_id
ORDER BY masterlist.dt, masterlist.activity_id
I have a table with a 'timestamp' column and a 'value' column where the values are roughly 3 seconds apart.
I'm trying to return a table that has daily average values.
So, something like this is what i'm looking for.
| timestamp | average |
| 2010-06-02 | 456.6 |
| 2010-06-03 | 589.4 |
| 2010-06-04 | 268.5 |
etc...
Any help on this would be greatly appreciated.
SELECT DATE(timestamp), AVG(value)
FROM table
GROUP BY DATE(timestamp)
Since you want the day instead of each timestamp
select DATE(timestamp), AVG(value)
from TABLE
group by DATE(timestamp)
This assumes that your timestamp column only contains information about the day, but not the time. That way, the dates can be grouped together:
select timestamp, AVG(value) as average
from TABLE_NAME
group by timestamp