Using a subquery in the WHERE clause of gapfill in TimescaleDB - sql

Question

I would like to run the gapfill function of timescaleDB in a way, where the start and end dates are generated automatically. For example, I would like to run the gapfill function between the largest and the lowest entries in the database.
Given dataset playground:
CREATE TABLE public.playground (
value1 numeric,
"timestamp" bigint,
name "char"
);
INSERT INTO playground(name, value1, timestamp)
VALUES ('test', 100, 1599100000000000000);
INSERT INTO playground(name, value1, timestamp)
VALUES ('test', 100, 1599100001000000000);
INSERT INTO playground(name, value1, timestamp)
VALUES ('test', 100, 1599300000000000000);
I have tried getting the data as such:
SELECT time_bucket_gapfill(300E9::BIGINT, timestamp) as bucket
FROM playground
WHERE
timestamp >= (SELECT COALESCE(MIN(timestamp), 0) FROM playground)
AND
timestamp < (SELECT COALESCE(MAX(timestamp), 0) FROM playground)
GROUP BY bucket
I get an error:
ERROR: missing time_bucket_gapfill argument: could not infer start from WHERE clause
If I try the query with hard coded timestamps, the query runs just fine.
For example:
SELECT time_bucket_gapfill(300E9::BIGINT, timestamp) as bucket
FROM playground
WHERE timestamp >= 0 AND timestamp < 15900000000000000
GROUP BY bucket
Another approach of providing the start and end dates as arguments in the gapfill function fails as well.
WITH bounds AS (
SELECT COALESCE(MIN(timestamp), 0) as min, COALESCE(MAX(timestamp), 0) as max
FROM playground
WHERE timestamp >= 0 AND timestamp < 15900000000000000
),
gapfill as(
SELECT time_bucket_gapfill(300E9::BIGINT, timestamp, bounds.min, bounds.max) as bucket
FROM playground, bounds
GROUP BY bucket
)
select * from gapfill
ERROR: invalid time_bucket_gapfill argument: start must be a simple expression

Answer 1

time_bucket_gapfill only accepts start and finish values, which can be evaluated to constants at the query planning time. So it works to provide expression with constants and now, however it doesn't work to access a table in the expressions.
While this limitation on time_bucket_gapfill is in place it is not possible to achieve the desired behaviour in a single query. The work around is to calculate values for start and finish separately and then provide the values into the query with time_bucket_gapfill, which can be done in a stored procedure or in the application.
A side note, if PREPARE statement will be used in PostgreSQL 12, it is important to explicitly disable generic plan for the same reason.

Answer 2

for inferring start and stop from WHERE clause only direct column references are supported
see : https://github.com/timescale/timescaledb/issues/1345
so something like that might work , ( I have no timescaleDB access to test)
but try this :
SELECT
time_bucket_gapfill(300E9::BIGINT, time_range.min , time_range.max ) AS bucket
FROM
(
SELECT
COALESCE(MIN(timestamp), 0) AS min
, COALESCE(MAX(timestamp), 0) AS max
FROM
playground
) AS time_range
, playground
WHERE
timestamp >= time_range.min
AND timestamp < time_range.max
GROUP BY
bucket;