Consume results from function lazily in PostgreSQL? - sql

Question

I have a function in my database that returns a lot of rows:
CREATE FUNCTION lots_of_rows(n integer) RETURNS SETOF integer
STABLE LANGUAGE plpgsql AS $$ BEGIN
FOR i IN 1..10000000 LOOP
RETURN NEXT i * n;
END LOOP;
END $$;
Unsurprisingly, queries that use this function are not very fast:
=# EXPLAIN ANALYZE SELECT n FROM lots_of_rows(4) as n;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------
Function Scan on lots_of_rows n (cost=0.25..10.25 rows=1000 width=4) (actual time=1867.135..2900.167 rows=10000000 loops=1)
Planning Time: 0.026 ms
Execution Time: 3494.365 ms
(3 rows)
That is to be expected. But what frustrates me is that I pay for the whole cost of this function even if I only use a tiny subset of the resulting rows:
=# EXPLAIN ANALYZE SELECT n FROM lots_of_rows(4) as n LIMIT 10;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------
Limit (cost=0.25..0.35 rows=10 width=4) (actual time=1863.679..1863.682 rows=10 loops=1)
-> Function Scan on lots_of_rows n (cost=0.25..10.25 rows=1000 width=4) (actual time=1863.675..1863.676 rows=10 loops=1)
Planning Time: 0.044 ms
Execution Time: 1872.395 ms
(4 rows)
Clearly, that is very wasteful. For comparison, if I do the same thing with a recursive view, it takes essentially zero time:
CREATE RECURSIVE VIEW lots_of_rows (n) AS
VALUES (1)
UNION ALL
SELECT n+1 FROM lots_of_rows WHERE n < 10000000;
=# EXPLAIN ANALYZE SELECT n * 4 FROM lots_of_rows LIMIT 10;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=2.95..3.28 rows=10 width=4) (actual time=0.005..0.027 rows=10 loops=1)
-> Subquery Scan on lots_of_rows (cost=2.95..3.96 rows=31 width=4) (actual time=0.005..0.023 rows=10 loops=1)
-> CTE Scan on lots_of_rows lots_of_rows_1 (cost=2.95..3.57 rows=31 width=4) (actual time=0.003..0.020 rows=10 loops=1)
CTE lots_of_rows
-> Recursive Union (cost=0.00..2.95 rows=31 width=4) (actual time=0.002..0.015 rows=10 loops=1)
-> Result (cost=0.00..0.01 rows=1 width=4) (actual time=0.001..0.001 rows=1 loops=1)
-> WorkTable Scan on lots_of_rows lots_of_rows_2 (cost=0.00..0.23 rows=3 width=4) (actual time=0.001..0.001 rows=1 loops=9)
Filter: (n < 10000000)
Planning Time: 0.213 ms
Execution Time: 0.089 ms
(10 rows)
But of course, my function takes an argument, n, but views cannot accept arguments, so some of the implementation details have to leak out into my individual queries.
Of course, this lots_of_rows function is very silly, and I do not actually literally use it anywhere. My real function is more complex: it accepts several different arguments and uses them to construct a SELECT query, iterates over the results using FOR, and for certain rows, returns records using RETURN NEXT. It is not nearly as simple to replace that particular function with a view.
Furthermore, it is not straightforward to move the limiting logic from my enclosing query into the function, since the enclosing queries sometimes add various WHERE conditions to the result:
SELECT r.id FROM complicated_function($1, $2, $3, $4) as r
WHERE r.is_public AND r.score > 0 LIMIT 20;
I guess I could always just add a ton of different arguments to the function for all the different conditions I need, but ideally, I’d like to be able to keep my function as it is (since it encapsulates precisely the abstraction I want), just somehow stream the results to the caller on-demand so that it acts a little bit more like a view (albeit still more or less opaque to the query planner). Is that at all possible, or must a function’s result be completely materialized in memory before it returns?

Answer 1

I believe you may be able to achieve what you're looking for by having the function return a cursor.
A cursor should allow the function caller to fetch rows in batches rather than all at once, allowing both for results to the caller more quickly and less in memory at once on both the client and the server.
Note: There is overhead on the server in terms of maintaining the cursor. The caller should close the cursor explicitly once done (otherwise it will close at the end of the transaction).
In particular, check out the section in the above link entitled 43.7.3.5. Returning Cursors.

Answer 2

Postgres has two possible implementation of table functions:
row based with persistent context - it returns only rows that are required - it is little bit more CPU expensive, but it stop early - with this implementation the function is called multiple times and returns only one row every time. For this implementation only C language can be used.
tuple store implementation - this is your case - PLpgSQL and other than C language use it. When the function is called, then special structure tuplestore is filled. All rows are generated - and all rows are returned. The reader of tuplestore (parent node), can read all rows (or not), but every time all rows are produced. Outer LIMIT clause is not push down inside function, so it has not any effect on speed.
There are not any other implementation - so if you need limit result, then you have to do explicitly (manually) (if you want higher programming language).