PostgreSQL - Shared temp table between functions - sql

I wnat to know if its possible to share a temporary table between functions that are called in a "main function", like this:
-- some sub function
create or replace function up_sub_function (str text)
returns table (id int, descr text) as $$
begin
return query select * from temp_table where descr like concat('%', str , '%');
end; $$
language plpgsql;
-- main function
create or replace function up_main_function ()
returns table (id int, descr text) as $$
begin
create temporary table temp_table if not exists (
id int,
descr text
);
insert into temp_campaigns select id, descr from test_table;
return query select * from up_sub_function('a');
end; $$
language plpgsql;
BEGIN;
select * from up_main_function();
drop table temp_table;
COMMIT;
If you can show me the correct way to achieve this, I want to be able to populate a temporary table and then filter rows by calling othe functions inside the main function.
Thanks ans happy programming! :)

See the documentation https://www.postgresql.org/docs/current/static/sql-createtable.html
temp tables are valid for the entire session. That is as long as you stay connected to the database.
In your case you only need it during the transaction. So you should create it with ON COMMIT DROP
create temporary table temp_table if not exists (
id int,
descr text
) ON COMMIT DROP;
Once you created the table you can use it within any function in the current transaction.
You do not need the BEGIN to start the transaction. A transaction is automatically started when the outer function is called.
Nested function calls share the same transaction. So they all see the table.

Related

Dynamic query that uses CTE gets "syntax error at end of input"

I have a table that looks like this:
CREATE TABLE label (
hid UUID PRIMARY KEY DEFAULT UUID_GENERATE_V4(),
name TEXT NOT NULL UNIQUE
);
I want to create a function that takes a list of names and inserts multiple rows into the table, ignoring duplicate names, and returns an array of the IDs generated for the rows it inserted.
This works:
CREATE OR REPLACE FUNCTION insert_label(nms TEXT[])
RETURNS UUID[]
AS $$
DECLARE
ids UUID[];
BEGIN
CREATE TEMP TABLE tmp_names(name TEXT);
INSERT INTO tmp_names SELECT UNNEST(nms);
WITH new_names AS (
INSERT INTO label(name)
SELECT tn.name
FROM tmp_names tn
WHERE NOT EXISTS(SELECT 1 FROM label h WHERE h.name = tn.name)
RETURNING hid
)
SELECT ARRAY_AGG(hid) INTO ids
FROM new_names;
DROP TABLE tmp_names;
RETURN ids;
END;
$$ LANGUAGE PLPGSQL;
I have many tables with the exact same columns as the label table, so I would like to have a function that can insert into any of them. I'd like to create a dynamic query to do that. I tried that, but this does not work:
CREATE OR REPLACE FUNCTION insert_label(h_tbl REGCLASS, nms TEXT[])
RETURNS UUID[]
AS $$
DECLARE
ids UUID[];
query_str TEXT;
BEGIN
CREATE TEMP TABLE tmp_names(name TEXT);
INSERT INTO tmp_names SELECT UNNEST(nms);
query_str := FORMAT('WITH new_names AS ( INSERT INTO %1$I(name) SELECT tn.name FROM tmp_names tn WHERE NOT EXISTS(SELECT 1 FROM %1$I h WHERE h.name = tn.name) RETURNING hid)', h_tbl);
EXECUTE query_str;
SELECT ARRAY_AGG(hid) INTO ids FROM new_names;
DROP TABLE tmp_names;
RETURN ids;
END;
$$ LANGUAGE PLPGSQL;
This is the output I get when I run that function:
psql=# select insert_label('label', array['how', 'now', 'brown', 'cow']);
ERROR: syntax error at end of input
LINE 1: ...SELECT 1 FROM label h WHERE h.name = tn.name) RETURNING hid)
^
QUERY: WITH new_names AS ( INSERT INTO label(name) SELECT tn.name FROM tmp_names tn WHERE NOT EXISTS(SELECT 1 FROM label h WHERE h.name = tn.name) RETURNING hid)
CONTEXT: PL/pgSQL function insert_label(regclass,text[]) line 19 at EXECUTE
The query generated by the dynamic SQL looks like it should be exactly the same as the query from static SQL.
I got the function to work by changing the return value from an array of UUIDs to a table of UUIDs and not using CTE:
CREATE OR REPLACE FUNCTION insert_label(h_tbl REGCLASS, nms TEXT[])
RETURNS TABLE (hid UUID)
AS $$
DECLARE
query_str TEXT;
BEGIN
CREATE TEMP TABLE tmp_names(name TEXT);
INSERT INTO tmp_names SELECT UNNEST(nms);
query_str := FORMAT('INSERT INTO %1$I(name) SELECT tn.name FROM tmp_names tn WHERE NOT EXISTS(SELECT 1 FROM %1$I h WHERE h.name = tn.name) RETURNING hid', h_tbl);
RETURN QUERY EXECUTE query_str;
DROP TABLE tmp_names;
RETURN;
END;
$$ LANGUAGE PLPGSQL;
I don't know if one way is better than the other, returning an array of UUIDs or a table of UUIDs, but at least I got it to work one of those ways. Plus, possibly not using a CTE is more efficient, so it may be better to stick with the version that returns a table of UUIDs.
What I would like to know is why the dynamic query did not work when using a CTE. The query it produced looked like it should have worked.
If anyone can let me know what I did wrong, I would appreciate it.
... why the dynamic query did not work when using a CTE. The query it produced looked like it should have worked.
No, it was only the CTE without (required) outer query. (You had SELECT ARRAY_AGG(hid) INTO ids FROM new_names in the static version.)
There are more problems, but just use this query instead:
INSERT INTO label(name)
SELECT unnest(nms)
ON CONFLICT DO NOTHING
RETURNING hid;
label.name is defined UNIQUE NOT NULL, so this simple UPSERT can replace your function insert_label() completely.
It's much simpler and faster. It also defends against possible duplicates from within your input array that you didn't cover, yet. And it's safe under concurrent write load - as opposed to your original, which might run into race conditions. Related:
How to use RETURNING with ON CONFLICT in PostgreSQL?
I would just use the simple query and replace the table name.
But if you still want a dynamic function:
CREATE OR REPLACE FUNCTION insert_label(_tbl regclass, _nms text[])
RETURNS TABLE (hid uuid)
LANGUAGE plpgsql AS
$func$
BEGIN
RETURN QUERY EXECUTE format(
$$
INSERT INTO %s(name)
SELECT unnest($1)
ON CONFLICT DO NOTHING
RETURNING hid
$$, _tbl)
USING _nms;
END
$func$;
If you don't need an array as result, stick with the set (RETURNS TABLE ...). Simpler.
Pass values (_nms) to EXECUTE in a USING clause.
The tablename (_tbl) is type regclass, so the format specifier %I for format() would be wrong. Use %s instead. See:
Table name as a PostgreSQL function parameter

Why is RETURN QUERY returning a string instead of a TABLE

This MWE is NOT how you would typically solve this problem, however, it is as simple as I can explain the problem I am encountering. I am merely trying to point out 2 things
I am doing more than simply returning the contents of a Table
What is being returned is NOT being returned as a Table but a String
Supporting SQL Statements:
DROP DATABASE IF EXISTS test;
CREATE DATABASE test;
\c test
CREATE TABLE credit_card(
id BIGSERIAL PRIMARY KEY,
balance BIGINT
);
Functions:
CREATE FUNCTION get_credit_card(
p_id BIGINT
)
RETURNS TABLE(
id BIGINT,
balance BIGINT
)
AS $$
DECLARE
BEGIN
RETURN QUERY
SELECT
credit_card.id,
credit_card.balance
FROM
credit_card
WHERE
credit_card.id = p_id;
END $$ LAnguage 'plpgsql';
CREATE FUNCTION pay_with_card(
p_id BIGINT,
p_amount BIGINT
)
RETURNS TABLE(
id BIGINT,
balance BIGINT
)
AS $$
DECLARE
v_balance BIGINT;
BEGIN
SELECT
credit_card.balance
FROM
credit_card
INTO
v_balance
WHERE
credit_card.id = p_id;
IF v_balance < p_amount
THEN
RETURN;
END IF;
UPDATE
credit_card
SET
balance = credit_card.balance - p_amount;
RETURN QUERY
SELECT get_credit_card (p_id);
END $$ LAnguage 'plpgsql';
Populate Table and Call function:
INSERT INTO credit_card
(balance)
VALUES
(100);
SELECT
pay_with_card (1, 100);
Error:
DROP DATABASE
CREATE DATABASE
You are now connected to database "test" as user "postgres".
CREATE TABLE
CREATE FUNCTION
CREATE FUNCTION
INSERT 0 1
psql:test.sql:74: ERROR: structure of query does not match function result type
DETAIL: Returned type record does not match expected type bigint in column 1.
CONTEXT: PL/pgSQL function pay_with_card(bigint,bigint) line 24 at RETURN QUERY
It took me a long time to figure out that pay_with_card is returning a String, or what appears to be a String, instead of a TABLE(id BIGINT, balance BIGINT). With the Python psycopg2 library, the returned query is
[('(1,100)'),]
So my entire code is breaking because I can't get the values (unless I hack it and use string manipulation.
Question:
How can I fix it so that it returns the correct query like so
[(1,100),]
An alternative to the hint in horse_with_no_name's comment, you can replace
RETURN QUERY
SELECT get_credit_card (p_id);
with
RETURN QUERY SELECT (get_credit_card(p_id)).*;
You need some way of expanding the returned record back into its constituent fields. (I think horse’s SELECT * … has the same effect.)

No data output from stored procedure (Postgresql)

I have a basic stored procedure for a select statement. The select statement by itself works and shows me the data output, but when I try calling it from a stored procedure, it says 'CALL Query returned successfully in 107 msec', but there's no data output. Is there something that I'm missing from my stored procedure?
(I'm also connecting a database on AWS with the basic tier, not sure if that makes a difference. Every other CRUD operation works except for select.)
CREATE PROCEDURE
experiment1()
LANGUAGE SQL
AS $$
SELECT * FROM assignment
$$
A Postgres procedure does not return anything. You can use a function instead, with the return query syntax. This requires enumerating the columns that the query returns. Assuming that your table has two columns, id and val, that would be:
create function experiment()
returns table (id int, val text)
as $$
begin
return query select * from assignment;
end;
$$ language plpgsql;
You can then invoke this set-returning function like so:
select * from experiment();
Demo on DB Fiddle:
create table assignment (id int, val text);
insert into assignment values(1, 'foo'), (2, 'bar');
-- 2 rows affected
create function experiment()
returns table (id int, val text)
as $$
begin
return query select * from assignment;
end;
$$ language plpgsql;
select * from experiment();
id | val
-: | :--
1 | foo
2 | bar

Filter table in a function using a table valued parameter

I have a table-valued function that uses a parameter to filter a table and returns the results:
drop table if exists mytbl;
create table mytbl (i int);
insert into mytbl (i) values (1),(2),(3);
CREATE FUNCTION filterer(
_filter int
)
RETURNS TABLE(i_ret int)
LANGUAGE 'plpgsql'
AS $BODY$
BEGIN
RETURN QUERY SELECT i
FROM mytbl
WHERE i = _filter;
END;
$BODY$;
select * from filterer(1);
it returns:
So far so good. However what I really want to do is filter on a list of values rather than just one. I figured the way to do it is create a table-valued parameter using a type that I define using CREATE TYPE, but I can't get it working. Here's what I have so far:
drop table if exists mytbl;
create table mytbl (i int);
DO $$
BEGIN
IF NOT EXISTS (SELECT * FROM pg_type t WHERE t.typname = 'tp') THEN
CREATE TYPE tp AS (i int);
END IF;
END$$;
insert into mytbl (i) values (1),(2),(3);
DROP FUNCTION public.filterer(tp);
CREATE FUNCTION filterer(
_filter tp
)
RETURNS TABLE(i_ret int)
LANGUAGE 'plpgsql'
AS $BODY$
BEGIN
RETURN QUERY SELECT i
FROM mytbl
WHERE i IN (_filter);
END;
$BODY$;
select * from filterer(1);
When I run that code it fails with:
ERROR: function filterer(integer) does not exist LINE 23: select *
from filterer(1);
How do I declare a variable of type tp and pass it into my function?
working example
Two things here:
WHERE i IN (_filter);
should be replaced with
WHERE row(i) IN (_filter);
and calling:
select * from filterer(row(1));
I assume the question is purely academic, as I can't see any use of such type?..
CREATE TYPE tp AS (i int);

Temporary table postgresql function

I can't find a clear explanation of the syntax to create (and use) tables just for the inside calculations of a function. Could anyone give me a syntax exemple please ?
From what I've found, I have tried this (with and without # before temp_table) :
CREATE FUNCTION test.myfunction()
RETURNS SETOF test.out_table
AS $$
DECLARE #temp_table TABLE
(
id int,
value text
)
BEGIN
INSERT INTO #temp_table
SELECT id, value
FROM test.another_table;
INSERT INTO test.out_table
SELECT id, value
FROM #temp_table;
RETURN END
$$ LANGUAGE SQL;
I get :
ERROR: syntax error at or near "DECLARE"
LINE 5: DECLARE #temp_table TABLE
-
I also tried the CREATE TABLE approach suggested here, this way :
CREATE FUNCTION test.myfunction()
RETURNS SETOF test.out_table
AS $$
CREATE TABLE temp_table AS
SELECT id, value
FROM test.another_table;
INSERT INTO test.out_table
SELECT id, value
FROM temp_table;
$$ LANGUAGE SQL;
And I get this :
ERROR: relation "temp_table " does not exist
LINE 11: FROM temp_table
(Obviously, I'm aware the temp_table is not necessary for what I'm doing in the code above, but that's not the point :) => I want to understand the syntax to get it to work)
The appropriate syntax for creating a temp table is
create temp table...
but you have to be sure to drop the temp table before existing out of the function. Also, I'd suggest this syntax instead:
CREATE TEMP TABLE IF NOT EXISTS temp_table AS
SELECT id, value
FROM test.another_table;
Thus your function will be like this:
CREATE FUNCTION test.myfunction()
RETURNS SETOF test.out_table
AS $$
CREATE TEMP TABLE IF NOT EXISTS temp_table AS
SELECT id, value
FROM test.another_table;
INSERT INTO test.out_table
SELECT id, value
FROM temp_table;
DROP TABLE temp_table;
$$ LANGUAGE SQL;
But if I can be so kind, I'd like to rewrite this function so it is more correct:
CREATE FUNCTION test.myfunction()
RETURNS TABLE (id int, value varchar) -- change your datatype as needed
AS $$
BEGIN;
CREATE TEMP TABLE IF NOT EXISTS temp_table AS
SELECT id, value
FROM test.another_table;
INSERT INTO test.out_table
SELECT id, value
FROM temp_table;
DROP TABLE temp_table;
RETURN QUERY
SELECT id, value
from temp_table;
END;
$$ LANGUAGE plpgsql;
Untested; let me know if this fails.