Why is RETURN QUERY returning a string instead of a TABLE - sql

This MWE is NOT how you would typically solve this problem, however, it is as simple as I can explain the problem I am encountering. I am merely trying to point out 2 things
I am doing more than simply returning the contents of a Table
What is being returned is NOT being returned as a Table but a String
Supporting SQL Statements:
DROP DATABASE IF EXISTS test;
CREATE DATABASE test;
\c test
CREATE TABLE credit_card(
id BIGSERIAL PRIMARY KEY,
balance BIGINT
);
Functions:
CREATE FUNCTION get_credit_card(
p_id BIGINT
)
RETURNS TABLE(
id BIGINT,
balance BIGINT
)
AS $$
DECLARE
BEGIN
RETURN QUERY
SELECT
credit_card.id,
credit_card.balance
FROM
credit_card
WHERE
credit_card.id = p_id;
END $$ LAnguage 'plpgsql';
CREATE FUNCTION pay_with_card(
p_id BIGINT,
p_amount BIGINT
)
RETURNS TABLE(
id BIGINT,
balance BIGINT
)
AS $$
DECLARE
v_balance BIGINT;
BEGIN
SELECT
credit_card.balance
FROM
credit_card
INTO
v_balance
WHERE
credit_card.id = p_id;
IF v_balance < p_amount
THEN
RETURN;
END IF;
UPDATE
credit_card
SET
balance = credit_card.balance - p_amount;
RETURN QUERY
SELECT get_credit_card (p_id);
END $$ LAnguage 'plpgsql';
Populate Table and Call function:
INSERT INTO credit_card
(balance)
VALUES
(100);
SELECT
pay_with_card (1, 100);
Error:
DROP DATABASE
CREATE DATABASE
You are now connected to database "test" as user "postgres".
CREATE TABLE
CREATE FUNCTION
CREATE FUNCTION
INSERT 0 1
psql:test.sql:74: ERROR: structure of query does not match function result type
DETAIL: Returned type record does not match expected type bigint in column 1.
CONTEXT: PL/pgSQL function pay_with_card(bigint,bigint) line 24 at RETURN QUERY
It took me a long time to figure out that pay_with_card is returning a String, or what appears to be a String, instead of a TABLE(id BIGINT, balance BIGINT). With the Python psycopg2 library, the returned query is
[('(1,100)'),]
So my entire code is breaking because I can't get the values (unless I hack it and use string manipulation.
Question:
How can I fix it so that it returns the correct query like so
[(1,100),]

An alternative to the hint in horse_with_no_name's comment, you can replace
RETURN QUERY
SELECT get_credit_card (p_id);
with
RETURN QUERY SELECT (get_credit_card(p_id)).*;
You need some way of expanding the returned record back into its constituent fields. (I think horse’s SELECT * … has the same effect.)

Related

Dynamic query that uses CTE gets "syntax error at end of input"

I have a table that looks like this:
CREATE TABLE label (
hid UUID PRIMARY KEY DEFAULT UUID_GENERATE_V4(),
name TEXT NOT NULL UNIQUE
);
I want to create a function that takes a list of names and inserts multiple rows into the table, ignoring duplicate names, and returns an array of the IDs generated for the rows it inserted.
This works:
CREATE OR REPLACE FUNCTION insert_label(nms TEXT[])
RETURNS UUID[]
AS $$
DECLARE
ids UUID[];
BEGIN
CREATE TEMP TABLE tmp_names(name TEXT);
INSERT INTO tmp_names SELECT UNNEST(nms);
WITH new_names AS (
INSERT INTO label(name)
SELECT tn.name
FROM tmp_names tn
WHERE NOT EXISTS(SELECT 1 FROM label h WHERE h.name = tn.name)
RETURNING hid
)
SELECT ARRAY_AGG(hid) INTO ids
FROM new_names;
DROP TABLE tmp_names;
RETURN ids;
END;
$$ LANGUAGE PLPGSQL;
I have many tables with the exact same columns as the label table, so I would like to have a function that can insert into any of them. I'd like to create a dynamic query to do that. I tried that, but this does not work:
CREATE OR REPLACE FUNCTION insert_label(h_tbl REGCLASS, nms TEXT[])
RETURNS UUID[]
AS $$
DECLARE
ids UUID[];
query_str TEXT;
BEGIN
CREATE TEMP TABLE tmp_names(name TEXT);
INSERT INTO tmp_names SELECT UNNEST(nms);
query_str := FORMAT('WITH new_names AS ( INSERT INTO %1$I(name) SELECT tn.name FROM tmp_names tn WHERE NOT EXISTS(SELECT 1 FROM %1$I h WHERE h.name = tn.name) RETURNING hid)', h_tbl);
EXECUTE query_str;
SELECT ARRAY_AGG(hid) INTO ids FROM new_names;
DROP TABLE tmp_names;
RETURN ids;
END;
$$ LANGUAGE PLPGSQL;
This is the output I get when I run that function:
psql=# select insert_label('label', array['how', 'now', 'brown', 'cow']);
ERROR: syntax error at end of input
LINE 1: ...SELECT 1 FROM label h WHERE h.name = tn.name) RETURNING hid)
^
QUERY: WITH new_names AS ( INSERT INTO label(name) SELECT tn.name FROM tmp_names tn WHERE NOT EXISTS(SELECT 1 FROM label h WHERE h.name = tn.name) RETURNING hid)
CONTEXT: PL/pgSQL function insert_label(regclass,text[]) line 19 at EXECUTE
The query generated by the dynamic SQL looks like it should be exactly the same as the query from static SQL.
I got the function to work by changing the return value from an array of UUIDs to a table of UUIDs and not using CTE:
CREATE OR REPLACE FUNCTION insert_label(h_tbl REGCLASS, nms TEXT[])
RETURNS TABLE (hid UUID)
AS $$
DECLARE
query_str TEXT;
BEGIN
CREATE TEMP TABLE tmp_names(name TEXT);
INSERT INTO tmp_names SELECT UNNEST(nms);
query_str := FORMAT('INSERT INTO %1$I(name) SELECT tn.name FROM tmp_names tn WHERE NOT EXISTS(SELECT 1 FROM %1$I h WHERE h.name = tn.name) RETURNING hid', h_tbl);
RETURN QUERY EXECUTE query_str;
DROP TABLE tmp_names;
RETURN;
END;
$$ LANGUAGE PLPGSQL;
I don't know if one way is better than the other, returning an array of UUIDs or a table of UUIDs, but at least I got it to work one of those ways. Plus, possibly not using a CTE is more efficient, so it may be better to stick with the version that returns a table of UUIDs.
What I would like to know is why the dynamic query did not work when using a CTE. The query it produced looked like it should have worked.
If anyone can let me know what I did wrong, I would appreciate it.
... why the dynamic query did not work when using a CTE. The query it produced looked like it should have worked.
No, it was only the CTE without (required) outer query. (You had SELECT ARRAY_AGG(hid) INTO ids FROM new_names in the static version.)
There are more problems, but just use this query instead:
INSERT INTO label(name)
SELECT unnest(nms)
ON CONFLICT DO NOTHING
RETURNING hid;
label.name is defined UNIQUE NOT NULL, so this simple UPSERT can replace your function insert_label() completely.
It's much simpler and faster. It also defends against possible duplicates from within your input array that you didn't cover, yet. And it's safe under concurrent write load - as opposed to your original, which might run into race conditions. Related:
How to use RETURNING with ON CONFLICT in PostgreSQL?
I would just use the simple query and replace the table name.
But if you still want a dynamic function:
CREATE OR REPLACE FUNCTION insert_label(_tbl regclass, _nms text[])
RETURNS TABLE (hid uuid)
LANGUAGE plpgsql AS
$func$
BEGIN
RETURN QUERY EXECUTE format(
$$
INSERT INTO %s(name)
SELECT unnest($1)
ON CONFLICT DO NOTHING
RETURNING hid
$$, _tbl)
USING _nms;
END
$func$;
If you don't need an array as result, stick with the set (RETURNS TABLE ...). Simpler.
Pass values (_nms) to EXECUTE in a USING clause.
The tablename (_tbl) is type regclass, so the format specifier %I for format() would be wrong. Use %s instead. See:
Table name as a PostgreSQL function parameter

Is SELECT "faster" than function with nested INSERT?

I'm using a function that inserts a row to a table if it doesn't exist, then returns the id of the row.
Whenever I put the function inside a SELECT statement, with values that don't exist in the table yet, e.g.:
SELECT * FROM table WHERE id = function(123);
... it returns an empty row. However, running it again with the same values will return the row with the values I want to see.
Why does this happen? Is the INSERT running behind the SELECT speed? Or does PostgreSQL cache the table when it didn't exist, and at next run, it displays the result?
Here's a ready to use example of how this issue can occur:
CREATE TABLE IF NOT EXISTS test_table(
id INTEGER,
tvalue boolean
);
CREATE OR REPLACE FUNCTION test_function(user_id INTEGER)
RETURNS integer
LANGUAGE 'plpgsql'
AS $$
DECLARE
__user_id INTEGER;
BEGIN
EXECUTE format('SELECT * FROM test_table WHERE id = $1')
USING user_id
INTO __user_id;
IF __user_id IS NOT NULL THEN
RETURN __user_id;
ELSE
INSERT INTO test_table(id, tvalue)
VALUES (user_id, TRUE)
RETURNING id
INTO __user_id;
RETURN __user_id;
END IF;
END;
$$;
Call:
SELECT * FROM test_table WHERE id = test_function(4);
To reproduce the issue, pass any integer that doesn't exist in the table, yet.
The example is broken in multiple places.
No need for dynamic SQL with EXECUTE.
SELECT * in the function is wrong.
Your table definition should have a UNIQUE or PRIMARY KEY constraint on (id).
Most importantly, the final SELECT statement is bound to fail. Since the function is VOLATILE (has to be), it is evaluated once for every existing row in the table. Even if that worked, it would be a performance nightmare. But it does not. Like #user2864740 commented, there is also a problem with visibility. Postgres checks every existing row against the result of the function, which in turn adds 1 or more rows, and those rows are not yet in the snapshot the SELECT is operating on.
SELECT * FROM test_table WHERE id = test_function(4);
This would work (but see below!):
CREATE TABLE test_table (
id int PRIMARY KEY --!
, tvalue bool
);
CREATE OR REPLACE FUNCTION test_function(_user_id int)
RETURNS test_table LANGUAGE sql AS
$func$
WITH ins AS (
INSERT INTO test_table(id, tvalue)
VALUES (_user_id, TRUE)
ON CONFLICT DO NOTHING
RETURNING *
)
TABLE ins
UNION ALL
SELECT * FROM test_table WHERE id = _user_id
LIMIT 1
$func$;
And replace your SELECT with just:
SELECT * FROM test_function(1);
db<>fiddle here
Related:
Return a value if no record is found
How to use RETURNING with ON CONFLICT in PostgreSQL?
There is still a race condition for concurrent calls. If that can happen, consider:
Is SELECT or INSERT in a function prone to race conditions?

Returning table with specific parameters from a PostgreSQL function

I have this function, with which I would like to return a table, which holds two columns: game_name and follow (this would be an integer 0 or 1):
CREATE OR REPLACE FUNCTION public.toggle2(uid numeric, gid NUMERIC)
RETURNS TABLE (
follow INT,
game_name TEXT
)
LANGUAGE plpgsql
AS $$
BEGIN
IF NOT EXISTS(SELECT *
FROM game_follows
WHERE user_id = uid and game_id = gid)
THEN
INSERT INTO game_follows(user_id, game_id) VALUES(uid, gid);
follow := 1;
ELSE
DELETE FROM game_follows WHERE user_id = uid and game_id = gid;
follow := 0;
END IF;
SELECT name INTO game_name FROM games WHERE id = gid;
END;
$$
;
Sadly, the function returns empty values. I am using it as this:
SELECT * FROM toggle2(83, 12);
A function declared to RETURN TABLE can return 0-n rows.
You must actively return rows, or nothing will be returned (no row). One way to do this:
RETURN NEXT; -- as last line before END;
There are other ways, see the manual.
However, it seems you want to return exactly one row every time. So rather use OUT parameters:
CREATE OR REPLACE FUNCTION public toggle2(uid numeric, gid numeric, OUT follow int, OUT game_name text) AS ...
Then it's enough to assign those OUT parameters, they are returned in the single result row automatically.
See:
Returning from a function with OUT parameter
plpgsql error "RETURN NEXT cannot have a parameter in function with OUT parameters" in table-returning function
How to return result of a SELECT inside a function in PostgreSQL?

PostgreSQL - Shared temp table between functions

I wnat to know if its possible to share a temporary table between functions that are called in a "main function", like this:
-- some sub function
create or replace function up_sub_function (str text)
returns table (id int, descr text) as $$
begin
return query select * from temp_table where descr like concat('%', str , '%');
end; $$
language plpgsql;
-- main function
create or replace function up_main_function ()
returns table (id int, descr text) as $$
begin
create temporary table temp_table if not exists (
id int,
descr text
);
insert into temp_campaigns select id, descr from test_table;
return query select * from up_sub_function('a');
end; $$
language plpgsql;
BEGIN;
select * from up_main_function();
drop table temp_table;
COMMIT;
If you can show me the correct way to achieve this, I want to be able to populate a temporary table and then filter rows by calling othe functions inside the main function.
Thanks ans happy programming! :)
See the documentation https://www.postgresql.org/docs/current/static/sql-createtable.html
temp tables are valid for the entire session. That is as long as you stay connected to the database.
In your case you only need it during the transaction. So you should create it with ON COMMIT DROP
create temporary table temp_table if not exists (
id int,
descr text
) ON COMMIT DROP;
Once you created the table you can use it within any function in the current transaction.
You do not need the BEGIN to start the transaction. A transaction is automatically started when the outer function is called.
Nested function calls share the same transaction. So they all see the table.

oracle inline function called multiple times

When calling a function via an inline select statement, when the function is returning a custom type, Oracle seems to execute the function equal to the number of arguments +1. This seems to happen when the select is included as a CTAS or an insert/select.
Has anyone seen this before? Is this an Oracle bug? I would expect the function to be called once per row in the table.
--Inline function gets called for the number of arguments +1
--drop table t
create table t(
id number,
l_blob blob
);
insert into t values(1, utl_raw.cast_to_raw('SampleString'));
COMMIT;
create table tmp_ts (c1 timestamp);
create or replace type test_type as object(
c1 varchar2(32)
,c2 varchar2(32)
);
/
create or replace FUNCTION test_function (p_blob blob, p_date date)
RETURN test_type
IS
BEGIN
--This could also be a DBMS_OUTPUT.PUT_LINE statement
insert into tmp_ts VALUES (systimestamp);
return test_type(null,null);
END test_function;
/
--0
select count(*) from tmp_ts;
--Call function on 1 row table - function should just insert 1 row into tmp_ts
create table tst_table as
select test_function(l_blob, '25-JAN-09') as c1
from t;
--it actually inserts 3
select count(*) from tmp_ts;
Example where increasing the argument call for the type increases the number of time the function is executed
--Same example with more arguements - 6 arguements here
--Inline function gets called for the number of arguments +1
--drop table t
create table t2(
id number,
l_blob blob
);
insert into t2 values(1, utl_raw.cast_to_raw('SampleString'));
COMMIT;
create table tmp_ts2 (c1 timestamp);
create or replace type test_type2 as object(
c1 varchar2(32)
,c2 varchar2(32)
,c3 varchar2(32)
,c4 varchar2(32)
,c5 varchar2(32)
,c6 varchar2(32)
);
/
create or replace FUNCTION test_function2 (p_blob blob, p_date date)
RETURN test_type2
IS
BEGIN
insert into tmp_ts2 VALUES (systimestamp);
return test_type2(null,null,null,null,null,null);
END test_function2;
/
--0
select count(*) from tmp_ts2;
--Call function on 1 row table - function should just insert 1 row into tmp_ts
create table tst_table2 as
select test_function2(l_blob, '25-JAN-09') as c1
from t;
--it actually inserts 7
select count(*) from tmp_ts2;
Any help/feedback is greatly appreciated.
First: It is a bug that you can even perform a DML inside a function which is called in a SELECT Statement. This should raise an exception.
Otherwise Oracle makes absolutely no guarantee how often Functions in a SQL-Select are executed, it could be once per row, ten times per row or just once for the whole query (with caching) - so however often it is called, this conforms to the specifications.
In this special case it will call the function for each attribute of the returning type, since oracle will not insert the object type as one memory-structure, but use the function like a table with multiple columns and read each column individually like this:
INSERT VALUES ( myFunc(x).attribute1, myFunc(x).attribute2 );
The important part: Never make any assumptions about how often a FUNCTION is called when you use it in an SQL Statement!!! At any time the function could be called again by the optimizer, maybe for sampling or caching...
Preferred solution: Pipelined Functions - a pipelined Function can be called like a Table and will only be called once. You can pass in a cursor which the function uses for input processing and do the whole data-transformation and logging and everything in the function.