How to assign result of table function to variable in PL/pgSQL - sql

Assume I have the following function declaration:
CREATE OR REPLACE FUNCTION build_org_branch(IN p_org_id organization.org_id%type,
IN p_padding text)
RETURNS table
(
object_id int,
parent_id int,
name text
)
Then I want to call build_org_branch with parameters and assign it to a variable inside of another function like this:
declare
l_table record[]; --??????
begin
l_table := build_org_branch(1, ' '); -- is it okay?
if l_table is not null then
-- do stuff with table rows
end if;
end;
Or should I use some another approach to pass tables of rows?

You have built a function that returns a table, so process the results that way.
do $$
declare
rec record;
begin
for rec in (select * from build_org_branch(101, ''))
loop
raise notice 'Returned Row: object_id=>%, name=>%, parent_id=>%'
, rec.object_id
, rec.name
, rec.parent_id ;
-- do stuff with table rows
end loop;
end;
$$;
I do not have your table, so I'll hard code some values but how they are populated is not the issue, but what you do afterward. See fiddle.

Postgres doesn't support table's variables. If you can pass a some relational content, then a) you can use a temporary table or b) you can pass a array of composite values:
CREATE TYPE branch_type AS
(
object_id int,
parent_id int,
name text
)
CREATE OR REPLACE FUNCTION build_org_branch(IN p_org_id organization.org_id%type,
IN p_padding text)
RETURNS branch_type[] AS ...
and then you can write
declare
l_table branch_type[];
begin
l_table := build_org_branch(1, ' ');
if l_table is not null then
-- do stuff with table rows
end if;
end;
This is array to array assignment. Table to array is possible too, but always it has to be static typed.
CREATE OR REPLACE FUNCTION build_org_branch(IN p_org_id organization.org_id%type,
IN p_padding text)
RETURNS SETOF branch_type AS ...
and processing:
declare
l_table branch_type[];
begin
l_table := ARRAY(SELECT build_org_branch(1, ' '));
if l_table is not null then
-- do stuff with table rows
end if;
end;
For smaller number of rows (to ten thousand) a arrays should be preferred. For high number of rows you should to use temp table.

Related

Postgres function to return Table into variables

How can I capture different columns into different variables like so (note this is only pseudocode so I am assuming it will cause errors. Example taken from here)
create or replace function get_film (
p_pattern varchar
)
returns table (
film_title varchar,
film_release_year int
)
language plpgsql
as $$
begin
return query
select
title,
release_year::integer
from
film
where
title ilike p_pattern;
end;$$
create or replace function get_film_into_variables (
p_pattern varchar
)
returns null
language plpgsql
as $$
declare
v_title varchar,
v_release_year integer
begin
SELECT
get_film (p_pattern)
INTO
v_title,
v_release_year;
end;$$
Assuming you have some purpose for the variables after retrieving them not just ending the function your "get_film_into_variables" is almost there. But first let's backup just a bit. A function that returns a table does just that, you can use the results just like a table stored on disk (it just goes away after query or calling function ends). To that end only a slight change to the "get_film_into_variables" function is required. The "get_film" becomes the object of the FROM clause. Also change the returns null, to returns void. So
create or replace function get_film_into_variables (
p_pattern varchar
)
returns void
language plpgsql
as $$
declare
v_title varchar;
v_release_year integer;
begin
select *
from get_film (p_pattern)
INTO
v_title,
v_release_year;
end;
$$;
The above works for a single row returned by a function returning table. However for a return of multiple rows you process the results of the table returning function just lake you would an actual table - with a cursor.
create or replace
function get_film_into_variables2(p_pattern varchar)
returns void
language plpgsql
as $$
declare
k_message_template constant text = 'The film "%s" was released in %s.';
v_title varchar;
v_release_year integer;
v_film_message varchar;
c_film cursor (c_pattern varchar) for
select * from get_film (c_pattern);
begin
open c_film (p_pattern);
loop
fetch c_film
into v_title
, v_release_year;
exit when not found;
v_film_message = format( k_message_template,v_title,v_release_year::text);
raise notice using
message = v_film_message;
end loop;
end;
$$;
BTW: the get_film function can be turned into a SQL function. See fiddle here. For demo purposes get_film_into_variable routines return a message.

Replacing Placeholder values with another table's data

I have 2 tables .The first table contains rows with placeholders and the second table contains those placeholders values.
I want a query which fetches data from the first table and replaces placeholders with actual values which are stored in the second table.
Ex:
Table1 Data
id value
608CB424-90BF-4B08-8CF8-241C7635434F jdbc:postgresql://{POSTGRESIP}:{POSTGRESPORT}/{TESTDB}
CDA4C3D4-72B5-4422-8071-A29D32BD14E0 https://{SERVICEIP}/svc/{TESTSERVICE}/
Table2 Data
id placeolder value
201FEBFE-DF92-4474-A945-A592D046CA02 POSTGRESIP 1.2.3.4
20D9DE14-643F-4CE3-B7BF-4B7E01963366 POSTGRESPORT 5432
45611605-F2D9-40C8-8C0C-251E300E183C TESTDB mytest
FA8E2E4E-014C-4C1C-907E-64BAE6854D72 SERVICEIP 10.90.30.40
45B76C68-8A0F-4FD3-882F-CA579EC799A6 TESTSERVICE mytest-service
Required output is
id value
608CB424-90BF-4B08-8CF8-241C7635434F jdbc:postgresql://1.2.3.4:5432/mytest
CDA4C3D4-72B5-4422-8071-A29D32BD14E0 https://10.90.30.40/svc/mytest-service/
If you want to use Python-like named placeholders then you need the helper function written on plpythonu:
create extension plpythonu;
create or replace function formatpystring( str text, a json ) returns text immutable language plpythonu as $$
import json
d = json.loads(a)
return str.format(**d)
$$;
Then simple test:
select formatpystring('{foo}.{bar}', '{"foo": "win", "bar": "amp"}');
formatpystring
----------------
win.amp
Finally you need to compose those arguments from your tables. It is simple:
select t1.id, formatpystring(t1.value, json_object_agg(t2.placeholder, t2.value)) as value
from table1 as t1, table2 as t2
group by t1.id, t1.value;
(Query was not tested but you have the direction)
(Clumsy) dynamic SQL implementation, featuring an outer join, but generating a recursive function call:
This function will not be very efficient, but probably the translation table is relatively small.
CREATE TABLE xlat_table (aa text ,bb text);
INSERT INTO xlat_table (aa ,bb ) VALUES( 'BBB', '/1.2.3.4/')
,( 'ccc', 'OMG') ,( 'ddd', '/4.3.2.1/') ;
CREATE FUNCTION dothe_replacements(_arg1 text) RETURNS text
AS
$func$
DECLARE
script text;
braced text;
res text;
found record; -- (aa text, bb text, xx text);
BEGIN
script := '';
res := format('%L', _arg1);
for found IN SELECT xy.aa,xy.bb
, regexp_matches(_arg1, '{\w+}','g' ) AS xx
FROM xlat_table xy
LOOP
-- RAISE NOTICE '#xx=%', found.xx[1];
-- RAISE NOTICE 'aa=%', found.aa;
-- RAISE NOTICE 'bb=%', found.bb;
braced := '{'|| found.aa || '}';
IF (found.xx[1] = braced ) THEN
-- RAISE NOTICE 'Res=%', res;
script := format ('replace(%s, %L, %L)'
,res,braced,found.bb);
res := format('%s', script);
END IF;
END LOOP;
if(length(script) =0) THEN return res; END IF;
script :='Select '|| script;
-- RAISE NOTICE 'script=%', script;
EXECUTE script INTO res;
return res;
END;
$func$
LANGUAGE plpgsql;
SELECT dothe_replacements( 'aaa{BBB}ccc{ddd}eee' );
SELECT dothe_replacements( '{AAA}bbb{CCC}DDD}{EEE}' );
Results:
CREATE TABLE
INSERT 0 3
CREATE FUNCTION
dothe_replacements
-----------------------------
aaa/1.2.3.4/ccc/4.3.2.1/eee
(1 row)
dothe_replacements
--------------------------
'{AAA}bbb{CCC}DDD}{EEE}'
(1 row)
The above method has quadratic behaviour(wrt the numberof xlat-entries); which is horrible.
But,we could dynamically create a function (once) and call it multiple times
(a poor man's generator)
Selecting only the relevant entries from the xlat table should probably be added.
And, you should of course re-create the function everytime the xlat table is changed.
CREATE FUNCTION create_replacement_function(_name text) RETURNS void
AS
$func$
DECLARE
argname text;
res text;
script text;
braced text;
found record; -- (aa text, bb text, xx text);
BEGIN
script := '';
argname := '_arg1';
res :=format('%I', argname);
for found IN SELECT xy.aa,xy.bb
FROM xlat_table xy
LOOP
-- RAISE NOTICE 'aa=%', found.aa;
-- RAISE NOTICE 'bb=%', found.bb;
-- RAISE NOTICE 'Res=%', res;
braced := '{'|| found.aa || '}';
script := format ('replace(%s, %L, %L)'
,res,braced,found.bb);
res := format('%s', script);
END LOOP;
script :=FORMAT('CREATE FUNCTION %I (_arg1 text) RETURNS text AS
$omg$
BEGIN
RETURN %s;
END;
$omg$ LANGUAGE plpgsql;', _name, script);
RAISE NOTICE 'script=%', script;
EXECUTE script ;
return ;
END;
$func$
LANGUAGE plpgsql;
SELECT create_replacement_function( 'my_function');
SELECT my_function('aaa{BBB}ccc{ddd}eee' );
SELECT my_function( '{AAA}bbb{CCC}DDD}{EEE}' );
And the result:
CREATE FUNCTION
NOTICE: script=CREATE FUNCTION my_function (_arg1 text) RETURNS text AS
$omg$
BEGIN
RETURN replace(replace(replace(_arg1, '{BBB}', '/1.2.3.4/'), '{ccc}', 'OMG'), '{ddd}', '/4.3.2.1/');
END;
$omg$ LANGUAGE plpgsql;
create_replacement_function
-----------------------------
(1 row)
my_function
-----------------------------
aaa/1.2.3.4/ccc/4.3.2.1/eee
(1 row)
my_function
------------------------
{AAA}bbb{CCC}DDD}{EEE}
(1 row)
The following offers a plpgsql solution in a with a single function.
You'll notice I've 'renamed' the value column. It's bad practice using rserved/key words as object names. Also soq is the schema I use for all SO code.
The process first takes the holder-values from table2 and generates a set of key-value pairs (in this case hstore, but jsonb would also work). It then builds an array from the value column (my column name: val_string) containing the place_holder name from the value. Finally, it iterates that array replacing the actual holder-name with the value from the key-values using the array value as the lookup key.
The performance would not be great with a larger volume from either table. If you need to process a large volume at a time to a single row temp table may yield better performance.
create or replace function soq.replace_holders( place_holder_line_in text)
returns text
language plpgsql
as $$
declare
l_holder_values hstore;
l_holder_line text;
l_holder_array text[];
l_indx integer;
begin
-- transform cloumns to key-value pairs of holder-value
select string_agg(place,',')::hstore
into l_holder_values
from (
select concat( '"',place_holder,'"=>"',place_value,'"') place
from soq.table2
) p;
-- raise notice 'holder_array_in==%',l_holder_values;
-- extract the text line and build array of place_holder names
select phv, string_to_array (string_agg(v,','),',')
into l_holder_line,l_holder_array
from (
select replace(replace(place_holder_line_in,'{',''),'}','') phv
, replace(replace(replace(regexp_matches(place_holder_line_in,'({[^}]+})','g')::text ,'{',''),'}',''),'"','') v
) s
group by phv;
-- raise notice 'Array==%',l_holder_array::text;
-- replace each key from text line with the corresponding value
for l_indx in 1 .. array_length(l_holder_array,1)
loop
l_holder_line = replace(l_holder_line,l_holder_array[l_indx],l_holder_values -> l_holder_array[l_indx]);
end loop;
-- done
return l_holder_line;
end;
$$;
-- Test driver
select id, soq.replace_holders(val_string) result_value from soq.table1;
I have created a simple query for this solution and it working as required.
WITH RECURSIVE cte(id, value, level) AS (
SELECT id,value, 0 as level
FROM Table1
UNION
SELECT ts.id,replace(ts.value,'{'||tp.placeholder||'}',tp.value) as value, level+1
FROM cte ts, Table2 tp WHERE ts.value LIKE CONCAT('%',tp.placeholder, '%')
)
SELECT id, value FROM cte c
where level =
(
select Max(level)
from cte c2 where c.id=c2.id
)
Output is
id value
CDA4C3D4-72B5-4422-8071-A29D32BD14E0 https://10.90.30.40/svc/mytest-service/
608CB424-90BF-4B08-8CF8-241C7635434F jdbc:postgresql://1.2.3.4:5432/mytest

using PERFORM to insert a string of SELECT statement into a temp table

I am trying to insert data into a temp_table and then truncating the table after analyzing the result.
Here is my code:
CREATE OR REPLACE FUNCTION validation()
RETURNS text AS $$
DECLARE counter INTEGER;
DECLARE minsid INTEGER;
DECLARE maxsid INTEGER;
DECLARE rec RECORD;
DECLARE stmt varchar;
BEGIN
SELECT MIN(sid) INTO minsid FROM staging.validation;
SELECT MAX(sid) INTO maxsid FROM staging.validation;
CREATE TEMPORARY TABLE temp_table (col1 TEXT, col2 INTEGER, col3 BOOLEAN) ON COMMIT DROP;
FOR counter IN minsid..maxsid LOOP
RAISE NOTICE 'Counter: %', counter;
SELECT sql INTO stmt FROM staging.validation WHERE sid = counter;
RAISE NOTICE 'sql: %', stmt;
PERFORM 'INSERT INTO temp_table (col1, col2, col3) ' || stmt;
IF temp_table.col3 = false THEN
RAISE NOTICE 'there is a false value';
END IF;
END LOOP;
END; $$
LANGUAGE plpgsql;
Whenever I run this function
SELECT * FROM validation();
I get an error:
ERROR: missing FROM-clause entry for table "temp_table" Where: PL/pgSQL function validation() line 21 at IF
Here is how my staging.validation table looks -
https://docs.google.com/spreadsheets/d/1bXO9gqFS-GtcC1qJtgNbFkR6ygOuPtR_RZoU7VNhgrI/edit?usp=sharing
First, you don't have to use DECLARE for every variable, it's enough with one.
DECLARE
counter INTEGER;
minsid INTEGER;
maxsid INTEGER;
rec RECORD;
stmt varchar;
Second, you can't use the temp_table.col3 just like that, you have to query it because it's a table. You could create a variable and query into the table or you could directly make the query.
Variable:
-- First you declare de varialbe:
DECLARE
temp BOOLEAN;
... -- rest of your code
temp := temp_table.col3 FROM temp_table WHERE temp_table.col2=counter;
-- you need something to make the query, here as a test I put col2=counter
IF temp=false THEN
... -- rest of your code
Directly:
... -- rest of your code
IF (SELECT temp_table.col3 FROM temp_table WHERE temp_table.col2=counter)=false THEN
-- Again, you need something to make the query
... -- rest of your code
And third, your function has RETURNS text, in your pl the return is missing;

How to insert rows to table in a loop

I have the following plpgsql function in PostgreSQL:
CREATE OR REPLACE FUNCTION func1()
RETURNS SETOF type_a AS
$BODY$
declare
param text;
sqls varchar;
row type_a;
begin
code.....
sqls='select * from func3(' || param || ') ';
for row in execute sqls LOOP
return next row;
END LOOP;
end if;
return;
end
$BODY$
LANGUAGE plpgsql VOLATILE
I want to add an insert statment into the loop, so that the loop will work as it is now but also all rows will be saved in a table.
for row in execute sqls LOOP
INSERT INTO TABLE new_tab(id, name)
return next row;
the thing is that I don't know how to do that... the insert statment normaly has syntax of:
INSERT INTO new_tab(id, name)
SELECT x.id, x.name
FROM y
but this syntax doesn't fit here. There is no query to select rows from.... the rows are in the loop.
Basic insert with values looks like this:
INSERT INTO table_name (column1,column2,column3,...)
VALUES (value1,value2,value3,...);
Based on the additional comments you need to use cursor instead of execute sqls.
No need for a loop, you can use insert .. select ... returning in dynamic SQL just as well:
create or replace function func1()
returns table (id integer, name text)
as
$$
declare
param text;
begin
param := ... ;
return query execute
'insert into new_tab (id, name)
select id, name
from func3($1)
returning *'
using param;
end;
$$
language plpgsql;
Note that I used a parameter placeholder and the USING clause instead of concatenating the parameter into the query - much more robust.

Looping on values, creating dynamic query and adding to result set

I have the following problem. I am an experienced Java programmer but am a bit of a n00b at SQL and PL/SQL.
I need to do the following.
1 Pass in a few arrays and some other variables into a procedure
2 Loop on the values in the arrays (they all have the same number of items) and dynamically create an SQL statement
3 Run this statement and add it to the result set (which is an OUT parameter of the procedure)
I already have experience of creating an SQL query on the fly, running it and adding the result to a result set (which is a REF CURSOR) but I'm not sure how I'd loop and add the results of each call to the query to the same result set. I'm not even sure if this is possible.
Here's what I have so far (code edited for simplicity). I know it's wrong because I'm just replacing the contents of the RESULT_SET with the most recent query result (and this is being confirmed in the Java which is calling this procedure).
Any and all help would be greatly appreciated.
TYPE REF_CURSOR IS REF CURSOR;
PROCEDURE GET_DATA_FASTER(in_seq_numbers IN seq_numbers_array, in_values IN text_array, in_items IN text_array, list IN VARCHAR2, RESULT_SET OUT REF_CURSOR) AS
query_str VARCHAR2(4000);
seq_number NUMBER;
the_value VARCHAR2(10);
the_item VARCHAR2(10);
BEGIN
FOR i IN 1..in_seq_numbers.COUNT
LOOP
seq_number := in_seq_numbers(i);
the_value := trim(in_values(i));
the_item := trim(in_items(i));
query_str := 'SELECT distinct '||seq_number||' as seq, value, item
FROM my_table ai';
query_str := query_str || '
WHERE ai.value = '''||the_value||''' AND ai.item = '''||the_item||'''
AND ai.param = ''BOOK''
AND ai.prod in (' || list || ');
OPEN RESULT_SET FOR query_str;
END LOOP;
EXCEPTION WHEN OTHERS THEN
RAISE;
END GET_DATA_FASTER;
A pipelined table function seems a better fit for what you want, especially if all you're doing is retrieving data. See http://www.oracle-base.com/articles/misc/pipelined-table-functions.php
What you do is create a type for your output row. So in your case you would create an object such as
CREATE TYPE get_data_faster_row AS OBJECT(
seq NUMBER(15,2),
value VARCHAR2(10),
item VARCHAR2(10)
);
Then create a table type which is a table made up of your row type above
CREATE TYPE get_data_faster_data IS TABLE OF get_data_faster_row;
Then create your table function that returns the data in a pipelined manner. Pipelined in Oracle is a bit like a yield return in .net (not sure if you're familiar with that). You find all of the rows that you want and "pipe" them out one at a time in a loop. When your function completes the table that's returned consists of all the rows you piped out.
CREATE FUNCTION Get_Data_Faster(params) RETURN get_data_faster_data PIPELINED AS
BEGIN
-- Iterate through your parameters
--Iterate through the results of the select using
-- the current parameters. You'll probably need a
-- cursor for this
PIPE ROW(get_data_faster_row(seq, value, item));
LOOP;
LOOP;
END;
EDIT: Following Alex's comment below, you need something like this. I haven't been able to test this but it should get you started:
CREATE FUNCTION Get_Data_Faster(in_seq_numbers IN seq_numbers_array, in_values IN text_array, in_items IN text_array, list IN VARCHAR2) RETURN get_data_faster_data PIPELINED AS
TYPE r_cursor IS REF CURSOR;
query_results r_cursor;
results_out get_data_faster_row := get_data_faster_row(NULL, NULL, NULL);
query_str VARCHAR2(4000);
seq_number NUMBER;
the_value VARCHAR2(10);
the_item VARCHAR2(10);
BEGIN
FOR i IN 1..in_seq_number.COUNT
LOOP
seq_number := in_seq_numbers(i);
the_value := trim(in_values(i));
the_item := trim(in_items(i));
query_str := 'SELECT distinct '||seq_number||' as seq, value, item
FROM my_table ai';
query_str := query_str || '
WHERE ai.value = '''||the_value||''' AND ai.item = '''||the_item||'''
AND ai.param = ''BOOK''
AND ai.prod in (' || list || ');
OPEN query_results FOR query_str;
LOOP
FETCH query_results INTO
results_out.seq,
results_out.value,
results_out.item;
EXIT WHEN query_results%NOTFOUND;
PIPE ROW(results_out);
END LOOP;
CLOSE query_results;
END LOOP;
END;
Extra info from Alex's comment below useful for the answer:
you can have multiple loops from different sources, and as long as the
data from each be put into the same object type, you can just keep
pumping them out with pipe row statements anywhere in the function.
The caller sees them as a table with the rows in the order you pipe
them. Rather than call a procedure and get a result set as an output
parameter, you can query as select seq, value, item from
table(package.get_data_faster(a, b, c, d)), and of course you can
still have an order by clause if the order they're piped isn't what
you want.