Using local variable in psql cli or declare and append in procedure (Postgresql) [duplicate] - sql

Here's what I would like to do:
\set values foo,bar,baz
DO $$
DECLARE
value TEXT;
values TEXT[] := string_to_array(:'values', ',');
BEGIN
FOREACH value IN ARRAY values LOOP
raise notice 'v: %', value;
END LOOP;
END $$ LANGUAGE plpgsql;
Which results in the following error:
ERROR: syntax error at or near ":"
SELECT string_to_array(:'values', ',') INTO values...
^
Here's the solution I have currently, but it feels hacky:
\set values foo,bar,baz
PREPARE get_values AS SELECT string_to_array(:'values', ',');
DO $$
DECLARE
value TEXT;
values TEXT[];
BEGIN
EXECUTE 'EXECUTE get_values' INTO values;
FOREACH value IN ARRAY values LOOP
raise notice 'v: %', value;
END LOOP;
END $$ LANGUAGE plpgsql;

Answer
DO expects a string literal with plpgsql code. Symbols are not substituted inside strings in psql.
You could concatenate the whole string into a psql variable and then execute it.
How to concatenate psql variables?
Pretty multi-line format is not possible, because (per documentation):
But in any case, the arguments of a meta-command cannot continue
beyond the end of the line.
Simple example:
test=# \set value foo
test=# \set do 'BEGIN\n RAISE NOTICE ''v: %'', ' :'value' ';\nEND'
test=# DO :'do';
NOTICE: v: foo
Replace line breaks with \n (or remove them if you don't care for pretty format). Based on this adapted code:
DO
'
DECLARE
_val text;
_vals text[] := string_to_array(>>values<<, '','');
BEGIN
FOREACH _val IN ARRAY _vals
LOOP
RAISE NOTICE ''v: %'', _val;
END LOOP;
END
'
It looks like this:
test=# \set do 'DECLARE\n _val text;\n _vals text[] := string_to_array(' :'values' ', '','');\nBEGIN\n FOREACH _val IN ARRAY _vals\n LOOP\n RAISE NOTICE ''v: %'', _val;\n END LOOP;\nEND'
test=# DO :'do';
NOTICE: v: foo
NOTICE: v: bar
NOTICE: v: baz
DO
I added bold emphasis to the variable to make it easier to spot.
Related answer by #Pavel (ab)using a server session variable:
Referring to session variables (\set var='value') from PL/PGSQL
Alternative solutions
Prepared statement
Your current solution doesn't look that bad. I would simplify:
PREPARE get_values AS SELECT * FROM regexp_split_to_table(:'values', ',');
DO
$do$
DECLARE
_val text;
BEGIN
FOR _val IN EXECUTE
'EXECUTE get_values'
LOOP
RAISE NOTICE 'v: %', _val;
END LOOP;
END
$do$;
Temporary table
Similar solution with a temporary table:
CREATE TEMP TABLE tmp AS SELECT * FROM regexp_split_to_table(:'values', ',') v;
DO
$do$
DECLARE
_val text;
BEGIN
FOR _val IN
TABLE tmp
LOOP
RAISE NOTICE 'v: %', _val;
END LOOP;
END
$do$;

Was able to take advantage of this solution:
Passing argument to a psql procedural script
Where I set the variable as such and retrieve it with current_setting()
\set values foo,bar,baz
SET vars.values TO :'values';
DO $$
DECLARE
value TEXT;
values TEXT[] := string_to_array(current_setting('vars.values'), ',');
BEGIN
FOREACH value IN ARRAY values LOOP
RAISE NOTICE 'v: %', value;
END LOOP;
END $$ LANGUAGE plpgsql

Related

Evaluate dynamic condition in PL/pgSQL using EXECUTE

I have a function that needs to dynamically validate the input based on the value type. It does this by finding the constraint in another table, but for simplicity I provide the constraint as well in the function below.
This other table contains (value_type, value_constraint), where value_constraint is a text field that contains e.g. value::int > 0. I need to dynamically check this constraint within my insert function. I was trying to do that using EXECUTE as seen below, but it's not working.
How do I dynamically execute a condition statement and get the value as a boolean into v_successful_insert?
CREATE OR REPLACE FUNCTION insert_value(p_value_type text, p_value_constraint text, p_value text) RETURNS boolean
AS $$
DECLARE
v_successful_insert bool;
BEGIN
EXECUTE p_value_constraint INTO v_successful_insert;
IF v_successful_insert THEN
INSERT INTO my_table (value_type, value)
VALUES (p_value_type, p_value);
END IF;
RETURN v_successful_insert;
END;
$$
LANGUAGE plpgsql volatile;
The code is run on Postgresql 10.6.
You need to do a SELECT in the execute. For example:
DO $$
DECLARE
p_value TEXT := 'x';
p_value_constraint TEXT := '::int > 0';
result BOOLEAN;
BEGIN
BEGIN
EXECUTE 'SELECT $1' || p_value_constraint
INTO result
USING p_value;
EXCEPTION
WHEN INVALID_TEXT_REPRESENTATION THEN
result := FALSE;
END;
RAISE NOTICE '%', result;
END $$
Prints FALSE

How can I refer to a variable in postgresql dynamic SQL?

I'm trying to write a PostgreSQL function for table upserts that can be used for any table. My starting point is taken from a concrete function for a specific table type:
CREATE TABLE doodad(id BIGINT PRIMARY KEY, data JSON);
CREATE OR REPLACE FUNCTION upsert_doodad(d doodad) RETURNS VOID AS
$BODY$
BEGIN
LOOP
UPDATE doodad
SET id = (d).id, data = (d).data
WHERE id = (d).id;
IF found THEN
RETURN;
END IF;
-- does not exist, or was just deleted.
BEGIN
INSERT INTO doodad SELECT d.*;
RETURN;
EXCEPTION when UNIQUE_VIOLATION THEN
-- do nothing, and loop to try the update again
END;
END LOOP;
END;
$BODY$
LANGUAGE plpgsql;
The dynamic SQL version of this for any table that I've come up with is here:
SQL Fiddle
CREATE OR REPLACE FUNCTION upsert(target ANYELEMENT) RETURNS VOID AS
$$
DECLARE
attr_name NAME;
col TEXT;
selectors TEXT[];
setters TEXT[];
update_stmt TEXT;
insert_stmt TEXT;
BEGIN
FOR attr_name IN SELECT a.attname
FROM pg_index i
JOIN pg_attribute a ON a.attrelid = i.indrelid
AND a.attnum = ANY(i.indkey)
WHERE i.indrelid = format_type(pg_typeof(target), NULL)::regclass
AND i.indisprimary
LOOP
selectors := array_append(selectors, format('%1$s = target.%1$s', attr_name));
END LOOP;
FOR col IN SELECT json_object_keys(row_to_json(target))
LOOP
setters := array_append(setters, format('%1$s = (target).%1$s', col));
END LOOP;
update_stmt := format(
'UPDATE %s SET %s WHERE %s',
pg_typeof(target),
array_to_string(setters, ', '),
array_to_string(selectors, ' AND ')
);
insert_stmt := format('INSERT INTO %s SELECT (target).*', pg_typeof(target));
LOOP
EXECUTE update_stmt;
IF found THEN
RETURN;
END IF;
BEGIN
EXECUTE insert_stmt;
RETURN;
EXCEPTION when UNIQUE_VIOLATION THEN
-- do nothing
END;
END LOOP;
END;
$$
LANGUAGE plpgsql;
When I attempt to use this function, I get an error:
SELECT * FROM upsert(ROW(1,'{}')::doodad);
ERROR: column "target" does not exist: SELECT * FROM upsert(ROW(1,'{}')::doodad)
I tried changing the upsert statement to use placeholders, but I can't figure out how to invoke it using the record:
EXECUTE update_stmt USING target;
ERROR: there is no parameter $2: SELECT * FROM upsert(ROW(1,'{}')::doodad)
EXECUTE update_stmt USING target.*;
ERROR: query "SELECT target.*" returned 2 columns: SELECT * FROM upsert(ROW(1,'{}')::doodad)
I feel really close to a solution, but I can't figure out the syntax issues.
Short answer: you can't.
Variable substitution does not happen in the command string given to EXECUTE or one of its variants. If you need to insert a varying value into such a command, do so as part of constructing the string value, or use USING, as illustrated in Section 40.5.4. 1
Longer answer:
SQL statements and expressions within a PL/pgSQL function can refer to variables and parameters of the function. Behind the scenes, PL/pgSQL substitutes query parameters for such references. 2
This was the first important piece to the puzzle: PL/pgSQL does magic transformations on function parameters that turn them into variable substitutions.
The second was that fields of variable substitutions can referenced:
Parameters to a function can be composite types (complete table rows). In that case, the corresponding identifier $n will be a row variable, and fields can be selected from it, for example $1.user_id. 3
This excerpt confused me, because it referred to function parameters, but knowing that function parameters are implemented as variable substitutions under the hood, it seemed that I should be able to use the same syntax in EXECUTE.
These two facts unlocked the solution: use the ROW variable in the USING clause, and dereference its fields in the dynamic SQL. The results (SQL Fiddle):
CREATE OR REPLACE FUNCTION upsert(v_target ANYELEMENT)
RETURNS SETOF ANYELEMENT AS
$$
DECLARE
v_target_name TEXT;
v_attr_name NAME;
v_selectors TEXT[];
v_colname TEXT;
v_setters TEXT[];
v_update_stmt TEXT;
v_insert_stmt TEXT;
v_temp RECORD;
BEGIN
v_target_name := format_type(pg_typeof(v_target), NULL);
FOR v_attr_name IN SELECT a.attname
FROM pg_index i
JOIN pg_attribute a ON a.attrelid = i.indrelid
AND a.attnum = ANY(i.indkey)
WHERE i.indrelid = v_target_name::regclass
AND i.indisprimary
LOOP
v_selectors := array_append(v_selectors, format('t.%1$I = $1.%1$I', v_attr_name));
END LOOP;
FOR v_colname IN SELECT json_object_keys(row_to_json(v_target))
LOOP
v_setters := array_append(v_setters, format('%1$I = $1.%1$I', v_colname));
END LOOP;
v_update_stmt := format(
'UPDATE %I t SET %s WHERE %s RETURNING t.*',
v_target_name,
array_to_string(v_setters, ','),
array_to_string(v_selectors, ' AND ')
);
v_insert_stmt = format('INSERT INTO %I SELECT $1.*', v_target_name);
LOOP
EXECUTE v_update_stmt INTO v_temp USING v_target;
IF v_temp IS NOT NULL THEN
EXIT;
END IF;
BEGIN
EXECUTE v_insert_stmt USING v_target;
EXIT;
EXCEPTION when UNIQUE_VIOLATION THEN
-- do nothing
END;
END LOOP;
RETURN QUERY SELECT v_target.*;
END;
$$
LANGUAGE plpgsql;
For writeable CTE fans, this is trivially convertible to CTE form:
v_cte_stmt = format(
'WITH up as (%s) %s WHERE NOT EXISTS (SELECT 1 from up t WHERE %s)',
v_update_stmt,
v_insert_stmt,
array_to_string(v_selectors, ' AND '));
LOOP
BEGIN
EXECUTE v_cte_stmt USING v_target;
EXIT;
EXCEPTION when UNIQUE_VIOLATION THEN
-- do nothing
END;
END LOOP;
RETURN QUERY SELECT v_target.*;
NB: I have done zero performance testing on this solution, and I am relying on the analysis of others for its correctness. For now it appears to run correctly on PostgreSQL 9.3 in my development environment. YMMV.

How to iterate over a record when the columns are dynamic

I have this function in postgres which takes PVH_COLS_DYNA that contains the columns that are going in to the query:
CREATE OR REPLACE FUNCTION DRYNAMIC_DATA_F(PVH_COLS_DYNA VARCHAR) RETURNS numeric AS $$
DECLARE
VV_QUERY_DINAMIC VARCHAR;
VV_ROW_RECORD record;
BEGIN
VV_QUERY_DINAMIC:=' SELECT '|| PVH_COLS_DYNA ||' FROM as_detalle_carga WHERE fk_id_carga_cartera = 1234 ;';
FOR VV_ROW_RECORD IN EXECUTE VV_QUERY_DINAMIC LOOP
raise notice ' data % ', VV_ROW_RECORD.???????;
END LOOP;
return 1;
END;
$$ LANGUAGE plpgsql;
How can I get the data from the record variable VV_ROW_RECORD, since the columns are dynamic?
VV_ROW_RECORD.1
VV_ROW_RECORD.?1
VV_ROW_RECORD.[1]
VV_ROW_RECORD.?????
You cannot reference columns like array items, columns have to be referenced by name.
The dynamic part is not getting the row in your example, but referencing each column.
CREATE OR REPLACE FUNCTION dynamic_data_f(pvh_cols_dyna text)
RETURNS numeric AS
$func$
DECLARE
_row as_detalle_carga%ROWTYPE;
_col text;
_data text;
BEGIN
SELECT *
INTO _row
FROM as_detalle_carga
WHERE fk_id_carga_cartera = 1234;
FOREACH _col IN ARRAY string_to_array(pvh_cols_dyna, ',')
LOOP
EXECUTE format('SELECT ($1).%I::text', trim(_col))
USING _row
INTO _data;
RAISE NOTICE 'data: % ', _data;
END LOOP;
RETURN 1;
END
$func$ LANGUAGE plpgsql;
%I is an argument to format(), properly escaping identifiers as needed.
$1 in the query string for EXECUTE is a parameter filled in by the USING clause (not to be confused with function parameters!).
Related answers (with more explanation):
Postgres pl/pgsql ERROR: column "column_name" does not exist
Iterating over integer[] in PL/pgSQL
How to use EXECUTE FORMAT ... USING in postgres function
You cannot iterate record columns directly. You have to convert it first into something iterable, like json or hstore.
FOR vv_row_record IN EXECUTE vv_query_dynamic LOOP
FOR vv_row_record_pairs IN SELECT * FROM json_each(row_to_json(vv_row_record)) LOOP
RAISE NOTICE ' field "%" in json is % ',
vv_row_record_pairs.key,
vv_row_record_pairs.value;
END LOOP;
-- OR
FOR vv_row_record_pairs IN SELECT * FROM each(hstore(vv_row_record)) LOOP
RAISE NOTICE ' field "%" in text representation is % ',
vv_row_record_pairs.key,
vv_row_record_pairs.value;
END LOOP;
END LOOP;

pl/pgsql: How show values of NEW & OLD trigger record if i don't know the name of original table colum?

Hi all,
I wrote a function in pl/pgsql and I have this problem:
I want use the values of NEW and OLD trigger record but I don't know the column name and number if the table.
for example:
CREATE OR REPLACE FUNCTION tt() RETURNS trigger AS $$
DECLARE
text1 text;
text2 text;
orig_name text =tg_argv[0];
orig_schema text =tg_argv[1];
log_name text =tg_argv[2];
log_schema text =tg_argv[3];
col pg_attribute.attname%TYPE;
[...]
BEGIN
orig_comp := quote_ident(orig_schema)||'.'||quote_ident(orig_name);
log_comp := quote_ident(log_schema)||'.'||quote_ident(log_name);
IF(trigger_mode='INSERT')
THEN
-- I want know the names of column
FOR colonna in
SELECT attname
FROM pg_attribute
WHERE attrelid = orig_comp::regclass
AND attnum > 0
AND NOT attisdropped
loop --for each column I want save the value like a string
text1=NEW||'.'||colonna; -- this don't work: error: record NEW don't have colonna like values
text2:=text2||' , '||text1;
end loop;
[...]
END IF;
[...]
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
you know a original name - it is a variable TG_TABLE_NAME. And dynamic access to record fields is possible with EXECUTE USING statement.
CREATE OR REPLACE FUNCTION dynamic_trigger()
RETURNS TRIGGER
LANGUAGE plpgsql
AS $$
DECLARE
ri RECORD;
t TEXT;
BEGIN
RAISE NOTICE E'\n Operation: %\n Schema: %\n Table: %',
TG_OP,
TG_TABLE_SCHEMA,
TG_TABLE_NAME;
FOR ri IN
SELECT ordinal_position, column_name, data_type
FROM information_schema.columns
WHERE
table_schema = quote_ident(TG_TABLE_SCHEMA)
AND table_name = quote_ident(TG_TABLE_NAME)
ORDER BY ordinal_position
LOOP
EXECUTE 'SELECT ($1).' || ri.column_name || '::text' INTO t USING NEW;
RAISE NOTICE E'Column\n number: %\n name: %\n type: %\n value: %.',
ri.ordinal_position,
ri.column_name,
ri.data_type,
t;
END LOOP;
RETURN NEW;
END; $$;
This code is wrote by Tom Lane and it is from postgresql tricks pages Iteration over RECORD variable inside trigger.

Testing PostgreSQL functions that consume and return refcursor

I want to test results of a Postgres function (changing the function is not a possibility).
The function receives as arguments a REFCURSOR and several other things and returns the same RECURSOR.
get_function_that_returns_cursor(ret, 4100, 'SOMETHING', 123465)
Now I want to create a small test in Postgres to get the results of this FUNCTION.
Something Like the code below (this is my approach but it is not working):
DO $$ DECLARE
ret REFCURSOR;
row_to_read table_it_will_return%ROWTYPE ;
BEGIN
PERFORM get_function_that_returns_cursor(ret, 4100, 'SOMETHING', 123465);
-- OR SELECT get_function_that_returns_cursor(ret, 4100, 'SOMETHING', 123465) INTO ret
FOR row_to_read IN SELECT * FROM ret LOOP
-- (...)
RAISE NOTICE 'Row read...';
END LOOP;
CLOSE ret;
END $$;
Any suggestion on how to get this to work? A generic solution that can be used for testing this type of functions (that get a Cursor and return a Cursor?
And if we don't know the rowtype that is being returned how could we do it?
Q1
Your "small test" can be plain SQL:
BEGIN;
SELECT get_function_that_returns_cursor('ret', 4100, 'foo', 123); -- note: 'ret'
FETCH ALL IN ret; -- works for any rowtype
COMMIT; -- or ROLLBACK;
Execute COMMIT / ROLLBACK after you inspected the results. Most clients only display the result of the lat command.
More in the chapter Returning Cursors of the manual.
Q2
And if we don't know the rowtype that is being returned how could we do it?
Since you only want to inspect the results, you could cast the whole record to text.
This way you avoid the problem with dynamic return types for the function altogether.
Consider this demo:
CREATE TABLE a (a_id int PRIMARY KEY, a text);
INSERT INTO a VALUES (1, 'foo'), (2, 'bar');
CREATE OR REPLACE FUNCTION reffunc(INOUT ret refcursor)
LANGUAGE plpgsql AS
$func$
BEGIN
OPEN ret FOR SELECT * FROM a;
END
$func$;
CREATE OR REPLACE FUNCTION ctest()
RETURNS SETOF text
LANGUAGE plpgsql AS
$func$
DECLARE
curs1 refcursor;
rec record;
BEGIN
curs1 := reffunc('ret'); -- simple assignment
LOOP
FETCH curs1 INTO rec;
EXIT WHEN NOT FOUND; -- note the placement!
RETURN NEXT rec::text;
END LOOP;
END
$func$;
db<>fiddle here
Old sqlfiddle
This worked for what I wanted:
DO $$ DECLARE
mycursor REFCURSOR;
rec RECORD;
BEGIN
SELECT 'ret' INTO mycursor FROM get_function_that_returns_cursor('ret'::REFCURSOR, 4100, 'SOMETHING', 123465);
WHILE (FOUND) LOOP
FETCH mycursor INTO rec;
RAISE NOTICE 'Row read. Data: % ', rec.collumn_name;
END LOOP;
END $$