plpgsql function issue - sql

I have the following plpgsql procedure;
DECLARE
_r record;
point varchar[] := '{}';
i int := 0;
BEGIN
FOR _r IN EXECUTE ' SELECT a.'|| quote_ident(column) || ' AS point,
FROM ' || quote_ident (table) ||' AS a'
LOOP
point[i] = _r;
i = i+1;
END LOOP;
RETURN 'OK';
END;
Which its main objective is to traverse a table and store each value of the row in an array. I am still new to plpgsql. Can anyone point out is the error as it is giving me the following error;

This is the complete syntax (note that I renamed the parameter column to col_name as column is reserved word. The same goes for table)
create or replace function foo(col_name text, table_name text)
returns text
as
$body$
DECLARE
_r record;
point character varying[] := '{}';
i int := 0;
BEGIN
FOR _r IN EXECUTE 'SELECT a.'|| quote_ident(col_name) || ' AS pt, FROM ' || quote_ident (table_name) ||' AS a'
loop
point[i] = _r;
i = i+1;
END LOOP;
RETURN 'OK';
END;
$body$
language plpgsql;
Although to be honest: I fail so see what you are trying to achieve here.

#a_horse fixes most of the crippling problems with your failed attempt.
However, nobody should use this. The following step-by-step instructions should lead to a sane implementation with modern PostgreSQL.
Phase 1: Remove errors and mischief
Remove the comma after the SELECT list to fix the syntax error.
You start your array with 0, while the default is to start with 1. Only do this if you need to do it. Leads to unexpected results if you operate with array_upper() et al. Start with 1 instead.
Change RETURN type to varchar[] to return the assembled array and make this demo useful.
What we have so far:
CREATE OR REPLACE FUNCTION foo(tbl varchar, col varchar)
RETURNS varchar[] LANGUAGE plpgsql AS
$BODY$
DECLARE
_r record;
points varchar[] := '{}';
i int := 0;
BEGIN
FOR _r IN
EXECUTE 'SELECT a.'|| quote_ident(col) || ' AS pt
FROM ' || quote_ident (tbl) ||' AS a'
LOOP
i = i + 1; -- reversed order to make array start with 1
points[i] = _r;
END LOOP;
RETURN points;
END;
$BODY$;
Phase 2: Remove cruft, make it useful
Use text instead of character varying / varchar for simplicity. Either works, though.
You are selecting a single column, but use a variable of type record. This way a whole record is being coerced to text, which includes surrounding parenthesis. Hardly makes any sense. Use a text variable instead. Works for any column if you explicitly cast to text (::text). Any type can be cast to text.
There is no point in initializing the variable point. It can start as NULL here.
Table and column aliases inside EXECUTE are of no use in this case. Dynamically executed SQL has its own scope!.
No semicolon (;) needed after final END in a plpgsql function.
It's simpler to just append each value to the array with || .
Almost sane:
CREATE OR REPLACE FUNCTION foo1(tbl text, col text)
RETURNS text[] LANGUAGE plpgsql AS
$func$
DECLARE
point text;
points text[];
BEGIN
FOR point IN
EXECUTE 'SELECT '|| quote_ident(col) || '::text FROM ' || quote_ident(tbl)
LOOP
points = points || point;
END LOOP;
RETURN points;
END
$func$;
Phase 3: Make it shine in modern PL/pgSQL
If you pass a table name as text, you create an ambiguous situation. You can prevent SQLi just fine with format() or quote_ident(), but this will fail with tables outside your search_path.
Then you need to add schema-qualification, which creates an ambiguous value. 'x.y' could stand for the table name "x.y" or the schema-qualified table name "x"."y". You can't pass "x"."y" since that will be escaped into """x"".""y""". You'd need to either use an additional parameter for the schema name or one parameter of type regclass regclass is automatically quoted as need when coerced to text and is the elegant solution here.
The new format() is simpler than multiple (or even a single) quote_ident() call.
You did not specify any order. SELECT returns rows in arbitrary order without ORDER BY. This may seem stable, since the result is generally reproducible as long as the underlying table doesn't change. But that's 100% unreliable. You probably want to add some kind of ORDER BY.
Finally, you don't need to loop at all. Use a plain SELECT with an Array constructor.
Use an OUT parameter to further simplify the code
Proper solution:
CREATE OR REPLACE FUNCTION f_arr(tbl regclass, col text, OUT arr text[])
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE format('SELECT ARRAY(SELECT %I::text FROM %s ORDER BY 1)', col, tbl)
INTO arr;
END
$func$;
Call:
SELECT f_arr('myschema.mytbl', 'mycol');

Related

Control sort by in postgres SQL using parameter [duplicate]

How can I write a stored procedure that contains a dynamically built SQL statement that returns a result set? Here is my sample code:
CREATE OR REPLACE FUNCTION reporting.report_get_countries_new (
starts_with varchar,
ends_with varchar
)
RETURNS TABLE (
country_id integer,
country_name varchar
) AS
$body$
DECLARE
starts_with ALIAS FOR $1;
ends_with ALIAS FOR $2;
sql VARCHAR;
BEGIN
sql = 'SELECT * FROM lookups.countries WHERE lookups.countries.country_name >= ' || starts_with ;
IF ends_with IS NOT NULL THEN
sql = sql || ' AND lookups.countries.country_name <= ' || ends_with ;
END IF;
RETURN QUERY EXECUTE sql;
END;
$body$
LANGUAGE 'plpgsql'
VOLATILE
CALLED ON NULL INPUT
SECURITY INVOKER
COST 100 ROWS 1000;
This code returns an error:
ERROR: syntax error at or near "RETURN"
LINE 1: RETURN QUERY SELECT * FROM omnipay_lookups.countries WHERE o...
^
QUERY: RETURN QUERY SELECT * FROM omnipay_lookups.countries WHERE omnipay_lookups.countries.country_name >= r
CONTEXT: PL/pgSQL function "report_get_countries_new" line 14 at EXECUTE statement
I have tried other ways instead of this:
RETURN QUERY EXECUTE sql;
Way 1:
RETURN EXECUTE sql;
Way 2:
sql = 'RETURN QUERY SELECT * FROM....
/*later*/
EXECUTE sql;
In all cases without success.
Ultimately I want to write a stored procedure that contains a dynamic sql statement and that returns the result set from the dynamic sql statement.
There is room for improvements:
CREATE OR REPLACE FUNCTION report_get_countries_new (starts_with text
, ends_with text = NULL)
RETURNS SETOF lookups.countries AS
$func$
DECLARE
sql text := 'SELECT * FROM lookups.countries WHERE country_name >= $1';
BEGIN
IF ends_with IS NOT NULL THEN
sql := sql || ' AND country_name <= $2';
END IF;
RETURN QUERY EXECUTE sql
USING starts_with, ends_with;
END
$func$ LANGUAGE plpgsql;
-- the rest is default settings
Major points
PostgreSQL 8.4 introduced the USING clause for EXECUTE, which is useful for several reasons. Recap in the manual:
The command string can use parameter values, which are referenced in
the command as $1, $2, etc. These symbols refer to values supplied in
the USING clause. This method is often preferable to inserting data
values into the command string as text: it avoids run-time overhead of
converting the values to text and back, and it is much less prone to
SQL-injection attacks since there is no need for quoting or escaping.
IOW, it is safer and faster than building a query string with text representation of parameters, even when sanitized with quote_literal().
Note that $1, $2 in the query string refer to the supplied values in the USING clause, not to the function parameters.
While you return SELECT * FROM lookups.countries, you can simplify the RETURN declaration like demonstrated:
RETURNS SETOF lookups.countries
In PostgreSQL there is a composite type defined for every table automatically. Use it. The effect is that the function depends on the type and you get an error message if you try to alter the table. Drop & recreate the function in such a case.
This may or may not be desirable - generally it is! You want to be made aware of side effects if you alter tables. The way you have it, your function would break silently and raise an exception on it's next call.
If you provide an explicit default for the second parameter in the declaration like demonstrated, you can (but don't have to) simplify the call in case you don't want to set an upper bound with ends_with.
SELECT * FROM report_get_countries_new('Zaire');
instead of:
SELECT * FROM report_get_countries_new('Zaire', NULL);
Be aware of function overloading in this context.
Don't quote the language name 'plpgsql' even if that's tolerated (for now). It's an identifier.
You can assign a variable at declaration time. Saves an extra step.
Parameters are named in the header. Drop the nonsensical lines:
starts_with ALIAS FOR $1;
ends_with ALIAS FOR $2;
Use quote_literal() to avoid SQL injection (!!!) and fix your quoting problem:
CREATE OR REPLACE FUNCTION report_get_countries_new (
starts_with varchar,
ends_with varchar
)
RETURNS TABLE (
country_id integer,
country_name varchar
) AS
$body$
DECLARE
starts_with ALIAS FOR $1;
ends_with ALIAS FOR $2;
sql VARCHAR;
BEGIN
sql := 'SELECT * FROM lookups.countries WHERE lookups.countries.country_name ' || quote_literal(starts_with) ;
IF ends_with IS NOT NULL THEN
sql := sql || ' AND lookups.countries.country_name <= ' || quote_literal(ends_with) ;
END IF;
RETURN QUERY EXECUTE sql;
END;
$body$
LANGUAGE 'plpgsql'
VOLATILE
CALLED ON NULL INPUT
SECURITY INVOKER
COST 100 ROWS 1000;
This is tested in version 9.1, works fine.

How to include quote in plpgsql function

The following function identifies columns with null values. How can I extend the where clause to check null or empty value?
coalesce(TRIM(string), '') = ''
CREATE OR REPLACE FUNCTION public.is_column_empty(IN table_name varchar, IN column_name varchar)
RETURNS bool
LANGUAGE plpgsql
AS $function$
declare
count integer;
BEGIN
execute FORMAT('SELECT COUNT(*) from %s WHERE %s IS NOT NULL', table_name, quote_ident(column_name)) into count;
RETURN (count = 0);
END;
$function$
;
There are more possibilities - for example you can use custom string separators:
CREATE OR REPLACE FUNCTION public.is_column_empty(IN table_name varchar,
IN column_name varchar)
RETURNS bool
LANGUAGE plpgsql
AS $function$
DECLARE _found boolean; /* attention "count" is keyword */
BEGIN
EXECUTE format($_$SELECT EXISTS(SELECT * FROM %I WHERE COALESCE(trim(%I), '') <> '')$_$,
table_name, column_name)
INTO _found;
RETURN NOT _found;
END;
$function$;
your example has more issues:
don't use count where you really need to know number of rows (items). This can be pretty slow on bigger tables
Usually for keywords are used uppercase chars
don't use variable names that are SQL, PL/pgSQL keywords (reserved or unreserved), there can be some problems in some contexts (count, user, ...)
this is classic example of some chaos in data - you should to disallow empty strings in data. Then you can use index and the predicate COLNAME IS NOT NULL. It can be pretty fast.
You need to double up the quotation marks, like this:
CREATE OR REPLACE FUNCTION public.is_column_empty(IN table_name varchar, IN column_name varchar)
RETURNS bool
LANGUAGE plpgsql
AS $function$
declare
count integer;
BEGIN
execute FORMAT('SELECT COUNT(*) from %s WHERE COALESCE(TRIM(%s),'''') <> ''''', table_name, quote_ident(column_name)) into count;
RETURN (count = 0);
END;
$function$
;
EDIT:
Re-reading your question, I was a little unsure that you are getting what you want. As it stands the function returns false if at least one row has a value in the given column, even if all the other rows are empty. Is this really what you want, or are you rather looking for columns where any row has this column empty?

How to iterate over a record when the columns are dynamic

I have this function in postgres which takes PVH_COLS_DYNA that contains the columns that are going in to the query:
CREATE OR REPLACE FUNCTION DRYNAMIC_DATA_F(PVH_COLS_DYNA VARCHAR) RETURNS numeric AS $$
DECLARE
VV_QUERY_DINAMIC VARCHAR;
VV_ROW_RECORD record;
BEGIN
VV_QUERY_DINAMIC:=' SELECT '|| PVH_COLS_DYNA ||' FROM as_detalle_carga WHERE fk_id_carga_cartera = 1234 ;';
FOR VV_ROW_RECORD IN EXECUTE VV_QUERY_DINAMIC LOOP
raise notice ' data % ', VV_ROW_RECORD.???????;
END LOOP;
return 1;
END;
$$ LANGUAGE plpgsql;
How can I get the data from the record variable VV_ROW_RECORD, since the columns are dynamic?
VV_ROW_RECORD.1
VV_ROW_RECORD.?1
VV_ROW_RECORD.[1]
VV_ROW_RECORD.?????
You cannot reference columns like array items, columns have to be referenced by name.
The dynamic part is not getting the row in your example, but referencing each column.
CREATE OR REPLACE FUNCTION dynamic_data_f(pvh_cols_dyna text)
RETURNS numeric AS
$func$
DECLARE
_row as_detalle_carga%ROWTYPE;
_col text;
_data text;
BEGIN
SELECT *
INTO _row
FROM as_detalle_carga
WHERE fk_id_carga_cartera = 1234;
FOREACH _col IN ARRAY string_to_array(pvh_cols_dyna, ',')
LOOP
EXECUTE format('SELECT ($1).%I::text', trim(_col))
USING _row
INTO _data;
RAISE NOTICE 'data: % ', _data;
END LOOP;
RETURN 1;
END
$func$ LANGUAGE plpgsql;
%I is an argument to format(), properly escaping identifiers as needed.
$1 in the query string for EXECUTE is a parameter filled in by the USING clause (not to be confused with function parameters!).
Related answers (with more explanation):
Postgres pl/pgsql ERROR: column "column_name" does not exist
Iterating over integer[] in PL/pgSQL
How to use EXECUTE FORMAT ... USING in postgres function
You cannot iterate record columns directly. You have to convert it first into something iterable, like json or hstore.
FOR vv_row_record IN EXECUTE vv_query_dynamic LOOP
FOR vv_row_record_pairs IN SELECT * FROM json_each(row_to_json(vv_row_record)) LOOP
RAISE NOTICE ' field "%" in json is % ',
vv_row_record_pairs.key,
vv_row_record_pairs.value;
END LOOP;
-- OR
FOR vv_row_record_pairs IN SELECT * FROM each(hstore(vv_row_record)) LOOP
RAISE NOTICE ' field "%" in text representation is % ',
vv_row_record_pairs.key,
vv_row_record_pairs.value;
END LOOP;
END LOOP;

Count rows affected by DELETE

I use this code to verify the DELETE sentence, but I am sure you know a better way:
CREATE OR REPLACE FUNCTION my_schema.sp_delete_row_table(table_name character varying
, id_column character varying
, id_value integer)
RETURNS integer AS
$BODY$
DECLARE
BEFORE_ROWS integer;
AFTER_ROWS integer;
BEGIN
EXECUTE 'SELECT count(*) FROM ' || TABLE_NAME INTO BEFORE_ROWS;
EXECUTE 'DELETE FROM ' || TABLE_NAME || ' WHERE ' || ID_COLUMN || ' = ' || (ID_VALUE)::varchar;
EXECUTE 'SELECT count(*) FROM ' || TABLE_NAME INTO AFTER_ROWS;
IF BEFORE_ROWS - AFTER_ROWS = 1 THEN
RETURN 1;
ELSE
RETURN 2;
END IF;
EXCEPTION WHEN OTHERS THEN
RETURN 0;
END;
$BODY$
LANGUAGE plpgsql VOLATILE;
How to improve this code? I need it to work in Postgres 8.4, 9.1 and 9.2.
Actually, you cannot use FOUND with EXECUTE. The manual:
Note in particular that EXECUTE changes the output of GET DIAGNOSTICS,
but does not change FOUND.
There are a couple of other things that might be improved. First of all, your original is open to SQL injection. I suggest:
CREATE OR REPLACE FUNCTION my_schema.sp_delete_row_table(table_name regclass
, id_column text
, id_value int
, OUT del_ct int) AS
$func$
BEGIN
EXECUTE format ('DELETE FROM %s WHERE %I = $1', table_name, id_column);
USING id_value; -- assuming integer columns
GET DIAGNOSTICS del_ct = ROW_COUNT; -- directly assign OUT parameter
EXCEPTION WHEN OTHERS THEN
del_ct := 0;
END
$func$ LANGUAGE plpgsql;
format() requires Postgres 9.1 or later. You can replace it with string concatenation, but be sure to use escape the column name properly with quote_ident()!
The rest works for 8.4 as well.
Closely related answers:
Dynamic SQL (EXECUTE) as condition for IF statement
Table name as a PostgreSQL function parameter
Look into the variables called found and row_count:
http://www.postgresql.org/docs/current/static/plpgsql-statements.html#PLPGSQL-STATEMENTS-DIAGNOSTICS
found is true if any rows were affected. row_count gives you the number of affected rows.
IF FOUND THEN
GET DIAGNOSTICS integer_var = ROW_COUNT;
END IF;

DROP FUNCTION without knowing the number/type of parameters?

I keep all my functions in a text file with 'CREATE OR REPLACE FUNCTION somefunction'.
So if I add or change some function I just feed the file to psql.
Now if I add or remove parameters to an existing function, it creates an overload with the same name and to delete the original I need type in all the parameter types in the exact order which is kind of tedious.
Is there some kind of wildcard I can use to DROP all functions with a given name so I can just add DROP FUNCTION lines to the top of my file?
Basic query
This query creates all necessary DDL statements:
SELECT 'DROP FUNCTION ' || oid::regprocedure
FROM pg_proc
WHERE proname = 'my_function_name' -- name without schema-qualification
AND pg_function_is_visible(oid); -- restrict to current search_path
Output:
DROP FUNCTION my_function_name(string text, form text, maxlen integer);
DROP FUNCTION my_function_name(string text, form text);
DROP FUNCTION my_function_name(string text);
Execute the commands after checking plausibility.
Pass the function name case-sensitive and with no added double-quotes to match against pg_proc.proname.
The cast to the object identifier type regprocedure (oid::regprocedure), and then to text implicitly, produces function names with argument types, automatically double-quoted and schema-qualified according to the current search_path where needed. No SQL injection possible.
pg_function_is_visible(oid) restricts the selection to functions in the current search_path ("visible"). You may or may not want this.
If you have multiple functions of the same name in multiple schemas, or overloaded functions with various function arguments, all of those will be listed separately. You may want to restrict to specific schema(s) or specific function parameter(s).
Related:
When / how are default value expression functions bound with regard to search_path?
Function
You can build a plpgsql function around this to execute the statements immediately with EXECUTE. For Postgres 9.1 or later:
Careful! It drops your functions!
CREATE OR REPLACE FUNCTION f_delfunc(_name text, OUT functions_dropped int)
LANGUAGE plpgsql AS
$func$
-- drop all functions with given _name in the current search_path, regardless of function parameters
DECLARE
_sql text;
BEGIN
SELECT count(*)::int
, 'DROP FUNCTION ' || string_agg(oid::regprocedure::text, '; DROP FUNCTION ')
FROM pg_catalog.pg_proc
WHERE proname = _name
AND pg_function_is_visible(oid) -- restrict to current search_path
INTO functions_dropped, _sql; -- count only returned if subsequent DROPs succeed
IF functions_dropped > 0 THEN -- only if function(s) found
EXECUTE _sql;
END IF;
END
$func$;
Call:
SELECT f_delfunc('my_function_name');
The function returns the number of functions found and dropped if no exceptions are raised. 0 if none were found.
Further reading:
How does the search_path influence identifier resolution and the "current schema"
Truncating all tables in a Postgres database
PostgreSQL parameterized Order By / Limit in table function
For Postgres versions older than 9.1 or older variants of the function using regproc and pg_get_function_identity_arguments(oid) check the edit history of this answer.
You would need to write a function that took the function name, and looked up each overload with its parameter types from information_schema, then built and executed a DROP for each one.
EDIT: This turned out to be a lot harder than I thought. It looks like information_schema doesn't keep the necessary parameter information in its routines catalog. So you need to use PostgreSQL's supplementary tables pg_proc and pg_type:
CREATE OR REPLACE FUNCTION udf_dropfunction(functionname text)
RETURNS text AS
$BODY$
DECLARE
funcrow RECORD;
numfunctions smallint := 0;
numparameters int;
i int;
paramtext text;
BEGIN
FOR funcrow IN SELECT proargtypes FROM pg_proc WHERE proname = functionname LOOP
--for some reason array_upper is off by one for the oidvector type, hence the +1
numparameters = array_upper(funcrow.proargtypes, 1) + 1;
i = 0;
paramtext = '';
LOOP
IF i < numparameters THEN
IF i > 0 THEN
paramtext = paramtext || ', ';
END IF;
paramtext = paramtext || (SELECT typname FROM pg_type WHERE oid = funcrow.proargtypes[i]);
i = i + 1;
ELSE
EXIT;
END IF;
END LOOP;
EXECUTE 'DROP FUNCTION ' || functionname || '(' || paramtext || ');';
numfunctions = numfunctions + 1;
END LOOP;
RETURN 'Dropped ' || numfunctions || ' functions';
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
I successfully tested this on an overloaded function. It was thrown together pretty fast, but works fine as a utility function. I would recommend testing more before using it in practice, in case I overlooked something.
Improving original answer in order to take schema into account, ie. schema.my_function_name,
select
format('DROP FUNCTION %s(%s);',
p.oid::regproc, pg_get_function_identity_arguments(p.oid))
FROM pg_catalog.pg_proc p
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = p.pronamespace
WHERE
p.oid::regproc::text = 'schema.my_function_name';
Slightly enhanced version of Erwin's answer. Additionally supports following
'like' instead of exact function name match
can run in 'dry-mode' and 'trace' the SQL for removing of the functions
Code for copy/paste:
/**
* Removes all functions matching given function name mask
*
* #param p_name_mask Mask in SQL 'like' syntax
* #param p_opts Combination of comma|space separated options:
* trace - output SQL to be executed as 'NOTICE'
* dryrun - do not execute generated SQL
* #returns Generated SQL 'drop functions' string
*/
CREATE OR REPLACE FUNCTION mypg_drop_functions(IN p_name_mask text,
IN p_opts text = '')
RETURNS text LANGUAGE plpgsql AS $$
DECLARE
v_trace boolean;
v_dryrun boolean;
v_opts text[];
v_sql text;
BEGIN
if p_opts is null then
v_trace = false;
v_dryrun = false;
else
v_opts = regexp_split_to_array(p_opts, E'(\\s*,\\s*)|(\\s+)');
v_trace = ('trace' = any(v_opts));
v_dryrun = ('dry' = any(v_opts)) or ('dryrun' = any(v_opts));
end if;
select string_agg(format('DROP FUNCTION %s(%s);',
oid::regproc, pg_get_function_identity_arguments(oid)), E'\n')
from pg_proc
where proname like p_name_mask
into v_sql;
if v_sql is not null then
if v_trace then
raise notice E'\n%', v_sql;
end if;
if not v_dryrun then
execute v_sql;
end if;
end if;
return v_sql;
END $$;
select mypg_drop_functions('fn_dosomething_%', 'trace dryrun');
Here is the query I built on top of #Сухой27 solution that generates sql statements for dropping all the stored functions in a schema:
WITH f AS (SELECT specific_schema || '.' || ROUTINE_NAME AS func_name
FROM information_schema.routines
WHERE routine_type='FUNCTION' AND specific_schema='a3i')
SELECT
format('DROP FUNCTION %s(%s);',
p.oid::regproc, pg_get_function_identity_arguments(p.oid))
FROM pg_catalog.pg_proc p
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = p.pronamespace
WHERE
p.oid::regproc::text IN (SELECT func_name FROM f);
As of Postgres 10 you can drop functions by name only, as long as the names are unique to their schema. Just place the following declaration at the top of your function file:
drop function if exists my_func;
Documentation here.
pgsql generates an error if there exists more than one procedure with the same name but different arguments when the procedure is deleted according to its name. Thus if you want to delete a single procedure without affecting others then simply use the following query.
SELECT 'DROP FUNCTION ' || oid::regprocedure
FROM pg_proc
WHERE oid = {$proc_oid}