Plpgsql; store all table names into an array - sql

My main purpose is actually, filtering all table names contain 'Messdaten' in it (for example "ID: 843063334 CH: 0001 Messdaten") and create new tables out of them with 'create table as' command as 'Backup_Messdaten1', 'Backup_Messdaten2', etc.
First I was trying to store all table names without filtering (there is maybe a way to retrieve all table names, contain 'Messdaten' in it by sql query, I don't know), and then storing the ones contain 'Messdaten' into another array and using that new array in the 'create table as' command.
But as I said my first goal is just to store all table names into an array;
Code itself;
CREATE OR REPLACE FUNCTION retrieve()
RETURNS text[] AS
$BODY$DECLARE
tbl_names text[];
BEGIN
tbl_names := array(SELECT table_name FROM information_schema.tables WHERE
table_schema='public' AND table_type='BASE TABLE');
SELECT tbl_names[i] FROM generate_subscripts(tbl_names, 1) g(i);
END;$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
ALTER FUNCTION retrieve()
OWNER TO postgres;
But for the code above, I am getting such an error;
Error;
ERROR: could not find array type for data type information_schema.sql_identifier
SQL state: 42704
Context: SQL statement "SELECT array(SELECT table_name FROM information_schema.tables WHERE table_schema='public' AND table_type='BASE TABLE')"
PL/pgSQL function retrieve() line 4 at assignment
Do you have any idea what is wrong with it and by the way I explained my main purpose, I would appreciate it If you point me to the right direction regarding that purpose.

SELECT array_agg(table_name::text)
FROM information_schema.tables
WHERE table_schema='public' AND table_type='BASE TABLE';
You need to cast the table name to text. The subquery is unnecessary, and you need to use array_agg not the array pseudo-function.
Personally I don't see why you need to aggregate them into an array at all, though. I'd just:
DECLARE
tablename text;
BEGIN
FOR tablename IN
SELECT table_name FROM information_schema.tables
WHERE table_schema='public' AND table_type='BASE TABLE'
AND ... my extra filters here ...
LOOP
EXECUTE format('CREATE TABLE %I AS TABLE %I', tablename || '_backup', tablename);
END LOOP;
END;

Your code contains more errors - basic error is missing any RETURN statement (for PL/pgSQL language). You can use SQL language too (see my example)
Postgres doesn't support arrays for some types - sql_identifier is one. You can try to use a casting to some basic type - in this case to text.
CREATE OR REPLACE FUNCTION names(filter text)
RETURNS text[] AS $$
SELECT array_agg(table_name::text)
FROM information_schema.tables
WHERE table_schema='public'
AND table_type='BASE TABLE' AND table_name LIKE $1;
$$ LANGUAGE sql;
postgres=# select names('foo%');
names
------------
{foo1,foo}
(1 row)

Related

Can I select data across multiple schemas within the same SQL database?

I have one database with multiple schemas. I would like to run SELECT * FROM info, across all schemas that starts with "team".
The schemas are not fixed, meaning that schemas are added and dropped continuously, so I can't hardcode the schemas in the query. But I'm only interested in schemas that starts with "team". How do I do this?
If all tables have an identical structure, you can write a PL/pgSQL function that does this:
create function get_info(p_schema_prefix text)
returns table (... column definitions go here ...)
as
$$
declare
l_rec record;
l_sql text;
begin
for l_rec in select table_schema, table_name
from information_schema.tables
where table_name = 'info'
and table_schema like p_schema_prefix||'%'
loop
l_sql := format('select id, data from %I.%I', l_rec.table_schema, l_rec.table_name);
return query execute l_sql;
end loop;
end;
$$
language plpgsql;
The use it like this:
select *
from get_info('team')

Truncate if exists in psql function and call function

I have the following code to create a function that truncates all rows from the table web_channel2 if the table is not empty:
create or replace function truncate_if_exists(tablename text)
returns void language plpgsql as $$
begin
select
from information_schema.tables
where table_name = tablename;
if found then
execute format('truncate %I', tablename);
end if;
end $$;
Unfortunately I don't know how should I continue ...
How to execute the function?
TLDR
To execute a Postgres function (returning void), call it with SELECT:
SELECT truncate_if_exists('web_channel2');
Proper solution
... how should I continue?
Delete the function again.
DROP FUNCTION truncate_if_exists(text);
It does not offer any way to schema-qualify the table. Using it might truncate the wrong table ...
Looks like you are trying to avoid an exception if the table is not there.
And you only want to truncate ...
if the table is not empty
To that end, I might use a safe function like this:
CREATE OR REPLACE FUNCTION public.truncate_if_exists(_table text, _schema text DEFAULT NULL)
RETURNS text
LANGUAGE plpgsql AS
$func$
DECLARE
_qual_tbl text := concat_ws('.', quote_ident(_schema), quote_ident(_table));
_row_found bool;
BEGIN
IF to_regclass(_qual_tbl) IS NOT NULL THEN -- table exists
EXECUTE 'SELECT EXISTS (SELECT FROM ' || _qual_tbl || ')'
INTO _row_found;
IF _row_found THEN -- table is not empty
EXECUTE 'TRUNCATE ' || _qual_tbl;
RETURN 'Table truncated: ' || _qual_tbl;
ELSE -- optional!
RETURN 'Table exists but is empty: ' || _qual_tbl;
END IF;
ELSE -- optional!
RETURN 'Table not found: ' || _qual_tbl;
END IF;
END
$func$;
To execute, call it with SELECT:
SELECT truncate_if_exists('web_channel2');
If no schema is provided, the function falls back to traversing the search_path - like your original did. If that's unreliable, or generally, to be safe (which seems prudent when truncating tables!) provide the schema explicitly:
SELECT truncate_if_exists('web_channel2', 'my_schema');
db<>fiddle here
When providing identifiers as strings, you need to use exact capitalization.
Why the custom variable _row_found instead of FOUND? See:
Dynamic SQL (EXECUTE) as condition for IF statement
Basics:
Table name as a PostgreSQL function parameter
How to check if a table exists in a given schema
PL/pgSQL checking if a row exists
How does the search_path influence identifier resolution and the "current schema"
Are PostgreSQL column names case-sensitive?

Access dynamic column name of row type in trigger function

I am trying to create a dynamic function to use for setting up triggers.
CREATE OR REPLACE FUNCTION device_bid_modifiers_count_per()
RETURNS TRIGGER AS
$$
DECLARE
devices_count INTEGER;
table_name regclass := TG_ARGV[0];
column_name VARCHAR := TG_ARGV[1];
BEGIN
LOCK TABLE device_types IN EXCLUSIVE MODE;
EXECUTE format('LOCK TABLE %s IN EXCLUSIVE MODE', table_name);
SELECT INTO devices_count device_types_count();
IF TG_OP = 'DELETE' THEN
SELECT format(
'PERFORM validate_bid_modifiers_count(%s, %s, OLD.%s, %s)',
table_name,
column_name,
column_name,
devices_count
);
ELSE
SELECT format(
'PERFORM validate_bid_modifiers_count(%s, %s, NEW.%s, %s)',
table_name,
column_name,
column_name,
devices_count
);
END IF;
RETURN NEW;
END;
$$ LANGUAGE plpgsql;
My issue is with the execution of the dynamic function validate_bid_modifiers_count(). Currently it throws:
ERROR: query has no destination for result data
HINT: If you want to discard the results of a SELECT, use PERFORM instead.
CONTEXT: PL/pgSQL function device_bid_modifiers_count_per() line 21 at SQL statement
I can't really wrap my head around this. I understand that format() returns the correct string of function call with arguments. How do I fix this and make it work?
This should do it:
CREATE OR REPLACE FUNCTION device_bid_modifiers_count_per()
RETURNS TRIGGER AS
$func$
DECLARE
devices_count int := device_types_count();
table_name regclass := TG_ARGV[0];
column_name text := TG_ARGV[1];
BEGIN
LOCK TABLE device_types IN EXCLUSIVE MODE;
EXECUTE format('LOCK TABLE %s IN EXCLUSIVE MODE', table_name);
IF TG_OP = 'DELETE' THEN
PERFORM validate_bid_modifiers_count(table_name
, column_name
, (row_to_json(OLD) ->> column_name)::bigint
, devices_count);
ELSE
PERFORM validate_bid_modifiers_count(table_name
, column_name
, (row_to_json(NEW) ->> column_name)::bigint
, devices_count);
END IF;
RETURN NEW;
END
$func$ LANGUAGE plpgsql;
The immediate cause for the error message was the outer SELECT. Without target, you need to replace it with PERFORM in plpgsql. But the inner PERFORM in the query string passed to EXECUTE was wrong, too. PERFORM is a plpgsql command, not valid in an SQL string passed to EXECUTE, which expects SQL code. You have to use SELECT there. Finally OLD and NEW are not visible inside EXECUTE and would each raise an exception of their own the way you had it. All issues are fixed by dropping EXECUTE.
A simple and fast way to get the value of a dynamic column name from the row types OLD and NEW: cast to json, then you can parameterize the key name like demonstrated. Should be a bit simpler and faster than the alternative with dynamic SQL - which is possible as well, like:
...
EXECUTE format('SELECT validate_bid_modifiers_count(table_name
, column_name
, ($1.%I)::bigint
, devices_count)', column_name)
USING OLD;
...
Related:
Get values from varying columns in a generic trigger
Trigger with dynamic field name
Aside: Not sure why you need the heavy locks.
Aside 2: Consider writing a separate trigger function for each trigger instead. More noisy DDL, but simpler and faster to execute.
As I pointed out in the comment to Erwin Brandstetter's answer, initially I have an almost identical solution.
But issue was that I was getting the error
ERROR: record "new" has no field "column_name"
CONTEXT: SQL statement "SELECT validate_bid_modifiers_count(table_name, column_name, NEW.column_name, devices_count)"
PL/pgSQL function device_bid_modifiers_count_per() line 15 at PERFORM
This is why I thought I needed a way to dynamically evaluate things.
Currently got this working with the following still ugly looking to me solution (ugly because I don't like 2 IF statements, I would like it to be super dynamic, but maybe I am asking for too much):
CREATE OR REPLACE FUNCTION device_bid_modifiers_count_per()
RETURNS TRIGGER AS
$func$
DECLARE
row RECORD;
table_name regclass := TG_ARGV[0];
column_name text := TG_ARGV[1];
devices_count INTEGER;
BEGIN
LOCK TABLE device_types IN EXCLUSIVE MODE;
EXECUTE format('LOCK TABLE %s IN EXCLUSIVE MODE', table_name);
devices_count := device_types_count();
IF TG_OP = 'DELETE' THEN
row := OLD;
ELSE
row := NEW;
END IF;
IF column_name = 'campaign_id' THEN
PERFORM validate_bid_modifiers_count(table_name, column_name, row.campaign_id, devices_count);
ELSIF column_name = 'adgroup_id' THEN
PERFORM validate_bid_modifiers_count(table_name, column_name, row.adgroup_id, devices_count);
ELSE
RAISE EXCEPTION 'invalid_column_name %', column_name;
END IF;
RETURN NEW;
END;
$func$ LANGUAGE plpgsql;
I am open to more robust solution suggestions.
Basically, the second condition kind'a almost defeats the purpose of having a single function, I could have at this point as well split it into two functions. Because the goal is to define multiple (2) triggers using this function (providing arguments to it).

Postgres dynamically select all text columns - subquery?

I want a select that returns all fields in a table that are of type = "character varying". It needs to run across multiple tables, so needs to be dynamic.
I was trying to use a subquery to first get the text columns, and then run the query:
SELECT (SELECT STRING_AGG(QUOTE_IDENT(column_name), ', ') FROM
information_schema.columns WHERE table_name = foo
AND data_type = 'character varying') FROM foo;
But that's not working, I just get a list of column names but not the values. Does anyone know how I can make it work or a better way to do it?
Thank you,
Ben
You need Pl/PgSQL for this, as PostgreSQL doesn't support dynamic SQL in its plain SQL dialect.
CREATE OR REPLACE FUNCTION get_cols(target_table text) RETURNS SETOF record AS $$
DECLARE
cols text;
BEGIN
cols := (SELECT STRING_AGG(QUOTE_IDENT(column_name), ', ')
FROM information_schema.columns
WHERE table_name = target_table
AND data_type = 'character varying');
RETURN QUERY EXECUTE 'SELECT '||cols||' FROM '||quote_ident(target_table)||';';
END;
$$
LANGUAGE plpgsql;
However, you'll find this hard to call, as you need to know the result column list to be able to call it. That kind of defeats the point. You'll need to massage the result into a concrete type. I convert to hstore here, but you could return json or an array or whatever, really:
CREATE OR REPLACE FUNCTION get_cols(target_table text) RETURNS SETOF hstore AS $$
DECLARE
cols text;
BEGIN
cols := (SELECT STRING_AGG(QUOTE_IDENT(column_name), ', ')
FROM information_schema.columns
WHERE table_name = target_table
AND data_type = 'character varying');
RETURN QUERY EXECUTE 'SELECT hstore(ROW('||cols||')) FROM '||quote_ident(target_table)||';';
END;
$$
LANGUAGE plpgsql;
Dynamic SQL is a pain, consider doing this at the application level.

Returning results from a function in 'select statement' format

I have a function that looks like this:
CREATE OR REPLACE FUNCTION mffcu.test_ty_hey()
RETURNS setof record
LANGUAGE plpgsql
AS $function$
Declare
cname1 text;
sql2 text;
Begin
for cname1 in
select array_to_string(useme, ', ') from (
select array_agg(column_name) as useme
from(
select column_name::text
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'crosstab_183'
and ordinal_position != 1
) as fin
) as fine
loop
sql2 := 'select distinct array['|| cname1 ||'] from mffcu.crosstab_183';
execute sql2;
end loop;
END;
$function$
I call the function with this:
select mffcu.test_ty_hey()
How do I return the results of the sql2 query without creating a table/temporary table?
While #Pavel is right, of course, your very convoluted function could be untangled to:
CREATE OR REPLACE FUNCTION mffcu.test_ty_hey()
RETURNS SETOF text[] LANGUAGE plpgsql
AS $func$
DECLARE
cname1 text;
BEGIN
FOR cname1 IN
SELECT column_name::text
FROM information_schema.columns
WHERE table_name = 'crosstab_183'
AND table_schema = 'mffcu'
AND ordinal_position <> 1
LOOP
RETURN QUERY
EXECUTE format('SELECT DISTINCT ARRAY[%I::text]
FROM mffcu.crosstab_183', cname1);
END LOOP;
END
$func$
format() requires PostgreSQL 9.1 or later. In 9.0 you can substitute with:
EXECUTE 'SELECT DISTINCT ARRAY['|| quote_ident(cname1) ||'::text]
FROM mffcu.crosstab_183';
Call:
select * FROM mffcu.test_ty_hey();
By casting each column to text we arrive at a consistent data type that can be used to declare the RETURN type. This compromise has to be made to return various data types from one function. Every data type can be cast to text, so that's the obvious common ground.
BTW, I have trouble imagining what the ARRAY wrapper around every single value should be good for. I suppose you could just drop that.
PostgreSQL functions should to have fixed result type before execution. You cannot specify type late in execution. There is only two workarounds - using temp tables or using cursors.
PLpgSQL language is not good for too generic routines - it good for implementation strict and clean business rules. And bad for generic crosstab calculations or generic auditing or similar generic task. It works, but code is slower and usually not well maintainable.
but reply for your query, you can use a output cursors
example http://okbob.blogspot.cz/2008/08/using-cursors-for-generating-cross.html