I have one database with multiple schemas. I would like to run SELECT * FROM info, across all schemas that starts with "team".
The schemas are not fixed, meaning that schemas are added and dropped continuously, so I can't hardcode the schemas in the query. But I'm only interested in schemas that starts with "team". How do I do this?
If all tables have an identical structure, you can write a PL/pgSQL function that does this:
create function get_info(p_schema_prefix text)
returns table (... column definitions go here ...)
as
$$
declare
l_rec record;
l_sql text;
begin
for l_rec in select table_schema, table_name
from information_schema.tables
where table_name = 'info'
and table_schema like p_schema_prefix||'%'
loop
l_sql := format('select id, data from %I.%I', l_rec.table_schema, l_rec.table_name);
return query execute l_sql;
end loop;
end;
$$
language plpgsql;
The use it like this:
select *
from get_info('team')
Related
I'm using django-tenants, and for some tests I need to delete all schemas at once, so I was wondering how could I delete all schemas with a single sentence/script from postgresql shell, because deleting one by one is not scalable.
Thx so much.
For deleting all schemas you must use dynamic SQL. And schema names you can get from statistic system tables (example: information_schema). Example Query:
do
$body$
declare
f_rec record;
begin
for f_rec in
SELECT schema_name::text
FROM information_schema.schemata
where schema_name <> 'public'
loop
execute 'DROP SCHEMA ' || f_rec.schema_name || ' CASCADE';
end loop;
end;
$body$
language 'plpgsql';
We have multiple schemas, I would like to run a simple count query across schemas such as:
SELECT COUNT(col_x) FROM schema1.table WHENRE col_x IS NOT NULL
I saw that I'm able to get all the schemas with:
SELECT schema_name FROM information_schema.schemata
So by using:
set search_path to schema1;
SELECT COUNT(col_x)
FROM table
WHERE col_x is not NULL;
I was able to run the query for schema1
The question is - is it possible to run in a loop and use the schema name as a parameter for search_path and run the query across all schemas? or any other efficient way to do so?
You will need some plpgsql and dynamic SQL for this. Here is an anonymous block for illustration:
do language plpgsql
$$
declare
v_schema_name text;
table_row_count bigint;
sysSchema text[] := array['pg_toast','pg_temp_1','pg_toast_temp_1','pg_catalog','public','information_schema'];
-- other declarations here
begin
for v_schema_name in SELECT schema_name FROM information_schema.schemata WHERE (schema_name != ALL(sysSchema)) loop
begin
execute format('select count(col_x) from %I.t_table', v_schema_name)
into table_row_count;
raise notice 'Schema % count %', v_schema_name, table_row_count;
exception when others then null; -- t_table may not exists in some schemata
end;
-- other statements here
end loop;
end;
$$;
And btw WHERE col_x is not NULL is redundant.
I have a 9 digit number, say "234234234", is there a way to find or check its appearance in my database, like in which particular schema or procedure does it fall? and list out all the tables and columns which has that value in pl/sql developer
This query only searches in stored objects that user is allowed to access (procedure, function, package, ...). You could refer to this
Not sure if there is one way to search for that value in all database table.
SELECT *
FROM all_source
WHERE text LIKE '%234234234%';
--AND owner = 'SCHEMA_NAME';
The below block identifies the given string's presence across all the tables in your DB.
declare
num_rows number;
sql_text varchar2(250);
sql_info varchar2(100);
begin
dbms_output.enable(1000000);
for x in (select table_name, column_name from dba_tab_columns
where data_type in ('VARCHAR','VARCHAR2','CHAR')
and owner <> 'SYSTEM')
loop
sql_text:='select count(*) into :num_rows from SYSTEM.'||x.table_name||' where '||x.column_name||' like ''%234234234%''';
-- dbms_output.put_line (sql_text);
execute immediate sql_text into num_rows;
if num_rows>0
then
sql_info:='Table: '||x.table_name||' contains the string';
dbms_output.put_line (sql_info);
end if;
end loop;
end;
/
My main purpose is actually, filtering all table names contain 'Messdaten' in it (for example "ID: 843063334 CH: 0001 Messdaten") and create new tables out of them with 'create table as' command as 'Backup_Messdaten1', 'Backup_Messdaten2', etc.
First I was trying to store all table names without filtering (there is maybe a way to retrieve all table names, contain 'Messdaten' in it by sql query, I don't know), and then storing the ones contain 'Messdaten' into another array and using that new array in the 'create table as' command.
But as I said my first goal is just to store all table names into an array;
Code itself;
CREATE OR REPLACE FUNCTION retrieve()
RETURNS text[] AS
$BODY$DECLARE
tbl_names text[];
BEGIN
tbl_names := array(SELECT table_name FROM information_schema.tables WHERE
table_schema='public' AND table_type='BASE TABLE');
SELECT tbl_names[i] FROM generate_subscripts(tbl_names, 1) g(i);
END;$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
ALTER FUNCTION retrieve()
OWNER TO postgres;
But for the code above, I am getting such an error;
Error;
ERROR: could not find array type for data type information_schema.sql_identifier
SQL state: 42704
Context: SQL statement "SELECT array(SELECT table_name FROM information_schema.tables WHERE table_schema='public' AND table_type='BASE TABLE')"
PL/pgSQL function retrieve() line 4 at assignment
Do you have any idea what is wrong with it and by the way I explained my main purpose, I would appreciate it If you point me to the right direction regarding that purpose.
SELECT array_agg(table_name::text)
FROM information_schema.tables
WHERE table_schema='public' AND table_type='BASE TABLE';
You need to cast the table name to text. The subquery is unnecessary, and you need to use array_agg not the array pseudo-function.
Personally I don't see why you need to aggregate them into an array at all, though. I'd just:
DECLARE
tablename text;
BEGIN
FOR tablename IN
SELECT table_name FROM information_schema.tables
WHERE table_schema='public' AND table_type='BASE TABLE'
AND ... my extra filters here ...
LOOP
EXECUTE format('CREATE TABLE %I AS TABLE %I', tablename || '_backup', tablename);
END LOOP;
END;
Your code contains more errors - basic error is missing any RETURN statement (for PL/pgSQL language). You can use SQL language too (see my example)
Postgres doesn't support arrays for some types - sql_identifier is one. You can try to use a casting to some basic type - in this case to text.
CREATE OR REPLACE FUNCTION names(filter text)
RETURNS text[] AS $$
SELECT array_agg(table_name::text)
FROM information_schema.tables
WHERE table_schema='public'
AND table_type='BASE TABLE' AND table_name LIKE $1;
$$ LANGUAGE sql;
postgres=# select names('foo%');
names
------------
{foo1,foo}
(1 row)
I have a function that loops through specific schema names and inserts data into a table. I would like to be able to truncate said table before the insert loop occurs. I've tried putting the truncate statement inside of the dynamic query and that caused it to only keep schema's data inside of the table. I also tried declaring it as it's own variable and then executing the statement separately from the looping statement -- but that resulted in the same.
So my question is -- Where exactly would I put a truncate table dwh.prod_table_notify statement within this function? So that every time I run this function the table would be truncated and then the insert would properly loop through each schema being returned from the FOR statement.
NOTE: I'm forced to use postgres 8.2
CREATE OR REPLACE FUNCTION dwh.dim_table_notification()
RETURNS void
LANGUAGE plpgsql
AS $function$
Declare
myschema varchar;
sql2 text;
Begin
for myschema in
select distinct table_schema
from information_schema.tables
where table_name in ('dim_loan_type', 'dim_acct_type')
and table_schema NOT LIKE 'pg_%'
and table_schema NOT IN ('information_schema', 'ad_delivery', 'dwh', 'users', 'wand', 'ttd')
order by table_schema
loop
sql2 ='insert into dwh.prod_table_notify
select '''|| myschema ||''' as userid, loan_type_id as acct_type_id, loan_type::varchar(10) as acct_type, loan_type_desc::varchar(50) as acct_type_desc, term_code, 1 as loan_type from '|| myschema || '.' ||'dim_loan_type where term_code is null
union
select '''|| myschema ||''' as userid, acct_type_id, acct_type::varchar(10), acct_type_desc::varchar(50), term_code, 0 as loan_type from '|| myschema || '.' ||'dim_acct_type where term_code is null';
execute sql2;
end loop;
END;
$function$
CREATE OR REPLACE FUNCTION dwh.dim_table_notification()
RETURNS void LANGUAGE plpgsql AS
$func$
DECLARE
myschema text;
BEGIN
-- truncate simply goes here:
TRUNCATE dwh.prod_table_notify;
FOR myschema IN
SELECT quote_ident(table_schema)
FROM information_schema.tables
WHERE table_name IN ('dim_loan_type', 'dim_acct_type')
AND table_schema NOT LIKE 'pg_%'
AND table_schema NOT IN
('information_schema', 'ad_delivery', 'dwh', 'users', 'wand', 'ttd')
ORDER BY table_schema
LOOP
EXECUTE '
INSERT INTO dwh.prod_table_notify
(userid, acct_type_id, acct_type, acct_type_desc, loan_type)
SELECT '''|| myschema ||''', loan_type_id, loan_type::varchar(10)
, loan_type_desc::varchar(50), term_code, 1 AS loan_type
FROM '|| myschema || '.dim_loan_type
WHERE term_code IS NULL
UNION ALL
SELECT '''|| myschema ||''' AS userid, acct_type_id, acct_type::varchar(10)
, acct_type_desc::varchar(50), term_code, 0 AS loan_type
FROM '|| myschema || '.dim_acct_type
WHERE term_code IS NULL';
END LOOP;
END
$func$
Are you sure, you can actually use TRUNCATE? Quoting the manual for 8.2:
TRUNCATE cannot be used on a table that has foreign-key references from other tables, unless all such tables are also truncated in the same command.
If tables are small, DELETE is faster than TRUNCATE to begin with:
DELETE FROM dwh.prod_table_notify;
You have to sanitize identifiers! Use quote_ident(), also available in pg 8.2.
No point in using DISTINCT here.
Provide a column definition list for your INSERT. Else it can break in confusing ways, when you change the table later.
If rows in the two legs of the SELECT are unique, use UNION ALL instead of UNION. No point in trying to fold duplicates.