SQL: send query to all database available - sql

How is it possible to send a query to all databases on a server? I do not want to input all databases names, the script should auto-detect them.
example query:
SELECT SUM(tourney_results.amt_won)-SUM((tourney_summary.amt_buyin+tourney_summary.amt_fee)) as results
FROM tourney_results
INNER JOIN tourney_summary
ON tourney_results.id_tourney=tourney_summary.id_tourney
Where id_player=(SELECT id_player FROM player WHERE player_name='Apple');
So what I want to achieve here, if there is 2 databases, the first one would result 60, the second one would result 50, I need the 55 output here.
All databeses would have the same structure, tables etc.

You can do it using plpgsql and db_link. First install the db_link extension in the database you are connecting to:
CREATE EXTENSION dblink;
Then use a plpgsql function which iterates over all database on the server and executes the query. See this example (see comments inline). Note that I used a sample query in the function. You have to adapt the function with your real query:
CREATE or REPLACE FUNCTION test_dblink() RETURNS BIGINT AS
$$
DECLARE pg_database_row record;
query_result BIGINT;
_dbname TEXT;
_conn_name TEXT;
return_value BIGINT;
BEGIN
--initialize the final value
return_value = 0;
--first iterate over the records in the meta table pg_database
FOR pg_database_row in SELECT * FROM pg_database WHERE (NOT datistemplate) AND (datallowconn) LOOP
_dbname = pg_database_row.datname;
--build a connection name for db_link
_conn_name=_dbname||'myconn';
--close the connection is already active:
IF array_contains(dblink_get_connections(),_conn_name) THEN
PERFORM dblink_disconnect(_conn_name);
END IF;
-- open the connection with the actual database name
PERFORM dblink_connect(_dbname||'myconn', 'dbname='||_dbname);
-- check if the table does exist in the database:
PERFORM * FROM dblink(_conn_name,'SELECT 1 from pg_tables where tablename = ''your_table''') AS t(id int) ;
IF FOUND THEN
-- if the table exist, perform the query and save the result in a variable
SELECT * FROM dblink(_conn_name,'SELECT sum(id) FROM your_table limit 1') AS t(total int) INTO query_result;
IF query_result IS NOT NULL THEN
return_value = return_value + query_result;
END IF;
END IF;
PERFORM dblink_disconnect(_conn_name);
END LOOP;
RETURN return_value;
END;
$$
LANGUAGE 'plpgsql';
Execute the function with
select test_dblink();

Related

Procedure in Redshift Return "SELECT query has no destination for result data" Error

I keep getting the "SELECT query has no destination for result data" error upon calling this test procedure. What am I doing wrong? I did try adding the RETURN() command prior to SELECT statement but that didn't work either.
CREATE OR REPLACE PROCEDURE SchemaName.SP_Testing_Creating_Procedure (OUT ColumnName VARCHAR(9))
AS $$
BEGIN
SELECT TOP 10 ColumnName FROM SchemaName.TableName where ColumnName is not null;
END;
$$
LANGUAGE plpgsql;
CALL SchemaName.SP_Testing_Creating_Procedure();
As John mentioned you need to put the result into OUT column, examples of using IN, OUT and INOUT parameters you can find here
But if you need to return a few rows as a result, you have to use refcursor
as it's described here
CREATE OR REPLACE PROCEDURE SchemaName.SP_Testing_Creating_Procedure (INOUT result refcursor)
AS $$
BEGIN
OPEN result FOR
SELECT TOP 10 ColumnName
FROM SchemaName.TableName
WHERE ColumnName IS NOT null;
END;
$$
LANGUAGE plpgsql;
Then you can call the stored procedure in a transaction
BEGIN;
CALL logs.SP_Testing_Creating_Procedure('mycursor');
FETCH ALL FROM mycursor;
COMMIT;
another option is temp table which is also described in the above doc
Your procedure is running a SELECT command, but it is not doing anything with the results.
If your intention was to return a result set, you will need to put data in the OUT column.
See: Returning a result set - Amazon Redshift

Infer row type from table in postgresql

My application uses multiple schemas to partition tenants across the database to improve performance. I am trying to create a plpgsql function that will give me an arbitrary result set based on the union of all application schemas given a table. Here is what I have so far (inspired by this blog post):
CREATE OR REPLACE FUNCTION app_union(tbl text) RETURNS SETOF RECORD AS $$
DECLARE
schema RECORD;
sql TEXT := '';
BEGIN
FOR schema IN EXECUTE 'SELECT distinct schema FROM tenants' LOOP
sql := sql || format('SELECT * FROM %I.%I %s UNION ALL ', schema.schema, tbl);
END LOOP;
RETURN QUERY EXECUTE left(sql, -11);
END
$$ LANGUAGE plpgsql;
This works great, but has to be called with a row type definition at the end:
select * from app_union('my_table') t(id uuid, name text, ...);
So, how can I call my function without providing a row type?
I know that I can introspect my tables using information_schema.columns, but I'm not sure how to dynamically generate the type declaration without a lot of case statements (columns doesn't report the definition sql the way that e.g., pg_indexes does).
Even if I could dynamically generate the row declaration, it seems I would have to append it to my former function call as dynamic sql anyway, which sort of chicken/eggs the problem of returning a result set of an arbitrary type from a function.
Instead of providing the table as a string, you could provide it as type anyelement to specify the actual type of the returning data, then infer the table's name using pg_typeof. You can also use string_agg rather than a loop to build your sql:
CREATE OR REPLACE FUNCTION app_union(tbl anyelement)
RETURNS setof anyelement AS $$
BEGIN
return query execute string_agg(
distinct format('select * from %I.%I', schema, pg_typeof(tbl)::text),
' union all '
) from tenants;
END
$$ LANGUAGE plpgsql;
select * from app_union(null::my_table);
Simplified example

Postgres SQL query across different schemas

We have multiple schemas, I would like to run a simple count query across schemas such as:
SELECT COUNT(col_x) FROM schema1.table WHENRE col_x IS NOT NULL
I saw that I'm able to get all the schemas with:
SELECT schema_name FROM information_schema.schemata
So by using:
set search_path to schema1;
SELECT COUNT(col_x)
FROM table
WHERE col_x is not NULL;
I was able to run the query for schema1
The question is - is it possible to run in a loop and use the schema name as a parameter for search_path and run the query across all schemas? or any other efficient way to do so?
You will need some plpgsql and dynamic SQL for this. Here is an anonymous block for illustration:
do language plpgsql
$$
declare
v_schema_name text;
table_row_count bigint;
sysSchema text[] := array['pg_toast','pg_temp_1','pg_toast_temp_1','pg_catalog','public','information_schema'];
-- other declarations here
begin
for v_schema_name in SELECT schema_name FROM information_schema.schemata WHERE (schema_name != ALL(sysSchema)) loop
begin
execute format('select count(col_x) from %I.t_table', v_schema_name)
into table_row_count;
raise notice 'Schema % count %', v_schema_name, table_row_count;
exception when others then null; -- t_table may not exists in some schemata
end;
-- other statements here
end loop;
end;
$$;
And btw WHERE col_x is not NULL is redundant.

Get column names for a given table name from a function

I want to call table name manually input type then result should be table's details, I tried those function
1st function is working.
2nd function is not working.
1)
DECLARE
All_columns varchar;
Tab_name ALIAS FOR $1 ;
BEGIN
FOR All_columns IN SELECT column_name
FROM information_schema.columns
WHERE table_name=Tab_name
loop
raise notice 'Columns:%',All_columns;
end loop;
return All_columns;
END;
select test_levelfunction1('country_table');
It shows all columns of country table
2)
DECLARE
All_columns varchar ;
Tab_name ALIAS FOR $1 ;
BEGIN
FOR All_columns IN SELECT Tab_name.*
FROM Tab_name
loop
raise notice 'Columns:%',All_columns;
end loop;
return All_columns;
END;
The call select test_levelfunction1('country_table'); results in an error.
I need all the details from country_table.
How can I fix this function?
Neither function works, insofar as I read them. Or then you expect the first to return your input instead of column names.
You probably want to be using dynamic sql in both functions, e.g.:
EXECUTE $x$SELECT * FROM $x$ || Tab_name::regclass
http://www.postgresql.org/docs/current/static/plpgsql-statements.html
You can largely simplify this task. This SQL function does the job:
CREATE OR REPLACE FUNCTION f_columns_of_tbl(_tbl regclass)
RETURNS SETOF text AS
$func$
SELECT quote_ident(attname) AS col
FROM pg_attribute
WHERE attrelid = $1 -- valid, visible table name
AND attnum >= 1 -- exclude tableoid & friends
AND NOT attisdropped -- exclude dropped columns
ORDER BY attnum
$func$ LANGUAGE sql;
Call:
SELECT f_columns_of_tbl('myschema.mytable'); -- optionally schema-qualified name
For more details, links and a plpgsql version consider the related answer to your last question:
PLpgSQL function to find columns with only NULL values in a given table

DROP FUNCTION without knowing the number/type of parameters?

I keep all my functions in a text file with 'CREATE OR REPLACE FUNCTION somefunction'.
So if I add or change some function I just feed the file to psql.
Now if I add or remove parameters to an existing function, it creates an overload with the same name and to delete the original I need type in all the parameter types in the exact order which is kind of tedious.
Is there some kind of wildcard I can use to DROP all functions with a given name so I can just add DROP FUNCTION lines to the top of my file?
Basic query
This query creates all necessary DDL statements:
SELECT 'DROP FUNCTION ' || oid::regprocedure
FROM pg_proc
WHERE proname = 'my_function_name' -- name without schema-qualification
AND pg_function_is_visible(oid); -- restrict to current search_path
Output:
DROP FUNCTION my_function_name(string text, form text, maxlen integer);
DROP FUNCTION my_function_name(string text, form text);
DROP FUNCTION my_function_name(string text);
Execute the commands after checking plausibility.
Pass the function name case-sensitive and with no added double-quotes to match against pg_proc.proname.
The cast to the object identifier type regprocedure (oid::regprocedure), and then to text implicitly, produces function names with argument types, automatically double-quoted and schema-qualified according to the current search_path where needed. No SQL injection possible.
pg_function_is_visible(oid) restricts the selection to functions in the current search_path ("visible"). You may or may not want this.
If you have multiple functions of the same name in multiple schemas, or overloaded functions with various function arguments, all of those will be listed separately. You may want to restrict to specific schema(s) or specific function parameter(s).
Related:
When / how are default value expression functions bound with regard to search_path?
Function
You can build a plpgsql function around this to execute the statements immediately with EXECUTE. For Postgres 9.1 or later:
Careful! It drops your functions!
CREATE OR REPLACE FUNCTION f_delfunc(_name text, OUT functions_dropped int)
LANGUAGE plpgsql AS
$func$
-- drop all functions with given _name in the current search_path, regardless of function parameters
DECLARE
_sql text;
BEGIN
SELECT count(*)::int
, 'DROP FUNCTION ' || string_agg(oid::regprocedure::text, '; DROP FUNCTION ')
FROM pg_catalog.pg_proc
WHERE proname = _name
AND pg_function_is_visible(oid) -- restrict to current search_path
INTO functions_dropped, _sql; -- count only returned if subsequent DROPs succeed
IF functions_dropped > 0 THEN -- only if function(s) found
EXECUTE _sql;
END IF;
END
$func$;
Call:
SELECT f_delfunc('my_function_name');
The function returns the number of functions found and dropped if no exceptions are raised. 0 if none were found.
Further reading:
How does the search_path influence identifier resolution and the "current schema"
Truncating all tables in a Postgres database
PostgreSQL parameterized Order By / Limit in table function
For Postgres versions older than 9.1 or older variants of the function using regproc and pg_get_function_identity_arguments(oid) check the edit history of this answer.
You would need to write a function that took the function name, and looked up each overload with its parameter types from information_schema, then built and executed a DROP for each one.
EDIT: This turned out to be a lot harder than I thought. It looks like information_schema doesn't keep the necessary parameter information in its routines catalog. So you need to use PostgreSQL's supplementary tables pg_proc and pg_type:
CREATE OR REPLACE FUNCTION udf_dropfunction(functionname text)
RETURNS text AS
$BODY$
DECLARE
funcrow RECORD;
numfunctions smallint := 0;
numparameters int;
i int;
paramtext text;
BEGIN
FOR funcrow IN SELECT proargtypes FROM pg_proc WHERE proname = functionname LOOP
--for some reason array_upper is off by one for the oidvector type, hence the +1
numparameters = array_upper(funcrow.proargtypes, 1) + 1;
i = 0;
paramtext = '';
LOOP
IF i < numparameters THEN
IF i > 0 THEN
paramtext = paramtext || ', ';
END IF;
paramtext = paramtext || (SELECT typname FROM pg_type WHERE oid = funcrow.proargtypes[i]);
i = i + 1;
ELSE
EXIT;
END IF;
END LOOP;
EXECUTE 'DROP FUNCTION ' || functionname || '(' || paramtext || ');';
numfunctions = numfunctions + 1;
END LOOP;
RETURN 'Dropped ' || numfunctions || ' functions';
END;
$BODY$
LANGUAGE plpgsql VOLATILE
COST 100;
I successfully tested this on an overloaded function. It was thrown together pretty fast, but works fine as a utility function. I would recommend testing more before using it in practice, in case I overlooked something.
Improving original answer in order to take schema into account, ie. schema.my_function_name,
select
format('DROP FUNCTION %s(%s);',
p.oid::regproc, pg_get_function_identity_arguments(p.oid))
FROM pg_catalog.pg_proc p
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = p.pronamespace
WHERE
p.oid::regproc::text = 'schema.my_function_name';
Slightly enhanced version of Erwin's answer. Additionally supports following
'like' instead of exact function name match
can run in 'dry-mode' and 'trace' the SQL for removing of the functions
Code for copy/paste:
/**
* Removes all functions matching given function name mask
*
* #param p_name_mask Mask in SQL 'like' syntax
* #param p_opts Combination of comma|space separated options:
* trace - output SQL to be executed as 'NOTICE'
* dryrun - do not execute generated SQL
* #returns Generated SQL 'drop functions' string
*/
CREATE OR REPLACE FUNCTION mypg_drop_functions(IN p_name_mask text,
IN p_opts text = '')
RETURNS text LANGUAGE plpgsql AS $$
DECLARE
v_trace boolean;
v_dryrun boolean;
v_opts text[];
v_sql text;
BEGIN
if p_opts is null then
v_trace = false;
v_dryrun = false;
else
v_opts = regexp_split_to_array(p_opts, E'(\\s*,\\s*)|(\\s+)');
v_trace = ('trace' = any(v_opts));
v_dryrun = ('dry' = any(v_opts)) or ('dryrun' = any(v_opts));
end if;
select string_agg(format('DROP FUNCTION %s(%s);',
oid::regproc, pg_get_function_identity_arguments(oid)), E'\n')
from pg_proc
where proname like p_name_mask
into v_sql;
if v_sql is not null then
if v_trace then
raise notice E'\n%', v_sql;
end if;
if not v_dryrun then
execute v_sql;
end if;
end if;
return v_sql;
END $$;
select mypg_drop_functions('fn_dosomething_%', 'trace dryrun');
Here is the query I built on top of #Сухой27 solution that generates sql statements for dropping all the stored functions in a schema:
WITH f AS (SELECT specific_schema || '.' || ROUTINE_NAME AS func_name
FROM information_schema.routines
WHERE routine_type='FUNCTION' AND specific_schema='a3i')
SELECT
format('DROP FUNCTION %s(%s);',
p.oid::regproc, pg_get_function_identity_arguments(p.oid))
FROM pg_catalog.pg_proc p
LEFT JOIN pg_catalog.pg_namespace n ON n.oid = p.pronamespace
WHERE
p.oid::regproc::text IN (SELECT func_name FROM f);
As of Postgres 10 you can drop functions by name only, as long as the names are unique to their schema. Just place the following declaration at the top of your function file:
drop function if exists my_func;
Documentation here.
pgsql generates an error if there exists more than one procedure with the same name but different arguments when the procedure is deleted according to its name. Thus if you want to delete a single procedure without affecting others then simply use the following query.
SELECT 'DROP FUNCTION ' || oid::regprocedure
FROM pg_proc
WHERE oid = {$proc_oid}