Consider this query.
SELECT app_label || '_' || model as name from django_content_type where id = 12;
name
-------------------
merc_benz
DJango people might have guessed, 'merc_benz' is a table name in same db. I want to write some SQL migrations and I need to select results from such dynamic table names.
How can i use variable name as a table name???
Something like this...(see RETURN QUERY EXECUTE in the plpgsql portion of the manual)
CREATE function dynamic_table_select(v_id int) returns setof text as $$
DECLARE
v_table_name text;
BEGIN
SELECT app_label || '_' || model into
v_table_name from django_content_type where id = v_id;
RETURN QUERY EXECUTE 'SELECT a_text_column from '||quote_ident(v_table_name);
RETURN;
END
$$ LANGUAGE plpgsql;
It becomes a little more complex if you want to return more than a single column of one type - either create a TYPE that is representative, or if you're using all the columns of a table, there's already a TYPE of that table name. You could also specify multiple OUT parameters.
http://www.postgresql.org/docs/8.1/static/ecpg-dynamic.html
The basic answer I think is EXECUTE IMMEDIATE
Related
How can I write a stored procedure that contains a dynamically built SQL statement that returns a result set? Here is my sample code:
CREATE OR REPLACE FUNCTION reporting.report_get_countries_new (
starts_with varchar,
ends_with varchar
)
RETURNS TABLE (
country_id integer,
country_name varchar
) AS
$body$
DECLARE
starts_with ALIAS FOR $1;
ends_with ALIAS FOR $2;
sql VARCHAR;
BEGIN
sql = 'SELECT * FROM lookups.countries WHERE lookups.countries.country_name >= ' || starts_with ;
IF ends_with IS NOT NULL THEN
sql = sql || ' AND lookups.countries.country_name <= ' || ends_with ;
END IF;
RETURN QUERY EXECUTE sql;
END;
$body$
LANGUAGE 'plpgsql'
VOLATILE
CALLED ON NULL INPUT
SECURITY INVOKER
COST 100 ROWS 1000;
This code returns an error:
ERROR: syntax error at or near "RETURN"
LINE 1: RETURN QUERY SELECT * FROM omnipay_lookups.countries WHERE o...
^
QUERY: RETURN QUERY SELECT * FROM omnipay_lookups.countries WHERE omnipay_lookups.countries.country_name >= r
CONTEXT: PL/pgSQL function "report_get_countries_new" line 14 at EXECUTE statement
I have tried other ways instead of this:
RETURN QUERY EXECUTE sql;
Way 1:
RETURN EXECUTE sql;
Way 2:
sql = 'RETURN QUERY SELECT * FROM....
/*later*/
EXECUTE sql;
In all cases without success.
Ultimately I want to write a stored procedure that contains a dynamic sql statement and that returns the result set from the dynamic sql statement.
There is room for improvements:
CREATE OR REPLACE FUNCTION report_get_countries_new (starts_with text
, ends_with text = NULL)
RETURNS SETOF lookups.countries AS
$func$
DECLARE
sql text := 'SELECT * FROM lookups.countries WHERE country_name >= $1';
BEGIN
IF ends_with IS NOT NULL THEN
sql := sql || ' AND country_name <= $2';
END IF;
RETURN QUERY EXECUTE sql
USING starts_with, ends_with;
END
$func$ LANGUAGE plpgsql;
-- the rest is default settings
Major points
PostgreSQL 8.4 introduced the USING clause for EXECUTE, which is useful for several reasons. Recap in the manual:
The command string can use parameter values, which are referenced in
the command as $1, $2, etc. These symbols refer to values supplied in
the USING clause. This method is often preferable to inserting data
values into the command string as text: it avoids run-time overhead of
converting the values to text and back, and it is much less prone to
SQL-injection attacks since there is no need for quoting or escaping.
IOW, it is safer and faster than building a query string with text representation of parameters, even when sanitized with quote_literal().
Note that $1, $2 in the query string refer to the supplied values in the USING clause, not to the function parameters.
While you return SELECT * FROM lookups.countries, you can simplify the RETURN declaration like demonstrated:
RETURNS SETOF lookups.countries
In PostgreSQL there is a composite type defined for every table automatically. Use it. The effect is that the function depends on the type and you get an error message if you try to alter the table. Drop & recreate the function in such a case.
This may or may not be desirable - generally it is! You want to be made aware of side effects if you alter tables. The way you have it, your function would break silently and raise an exception on it's next call.
If you provide an explicit default for the second parameter in the declaration like demonstrated, you can (but don't have to) simplify the call in case you don't want to set an upper bound with ends_with.
SELECT * FROM report_get_countries_new('Zaire');
instead of:
SELECT * FROM report_get_countries_new('Zaire', NULL);
Be aware of function overloading in this context.
Don't quote the language name 'plpgsql' even if that's tolerated (for now). It's an identifier.
You can assign a variable at declaration time. Saves an extra step.
Parameters are named in the header. Drop the nonsensical lines:
starts_with ALIAS FOR $1;
ends_with ALIAS FOR $2;
Use quote_literal() to avoid SQL injection (!!!) and fix your quoting problem:
CREATE OR REPLACE FUNCTION report_get_countries_new (
starts_with varchar,
ends_with varchar
)
RETURNS TABLE (
country_id integer,
country_name varchar
) AS
$body$
DECLARE
starts_with ALIAS FOR $1;
ends_with ALIAS FOR $2;
sql VARCHAR;
BEGIN
sql := 'SELECT * FROM lookups.countries WHERE lookups.countries.country_name ' || quote_literal(starts_with) ;
IF ends_with IS NOT NULL THEN
sql := sql || ' AND lookups.countries.country_name <= ' || quote_literal(ends_with) ;
END IF;
RETURN QUERY EXECUTE sql;
END;
$body$
LANGUAGE 'plpgsql'
VOLATILE
CALLED ON NULL INPUT
SECURITY INVOKER
COST 100 ROWS 1000;
This is tested in version 9.1, works fine.
My application uses multiple schemas to partition tenants across the database to improve performance. I am trying to create a plpgsql function that will give me an arbitrary result set based on the union of all application schemas given a table. Here is what I have so far (inspired by this blog post):
CREATE OR REPLACE FUNCTION app_union(tbl text) RETURNS SETOF RECORD AS $$
DECLARE
schema RECORD;
sql TEXT := '';
BEGIN
FOR schema IN EXECUTE 'SELECT distinct schema FROM tenants' LOOP
sql := sql || format('SELECT * FROM %I.%I %s UNION ALL ', schema.schema, tbl);
END LOOP;
RETURN QUERY EXECUTE left(sql, -11);
END
$$ LANGUAGE plpgsql;
This works great, but has to be called with a row type definition at the end:
select * from app_union('my_table') t(id uuid, name text, ...);
So, how can I call my function without providing a row type?
I know that I can introspect my tables using information_schema.columns, but I'm not sure how to dynamically generate the type declaration without a lot of case statements (columns doesn't report the definition sql the way that e.g., pg_indexes does).
Even if I could dynamically generate the row declaration, it seems I would have to append it to my former function call as dynamic sql anyway, which sort of chicken/eggs the problem of returning a result set of an arbitrary type from a function.
Instead of providing the table as a string, you could provide it as type anyelement to specify the actual type of the returning data, then infer the table's name using pg_typeof. You can also use string_agg rather than a loop to build your sql:
CREATE OR REPLACE FUNCTION app_union(tbl anyelement)
RETURNS setof anyelement AS $$
BEGIN
return query execute string_agg(
distinct format('select * from %I.%I', schema, pg_typeof(tbl)::text),
' union all '
) from tenants;
END
$$ LANGUAGE plpgsql;
select * from app_union(null::my_table);
Simplified example
I want a select that returns all fields in a table that are of type = "character varying". It needs to run across multiple tables, so needs to be dynamic.
I was trying to use a subquery to first get the text columns, and then run the query:
SELECT (SELECT STRING_AGG(QUOTE_IDENT(column_name), ', ') FROM
information_schema.columns WHERE table_name = foo
AND data_type = 'character varying') FROM foo;
But that's not working, I just get a list of column names but not the values. Does anyone know how I can make it work or a better way to do it?
Thank you,
Ben
You need Pl/PgSQL for this, as PostgreSQL doesn't support dynamic SQL in its plain SQL dialect.
CREATE OR REPLACE FUNCTION get_cols(target_table text) RETURNS SETOF record AS $$
DECLARE
cols text;
BEGIN
cols := (SELECT STRING_AGG(QUOTE_IDENT(column_name), ', ')
FROM information_schema.columns
WHERE table_name = target_table
AND data_type = 'character varying');
RETURN QUERY EXECUTE 'SELECT '||cols||' FROM '||quote_ident(target_table)||';';
END;
$$
LANGUAGE plpgsql;
However, you'll find this hard to call, as you need to know the result column list to be able to call it. That kind of defeats the point. You'll need to massage the result into a concrete type. I convert to hstore here, but you could return json or an array or whatever, really:
CREATE OR REPLACE FUNCTION get_cols(target_table text) RETURNS SETOF hstore AS $$
DECLARE
cols text;
BEGIN
cols := (SELECT STRING_AGG(QUOTE_IDENT(column_name), ', ')
FROM information_schema.columns
WHERE table_name = target_table
AND data_type = 'character varying');
RETURN QUERY EXECUTE 'SELECT hstore(ROW('||cols||')) FROM '||quote_ident(target_table)||';';
END;
$$
LANGUAGE plpgsql;
Dynamic SQL is a pain, consider doing this at the application level.
I want to call table name manually input type then result should be table's details, I tried those function
1st function is working.
2nd function is not working.
1)
DECLARE
All_columns varchar;
Tab_name ALIAS FOR $1 ;
BEGIN
FOR All_columns IN SELECT column_name
FROM information_schema.columns
WHERE table_name=Tab_name
loop
raise notice 'Columns:%',All_columns;
end loop;
return All_columns;
END;
select test_levelfunction1('country_table');
It shows all columns of country table
2)
DECLARE
All_columns varchar ;
Tab_name ALIAS FOR $1 ;
BEGIN
FOR All_columns IN SELECT Tab_name.*
FROM Tab_name
loop
raise notice 'Columns:%',All_columns;
end loop;
return All_columns;
END;
The call select test_levelfunction1('country_table'); results in an error.
I need all the details from country_table.
How can I fix this function?
Neither function works, insofar as I read them. Or then you expect the first to return your input instead of column names.
You probably want to be using dynamic sql in both functions, e.g.:
EXECUTE $x$SELECT * FROM $x$ || Tab_name::regclass
http://www.postgresql.org/docs/current/static/plpgsql-statements.html
You can largely simplify this task. This SQL function does the job:
CREATE OR REPLACE FUNCTION f_columns_of_tbl(_tbl regclass)
RETURNS SETOF text AS
$func$
SELECT quote_ident(attname) AS col
FROM pg_attribute
WHERE attrelid = $1 -- valid, visible table name
AND attnum >= 1 -- exclude tableoid & friends
AND NOT attisdropped -- exclude dropped columns
ORDER BY attnum
$func$ LANGUAGE sql;
Call:
SELECT f_columns_of_tbl('myschema.mytable'); -- optionally schema-qualified name
For more details, links and a plpgsql version consider the related answer to your last question:
PLpgSQL function to find columns with only NULL values in a given table
I want to create a postgres function that builds the set of columns it
returns on-the-fly; in short, it should take in a list of keys, build
one column per-key, and return a record consisting of whatever that set
of columns was. Briefly, here's the code:
CREATE OR REPLACE FUNCTION reports.get_activities_for_report() RETURNS int[] AS $F$
BEGIN
RETURN ARRAY(SELECT activity_id FROM public.activity WHERE activity_id NOT IN (1, 2));
END;
$F$
LANGUAGE plpgsql
STABLE;
CREATE OR REPLACE FUNCTION reports.get_amount_of_time_query(format TEXT, _activity_id INTEGER) RETURNS TEXT AS $F$
DECLARE
_label TEXT;
BEGIN
SELECT label INTO _label FROM public.activity WHERE activity_id = _activity_id;
IF _label IS NOT NULL THEN
IF lower(format) = 'percentage' THEN
RETURN $$TO_CHAR(100.0 *$$ ||
$$ (SUM(CASE WHEN activity_id = $$ || _activity_id || $$ THEN EXTRACT(EPOCH FROM ended - started) END) /$$ ||
$$ SUM(EXTRACT(EPOCH FROM ended - started))),$$ ||
$$ '990.99 %') AS $$ || quote_ident(_label);
ELSE
RETURN $$SUM(CASE WHEN activity_id = $$ || _activity_id || $$ THEN ended - started END)$$ ||
$$ AS $$ || quote_ident(_label);
END IF;
END IF;
END;
$F$
LANGUAGE plpgsql
STABLE;
CREATE OR REPLACE FUNCTION reports.build_activity_query(format TEXT, activities int[]) RETURNS TEXT AS $F$
DECLARE
_activity_id INT;
query TEXT;
_activity_count INT;
BEGIN
_activity_count := array_upper(activities, 1);
query := $$SELECT agent_id, portal_user_id, SUM(ended - started) AS total$$;
FOR i IN 1.._activity_count LOOP
_activity_id := activities[i];
query := query || ', ' || reports.get_amount_of_time_query(format, _activity_id);
END LOOP;
query := query || $$ FROM public.activity_log_final$$ ||
$$ LEFT JOIN agent USING (agent_id)$$ ||
$$ WHERE started::DATE BETWEEN actual_start_date AND actual_end_date$$ ||
$$ GROUP BY agent_id, portal_user_id$$ ||
$$ ORDER BY agent_id$$;
RETURN query;
END;
$F$
LANGUAGE plpgsql
STABLE;
CREATE OR REPLACE FUNCTION reports.get_agent_activity_breakdown(format TEXT, start_date DATE, end_date DATE) RETURNS SETOF RECORD AS $F$
DECLARE
actual_end_date DATE;
actual_start_date DATE;
query TEXT;
_rec RECORD;
BEGIN
actual_start_date := COALESCE(start_date, '1970-01-01'::DATE);
actual_end_date := COALESCE(end_date, now()::DATE);
query := reports.build_activity_query(format, reports.get_activities_for_report());
FOR _rec IN EXECUTE query LOOP
RETURN NEXT _rec;
END LOOP;
END
$F$
LANGUAGE plpgsql;
This builds queries that look (roughly) like this:
SELECT agent_id,
portal_user_id,
SUM(ended - started) AS total,
SUM(CASE WHEN activity_id = 3 THEN ended - started END) AS "Label 1"
SUM(CASE WHEN activity_id = 4 THEN ended - started END) AS "Label 2"
FROM public.activity_log_final
LEFT JOIN agent USING (agent_id)
WHERE started::DATE BETWEEN actual_start_date AND actual_end_date
GROUP BY agent_id, portal_user_id
ORDER BY agent_id
When I try to call the get_agent_activity_breakdown() function, I get this error:
psql:2009-10-22_agent_activity_report_test.sql:179: ERROR: a column definition list is required for functions returning "record"
CONTEXT: SQL statement "SELECT * FROM reports.get_agent_activity_breakdown('percentage', NULL, NULL)"
PL/pgSQL function "test_agent_activity" line 92 at SQL statement
The trick is, of course, that the columns labeled 'Label 1' and 'Label
2' are dependent on the set of activities defined in the contents of the
activity table, which I cannot predict when calling the function. How
can I create a function to access this information?
If you really want to create such table dynamically, maybe just create a temporary table within the function so it can have any columns you want. Let the function insert all rows into the table instead of returning them. The function can return only the name of the table or you can just have one exact table name that you know. After running that function you can just select data from the table. The function should also check if the temporary table exists so it should delete or truncate it.
Simon's answer might be better overall in the end, I'm just telling you how to do it without changing what you've got.
From the docs:
from_item can be one of:
...
function_name ( [ argument [, ...] ] ) [ AS ] alias [ ( column_alias [, ...] | column_definition [, ...] ) ]
function_name ( [ argument [, ...] ] ) AS ( column_definition [, ...] )
In other words, later it says:
If the function has been defined as
returning the record data type, then
an alias or the key word AS must be
present, followed by a column
definition list in the form (
column_name data_type [, ... ] ). The
column definition list must match the
actual number and types of columns
returned by the function.
I think the alias thing is only an option if you've predefined a type somewhere (like if you're mimicing the output of a predefined table, or have actually used CREATE TYPE...don't quote me on that, though.)
So, I think you would need something like:
SELECT *
FROM reports.get_agent_activity_breakdown('percentage', NULL, NULL)
AS (agent_id integer, portal_user_id integer, total something, ...)
The problem for you lies in the .... You'll need to know before you execute the query the names and types of all the columns--so you'll end up selecting on public.activity twice.
Both Simon's and Kev's answers are good ones, but what I ended up doing was splitting the calls to the database into two queries:
Build the query using the query constructor methods I included in the question, returning that to the application.
Call the query directly, and return that data.
This is safe in my case because the dynamic column list is not subject to frequent change, so I don't need to worry about the query's target data changing in between these calls. Otherwise, though, my method might not work.
you cannot change number of output columns, but you can to use refcursor, and you can return opened cursor.
more on http://okbob.blogspot.com/2008/08/using-cursors-for-generating-cross.html