Let's say I want to create a new table from an existing table in SQL (postgres). I want the new table to have the same name as the old table but I want it to be in a different schema.
Is there a way to do this without having to repeat the name of the two tables (who share one name?)
Let's say the name of the original table is public.student
CREATE TABLE student(
student_id INT PRIMARY KEY,
last_name VARCHAR(30),
major VARCHAR(30))
Now I want to have the exact table but I want it to be in test.student
I know I would "clone" that table via
CREATE TABLE test.student AS
SELECT *
FROM public.student;
but I would like to write this without having to repeat writing "student".
Is there a way to write a function for this?
I'm quite new to SQL, so I'm thankful for any help! I looked into functions and I wasn't able to make it work.
You could create a procedure (or a function) with dynamic SQL:
CREATE OR REPLACE PROCEDURE foo(_schema text, _table text)
LANGUAGE plpgsql AS
$func$
BEGIN
EXECUTE format('CREATE TABLE %1$I.%2$I AS TABLE public.%2$I'
, _schema, _table);
END
$func$;
Call:
CALL foo('test', 'student');
Note that identifers are case sensitive here!
Be wary of possible SQL injection. format() with the format specifier %I (for identifier) is safe. (nested $1, $2 are ordinal references to format input)
See:
Define table and column names as arguments in a plpgsql function?
Table name as a PostgreSQL function parameter
Related
I have created a PL/pgSQL function that accepts two column names, a "relation", and two table names. It finds distinct rows in one table and inserts them in to a temporary table, deletes any row with a null value, and sets all values of one column to relation. I have the first part of the process using this function.
create or replace function alt_edger(s text, v text, relation text, tbl text, tbl_src text)
returns void
language plpgsql as
$func$
begin
raise notice 's: %, v: %, tbl: %, tbl_src: %', s,v,tbl,tbl_src;
execute ('insert into '||tbl||' ("source", "target") select distinct "'||s||'","'||v||'" from '||tbl_src||'');
execute ('DELETE FROM '||tbl||' WHERE "source" IS null or "target" is null');
end
$func$;
It is executed as follows:
-- create a temporary table and execute the function twice
drop table if exists temp_stack;
create temporary table temp_stack("label" text, "source" text, "target" text, "attr" text, "graph" text);
select alt_edger('x_x', 'y_y', ':associated_with', 'temp_stack','pg_check_table' );
select alt_edger('Document Number', 'x_x', ':documents', 'temp_stack','pg_check_table' );
select * from temp_stack;
Note that I didn't use relation, yet. The INSERT shall also assign relation, but I can't figure out how to make that happen to get something like:
label
source
target
attr
graph
:associated_with
638000
ARAS
:associated_with
202000
JASE
:associated_with
638010
JASE
:associated_with
638000
JASE
:associated_with
202100
JASE
:documents
A
638010
:documents
A
202000
:documents
A
202100
:documents
B
638000
:documents
A
638000
:documents
B
124004
:documents
B
202100
My challenges are:
How to integrate relation in the INSERT? When I try to use VALUES and comma separation I get an "error near select".
How to allow strings starting with ":" in relation? I'm anticipating here, the inclusion of the colon has given me challenges in the past.
How can I do this? Or is there a better approach?
Toy data model:
drop table if exists pg_check_table;
create temporary table pg_check_table("Document Number" text, x_x int, y_y text);
insert into pg_check_table values ('A',202000,'JASE'),
('A',202100,'JASE'),
('A',638010,'JASE'),
('A',Null,'JASE'),
('A',Null,'JASE'),
('A',202100,'JASE'),
('A',638000,'JASE'),
('A',202100,'JASE'),
('B',638000,'JASE'),
('B',202100,null),
('B',638000,'JASE'),
('B',null,'ARAS'),
('B',638000,'ARAS'),
('B',null,'ARAS'),
('B',638000,null),
('B',124004,null);
alter table pg_check_table add row_num serial;
select * from pg_check_table;
-- DROP FUNCTION alt_edger(_s text, _v text, _relation text, _tbl text, _tbl_src text)
CREATE OR REPLACE FUNCTION alt_edger(_s text, _v text, _relation text, _tbl text, _tbl_src text, OUT row_count int)
LANGUAGE plpgsql AS
$func$
DECLARE
_sql text := format(
'INSERT INTO pg_temp.%3$I (label, source, target)
SELECT DISTINCT $1, %1$I, %2$I FROM pg_temp.%4$I
WHERE (%1$I, %2$I) IS NOT NULL'
, _s, _v, _tbl, _tbl_src);
BEGIN
-- RAISE NOTICE '%', _sql; -- debug
EXECUTE _sql USING _relation;
GET DIAGNOSTICS row_count = ROW_COUNT; -- return number of inserted rows
END
$func$;
db<>fiddle here
Most importantly, use format() to concatenate your dynamic SQL commands safely. And use the format specifier %I for identifiers. This way, SQL injection is not possible and identifiers are double-quoted properly - preserving non-standard names like Document Number. That's where your original failed.
We could concatenate _relation as string to be inserted into label, too. But the preferable way to pass values to EXECUTE is with the USING clause. $1 inside the SQL string passed to EXECUTE is a placeholder for the first USING argument. Not to be confused with $1 referencing function parameters in the context of the function body outside EXECUTE! (You can pass any string, leading colon (:) does not matter, the string is not interpreted when done right.)
See:
Format specifier for integer variables in format() for EXECUTE?
Table name as a PostgreSQL function parameter
I replaced the DELETE in your original with a WHERE clause to the SELECT of the INSERT. Don't insert rows in the first place, instead of deleting them again later.
(%1$I, %2$I) IS NOT NULL only qualifies when both values are NOT NULL.
About that:
Check if a Postgres composite field is null/empty
Don't use the prefix "pg_" for your table names. That's what Postgres uses for system tables. Don't mess with those.
I schema-qualify known temporary tables with pg_temp. That's typically optional as the temporary schema comes first in the search_path by default. But that can be changed (maliciously), and then the table name would resolve to any existing regular table of the same name in the search_path. So better safe than sorry. See:
How does the search_path influence identifier resolution and the "current schema"
I made the function return the number of inserted rows. That's totally optional!
Since I do that with an OUT parameter, I am allowed to skip the RETURNS clause. See:
Can I make a plpgsql function return an integer without using a variable?
So I have a table in SQL server that is defined as such
create table test(value varchar(200), Reverse(value) as valueReverse);
now when I insert something in this table lets say I insert the string hello, it will store the value in the table as such.
value | valueReverse
--------------------
hello | olleh
I am trying to convert the table into PostgreSQL however the reverse() function is not working and it's giving me errors. What is the correct way to create this table in postgres?
For PostgreSQL 12 and above
If you are using Postgres 12 or higher then you can use GENERATED ALWAYS AS for the column valueReverse like below: Manual
create table test(value varchar(200),
valueReverse varchar(200) generated always as (reverse(value)) STORED );
DEMO
For PostgreSQL 11 or below
For earlier version you can use Triggers like below.
Creating Trigger Function
create or replace function trig_reverse() returns trigger as
$$
begin
new.valueReverse=reverse(new.value);
return new;
end;
$$
language plpgsql
Creating Trigger
create trigger trig_rev
before insert or update on test
for each row
execute procedure trig_reverse();
DEMO
Do not store string twice (redundantly). It will be much cleaner and cheaper overall to store it once and produce the reverted copy on the fly. You can use a VIEW if you need a drop-in replacement for your table:
CREATE TABLE base_test(value varchar(200));
INSERT INTO base_test VALUES ('hello');
CREATE VIEW test AS
SELECT *, reverse(value) AS value_reverse
FROM base_test;
db<>fiddle here
Related:
Computed / calculated / virtual / derived columns in PostgreSQL
Lets say we have a table A with 200 different columns. We want to select columns that contain a certain substring (e.g. the "host" substring in "host_id", "host_name", "average_host_rating"), and create a new table B with only those columns and their data imported from a .csv file.
I tried creating the new table manually, however this is not good practice and i want to improve the code by making it valid and functional even if i add more columns to table A.
Creating the table manually:
SELECT
listings.host_id,
listings.host_url ,
..
..
listings.host_name ,
listings.host_since ,
INTO host_table
FROM listings
WHERE TRUE;
Trying to create the table in a better way:
CREATE TABLE B AS
SELECT *
FROM A
WHERE A::text LIKE '%host%'
I expected it to create table B with every column that contains 'host' in its name however it returned an exact copy of table A (and all its data). I tried different ways and methods of creating new tables, however the problem always was that i could not isolate only the columns with the specified substring ('host').
What could be wrong in my syntax, way of thinking or anything else?
Thanks in advance!
You may create and call a function with parameters. The function will dynamically chose the columns from information_schema.columns.
Note that where false is used because you mentioned data will come from a csv file and not from the original table.
create or replace function
fn_gen_tab_text ( curr_tab_in text,tab_text_in TEXT, new_tab_in text)
RETURNS void AS
$body$
declare v_sql TEXT;
BEGIN
select 'CREATE TABLE %I AS select ' || string_agg(column_name,',')
||' from %I where false'
into v_sql from information_schema.columns
where table_name = curr_tab_in
and column_name like '%'||tab_text_in||'%';
EXECUTE format (v_sql,new_tab_in,curr_tab_in);
END $body$ language plpgsql
Call it as
select fn_gen_tab_text('host_table','host','new_table' );
DEMO
I want to call an existing procedure and store its table-typed OUT parameters to new physical tables, without having to repeat the definitions of the output types when creating the new tables. For example, if the procedure were
CREATE PROCEDURE MYPROC
(IN X INTEGER, OUT Y TABLE(A INTEGER, B DOUBLE, C NVARCHAR(25)))
LANGUAGE SQLSCRIPT AS BEGIN
...
END;
I would want to create a physical table for the output without repeating the (A INTEGER, B DOUBLE, C NVARCHAR(25)) part.
If I already had a table with the structure I want my result to have, I could CREATE TABLE MY_OUTPUT LIKE EXISTING_TABLE, but I don't.
If I already had a named type defined for the procedure's output type, I could create my table based on that type, but I don't.
If it were a subquery instead of a procedure output parameter, I could CREATE TABLE MY_OUTPUT AS (<subquery>), but it's not a subquery, and I don't know how to express it as a subquery. Also, there could be multiple output parameters, and I don't know how you'd make this work with multiple output parameters.
In my specific case, the functions come from the SAP HANA Predictive Analysis Library, so I don't have the option of changing how the functions are defined. Additionally, I suspect that PAL's unusually flexible handling of parameter types might prevent me from using solutions that would work for ordinary SQLScript procedures, but I'm still interested in solutions that would work for regular procedures, even if they fail on PAL.
Is there a way to do this?
It's possible, with limitations, to do this by using a SQLScript anonymous block:
DO BEGIN
CALL MYPROC(5, Y);
CREATE TABLE BLAH AS (SELECT * FROM :Y);
END;
We store the output to a table variable in the anonymous block, then create a physical table with data taken from the table variable. This even works with PAL! It's a lot of typing, though.
The limitation I've found is that the body of an anonymous block can't refer to local temporary tables created outside the anonymous block, so it's awkward to pass local temporary tables to the procedure this way. It's possible to do it anyway by passing the local temporary table as a parameter to the anonymous block itself, but that requires writing out the type of the local temporary table, and we were trying to avoid writing table types manually.
As far as I understand, you want to use your database tables as output parameter types.
In my default schema, I have a database table named CITY
I can create a stored procedure as follows using the table as output parameter type
CREATE PROCEDURE MyCityList (
OUT CITYLIST CITY
)
LANGUAGE SQLSCRIPT
AS
BEGIN
CITYLIST = SELECT * FROM CITY;
END;
After procedure is created, you can execute it as follows
do
begin
declare myList CITY;
call MyCityList(:myList);
select * from :myList;
end;
Here is the result where the output data is in a database table format, namely as CITY table
I hope this answers your question,
Update after first comment
If the scenario is the opposite as mentioned in the first comment, you can query system view PROCEDURE_PARAMETER_COLUMNS and create dynamic SQL statements that will generate tables with definitions in procedure table type parameters
Here is the SQL query
select
parameter_name,
'CREATE Column Table ' ||
procedure_name || '_'
|| parameter_name || ' ( ' ||
string_agg(
column_name || ' ' ||
data_type_name ||
case when data_type_name = 'INTEGER' then '' else
'(' || length || ')'
end
, ','
) || ' );'
from PROCEDURE_PARAMETER_COLUMNS
where
schema_name = 'A00077387'
group by procedure_name, parameter_name
You need to replace the WHERE clause according to your case.
Each line will have such an output
CREATE Column Table LISTCITIESBYCOUNTRYID_CITYLIST ( CITYID INTEGER,NAME NVARCHAR(40) );
The format for table name is concatenation of procedure name and parameter name
One last note, some data types integer, decimal, etc requires special code like excluding length or adding of scale , etc. Some are not handled in this SQL.
I'll try to enhance the query soon and publish an update
I've been defining the output of methods as...
CREATE TABLE person AS (
id serial PRIMARY KEY,
name text NOT NULL,
fingerprint text
);
CREATE TYPE person_output AS (
name text
);
CREATE OR REPLACE FUNCTION my_schema.create_person(_name text)
RETURNS person_output AS $$
DECLARE
output person_output;
BEGIN
INSERT INTO person (name) VALUES (_name) RETURNING name INTO person_output;
RETURN person_output;
END;
$$ LANGUAGE plpgsql;
I'm wondering if this rule of creating a type for what the function should output is standard within industry? Or, would it be better to create a view and just return 1 or raised errors depending if it succeeded or failed, and then calling the view in server-side python code to return?
I could see type-based outputs being good for creating "upgrades" like, CREATE TYPE person_output_v002 for the next-version, and creating a template for adapting, but then again, that wouldn't be needed if there were views.
The only reason I have decided not to use views is because I have things like groups of people and I'd like to create views for the groups, but I'd need to add a WHERE group_id = _GROUP_ID type of thing when selecting the view which I think isn't possible.
Might there be a way to select VIEWS with additional query parameters?
---- UPDATE ------
When there are certain outputs like current_user repeated as RETURN _current_user_output across multiple FUNCTIONs, is it a good idea to create one TYPE current_user_output so that we can be sure all FUNCTIONs which output the current_user are returning the same thing?
You do not need a row type for that at all. Not even a PL/pgSQL function. Simplify:
CREATE OR REPLACE FUNCTION my_schema.create_person(_name text)
RETURNS text AS
$func$
INSERT INTO person (name)
VALUES (_name)
RETURNING name;
$func$ LANGUAGE sql;
There is no rule of creating a type for what the function should output. The requirement is to declare the return type, and that's required by SQL which demands to know the type at execution time. To return multiple columns, you don't have to create a type. You can use RETURNS TABLE:
RETURNS TABLE (col1 int, col2 text, ...)
I suggest to read these two chapters of the manual:
Returning From a Function
CREATE FUNCTION
And you can use the readily defined row type of existing tables or polymorphic types:
Refactor a PL/pgSQL function to return the output of various SELECT queries
Added question:
... so that we can be sure all FUNCTIONs which output the current_user are returning the same thing?
If your functions just return text, you can be just as sure. It might make sense for row-types. Then it can be useful to define a composite type to share. Complicates maintenance. If you want to change the return type, you have to change alls functions at once. But that may be as intended ...