Drop all tables in a Redshift schema - without dropping permissions - sql

I would be interested to drop all tables in a Redshift schema. Even though this solution works
DROP SCHEMA public CASCADE;
CREATE SCHEMA public;
is NOT good for me since that it drops SCHEMA permissions as well.
A solution like
DO $$ DECLARE
r RECORD;
BEGIN
-- if the schema you operate on is not "current", you will want to
-- replace current_schema() in query with 'schematodeletetablesfrom'
-- *and* update the generate 'DROP...' accordingly.
FOR r IN (SELECT tablename FROM pg_tables WHERE schemaname = current_schema()) LOOP
EXECUTE 'DROP TABLE IF EXISTS ' || quote_ident(r.tablename) || ' CASCADE';
END LOOP;
END $$;
as reported in this thread How can I drop all the tables in a PostgreSQL database?
would be ideal. Unfortunately it doesn't work on Redshift (apparently there is no support for for loops).
Is there any other solution to achieve it?

Run this SQL and copy+paste the result on your SQL client.
If you want to do it programmatically you need to built little bit code around it.
SELECT 'DROP TABLE IF EXISTS ' || tablename || ' CASCADE;'
FROM pg_tables
WHERE schemaname = '<your_schema>'

I solved it through a procedure that deletes all records. Using this technique to truncate fails but deleting it works fine for my intents and purposes.
create or replace procedure sp_truncate_dwh() as $$
DECLARE
tables RECORD;
BEGIN
FOR tables in SELECT tablename
FROM pg_tables
WHERE schemaname = 'dwh'
order by tablename
LOOP
EXECUTE 'delete from dwh.' || quote_ident(tables.tablename) ;
END LOOP;
RETURN;
END;
$$ LANGUAGE plpgsql;
--call sp_truncate_dwh()

In addition to demircioglu's answer, I had to add Commit after every drop statement to drop all tables in my schema. SELECT 'DROP TABLE IF EXISTS ' || tablename || ' CASCADE; COMMIT;' FROM pg_tables WHERE schemaname = '<your_schema>'
P.S.: I do not have required reputation to add this note as a comment and had to add as an answer.

Using Python and pyscopg2 locally on my PC I came up with this script to delete all tables in schema:
import psycopg2
schema = "schema_to_be_deleted"
try:
conn = psycopg2.connect("dbname='{}' port='{}' host='{}' user='{}' password='{}'".format("DB_NAME", "DB_PORT", "DB_HOST", "DB_USER", "DB_PWD"))
cursor = conn.cursor()
cursor.execute("SELECT tablename FROM pg_tables WHERE schemaname = '%s'" % schema)
rows = cursor.fetchall()
for row in rows:
cursor.execute("DROP TABLE {}.{}".format(schema, row[0]))
cursor.close()
conn.commit()
except psycopg2.DatabaseError as error:
logger.error(error)
finally:
if conn is not None:
conn.close()
Replace correctly values for DB_NAME, DB_PORT, DB_HOST, DB_USER and DB_PWD to connect to the Redshift DB

The following recipe differs from other answers in the regard that it generates one SQL statement for all tables we're going to delete.
SELECT
'DROP TABLE ' ||
LISTAGG("table", ', ') ||
';'
FROM
svv_table_info
WHERE
"table" LIKE 'staging_%';
Example result:
DROP TABLE staging_077815128468462e9de8ca6fec22f284, staging_abc, staging_123;
As in other answers, you will need to copy the generated SQL and execute it separately.
References
|| operator concatenates strings
LISTAGG function concatenates every table name into a string with a separator
The table svv_table_info is used because LISTAGG doesn't want to work with pg_tables for me. Complaint:
One or more of the used functions must be applied on at least one user created tables. Examples of user table only functions are LISTAGG, MEDIAN, PERCENTILE_CONT, etc
UPD. I just now noticed that SVV_TABLE_INFO page says:
The SVV_TABLE_INFO view doesn't return any information for empty tables.
...which means empty tables will not be in the list returned by this query. I usually delete transient tables to save disk space, so this does not bother me much; but in general this factor should be considered.

Related

Get DDL of all tables under schema in one database in Snowflake

Can anyone tell me a query to DDL of all tables under schema in database.
I know how to get a DDL of a table.
select get_ddl('table', 'ods.users');
It gives us only one table DDL. But in my I have around 40 tables.
I want to get all tables DDL at a time instead of getting one by one. Is there any query available. If so please guide me.
Regards,
Kathija.
GET_DDL can script entire schema(with all objects inside):
select get_ddl('SCHEMA', 'ods');
select get_ddl('SCHEMA', 'ods', true); -- to get qualified names
With Snowflake Scripting block it is possible to list over tables/views/functions/... only:
DECLARE
CUR CURSOR FOR SELECT CONCAT_WS('.',TABLE_CATALOG,TABLE_SCHEMA,TABLE_NAME) AS name
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_TYPE = 'BASE TABLE'
AND TABLE_SCHEMA ILIKE 'ODS';
BEGIN
CREATE OR REPLACE TEMPORARY TABLE temp_view_defs(view_name TEXT, definition TEXT);
FOR rec IN CUR DO
EXECUTE IMMEDIATE REPLACE('INSERT INTO temp_view_defs(view_name, definition)
SELECT ''<view_name>'', GET_DDL(''TABLE'', ''<view_name>'')'
,'<view_name>'
,rec.name);
END FOR;
LET rs RESULTSET := (SELECT * FROM temp_view_defs);
RETURN TABLE(rs);
END;
For sample:
CREATE SCHEMA ODS;
CREATE TABLE ODS.TAB1(i INT);
CREATE TABLE ODS.TAB2(id INT DEFAULT 0) COMMENT = 'Test';
Output:

Oracle SQL only drop a column if a table exists

For Microsoft SQL Server I have following statement to only drop a column if the table exist.
IF EXISTS(SELECT 1
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'TEST_TABLE')
ALTER TABLE TEST_TABLE DROP COLUMN LEGACY_VALUE
GO
I was wondering if there was a related IF-EXISTS mechanism is present in Oracle.
All the metadata about the columns in Oracle Database is accessible using one of the following views.
user_tab_cols; -- For all tables owned by the user
all_tab_cols ; -- For all tables accessible to the user
dba_tab_cols; -- For all tables in the Database.
So, if you are looking for a column that exists and want to drop it, your code may look something like this ( see below).
Since this appears to be a one time task, is the effort really worth it?
DECLARE
v_column_exists number := 0;
BEGIN
Select count(*) into v_column_exists
from user_tab_cols
where upper(column_name) = 'LEGACY_VALUE''
and upper(table_name) = 'TEST_TABLE';
--and owner = 'SCOTT --*might be required if you are using all/dba views
if (v_column_exists = 1) then
execute immediate 'alter table test_table drop column legacy_value';
end if;
end;
/

Postgresql - Union table values from different schemas in a database into one table

I have a Database called Knowledge in postgres. It has multiple schemas and every schema has same number of tables, table has same columns as well.
Now I want to create a new schema called Aggregate, table called aggregate.table1 and put values from schema1.table1 and schema2.table1 in it.
I need to add another column in Aggregate.table1 which holds the value representing schema.
If any value in schema1.table1 is updated then aggregate.table1 should get the updated values.
Question,
Is it possible in Postgresql? if so please help me with this.
I need this aggregated table for further processing
You can try writing an anonymous code block to iterate over all schemas and tables, so that you can import your data into the aggregate schema. The following block search for all tables contained in the schemas s1 and s2, creates a corresponding table in the schema s_agg and finally copies its records.
DO $$
DECLARE row record;
BEGIN
FOR row IN SELECT * FROM pg_tables WHERE schemaname IN ('s1','s2') LOOP
EXECUTE 'CREATE TABLE IF NOT EXISTS s_agg.'||quote_ident(row.tablename)||
' AS TABLE ' || quote_ident(row.schemaname)||'.'|| quote_ident(row.tablename) ||
' WITH NO DATA;';
EXECUTE 'INSERT INTO s_agg.' || quote_ident(row.tablename)
|| ' SELECT * FROM '||quote_ident(row.schemaname)||'.'||quote_ident(row.tablename);
END LOOP;
END;
$$;
Demo
CREATE SCHEMA s1;
CREATE SCHEMA s2;
CREATE SCHEMA s_agg;
CREATE TABLE s1.t1 (id int);
INSERT INTO s1.t1 VALUES (1);
CREATE TABLE s2.t1 (id int);
INSERT INTO s2.t1 VALUES (42);
DO $$
DECLARE row record;
BEGIN
FOR row IN SELECT * FROM pg_tables WHERE schemaname IN ('s1','s2') LOOP
EXECUTE 'CREATE TABLE IF NOT EXISTS s_agg.'||quote_ident(row.tablename)||
' AS TABLE ' || quote_ident(row.schemaname)||'.'|| quote_ident(row.tablename) ||
' WITH NO DATA;';
EXECUTE 'INSERT INTO s_agg.' || quote_ident(row.tablename)
|| ' SELECT * FROM '||quote_ident(row.schemaname)||'.'||quote_ident(row.tablename);
END LOOP;
END;
$$;
-- contains values of t1 from s1 and s2
SELECT * FROM s_agg.t1;
id
----
1
42
Note: This code works with the assumption that the aggregate schema is either empty or it has empty tables, otherwise data will be duplicated. If you run this periodically and the size of your tables isn't too large, you can add a DROP TABLE before the CREATE TABLE statement. To make it work on every commit on all tables of all schemas you have to take a look at TRIGGERS or even logical replication.

How to delete every table in a specific schema in postgres?

How do I delete all the tables I have in a specific schema? Only the tables in the schema should be deleted.
I already have all the table names that I fetched with the code below, but how do delete all those tables?
The following is some psycopg2 code, and below that is the SQL generated
writeCon.execute("SELECT table_name FROM information_schema.tables WHERE table_schema='mySchema'")
SELECT table_name FROM information_schema.tables WHERE table_schema='mySchema'
You can use an anonymous code block for that.
WARNING: This code is playing with DROP TABLE statements, and they are really mean if you make a mistake ;) The CASCADE option drops all depending objects as well. Use it with care!
DO $$
DECLARE
row record;
BEGIN
FOR row IN SELECT * FROM pg_tables WHERE schemaname = 'mySchema'
LOOP
EXECUTE 'DROP TABLE mySchema.' || quote_ident(row.tablename) || ' CASCADE';
END LOOP;
END;
$$;
In case you want to drop everything in your schema, including wrappers, sequences, etc., consider dropping the schema itself and creating it again:
DROP SCHEMA mySchema CASCADE;
CREATE SCHEMA mySchema;
For a single-line command, you can use psql and its \gexec functionality:
SELECT format('DROP TABLE %I.%I', table_schema, table_name)
FROM information_schema.tables
WHERE table_schema= 'mySchema';\gexec
That will run the query and execute each result string as SQL command.

PostgreSQL Function returning result set from dynamic tables names

In my database, I have the standard app tables and backup tables. Eg. for a table "employee", I have a table called "bak_employee". The bak_employee table is a backup of the employee table. I use it to restore the employee table between tests.
I'd figure I can use these "bak_" tables to see the changes that have occurred during the test like this:
SELECT * FROM employee EXCEPT SELECT * FROM bak_employee
This will show me the inserted and updated records. I'll ignore the deleted records for now.
Now, what I would like to do is go through all the tables in my database to see if there's any changes in any of the tables. I was thinking of doing this as a function so it's easy to call over and over. This is what I have so far:
CREATE OR REPLACE FUNCTION public.show_diff()
RETURNS SETOF diff_tables AS
$BODY$
DECLARE
app_tables text;
BEGIN
FOR app_tables IN
SELECT table_name
FROM information_schema.tables
WHERE table_catalog = 'myDatabase'
AND table_schema = 'public'
AND table_name not like 'bak_%' -- exclude existing backup tables
LOOP
-- somehow loop through tables to see what's changed something like:
EXECUTE 'SELECT * FROM ' || app_tables || ' EXCEPT SELECT * FROM bak_' || app_tables;
END LOOP;
RETURN;
END;
$BODY$
LANGUAGE plpgsql;
But obviously this isn't going to return me any useful information. Any help would be appreciated.
You cannot return various well-known row types from the same function in the same call. A cheap fix is to cast each row type to text, so we have a common return type.
CREATE OR REPLACE FUNCTION public.show_diff()
RETURNS SETOF text AS -- text!!
$func$
DECLARE
app_table text;
BEGIN
FOR app_table IN
SELECT table_name
FROM information_schema.tables
WHERE table_catalog = 'myDatabase'
AND table_schema = 'public'
AND table_name NOT LIKE 'bak_%' -- exclude existing backup tables
LOOP
RETURN NEXT ' ';
RETURN NEXT '=== ' || app_table || ' ===';
RETURN QUERY EXECUTE format(
'SELECT x::text FROM (TABLE %I EXCEPT ALL TABLE %I) x'
, app_table, 'bak_' || app_table);
END LOOP;
RETURN;
END
$func$ LANGUAGE plpgsql;
Call:
SELECT * FROM public.show_diff();
I had the test suggested by #a_horse at first, but after your comment I realized that there is no need for this. EXCEPT considers NULL values to be equal and shows all differences.
While being at it, I improved and simplified your solution some more. Use EXCEPT ALL: cheaper and does not run the risk of folding complete duplicates.
Using EXCEPT clause in PostgreSQL
TABLE is just syntactical sugar.
Is there a shortcut for SELECT * FROM in psql?
However, if you have an index on a unique (combination of) column(s), a JOIN like I suggested before should be faster: finding the only possible duplicate via index should be substantially cheaper.
Crucial element is the cast the row type to text (x::text).
You can even make the function work for any table - but never more than one at a time: With a polymorphic parameter type:
Refactor a PL/pgSQL function to return the output of various SELECT queries