I am trying to set integrity to about 1000 tables which error out because of Pending State. I cannot go 1 by 1 as there are too many tables. The subquery returns the name of all the tables. This is the query that I use which is not working right now:
SET INTEGRITY FOR TABSCHEMA.TABNAME IMMEDIATE CHECKED IN
( SELECT TABNAME
FROM SYSCAT.TABLES
WHERE ( CONST_CHECKED LIKE '%N%' AND TABSCHEMA = 'FINANCE')
WITH ur
)
Any idea?
The main problem of such a massive SET INTEGRITYs is, that if you have a parent-child pair in check-pending state, that you have to either include both tables into a single SET INTEGRITY command or run it on the parent table first, and on the child table with a subsequent command. You get an error, if you run SET INTEGRITY on a child table only, if the corresponding parent table is in the check pending state.
It's quite a non-trivial task to split all tables in check pending to distinct non-relative groups to run a single SET INTEGRITY on each such a group of tables.
This is why it's better to run a script like below:
--#SET TERMINATOR #
SET SERVEROUTPUT ON#
DECLARE GLOBAL TEMPORARY TABLE SESSION.BAD_TABLES
(
TABSCHEMA VARCHAR (128) NOT NULL
, TABNAME VARCHAR (128) NOT NULL
) WITH REPLACE ON COMMIT PRESERVE ROWS NOT LOGGED#
BEGIN
--DECLARE L_ITER INT DEFAULT 0;
DECLARE L_PROCESSED INT;
DECLARE L_TABSCHEMA VARCHAR (128);
DECLARE L_TABNAME VARCHAR (128);
DECLARE CONTINUE HANDLER FOR SQLSTATE '23514'
BEGIN
INSERT INTO SESSION.BAD_TABLES (TABSCHEMA, TABNAME) VALUES (L_TABSCHEMA, L_TABNAME);
END;
-- Ordinal tables processing
L1: LOOP
--SET L_ITER = L_ITER + 1;
--CALL DBMS_OUTPUT.PUT_LINE ('Iteration ' || L_ITER);
SET L_PROCESSED = 0;
FOR V AS C1 INSENSITIVE CURSOR WITH HOLD FOR
SELECT
'SET INTEGRITY FOR "' || T.TABSCHEMA || '"."' || T.TABNAME || '" IMMEDIATE CHECKED' AS CMD
, T.TABSCHEMA
, T.TABNAME
FROM SYSCAT.TABLES T
WHERE T.TYPE = 'T' AND T.STATUS = 'C'
AND NOT EXISTS
(
SELECT 1
FROM SYSCAT.REFERENCES R
JOIN SYSCAT.TABLES P ON P.TABSCHEMA = R.REFTABSCHEMA AND P.TABNAME = R.REFTABNAME
WHERE R.TABSCHEMA = T.TABSCHEMA AND R.TABNAME = T.TABNAME
AND P.STATUS = 'C'
)
AND NOT EXISTS
(
SELECT 1 FROM SESSION.BAD_TABLES B WHERE B.TABSCHEMA = T.TABSCHEMA AND B.TABNAME = T.TABNAME
)
DO
SET (L_TABSCHEMA, L_TABNAME) = (V.TABSCHEMA, V.TABNAME);
EXECUTE IMMEDIATE V.CMD;
COMMIT;
CALL DBMS_OUTPUT.PUT_LINE (V.CMD);
SET L_PROCESSED = L_PROCESSED + 1;
END FOR;
CALL DBMS_OUTPUT.PUT_LINE ('Tables processed: ' || L_PROCESSED);
IF L_PROCESSED = 0 THEN LEAVE L1; END IF;
END LOOP L1;
-- MQTs processing
SET L_PROCESSED = 0;
FOR V AS C1 INSENSITIVE CURSOR WITH HOLD FOR
SELECT
'SET INTEGRITY FOR "' || T.TABSCHEMA || '"."' || T.TABNAME || '" IMMEDIATE CHECKED' AS CMD
, T.TABSCHEMA
, T.TABNAME
FROM SYSCAT.TABLES T
WHERE T.TYPE = 'S' AND T.STATUS = 'C'
DO
SET (L_TABSCHEMA, L_TABNAME) = (V.TABSCHEMA, V.TABNAME);
EXECUTE IMMEDIATE V.CMD;
COMMIT;
CALL DBMS_OUTPUT.PUT_LINE (V.CMD);
SET L_PROCESSED = L_PROCESSED + 1;
END FOR;
CALL DBMS_OUTPUT.PUT_LINE ('MQTs processed: ' || L_PROCESSED);
END
#
SET SERVEROUTPUT OFF#
Ordinal tables are processed iteratively. Each iteration processes a table, if it doesn't have its parent table in check pending at the moment.
MQTs are processed afterwards.
The table name is inserted into a session table, if SET INTEGRITY failed on it.
Related
I need to know what columns of one table have only null values. I understand that I should do a loop in user_tab_columns. But how detect only columns with null value?
Thanks and sorry for my English
To perform a query where you don't know the column identifies in advance, you need to use dynamic SQL. Assuming you already know the table is not empty, you could do something like:
declare
l_count pls_integer;
begin
for r in (
select table_name, column_name
from user_tab_columns
where table_name = 'T42'
and nullable = 'Y'
)
loop
execute immediate 'select count(*) '
|| ' from "' || r.table_name || '"'
|| ' where "' || r.column_name || '" is not null'
into l_count;
if l_count = 0 then
dbms_output.put_line('Table ' || r.table_name
|| ' column ' || r.column_name || ' only has nulls');
end if;
end loop;
end;
/
Remember to set serveroutput on or your client's equivalent before executing.
The cursor gets the columns from the table which are declared as nullable (if they aren't, not much point checking them; though this won't catch explicit check constraints). For each column it builds a query to count the rows where that column is not null. If that count is zero then it didn't find any that are not null, therefore they all are. Again, assuming you know the table isn't empty before you start.
I've included the table name in the cursor select list and references so you only need to change the name in one place to search a different table, or you could use a variable for that name. Or check multiple tables at once by changing that filter.
You may get better performance by selecting a dummy value from any non-null row, with a rownum stop check - which means it will stop as soon as it finds a non-null value, rather than having to check every row to get an actual count:
declare
l_flag pls_integer;
begin
for r in (
select table_name, column_name
from user_tab_columns
where table_name = 'T42'
and nullable = 'Y'
)
loop
begin -- inner block to allow exception trapping within loop
execute immediate 'select 42 '
|| ' from "' || r.table_name || '"'
|| ' where "' || r.column_name || '" is not null'
|| ' and rownum < 2'
into l_flag;
-- if this foudn anything there is a non-null value
exception
when no_data_found then
dbms_output.put_line('Table ' || r.table_name
|| ' column ' || r.column_name || ' only has nulls');
end;
end loop;
end;
/
or you could do something similar with an exists() check.
If you don't know that the table has data then you can do a simple count(*) from the table before the loop to check if it is empty, and report that instead:
...
begin
if l_count = 0 then
dbms_output.put_line('Table is empty');
return;
end if;
...
Or you could combine it with the cursor query, but this would need some work if you wanted to check multiple tables at once as it would stop as soon as it found any empty one (have to leave you something to do... *8-)
declare
l_count_any pls_integer;
l_count_not_null pls_integer;
begin
for r in (
select table_name, column_name
from user_tab_columns
where table_name = 'T42'
and nullable = 'Y'
)
loop
execute immediate 'select count(*),'
|| ' count(case when "' || r.column_name || '" is not null then 1 end)'
|| ' from "' || r.table_name || '"'
into l_count_any, l_count_not_null;
if l_count_any = 0 then
dbms_output.put_line('Table ' || r.table_name || ' is empty');
exit; -- only report once
elsif l_count_not_null = 0 then
dbms_output.put_line('Table ' || r.table_name
|| ' column ' || r.column_name || ' only has nulls');
end if;
end loop;
end;
/
You could of course populate a collection or make it a pipelined function or whatever if you didn't want to reply on dbms_output, but I assume this is a one-off check so it is probably acceptable.
You can loop though your columns and count null rows. If it is same with your table count, then that column has only null values.
The first question is: one column with zero row could be regarded as only (null) value containing column. But it can remain your decision: the scripts below provide solutions to both ways. (In my opinion: no. The empty columns is not a column with only (null) value)
If you want to know the (null) values about one table you can it with count(column):
select count(column) from table
and when the count(column) = 0 then the column has only (null) value or has no value. (So, you cannot make a correct decision).
E.g. The following three tables (x, y and z) has the following columns:
select * from x;
N_X M_X
---------------
100 (null)
200 (null)
300 (null)
select * from y;
N_Y M_Y
---------------
101 (null)
202 (null)
303 apple
select * from z;
N_Z M_Z
---------------
The count() selects:
select count(n_x), count(m_x) from x;
COUNT(N_X) COUNT(M_X)
-----------------------
3 0
select count(n_y), count(m_y) from y;
COUNT(N_Y) COUNT(M_Y)
-----------------------
3 1
select count(n_z), count(m_Z) from z;
COUNT(N_Z) COUNT(M_Z)
-----------------------
0 0
As you can see, the difference between x and y is appears but you cannot decide that the table z has no rows or only full of (null) values.
The general solution:
I have separeted the schema and the db level but the basic idea is common:
Schema level: the current user’s table
DB level: all users or a chosen schema
The number of (null) in one columns:
all_tab_columns.num_nulls
(Or: user_tab_columns, num_nulls).
And we need the num_rows of the table:
all_all_tables.num_rows
(Or: user_all_tables.num_rows)
Where the num_nulls equals to num_rows there are only (null) values.
Firstly, you need to run the DBMS_STATS for refresh the statistics.
on database level:
exec DBMS_STATS.GATHER_DATABASE_STATS;
(it can use a lot of resources)
on schema level:
EXEC DBMS_STATS.gather_schema_stats('TRANEE',DBMS_STATS.AUTO_SAMPLE_SIZE); (owner = tranee)
-- column with zero row = column has only (null) values -> exclude num_nulls > 0 condition
-- column with zero row <> column has only (null) values -> include num_nulls > 0 condition
the scripts:
-- 1. current user
select
a.table_name,
a.column_name,
a.num_nulls,
b.num_rows
from user_tab_columns a, user_all_tables b
where a.table_name = b.table_name
and num_nulls = num_rows
and num_nulls > 0;
-- 2. chosen user / all user -> exclude the a.owner = 'TRANEE' condition
select
a.owner,
a.table_name,
a.column_name,
a.num_nulls,
b.num_rows
from all_tab_columns a, all_all_tables b
where a.owner = b.owner
and a.table_name = b.table_name
and a.owner = 'TRANEE'
and num_nulls = num_rows
and num_nulls > 0;
TABLE_NAME COLUMN_NAME NUM_NULLS NUM_ROWS
----------------------------------------------------
LEADERS COMM 4 4
EMP_ACTION ACTION 12 12
X M_X 3 3
These tables and columns have only (null) values in tranee schema.
postgres 12
I am trying to loop through a table which has schema , table_names and columns
I want to do various things like finding nulls ,row count etc. I failed at the first hurdle trying to update the col records.
table i am using
CREATE TABLE test.table_study (
table_schema text,
table_name text,
column_name text,
records int,
No_Nulls int,
No_Blanks int,
per_pop int
);
I populate the table with some schema names ,tables and columns from information_schema.columns
insert into test.table_study select table_schema, table_name, column_name
from information_schema.columns
where table_schema like '%white'
order by table_schema, table_name, ordinal_position;
I want to populate the rest with a function
function :-
CREATE OR REPLACE PROCEDURE test.insert_data_population()
as $$
declare s record;
declare t record;
declare c record;
BEGIN
FOR s IN SELECT distinct table_schema FROM test.table_study
LOOP
FOR t IN SELECT distinct table_name FROM test.table_study where table_schema = s.table_schema
loop
FOR c IN SELECT column_name FROM test.table_study where table_name = t.table_name
LOOP
execute 'update test.table_study set records = (select count(*) from ' || s.table_schema || '.' || t.table_name || ') where table_study.table_name = '|| t.table_name ||';';
END LOOP;
END LOOP;
END LOOP;
END;
$$
LANGUAGE plpgsql;
I get this error SQL Error [42703]: ERROR: column "age" does not exist. the table age does exist.
when I take out the where clause
execute 'update referralunion.testinsert ti set records = (select count(*) from ' || s.table_schema || '.' || t.table_name || ') ;';
it works, I just cant figure out whats wrong?
Your procedure is structured entirely wrong. What it results in is an attempt to get every column name for every table name in every schema. I would guess results in your column does not exist error. Further is shows procedural thinking. SQL requires think in terms of sets. Below I use basically your query to demonstrate then a revised version which uses a single loop.
-- setup (dropping schema references)
create table table_study (
table_schema text,
table_name text,
column_name text,
records int,
no_nulls int,
no_blanks int,
per_pop int
);
insert into table_study(table_schema, table_name, column_name)
values ('s1','t1','age')
, ('s2','t1','xyz');
-- procedure replacing EXECUTE with Raise Notice.
create or replace procedure insert_data_population()
as $$
declare
s record;
t record;
c record;
line int = 0;
begin
for s in select distinct table_schema from table_study
loop
for t in select distinct table_name from table_study where table_schema = s.table_schema
loop
for c in select column_name from table_study where table_name = t.table_name
loop
line = line+1;
raise notice '%: update table_study set records = (select count(*) from %.% where table_study.table_name = %;'
, line, s.table_schema, t.table_name, c.column_name;
end loop;
end loop;
end loop;
end;
$$
language plpgsql;
Run procedure
do $$
begin
call insert_data_population();
end;
$$;
RESULTS
1: update table_study set records = (select count(*) from s2.t1 where table_study.table_name = age; 2: update table_study set records = (select count(*) from s2.t1 where table_study.table_name = xyz; 3: update table_study set records = (select count(*) from s1.t1 where table_study.table_name = age; 4: update table_study set records = (select count(*) from s1.t1 where table_study.table_name = xyz;
Notice lines 2 and 3. Each references a column name that does not exist in the table. This results from the FOR structure with the same table name in different schema.
Revision for Single Select statement with Single For loop.
create or replace
procedure insert_data_population()
language plpgsql
as $$
declare
s record;
line int = 0;
begin
for s in select distinct table_schema, table_name, column_name from table_study
loop
line = line+1;
raise notice '%: update table_study set records = (select count(*) from %.% where table_study.table_name = %;'
, line, s.table_schema, s.table_name, s.column_name;
end loop;
end;
$$;
do $$
begin
call insert_data_population();
end;
$$;
RESULTS
1: update table_study set records = (select count(*) from s2.t1 where table_study.table_name = xyz;
2: update table_study set records = (select count(*) from s1.t1 where table_study.table_name = age;
Note: In Postgres DECLARE begins a block. It is not necessary to declared each variable. I would actually consider it bad practice. In theory it could require an end for each declare as each could be considered a nested block. Fortunately Postgres does not require this.
How to add a new column in a table after the 2nd or 3rd column in the table using postgres?
My code looks as follows
ALTER TABLE n_domains ADD COLUMN contract_nr int after owner_id
No, there's no direct way to do that. And there's a reason for it - every query should list all the fields it needs in whatever order (and format etc) it needs them, thus making the order of the columns in one table insignificant.
If you really need to do that I can think of one workaround:
dump and save the description of the table in question (using pg_dump --schema-only --table=<schema.table> ...)
add the column you want where you want it in the saved definition
rename the table in the saved definition so not to clash with the name of the old table when you attempt to create it
create the new table using this definition
populate the new table with the data from the old table using 'INSERT INTO <new_table> SELECT field1, field2, <default_for_new_field>, field3,... FROM <old_table>';
rename the old table
rename the new table to the original name
eventually drop the old, renamed table after you make sure everything's alright
The order of columns is not irrelevant, putting fixed width columns at the front of the table can optimize the storage layout of your data, it can also make working with your data easier outside of your application code.
PostgreSQL does not support altering the column ordering (see Alter column position on the PostgreSQL wiki); if the table is relatively isolated, your best bet is to recreate the table:
CREATE TABLE foobar_new ( ... );
INSERT INTO foobar_new SELECT ... FROM foobar;
DROP TABLE foobar CASCADE;
ALTER TABLE foobar_new RENAME TO foobar;
If you have a lot of views or constraints defined against the table, you can re-add all the columns after the new column and drop the original columns (see the PostgreSQL wiki for an example).
The real problem here is that it's not done yet. Currently PostgreSQL's logical ordering is the same as the physical ordering. That's problematic because you can't get a different logical ordering, but it's even worse because the table isn't physically packed automatically, so by moving columns you can get different performance characteristics.
Arguing that it's that way by intent in design is pointless. It's somewhat likely to change at some point when an acceptable patch is submitted.
All of that said, is it a good idea to rely on the ordinal positioning of columns, logical or physical? Hell no. In production code you should never be using an implicit ordering or *. Why make the code more brittle than it needs to be? Correctness should always be a higher priority than saving a few keystrokes.
As a work around, you can in fact modify the column ordering by recreating the table, or through the "add and reorder" game
See also,
Column tetris reordering in order to make things more space-efficient
The column order is relevant to me, so I created this function. See if it helps. It works with indexes, primary key, and triggers. Missing Views and Foreign Key and other features are missing.
Example:
SELECT xaddcolumn('table', 'col3 int NOT NULL DEFAULT 0', 'col2');
Source code:
CREATE OR REPLACE FUNCTION xaddcolumn(ptable text, pcol text, pafter text) RETURNS void AS $BODY$
DECLARE
rcol RECORD;
rkey RECORD;
ridx RECORD;
rtgr RECORD;
vsql text;
vkey text;
vidx text;
cidx text;
vtgr text;
ctgr text;
etgr text;
vseq text;
vtype text;
vcols text;
BEGIN
EXECUTE 'CREATE TABLE zzz_' || ptable || ' AS SELECT * FROM ' || ptable;
--colunas
vseq = '';
vcols = '';
vsql = 'CREATE TABLE ' || ptable || '(';
FOR rcol IN SELECT column_name as col, udt_name as coltype, column_default as coldef,
is_nullable as is_null, character_maximum_length as len,
numeric_precision as num_prec, numeric_scale as num_scale
FROM information_schema.columns
WHERE table_name = ptable
ORDER BY ordinal_position
LOOP
vtype = rcol.coltype;
IF (substr(rcol.coldef,1,7) = 'nextval') THEN
vtype = 'serial';
vseq = vseq || 'SELECT setval(''' || ptable || '_' || rcol.col || '_seq'''
|| ', max(' || rcol.col || ')) FROM ' || ptable || ';';
ELSIF (vtype = 'bpchar') THEN
vtype = 'char';
END IF;
vsql = vsql || E'\n' || rcol.col || ' ' || vtype;
IF (vtype in ('varchar', 'char')) THEN
vsql = vsql || '(' || rcol.len || ')';
ELSIF (vtype = 'numeric') THEN
vsql = vsql || '(' || rcol.num_prec || ',' || rcol.num_scale || ')';
END IF;
IF (rcol.is_null = 'NO') THEN
vsql = vsql || ' NOT NULL';
END IF;
IF (rcol.coldef <> '' AND vtype <> 'serial') THEN
vsql = vsql || ' DEFAULT ' || rcol.coldef;
END IF;
vsql = vsql || E',';
vcols = vcols || rcol.col || ',';
--
IF (rcol.col = pafter) THEN
vsql = vsql || E'\n' || pcol || ',';
END IF;
END LOOP;
vcols = substr(vcols,1,length(vcols)-1);
--keys
vkey = '';
FOR rkey IN SELECT constraint_name as name, column_name as col
FROM information_schema.key_column_usage
WHERE table_name = ptable
LOOP
IF (vkey = '') THEN
vkey = E'\nCONSTRAINT ' || rkey.name || ' PRIMARY KEY (';
END IF;
vkey = vkey || rkey.col || ',';
END LOOP;
IF (vkey <> '') THEN
vsql = vsql || substr(vkey,1,length(vkey)-1) || ') ';
END IF;
vsql = substr(vsql,1,length(vsql)-1) || ') WITHOUT OIDS';
--index
vidx = '';
cidx = '';
FOR ridx IN SELECT s.indexrelname as nome, a.attname as col
FROM pg_index i LEFT JOIN pg_class c ON c.oid = i.indrelid
LEFT JOIN pg_attribute a ON a.attrelid = c.oid AND a.attnum = ANY(i.indkey)
LEFT JOIN pg_stat_user_indexes s USING (indexrelid)
WHERE c.relname = ptable AND i.indisunique != 't' AND i.indisprimary != 't'
ORDER BY s.indexrelname
LOOP
IF (ridx.nome <> cidx) THEN
IF (vidx <> '') THEN
vidx = substr(vidx,1,length(vidx)-1) || ');';
END IF;
cidx = ridx.nome;
vidx = vidx || E'\nCREATE INDEX ' || cidx || ' ON ' || ptable || ' (';
END IF;
vidx = vidx || ridx.col || ',';
END LOOP;
IF (vidx <> '') THEN
vidx = substr(vidx,1,length(vidx)-1) || ')';
END IF;
--trigger
vtgr = '';
ctgr = '';
etgr = '';
FOR rtgr IN SELECT trigger_name as nome, event_manipulation as eve,
action_statement as act, condition_timing as cond
FROM information_schema.triggers
WHERE event_object_table = ptable
LOOP
IF (rtgr.nome <> ctgr) THEN
IF (vtgr <> '') THEN
vtgr = replace(vtgr, '_#eve_', substr(etgr,1,length(etgr)-3));
END IF;
etgr = '';
ctgr = rtgr.nome;
vtgr = vtgr || 'CREATE TRIGGER ' || ctgr || ' ' || rtgr.cond || ' _#eve_ '
|| 'ON ' || ptable || ' FOR EACH ROW ' || rtgr.act || ';';
END IF;
etgr = etgr || rtgr.eve || ' OR ';
END LOOP;
IF (vtgr <> '') THEN
vtgr = replace(vtgr, '_#eve_', substr(etgr,1,length(etgr)-3));
END IF;
--exclui velha e cria nova
EXECUTE 'DROP TABLE ' || ptable;
IF (EXISTS (SELECT sequence_name FROM information_schema.sequences
WHERE sequence_name = ptable||'_id_seq'))
THEN
EXECUTE 'DROP SEQUENCE '||ptable||'_id_seq';
END IF;
EXECUTE vsql;
--dados na nova
EXECUTE 'INSERT INTO ' || ptable || '(' || vcols || ')' ||
E'\nSELECT ' || vcols || ' FROM zzz_' || ptable;
EXECUTE vseq;
EXECUTE vidx;
EXECUTE vtgr;
EXECUTE 'DROP TABLE zzz_' || ptable;
END;
$BODY$ LANGUAGE plpgsql VOLATILE COST 100;
#Jeremy Gustie's solution above almost works, but will do the wrong thing if the ordinals are off (or fail altogether if the re-ordered ordinals make incompatible types match). Give it a try:
CREATE TABLE test1 (one varchar, two varchar, three varchar);
CREATE TABLE test2 (three varchar, two varchar, one varchar);
INSERT INTO test1 (one, two, three) VALUES ('one', 'two', 'three');
INSERT INTO test2 SELECT * FROM test1;
SELECT * FROM test2;
The results show the problem:
testdb=> select * from test2;
three | two | one
-------+-----+-------
one | two | three
(1 row)
You can remedy this by specifying the column names in the insert:
INSERT INTO test2 (one, two, three) SELECT * FROM test1;
That gives you what you really want:
testdb=> select * from test2;
three | two | one
-------+-----+-----
three | two | one
(1 row)
The problem comes when you have legacy that doesn't do this, as I indicated above in my comment on peufeu's reply.
Update: It occurred to me that you can do the same thing with the column names in the INSERT clause by specifying the column names in the SELECT clause. You just have to reorder them to match the ordinals in the target table:
INSERT INTO test2 SELECT three, two, one FROM test1;
And you can of course do both to be very explicit:
INSERT INTO test2 (one, two, three) SELECT one, two, three FROM test1;
That gives you the same results as above, with the column values properly matched.
The order of the columns is totally irrelevant in relational databases
Yes.
For instance if you use Python, you would do :
cursor.execute( "SELECT id, name FROM users" )
for id, name in cursor:
print id, name
Or you would do :
cursor.execute( "SELECT * FROM users" )
for row in cursor:
print row['id'], row['name']
But no sane person would ever use positional results like this :
cursor.execute( "SELECT * FROM users" )
for id, name in cursor:
print id, name
Well, it's a visual goody for DBA's and can be implemented to the engine with minor performance loss. Add a column order table to pg_catalog or where it's suited best. Keep it in memory and use it before certain queries. Why overthink such a small eye candy.
# Milen A. Radev
The irrelevant need from having a set order of columns is not always defined by the query that pulls them. In the values from pg_fetch_row does not include the associated column name and therefore would require the columns to be defined by the SQL statement.
A simple select * from would require innate knowledge of the table structure, and would sometimes cause issues if the order of the columns were to change.
Using pg_fetch_assoc is a more reliable method as you can reference the column names, and therefore use a simple select * from.
I have some code like this:
declare
p_vara varchar2;
p_varb varchar2;
p_varc varchar2;
begin
INSERT INTO MY_INSERTABLE_TABLE
SELECT a.id,b.id,c.id
FROM table_a a, table_b b, table_c c
WHERE a.id is not null
and a.id = b.id
and c.id = 'cat'
end;
Now based on the the variable to make it conditional so that only certain parts of the query get called based on the variable.
declare
p_vara varchar2;
p_varb varchar2;
p_varc varchar2;
begin
INSERT INTO MY_INSERTABLE_TABLE
SELECT a.id, -- Show only if p_vara = 'yes'
b.id, -- Show only if p_varb = 'yes'
c.id -- Show only if p_varc = 'yes'
FROM table_a a, -- Use only if p_vara = 'yes'
table_b b, -- Use only if p_varb = 'yes'
table_c c -- Use only if p_varc = 'yes'
WHERE a.id is not null -- Use only if p_vara = 'yes'
and a.id = b.id -- Use only if p_vara = 'yes' and p_varb = 'yes'
and c.id = 'cat' -- Use only if p_varc = 'yes'
end;
So for example if the variables are set as this:
p_vara = 'yes'
p_varb = 'no'
p_varc = 'no'
Then the query should look like this:
SELECT a.id
FROM table_a
WHERE a.id is null;
As written, your requirements do not appear to be complete. If all three variables are yes, for example, your full statement would have three table joins but only one join condition so you'd generate a Cartesian product with table_c. If p_vara = 'yes' and p_varc = 'yes', you'd have two tables joined with no join condition so you'd again have a Cartesian product. It seems unlikely to me that you really want to generate a Cartesian product.
In general, you can build a SQL statement in a string variable and then pass that to EXECUTE IMMEDIATE. If you have 25 boolean variables, that implies that your code could generate a total of 33.55 million distinct SQL statements. Just verifying that none of those paths generate a statement with syntax errors would be non-trivial. Combined with the fact that resorting to dynamic SQL generally makes code harder to read, maintain, and support in addition to creating opportunities for performance and security issues, I would tend to push back on any design that contemplates something as complex as what you are describing.
That said, you could do something like this (I'm not building the WHERE clause out completely but I trust you get the jist)
declare
l_vara boolean;
l_varb boolean;
l_varc boolean;
l_sql_stmt varchar(4000);
begin
l_sql_stmt := 'INSERT INTO my_insertable_table( col1, col2, col3 ) ';
l_sql_stmt := l_sql_stmt || ' SELECT ' ||
(case when l_vara then ' a.id, ' else ' null, ' end) ||
(case when l_varb then ' b.id, ' else ' null, ' end) ||
(case when l_varc then ' c.id, ' else ' null, ' end);
l_sql_stmt := rtrim( l_sql_stmt, ',' ); -- remove the extra trailing comma
l_sql_stmt := l_sql_stmt || ' FROM ';
if( l_vara )
then
l_sql_stmt := l_sql_stmt || ' table_a a, ';
end if;
if( l_varb )
then
l_sql_stmt := l_sql_stmt || ' table_b b, ';
end if;
if( l_varc )
then
l_sql_stmt := l_sql_stmt || ' table_c c, ';
end if;
-- again remove the extra trailing comma
l_sql_stmt := rtrim( l_sql_stmt, ',' );
<<build out the WHERE clause similarly>>
-- Log the SQL statement so you can debug it when it fails
insert into some_log_table( sql_stmt ) values( l_sql_stmt );
EXECUTE IMMEDIATE l_sql_stmt;
end;
I have a function in plpgsql where it creates a temporary table and then it has a loop. The thing is that each time it loops it executes also the part where it creates the temporary table and therefore an error pops up saying;
ERROR: relation "tmpr" already exists
CONTEXT: SQL statement "CREATE TEMPORARY TABLE tmpr (
id int,
source geometry,
target geometry,
dist_ft character varying
)"
Is there any way to prevent part of the code from executing more than once?
Below you can find the code:
DECLARE
_r record;
t record;
i int := 0;
j int := 1;
count int := 0;
source_geom character varying;
target_geom character varying;
BEGIN
BEGIN
CREATE TEMPORARY TABLE tmpr (
id int,
source geometry,
target geometry,
dist_ft character varying
);
END;
BEGIN
CREATE TEMPORARY TABLE tmp (
ogc_fid int,
wkb_geometry character varying,
track_fid int
);
END;
-- END IF;
WHILE i < 3 --DEPENDS ON THE NUMBER OF TRACKS
LOOP
--j := 1;
--WHILE j < 29 --DEPENDS ON THE NUMBER OF TRACK POINTS
--LOOP
EXECUTE 'INSERT INTO tmp (ogc_fid, wkb_geometry, track_fid)
SELECT '|| quote_ident(gid_cname) ||' , ' ||quote_ident(geo_cname)||' , ' || quote_ident(tid_cname) ||'
FROM ' ||quote_ident(geom_table)|| '
WHERE ' ||quote_ident(tid_cname)|| ' = ' || i;
FOR _r IN EXECUTE
' SELECT *'
||' FROM tmp'
LOOP
EXECUTE 'INSERT INTO tmpr (id, source, target, dist_ft)
SELECT a.'|| quote_ident(gid_cname) || ' AS id,'
|| ' st_astext( a.'||quote_ident(geo_cname)||') AS source,'
|| ' st_astext(b.'||quote_ident(geo_cname)||') AS target, '
|| ' ST_Distance(a.'||quote_ident(geo_cname) || ' , b.'||quote_ident(geo_cname)||') As dist_ft '
|| ' FROM tmp AS a INNER JOIN tmp As b ON ST_DWithin(a.'||quote_ident(geo_cname)|| ', b.'||quote_ident(geo_cname)|| ',1000)'
|| ' WHERE b.'||quote_ident(gid_cname)|| ' > a.'||quote_ident(gid_cname)|| ' AND b.'||quote_ident(tid_cname)|| ' = '||i|| 'AND a.'||quote_ident(tid_cname)|| ' = '||i||
' ORDER BY dist_ft '
|| ' Limit 1 ';
--source_geom := temp.source;
--target_geom := temp.target;
EXECUTE 'update ' || quote_ident(geom_table) ||
' SET source = tmpr.source
, target = tmpr.target
FROM tmpr
WHERE ' || quote_ident(gid_cname) || ' = tmpr.id';
EXECUTE 'delete from tmpr';
END LOOP;
--j = j + 1;
--END LOOP;
EXECUTE 'delete from tmp';
i = i + 1;
END LOOP;
RETURN 'OK';
END;
You can use the IF NOT EXISTS clause to avoid an exception (introduced with pg 9.1):
CREATE TEMPORARY TABLE IF NOT EXISTS tmpr (...);
You'd better check if there are rows in the table in this case:
IF EXISTS (SELECT 1 FROM tmpr) THEN -- table itself exists after above command
DELETE FROM tmpr;
END IF;
To avoid that a subsequent call of the function conflicts, or generally, if you don't need the temp table any more after the function finishes, add ON COMMIT DROP:
CREATE TEMPORARY TABLE IF NOT EXISTS tmpr (...) ON COMMIT DROP;
This would still fail if you call the function repeatedly inside a single transaction. In this case, you can add explicit DROP TABLE statements to the end of your function instead.