I wanted to make a table that sanity checked record integrity for any duplications among my db.
I have a table currently with object names (tables) and their primary keys:
I want to create a procedure that loops through those objects with their keys, and inserts into a separate table the count of duplicates:
below is my code, but I've never done anything like this before and am new to postgres. What I have is from hours of googling/researching but every time I get closer I get a new error and am quite stuck :( Any insights would be greatly appreciated.
My newest error is I believe from my quote_ident(object_names). I don't want to query the column as postgres is reading it, I'd want that to be a raw string:
code:
do $$
declare
object_names varchar;
keys varchar;
rec record;
begin
for rec in select object_name, key from mfr_incentives.public.t_jh_dup_check
loop
object_names = rec.object_name;
keys = rec.key;
execute 'insert into mfr_incentives.public.t_jh_dup_check_final_output
select * from
(select ' || quote_ident(object_names) || ', ' || quote_ident(keys) || ', ' || ' count(*), current_date from
( select ' || keys || ', count(*)
from ' || object_names ||
' group by ' || keys || ' having count(*) > 1
) a
) a';
end loop;
end;
$$;
Found out my problem!
Being unfamiliar with the topic I finally found that I wanted quote_literal() instead of quote_ident().
The below works:
create or replace procedure public.proc_jh_dup_check()
language plpgsql
--IT WORKS NOW
as $$
declare
rec record;
begin
for rec in select object_name, key from mfr_incentives.public.t_jh_dup_check
loop
execute 'insert into mfr_incentives.public.t_jh_dup_check_final_output
select * from
(select ' || quote_literal(rec.object_name) || ', ' || quote_literal(rec.key) || ', ' || ' count(*), current_date from
( select ' || rec.key || ', count(*)
from ' || rec.object_name ||
' group by ' || rec.key || ' having count(*) > 1
) a
) a';
end loop;
end;
$$;
Related
I would like to display results of dynamic sql based on antoher sql, but get error message. My code:
DECLARE
sql_qry VARCHAR2(1000) := NULL;
TYPE results IS
TABLE OF all_tab_columns%rowtype;
results_tbl results;
BEGIN
FOR i IN (
SELECT
*
FROM
all_tab_columns
WHERE
owner = 'OWNER_XYZ'
AND upper(column_name) LIKE '%COLUMN_XYZ%'
ORDER BY
table_name,
column_name
) LOOP
sql_qry := ' SELECT DISTINCT '
|| i.column_name
|| ' as column_name '
|| ' FROM '
|| i.owner
|| '.'
|| i.table_name
|| ' WHERE SUBSTR('
|| i.column_name
|| ',1,1) = ''Y''';
EXECUTE IMMEDIATE sql_qry BULK COLLECT
INTO results_tbl;
dbms_output.put_line(results_tbl);
END LOOP;
END;
I get the error message:
PLS-00306: wrong number or types of arguments in call to 'PUT_LINE'
In fact I need the results of all queries with an union between them like that
[1] [1]: https://i.stack.imgur.com/llxzr.png
Your dynamic query is selecting a single column, but you are bulk collecting into results_tbl, which is of type results, based on all_tab_columns%rowtype - which has lots of columns.
If you define your collection type as a single column of the data type you need, i.e. some length of string:
DECLARE
sql_qry VARCHAR2(1000) := NULL;
TYPE results IS
TABLE OF varchar2(4000);
results_tbl results;
...
You will then bulk collect a single column into that single-column collection. To display the results you need to loop over the collection:
FOR j IN 1..results_tbl.COUNT loop
dbms_output.put_line(results_tbl(j));
END LOOP;
so the whole block becomes:
DECLARE
sql_qry VARCHAR2(1000) := NULL;
TYPE results IS
TABLE OF varchar2(4000);
results_tbl results;
BEGIN
FOR i IN (
SELECT
*
FROM
all_tab_columns
WHERE
owner = 'OWNER_XYZ'
AND upper(column_name) LIKE '%COLUMN_XYZ%'
ORDER BY
table_name,
column_name
) LOOP
sql_qry := ' SELECT DISTINCT '
|| i.column_name
|| ' as column_name '
|| ' FROM '
|| i.owner
|| '.'
|| i.table_name
|| ' WHERE SUBSTR('
|| i.column_name
|| ',1,1) = ''Y''';
EXECUTE IMMEDIATE sql_qry BULK COLLECT
INTO results_tbl;
FOR j IN 1..results_tbl.COUNT loop
dbms_output.put_line(results_tbl(j));
END LOOP;
END LOOP;
END;
/
However, you can also do this without PL/SQL, using a variation on an XML trick. You can get the results of the dynamic query as an XML document using something like:
select dbms_xmlgen.getxmltype(
'select distinct "' || column_name || '" as value'
|| ' from "' || owner || '"."' || table_name || '"'
|| ' where substr("' || column_name || '", 1, 1) = ''Y'''
)
from all_tab_columns
where owner = 'OWNER_XYZ'
and upper(column_name) like '%COLUMN_XYZ%';
and then extract the value you want from that:
with cte (xml) as (
select dbms_xmlgen.getxmltype(
'select distinct "' || column_name || '" as value'
|| ' from "' || owner || '"."' || table_name || '"'
|| ' where substr("' || column_name || '", 1, 1) = ''Y'''
)
from all_tab_columns
where owner = 'OWNER_XYZ'
and upper(column_name) like '%COLUMN_XYZ%'
)
select x.value
from cte
cross apply xmltable(
'/ROWSET/ROW'
passing cte.xml
columns value varchar2(4000) path 'VALUE'
) x;
You can also easily include the table each value came from if you want that information (and the owner, and actual column name, etc.).
db<>fiddle showing both approaches.
How to add a new column in a table after the 2nd or 3rd column in the table using postgres?
My code looks as follows
ALTER TABLE n_domains ADD COLUMN contract_nr int after owner_id
No, there's no direct way to do that. And there's a reason for it - every query should list all the fields it needs in whatever order (and format etc) it needs them, thus making the order of the columns in one table insignificant.
If you really need to do that I can think of one workaround:
dump and save the description of the table in question (using pg_dump --schema-only --table=<schema.table> ...)
add the column you want where you want it in the saved definition
rename the table in the saved definition so not to clash with the name of the old table when you attempt to create it
create the new table using this definition
populate the new table with the data from the old table using 'INSERT INTO <new_table> SELECT field1, field2, <default_for_new_field>, field3,... FROM <old_table>';
rename the old table
rename the new table to the original name
eventually drop the old, renamed table after you make sure everything's alright
The order of columns is not irrelevant, putting fixed width columns at the front of the table can optimize the storage layout of your data, it can also make working with your data easier outside of your application code.
PostgreSQL does not support altering the column ordering (see Alter column position on the PostgreSQL wiki); if the table is relatively isolated, your best bet is to recreate the table:
CREATE TABLE foobar_new ( ... );
INSERT INTO foobar_new SELECT ... FROM foobar;
DROP TABLE foobar CASCADE;
ALTER TABLE foobar_new RENAME TO foobar;
If you have a lot of views or constraints defined against the table, you can re-add all the columns after the new column and drop the original columns (see the PostgreSQL wiki for an example).
The real problem here is that it's not done yet. Currently PostgreSQL's logical ordering is the same as the physical ordering. That's problematic because you can't get a different logical ordering, but it's even worse because the table isn't physically packed automatically, so by moving columns you can get different performance characteristics.
Arguing that it's that way by intent in design is pointless. It's somewhat likely to change at some point when an acceptable patch is submitted.
All of that said, is it a good idea to rely on the ordinal positioning of columns, logical or physical? Hell no. In production code you should never be using an implicit ordering or *. Why make the code more brittle than it needs to be? Correctness should always be a higher priority than saving a few keystrokes.
As a work around, you can in fact modify the column ordering by recreating the table, or through the "add and reorder" game
See also,
Column tetris reordering in order to make things more space-efficient
The column order is relevant to me, so I created this function. See if it helps. It works with indexes, primary key, and triggers. Missing Views and Foreign Key and other features are missing.
Example:
SELECT xaddcolumn('table', 'col3 int NOT NULL DEFAULT 0', 'col2');
Source code:
CREATE OR REPLACE FUNCTION xaddcolumn(ptable text, pcol text, pafter text) RETURNS void AS $BODY$
DECLARE
rcol RECORD;
rkey RECORD;
ridx RECORD;
rtgr RECORD;
vsql text;
vkey text;
vidx text;
cidx text;
vtgr text;
ctgr text;
etgr text;
vseq text;
vtype text;
vcols text;
BEGIN
EXECUTE 'CREATE TABLE zzz_' || ptable || ' AS SELECT * FROM ' || ptable;
--colunas
vseq = '';
vcols = '';
vsql = 'CREATE TABLE ' || ptable || '(';
FOR rcol IN SELECT column_name as col, udt_name as coltype, column_default as coldef,
is_nullable as is_null, character_maximum_length as len,
numeric_precision as num_prec, numeric_scale as num_scale
FROM information_schema.columns
WHERE table_name = ptable
ORDER BY ordinal_position
LOOP
vtype = rcol.coltype;
IF (substr(rcol.coldef,1,7) = 'nextval') THEN
vtype = 'serial';
vseq = vseq || 'SELECT setval(''' || ptable || '_' || rcol.col || '_seq'''
|| ', max(' || rcol.col || ')) FROM ' || ptable || ';';
ELSIF (vtype = 'bpchar') THEN
vtype = 'char';
END IF;
vsql = vsql || E'\n' || rcol.col || ' ' || vtype;
IF (vtype in ('varchar', 'char')) THEN
vsql = vsql || '(' || rcol.len || ')';
ELSIF (vtype = 'numeric') THEN
vsql = vsql || '(' || rcol.num_prec || ',' || rcol.num_scale || ')';
END IF;
IF (rcol.is_null = 'NO') THEN
vsql = vsql || ' NOT NULL';
END IF;
IF (rcol.coldef <> '' AND vtype <> 'serial') THEN
vsql = vsql || ' DEFAULT ' || rcol.coldef;
END IF;
vsql = vsql || E',';
vcols = vcols || rcol.col || ',';
--
IF (rcol.col = pafter) THEN
vsql = vsql || E'\n' || pcol || ',';
END IF;
END LOOP;
vcols = substr(vcols,1,length(vcols)-1);
--keys
vkey = '';
FOR rkey IN SELECT constraint_name as name, column_name as col
FROM information_schema.key_column_usage
WHERE table_name = ptable
LOOP
IF (vkey = '') THEN
vkey = E'\nCONSTRAINT ' || rkey.name || ' PRIMARY KEY (';
END IF;
vkey = vkey || rkey.col || ',';
END LOOP;
IF (vkey <> '') THEN
vsql = vsql || substr(vkey,1,length(vkey)-1) || ') ';
END IF;
vsql = substr(vsql,1,length(vsql)-1) || ') WITHOUT OIDS';
--index
vidx = '';
cidx = '';
FOR ridx IN SELECT s.indexrelname as nome, a.attname as col
FROM pg_index i LEFT JOIN pg_class c ON c.oid = i.indrelid
LEFT JOIN pg_attribute a ON a.attrelid = c.oid AND a.attnum = ANY(i.indkey)
LEFT JOIN pg_stat_user_indexes s USING (indexrelid)
WHERE c.relname = ptable AND i.indisunique != 't' AND i.indisprimary != 't'
ORDER BY s.indexrelname
LOOP
IF (ridx.nome <> cidx) THEN
IF (vidx <> '') THEN
vidx = substr(vidx,1,length(vidx)-1) || ');';
END IF;
cidx = ridx.nome;
vidx = vidx || E'\nCREATE INDEX ' || cidx || ' ON ' || ptable || ' (';
END IF;
vidx = vidx || ridx.col || ',';
END LOOP;
IF (vidx <> '') THEN
vidx = substr(vidx,1,length(vidx)-1) || ')';
END IF;
--trigger
vtgr = '';
ctgr = '';
etgr = '';
FOR rtgr IN SELECT trigger_name as nome, event_manipulation as eve,
action_statement as act, condition_timing as cond
FROM information_schema.triggers
WHERE event_object_table = ptable
LOOP
IF (rtgr.nome <> ctgr) THEN
IF (vtgr <> '') THEN
vtgr = replace(vtgr, '_#eve_', substr(etgr,1,length(etgr)-3));
END IF;
etgr = '';
ctgr = rtgr.nome;
vtgr = vtgr || 'CREATE TRIGGER ' || ctgr || ' ' || rtgr.cond || ' _#eve_ '
|| 'ON ' || ptable || ' FOR EACH ROW ' || rtgr.act || ';';
END IF;
etgr = etgr || rtgr.eve || ' OR ';
END LOOP;
IF (vtgr <> '') THEN
vtgr = replace(vtgr, '_#eve_', substr(etgr,1,length(etgr)-3));
END IF;
--exclui velha e cria nova
EXECUTE 'DROP TABLE ' || ptable;
IF (EXISTS (SELECT sequence_name FROM information_schema.sequences
WHERE sequence_name = ptable||'_id_seq'))
THEN
EXECUTE 'DROP SEQUENCE '||ptable||'_id_seq';
END IF;
EXECUTE vsql;
--dados na nova
EXECUTE 'INSERT INTO ' || ptable || '(' || vcols || ')' ||
E'\nSELECT ' || vcols || ' FROM zzz_' || ptable;
EXECUTE vseq;
EXECUTE vidx;
EXECUTE vtgr;
EXECUTE 'DROP TABLE zzz_' || ptable;
END;
$BODY$ LANGUAGE plpgsql VOLATILE COST 100;
#Jeremy Gustie's solution above almost works, but will do the wrong thing if the ordinals are off (or fail altogether if the re-ordered ordinals make incompatible types match). Give it a try:
CREATE TABLE test1 (one varchar, two varchar, three varchar);
CREATE TABLE test2 (three varchar, two varchar, one varchar);
INSERT INTO test1 (one, two, three) VALUES ('one', 'two', 'three');
INSERT INTO test2 SELECT * FROM test1;
SELECT * FROM test2;
The results show the problem:
testdb=> select * from test2;
three | two | one
-------+-----+-------
one | two | three
(1 row)
You can remedy this by specifying the column names in the insert:
INSERT INTO test2 (one, two, three) SELECT * FROM test1;
That gives you what you really want:
testdb=> select * from test2;
three | two | one
-------+-----+-----
three | two | one
(1 row)
The problem comes when you have legacy that doesn't do this, as I indicated above in my comment on peufeu's reply.
Update: It occurred to me that you can do the same thing with the column names in the INSERT clause by specifying the column names in the SELECT clause. You just have to reorder them to match the ordinals in the target table:
INSERT INTO test2 SELECT three, two, one FROM test1;
And you can of course do both to be very explicit:
INSERT INTO test2 (one, two, three) SELECT one, two, three FROM test1;
That gives you the same results as above, with the column values properly matched.
The order of the columns is totally irrelevant in relational databases
Yes.
For instance if you use Python, you would do :
cursor.execute( "SELECT id, name FROM users" )
for id, name in cursor:
print id, name
Or you would do :
cursor.execute( "SELECT * FROM users" )
for row in cursor:
print row['id'], row['name']
But no sane person would ever use positional results like this :
cursor.execute( "SELECT * FROM users" )
for id, name in cursor:
print id, name
Well, it's a visual goody for DBA's and can be implemented to the engine with minor performance loss. Add a column order table to pg_catalog or where it's suited best. Keep it in memory and use it before certain queries. Why overthink such a small eye candy.
# Milen A. Radev
The irrelevant need from having a set order of columns is not always defined by the query that pulls them. In the values from pg_fetch_row does not include the associated column name and therefore would require the columns to be defined by the SQL statement.
A simple select * from would require innate knowledge of the table structure, and would sometimes cause issues if the order of the columns were to change.
Using pg_fetch_assoc is a more reliable method as you can reference the column names, and therefore use a simple select * from.
I'm trying to learn plsql and got stuck in understanding some basic stuff. Here is a challenge that I'm trying to solve. I have two tables. One holds information about owners and the other is information about cars.
I want to to write an anonymous block that joins these two tables and with a for loop based on amount of cars that is registered to each owner prints how many cars each person own. furthermore I want an if statement which distinguishes between 1 Car (singular) and 2, 3 Cars (plural).
the tables are these:
CREATE TABLE owners(
id_nr VARCHAR2(13) PRIMARY KEY,
f_name VARCHAR2(20),
s_name VARCHAR2(20)
);
CREATE TABLE cars(
reg_nr VARCHAR2(6) PRIMARY KEY,
id_nr REFERENCES owners(pnr),
model VARCHAR2(20),
year NUMBER(4),
date DATE
);
The result may look like something like this:
19380321-7799, Hans, Anderson, Owns: 1 car
19490321-7899, Mike, Erikson, Owns: 2 cars
. . . etc
I know the this question was already answered but when I try following:
declare
v_suffix varchar2(1);
begin
for o in (select bilägare_pnr, fnamn, enamn,
(select count(1) from fordon where fordon_pnr = bilägare_pnr) as bilar_ägda
from bilägare)
loop
if o.pnr_fordon = 1
then v_suffix = 'bil'
else v_suffix = 'bilar'
end if;
dbms_output.put_line(o.pnr || ', ' || o.fnamn || ', ' || o.enamn
|| ' Äger: ' || o.pnr_fordon || ' bil' || v_suffix);
end loop;
end;
/
I get:
ORA-06550: line 9, column 27:
PLS-00103: Encountered the symbol "=" when expecting one of the following:
:= . ( # % ;
any tips? Im not sure how to declare v_suffix
EDIT (copied from comment on answer below):
Updating my code:
declare
cursor c_BILÄGARE is
select fnamn,enamn,pnr
from BILÄGARE;
begin
for rec in c_BILÄGARE loop
if (rec.antal>1) then
dbms_output.put_line (rec.pnr||','|| rec.fnamn || ',' ||
rec.enamn || ',' || rec.antal || 'bilar');
else
dbms_output.put_line (rec.pnr||','|| rec.fnamn || ',' ||
rec.enamn || ',' || rec.antal || 'bil');
end if;
end loop;
end;
getting:
ORA-06550: line 9, column 9: PLS-00302: component 'ANTAL' must be declared (antal=Quantity)
As noted above, you need to use := as the assignment operator. You also need a semi-colon after each statement - thus, you should use
then v_suffix := 'bil';
instead of
then v_suffix := 'bil'
So your code should look like:
declare
v_suffix varchar2(1);
begin
for o in (select bilägare_pnr, fnamn, enamn,
(select count(1)
from fordon
where fordon_pnr = bilägare_pnr) as bilar_ägda
from bilägare)
loop
if o.pnr_fordon = 1
then v_suffix := 'bil';
else v_suffix := 'bilar';
end if;
dbms_output.put_line(o.pnr || ', ' || o.fnamn || ', ' || o.enamn
|| ' Äger: ' || o.pnr_fordon || ' bil' || v_suffix);
end loop;
end;
Please try this piece of code:
begin
for rec in (select
o.id_nr || ',' || o.f_name || ',' || o.s_name || ', Owns: ' || coalesce(car.cnt, 0) || ' ' || decode(car.cnt , 1, 'Car', 'Cars') as res
from owners o
left outer join (select id_nr, count(1) as cnt from cars group by id_nr) car on (car.id_nr = o.id_nr)) loop
dbms_output.put_line(rec.res);
end loop;
end;
Thanks.
There are multiple issues in your code which is highlighted and fixed in following code:
declare
v_suffix varchar2(5); -- data length should be 5 or more
begin
for o in (select bilägare_pnr, fnamn, enamn,
(select count(1) from fordon where fordon_pnr = bilägare_pnr) as bilar_ägda
from bilägare)
loop
if o.bilar_ägda <= 1 -- replaced it from pnr_fordon and <= is used for 0 or 1 car
then v_suffix := 'bil'; -- := and ; is used here and in next statement
else v_suffix := 'bilar';
end if;
dbms_output.put_line(o.pnr || ', ' || o.fnamn || ', ' || o.enamn
|| ' Äger: ' || o.pnr_fordon || ' bil' || v_suffix);
end loop;
end;
/
I have been stuck into a question.
The question is I want to get all Table name with their Row Count from Teradata.
I have this query which gives me all View Name from a specific Schema.
I ] SELECT TableName FROM dbc.tables WHERE tablekind='V' AND databasename='SCHEMA' order by TableName;
& I have this query which gives me row count for a specific Table/View in Schema.
II ] SELECT COUNT(*) as RowsNum FROM SCHEMA.TABLE_NAME;
Now can anyone tell me what to do to get the result from Query I (TableName) and put it into QUERY II (TABLE_NAME)
You help will be appreciated.
Thanks in advance,
Vrinda
This is a SP to collect row counts from all tables within a database, it's very basic, no error checking etc.
It shows a cursor and dynamic SQL using dbc.SysExecSQL or EXECUTE IMMEDIATE:
CREATE SET TABLE RowCounts
(
DatabaseName VARCHAR(30) CHARACTER SET LATIN NOT CASESPECIFIC,
TableName VARCHAR(30) CHARACTER SET LATIN NOT CASESPECIFIC,
RowCount BIGINT,
COllectTimeStamp TIMESTAMP(2))
PRIMARY INDEX ( DatabaseName ,TableName )
;
REPLACE PROCEDURE GetRowCounts(IN DBName VARCHAR(30))
BEGIN
DECLARE SqlTxt VARCHAR(500);
FOR cur AS
SELECT
TRIM(DatabaseName) AS DBName,
TRIM(TableName) AS TabName
FROM dbc.Tables
WHERE DatabaseName = :DBName
AND TableKind = 'T'
DO
SET SqlTxt =
'INSERT INTO RowCounts ' ||
'SELECT ' ||
'''' || cur.DBName || '''' || ',' ||
'''' || cur.TabName || '''' || ',' ||
'CAST(COUNT(*) AS BIGINT)' || ',' ||
'CURRENT_TIMESTAMP(2) ' ||
'FROM ' || cur.DBName ||
'.' || cur.TabName || ';';
--CALL dbc.sysexecsql(:SqlTxt);
EXECUTE IMMEDIATE sqlTxt;
END FOR;
END;
If you can't create a table or SP you might use a VOLATILE TABLE (as DrBailey suggested) and run the INSERTs returned by following query:
SELECT
'INSERT INTO RowCounts ' ||
'SELECT ' ||
'''' || DatabaseName || '''' || ',' ||
'''' || TableName || '''' || ',' ||
'CAST(COUNT(*) AS BIGINT)' || ',' ||
'CURRENT_TIMESTAMP(2) ' ||
'FROM ' || DatabaseName ||
'.' || TableName || ';'
FROM dbc.tablesV
WHERE tablekind='V'
AND databasename='schema'
ORDER BY TableName;
But a routine like this might already exist on your system, you might ask you DBA. If it dosn't have to be 100% accurate this info might also be extracted from collected statistics.
Use dnoeth's answer but instead use create "create volatile table" this will use your spool to create the table and will delete all data when your session is closed. You need no write access to use volatile tables.
Is there any way to find the row count of a table/view from system tables with out using count(). Count() took ages to run for big tables. dbc.statsv view only include tables as we are not able to collect the stat of a view.
I need to run a query on generated generated column names.
Here's the query:
select 'col_'||4 from MY_TABLE
Note:
"4" is a variable that is passed to this query from within the Java code
MY_TABLE is a table that contain columns with names (col_4, col_5, etc..)
Inside Oracle you need use dynamic SQL. (YourVariable value is 4 for your example)
EXECUTE IMMEDIATE ' select col_' || YourVariable || ' from MY_TABLE ';
From Java you can build any SQL and execute them
To run a dynamic SELECT statement, you have two choices:
For single row selects, you use EXECUTE IMMEDIATE ... INTO:
EXECUTE IMMEDIATE 'select col_' || l_num || ' from MY_TABLE WHERE id = 37' INTO l_result;
For selecting multiple rows, you can use a dynamic cursor:
DECLARE
TYPE MyCurType IS REF CURSOR;
my_cv MyCurType;
BEGIN
OPEN emp_cv FOR 'select col_' || l_num || ' from MY_TABLE';
...
END;
This code generates a SELECT that returns the tables with their column name:
SELECT
'SELECT ' ||(
SELECT
LISTAGG(
c.TABLE_NAME || '.' || c.COLUMN_NAME || ' AS "' || c.TABLE_NAME || '.' || c.COLUMN_NAME || '"',
', '
) WITHIN GROUP(
ORDER BY
c.TABLE_NAME
) "TABLE_NAMES"
FROM
USER_TAB_COLS c
WHERE
TABLE_NAME IN(
'PESSOA',
'PESSOA_FISICA',
'PESSOA_JURIDICA'
)
)|| 'FROM PERSON;'
FROM
DUAL;