I have a requirement to translate it to an SQL script.
I am using the information schema to get all the columns of a table and print their distinct count.
I was able to get the count, but not able to print the column name properly,
PFA the below code.
I have to pass the value of the "colum_lbl" to my select clause, if I do so it is giving me an group by error.
So I passed the "colum_lbl" within quotes. now all the values of the result has hardcoded 'colum_lbl' as value, I have to replace it with the original value I read from the for Loop
Any other efficient method for this requirement will be very much appreciated. Thanks in advance
do $$
DECLARE
colum_lbl text;
BEGIN
DROP TABLE IF EXISTS tmp_table;
CREATE TABLE tmp_table
(
colnm varchar(50),
cnt integer
);
FOR colum_lbl IN
SELECT distinct column_name
FROM information_schema.columns
WHERE table_schema = 'cva_aggr'
AND table_name = 'employee' AND column_name in ('empid','empnm')
LOOP
EXECUTE
'Insert into tmp_table
SELECT '' || colum_lbl || '',count(distinct ' || colum_lbl || ')
FROM employee ';
END LOOP;
END; $$
Related
postgres 12
I am trying to loop through a table which has schema , table_names and columns
I want to do various things like finding nulls ,row count etc. I failed at the first hurdle trying to update the col records.
table i am using
CREATE TABLE test.table_study (
table_schema text,
table_name text,
column_name text,
records int,
No_Nulls int,
No_Blanks int,
per_pop int
);
I populate the table with some schema names ,tables and columns from information_schema.columns
insert into test.table_study select table_schema, table_name, column_name
from information_schema.columns
where table_schema like '%white'
order by table_schema, table_name, ordinal_position;
I want to populate the rest with a function
function :-
CREATE OR REPLACE PROCEDURE test.insert_data_population()
as $$
declare s record;
declare t record;
declare c record;
BEGIN
FOR s IN SELECT distinct table_schema FROM test.table_study
LOOP
FOR t IN SELECT distinct table_name FROM test.table_study where table_schema = s.table_schema
loop
FOR c IN SELECT column_name FROM test.table_study where table_name = t.table_name
LOOP
execute 'update test.table_study set records = (select count(*) from ' || s.table_schema || '.' || t.table_name || ') where table_study.table_name = '|| t.table_name ||';';
END LOOP;
END LOOP;
END LOOP;
END;
$$
LANGUAGE plpgsql;
I get this error SQL Error [42703]: ERROR: column "age" does not exist. the table age does exist.
when I take out the where clause
execute 'update referralunion.testinsert ti set records = (select count(*) from ' || s.table_schema || '.' || t.table_name || ') ;';
it works, I just cant figure out whats wrong?
Your procedure is structured entirely wrong. What it results in is an attempt to get every column name for every table name in every schema. I would guess results in your column does not exist error. Further is shows procedural thinking. SQL requires think in terms of sets. Below I use basically your query to demonstrate then a revised version which uses a single loop.
-- setup (dropping schema references)
create table table_study (
table_schema text,
table_name text,
column_name text,
records int,
no_nulls int,
no_blanks int,
per_pop int
);
insert into table_study(table_schema, table_name, column_name)
values ('s1','t1','age')
, ('s2','t1','xyz');
-- procedure replacing EXECUTE with Raise Notice.
create or replace procedure insert_data_population()
as $$
declare
s record;
t record;
c record;
line int = 0;
begin
for s in select distinct table_schema from table_study
loop
for t in select distinct table_name from table_study where table_schema = s.table_schema
loop
for c in select column_name from table_study where table_name = t.table_name
loop
line = line+1;
raise notice '%: update table_study set records = (select count(*) from %.% where table_study.table_name = %;'
, line, s.table_schema, t.table_name, c.column_name;
end loop;
end loop;
end loop;
end;
$$
language plpgsql;
Run procedure
do $$
begin
call insert_data_population();
end;
$$;
RESULTS
1: update table_study set records = (select count(*) from s2.t1 where table_study.table_name = age; 2: update table_study set records = (select count(*) from s2.t1 where table_study.table_name = xyz; 3: update table_study set records = (select count(*) from s1.t1 where table_study.table_name = age; 4: update table_study set records = (select count(*) from s1.t1 where table_study.table_name = xyz;
Notice lines 2 and 3. Each references a column name that does not exist in the table. This results from the FOR structure with the same table name in different schema.
Revision for Single Select statement with Single For loop.
create or replace
procedure insert_data_population()
language plpgsql
as $$
declare
s record;
line int = 0;
begin
for s in select distinct table_schema, table_name, column_name from table_study
loop
line = line+1;
raise notice '%: update table_study set records = (select count(*) from %.% where table_study.table_name = %;'
, line, s.table_schema, s.table_name, s.column_name;
end loop;
end;
$$;
do $$
begin
call insert_data_population();
end;
$$;
RESULTS
1: update table_study set records = (select count(*) from s2.t1 where table_study.table_name = xyz;
2: update table_study set records = (select count(*) from s1.t1 where table_study.table_name = age;
Note: In Postgres DECLARE begins a block. It is not necessary to declared each variable. I would actually consider it bad practice. In theory it could require an end for each declare as each could be considered a nested block. Fortunately Postgres does not require this.
I have a timescale db with multiple tables having the same structure.
I want to retrieve the recent row from each table where a value is true.
My logic is to
retrieve all the tablenames for the tables where this condition can be true
loop over list of tablenames and select the rows where the condition is met
I get an syntax error on the FOR loop but I expect that I do more things wrong.
Can someone suggest a solution please? Thank you in advance.
DECLARE
tablename text;
BEGIN
FOR tablename IN
SELECT table_name FROM information_schema.tables
WHERE table_name LIKE 'ohlc%'
LOOP
SELECT WHERE tablename.is_active is TRUE
ORDER BY time_stamp DESC
Limit 1
END LOOP;
END;
translate your problem
find table that have specific column name in schema.
How to find a table having a specific column in postgresql
first condition meet then loop. Function to loop through and select data from multiple tables
most tricky issue is quote_ident.
create or replace function test0()
returns table (_is_active boolean, id int) as
$$
declare tbl text;
begin
for tbl in
select quote_ident( table_name)
from information_schema.columns
where table_schema = 'public'
and table_name ilike 'ohlc%'
and column_name = 'is_active'
loop
return query EXECUTE
'select ' || quote_ident('is_active') || ' , ' || quote_ident('id') || ' from ' || tbl || ' where '|| quote_ident('is_active') ||' is true';
end loop;
end
$$ language plpgsql;
I have table like this:
Table-1
Table-2
Table-3
Table-4
Table-5
each table is having many columns and one of the column name is employee_id.
Now, I want to write a query which will
1) return all the tables which is having this columns and
2) results should show the tables if the column is having values or empty values by passing employee_id.
e.g. show table name, column name from Table-1, Table-2,Table-3,... where employee_id='1234'.
If one of the table doesn't have this column, then it is not required to show.
I have verified with link, but it shows only table name and column name and not by passing some column values to it.
Also verified this, but here verifies from entire schema which I dont want to do it.
UPDATE:
Found a solution, but by using xmlsequence which is deprecated,
1)how do I make this code as xmltable?
2) If there are no values in the table, then output should have empty/null. or default as "YES" value
WITH char_cols AS
(SELECT /*+materialize */ table_name, column_name
FROM cols
WHERE data_type IN ('CHAR', 'VARCHAR2') and table_name in ('Table-1','Table-2','Table-3','Table-4','Table-5'))
SELECT DISTINCT SUBSTR (:val, 1, 11) "Employee_ID",
SUBSTR (table_name, 1, 14) "Table",
SUBSTR (column_name, 1, 14) "Column"
FROM char_cols,
TABLE (xmlsequence (dbms_xmlgen.getxmltype ('select "'
|| column_name
|| '" from "'
|| table_name
|| '" where upper("'
|| column_name
|| '") like upper(''%'
|| :val
|| '%'')' ).extract ('ROWSET/ROW/*') ) ) t ORDER BY "Table"
/
This query can be done in one step using the (non-deprecated) XMLTABLE.
Sample Schema
--Table-1 and Table-2 match the criteria.
--Table-3 has the right column but not the right value.
--Table-4 does not have the right column.
create table "Table-1" as select '1234' employee_id from dual;
create table "Table-2" as select '1234' employee_id from dual;
create table "Table-3" as select '4321' employee_id from dual;
create table "Table-4" as select 1 id from dual;
Query
--All tables with the column EMPLOYEE_ID, and the number of rows where EMPLOYEE_ID = '1234'.
select table_name, total
from
(
--Get XML results of dynamic query on relevant tables and columns.
select
dbms_xmlgen.getXMLType(
(
--Create a SELECT statement on each table, UNION ALL'ed together.
select listagg(
'select '''||table_name||''' table_name, count(*) total
from "'||table_name||'" where employee_id = ''1234'''
,' union all'||chr(10)) within group (order by table_name) v_sql
from user_tab_columns
where column_name = 'EMPLOYEE_ID'
)
) xml
from dual
) x
cross join
--Convert the XML data to relational.
xmltable('/ROWSET/ROW'
passing x.xml
columns
table_name varchar2(128) path 'TABLE_NAME',
total number path 'TOTAL'
);
Results
TABLE_NAME TOTAL
---------- -----
Table-1 1
Table-2 1
Table-3 0
Just try to use code below.
Pay your attention that may be nessecery clarify scheme name in loop.
This code works for my local db.
set serveroutput on;
DECLARE
ex_query VARCHAR(300);
num NUMBER;
emp_id number;
BEGIN
emp_id := <put your value>;
FOR rec IN
(SELECT table_name
FROM all_tab_columns
WHERE column_name LIKE upper('employee_id')
)
LOOP
num :=0;
ex_query := 'select count(*) from ' || rec.table_name || ' where employee_id = ' || emp_id;
EXECUTE IMMEDIATE ex_query into num;
if (num>0) then
DBMS_OUTPUT.PUT_LINE(rec.table_name);
end if;
END LOOP;
END;
I tried with the xml thing, but I get an error I cannot solve. Something about a zero size result. How difficult is it to solve this instead of raising exception?! Ask Oracle.
Anyway.
What you can do is use the COLS table to know what table has the employee_id column.
1) what table from table TABLE_LIKE_THIS (I assume column with table names is C) has this column?
select *
from COLS, TABLE_LIKE_THIS t
where cols.table_name = t
and cols.column_name = 'EMPLOYEE_ID'
-- think Oracle metadata/ think upper case
2) Which one has the value you are looking for: write a little chunk of Dynamic PL/SQL with EXECUTE IMMEDIATE to count the tables matching above condition
declare
v_id varchar2(10) := 'JP1829'; -- value you are looking for
v_col varchar2(20) := 'EMPLOYEE_ID'; -- column
n_c number := 0;
begin
for x in (
select table_name
from all_tab_columns cols
, TABLE_LIKE_THIS t
where cols.table_name = t.c
and cols.column_name = v_col
) loop
EXECUTE IMMEDIATE
'select count(1) from '||x.table_name
||' where Nvl('||v_col||', ''##'') = ''' ||v_id||'''' -- adding quotes around string is a little specific
INTO n_c;
if n_c > 0 then
dbms_output.put_line(n_C|| ' in ' ||x.table_name||' has '||v_col||'='||v_id);
end if;
-- idem for null values
-- ... ||' where '||v_col||' is null '
-- or
-- ... ||' where Nvl('||v_col||', ''##'') = ''##'' '
end loop;
dbms_output.put_line('done.');
end;
/
Hope this helps
This question already has answers here:
Can you SELECT everything, but 1 or 2 fields, without writer's cramp?
(12 answers)
Closed 6 years ago.
I want to write a query to fetch data from a table except some columns which start name like given in the wildcard criteria as bellow example pseudo code. Is it possible on oracle?
(this can be done as adding column names for the select clause. but assuming there will be new columns will add in future , i want to write a more generic code)
example
Employee(id , name , age, gender)
select *
from table_name
where column_name not like a%
after query it should display a table with
Employee(id , name . gender)
the age column is not there because we are not include in the result
You can try with some dynamic SQL:
declare
vSQL varchar2(32767);
vClob clob;
begin
/* build the query */
select distinct 'select ' || listagg(column_name, ',') within group (order by column_name) over (partition by table_name)|| ' from ' || table_name
into vSQL
from user_tab_columns
where table_name = 'EMPLOYEE'
and column_name not like 'A%';
/* print the query */
dbms_output.put_line(vSQL);
/* build an XML */
select DBMS_XMLGEN.getXML(vSQL)
into vClob
from dual;
dbms_output.put_line(vClob);
/* build a CLOB with all the columns */
vSQL := replace (vSQL, ',', ' || '' | '' || ' );
execute immediate vSQL into vClob;
dbms_output.put_line(vClob);
end;
In this way you can dynamically build a query that extracts all the columns exept those matching a pattern.
After building the query, the question is how to fetch it, given that you don't know in advance what columns you are fetching.
In the example I make an XML and a single row; you can use the query in different ways, depending on your needs.
Duplicate, but I like writing PL, so here is how you could, creating a temp table, then select * from it:
declare
your_table varchar2(40) := 'CHEMIN';
select_to_tmp varchar2(4000) := 'create table ttmp as select ';
begin
-- drop temporary table if exists
begin
execute immediate 'drop table ttmp';
Exception
When others Then
dbms_output.put_line(SQLERRM);
end;
for x in (
select column_name from all_tab_columns
where table_name=your_table
and column_name not in (
-- list columns you want to exclude
'COL_A'
, 'COL_B'
)
)
loop
select_to_tmp := select_to_tmp|| x.column_name ||',';
dbms_output.put_line(x.column_name);
end loop;
-- remove last ','
select_to_tmp := substr(select_to_tmp, 1, length(select_to_tmp) -1);
-- from your table
select_to_tmp := select_to_tmp||' from '||your_table;
-- add conditions if necessary
-- select_to_tmp := select_to_tmp|| ' where rownum < 1 '
dbms_output.put_line(select_to_tmp);
-- then create the temporary table using the query you generated:
execute immediate select_to_tmp;
end;
/
SELECT * FROM ttmp;
I would love to be able to use the system tables (Oracle in this case) to drive which fields are used in a SELECT statement. Something like:
SELECT
(
select column_name
from all_tab_cols
where table_Name='CLARITY_SER'
AND OWNER='CLARITY'
AND data_type='DATE'
)
FROM CLARITY_SER
This syntax doesn't work, as the subquery returns multiple rows, instead of one row with multiple columns.
Is it possible to generate a SQL statement dynamically by querying the table schema information in order to select only certain columns?
** edit **
Do this without using a function or procedure, if possible.
You can do this:
declare
l_sql varchar2(32767);
rc sys_refcursor;
begin
l_sql := 'select ';
for r in
( select column_name
from all_tab_cols
where table_Name='CLARITY_SER'
AND OWNER='CLARITY'
AND data_type='DATE'
)
loop
l_sql := l_sql || r.column_name || ',';
end loop;
l_sql := rtrim(l_sql,',') || ' from clarity_ser';
open rc for l_sql;
...
end;
No, it's not possible to specify a column list dynamically in SQL. You'll need to use a procedural language to run the first query, use that to construct a second query, then run the second query.
You could use dynamic SQL. Create a function that takes the table name, owner, data type, executes the inner query and returns a comma-separated list of column names, or an array table if you prefer. Then construct the outer query and execute it with execute immediate.
CREATE FUNCTION get_column_list(
table_name IN varchar2,
owner_name IN varchar2,
data_type IN varchar2)
RETURN varchar2
IS
BEGIN
...... (get columns and return comma-separated list)
END;
/
If your function returns a comma-separated list you can inline it:
execute immediate 'select ' || get_column_list(table_name, owner_name, datatype) || ' from ' || table_name
Admittedly it's a long time since I played with oracle so I may be a bit off but I'm pretty sure this is quite doable.
In SQLPlus you could do this:
COLUMN cols NEW_VALUE cols
SELECT max( ltrim( sys_connect_by_path( column_name, ',' ), ',' ) ) cols
FROM
(
select rownum rn, column_name
from all_tab_cols
where table_Name='CLARITY_SER'
and OWNER='CLARITY'
AND data_type='DATE'
)
start with rn = 1 connect by rn = prior rn +1
;
select &cols from clarity.clarity_ser;