How to get data from a Oracle database table except specific columns? [duplicate] - sql

This question already has answers here:
Can you SELECT everything, but 1 or 2 fields, without writer's cramp?
(12 answers)
Closed 6 years ago.
I want to write a query to fetch data from a table except some columns which start name like given in the wildcard criteria as bellow example pseudo code. Is it possible on oracle?
(this can be done as adding column names for the select clause. but assuming there will be new columns will add in future , i want to write a more generic code)
example
Employee(id , name , age, gender)
select *
from table_name
where column_name not like a%
after query it should display a table with
Employee(id , name . gender)
the age column is not there because we are not include in the result

You can try with some dynamic SQL:
declare
vSQL varchar2(32767);
vClob clob;
begin
/* build the query */
select distinct 'select ' || listagg(column_name, ',') within group (order by column_name) over (partition by table_name)|| ' from ' || table_name
into vSQL
from user_tab_columns
where table_name = 'EMPLOYEE'
and column_name not like 'A%';
/* print the query */
dbms_output.put_line(vSQL);
/* build an XML */
select DBMS_XMLGEN.getXML(vSQL)
into vClob
from dual;
dbms_output.put_line(vClob);
/* build a CLOB with all the columns */
vSQL := replace (vSQL, ',', ' || '' | '' || ' );
execute immediate vSQL into vClob;
dbms_output.put_line(vClob);
end;
In this way you can dynamically build a query that extracts all the columns exept those matching a pattern.
After building the query, the question is how to fetch it, given that you don't know in advance what columns you are fetching.
In the example I make an XML and a single row; you can use the query in different ways, depending on your needs.

Duplicate, but I like writing PL, so here is how you could, creating a temp table, then select * from it:
declare
your_table varchar2(40) := 'CHEMIN';
select_to_tmp varchar2(4000) := 'create table ttmp as select ';
begin
-- drop temporary table if exists
begin
execute immediate 'drop table ttmp';
Exception
When others Then
dbms_output.put_line(SQLERRM);
end;
for x in (
select column_name from all_tab_columns
where table_name=your_table
and column_name not in (
-- list columns you want to exclude
'COL_A'
, 'COL_B'
)
)
loop
select_to_tmp := select_to_tmp|| x.column_name ||',';
dbms_output.put_line(x.column_name);
end loop;
-- remove last ','
select_to_tmp := substr(select_to_tmp, 1, length(select_to_tmp) -1);
-- from your table
select_to_tmp := select_to_tmp||' from '||your_table;
-- add conditions if necessary
-- select_to_tmp := select_to_tmp|| ' where rownum < 1 '
dbms_output.put_line(select_to_tmp);
-- then create the temporary table using the query you generated:
execute immediate select_to_tmp;
end;
/
SELECT * FROM ttmp;

Related

How to get value printed on Postgres

I have a requirement to translate it to an SQL script.
I am using the information schema to get all the columns of a table and print their distinct count.
I was able to get the count, but not able to print the column name properly,
PFA the below code.
I have to pass the value of the "colum_lbl" to my select clause, if I do so it is giving me an group by error.
So I passed the "colum_lbl" within quotes. now all the values of the result has hardcoded 'colum_lbl' as value, I have to replace it with the original value I read from the for Loop
Any other efficient method for this requirement will be very much appreciated. Thanks in advance
do $$
DECLARE
colum_lbl text;
BEGIN
DROP TABLE IF EXISTS tmp_table;
CREATE TABLE tmp_table
(
colnm varchar(50),
cnt integer
);
FOR colum_lbl IN
SELECT distinct column_name
FROM information_schema.columns
WHERE table_schema = 'cva_aggr'
AND table_name = 'employee' AND column_name in ('empid','empnm')
LOOP
EXECUTE
'Insert into tmp_table
SELECT '' || colum_lbl || '',count(distinct ' || colum_lbl || ')
FROM employee ';
END LOOP;
END; $$

Oracle get table names based on column value

I have table like this:
Table-1
Table-2
Table-3
Table-4
Table-5
each table is having many columns and one of the column name is employee_id.
Now, I want to write a query which will
1) return all the tables which is having this columns and
2) results should show the tables if the column is having values or empty values by passing employee_id.
e.g. show table name, column name from Table-1, Table-2,Table-3,... where employee_id='1234'.
If one of the table doesn't have this column, then it is not required to show.
I have verified with link, but it shows only table name and column name and not by passing some column values to it.
Also verified this, but here verifies from entire schema which I dont want to do it.
UPDATE:
Found a solution, but by using xmlsequence which is deprecated,
1)how do I make this code as xmltable?
2) If there are no values in the table, then output should have empty/null. or default as "YES" value
WITH char_cols AS
(SELECT /*+materialize */ table_name, column_name
FROM cols
WHERE data_type IN ('CHAR', 'VARCHAR2') and table_name in ('Table-1','Table-2','Table-3','Table-4','Table-5'))
SELECT DISTINCT SUBSTR (:val, 1, 11) "Employee_ID",
SUBSTR (table_name, 1, 14) "Table",
SUBSTR (column_name, 1, 14) "Column"
FROM char_cols,
TABLE (xmlsequence (dbms_xmlgen.getxmltype ('select "'
|| column_name
|| '" from "'
|| table_name
|| '" where upper("'
|| column_name
|| '") like upper(''%'
|| :val
|| '%'')' ).extract ('ROWSET/ROW/*') ) ) t ORDER BY "Table"
/
This query can be done in one step using the (non-deprecated) XMLTABLE.
Sample Schema
--Table-1 and Table-2 match the criteria.
--Table-3 has the right column but not the right value.
--Table-4 does not have the right column.
create table "Table-1" as select '1234' employee_id from dual;
create table "Table-2" as select '1234' employee_id from dual;
create table "Table-3" as select '4321' employee_id from dual;
create table "Table-4" as select 1 id from dual;
Query
--All tables with the column EMPLOYEE_ID, and the number of rows where EMPLOYEE_ID = '1234'.
select table_name, total
from
(
--Get XML results of dynamic query on relevant tables and columns.
select
dbms_xmlgen.getXMLType(
(
--Create a SELECT statement on each table, UNION ALL'ed together.
select listagg(
'select '''||table_name||''' table_name, count(*) total
from "'||table_name||'" where employee_id = ''1234'''
,' union all'||chr(10)) within group (order by table_name) v_sql
from user_tab_columns
where column_name = 'EMPLOYEE_ID'
)
) xml
from dual
) x
cross join
--Convert the XML data to relational.
xmltable('/ROWSET/ROW'
passing x.xml
columns
table_name varchar2(128) path 'TABLE_NAME',
total number path 'TOTAL'
);
Results
TABLE_NAME TOTAL
---------- -----
Table-1 1
Table-2 1
Table-3 0
Just try to use code below.
Pay your attention that may be nessecery clarify scheme name in loop.
This code works for my local db.
set serveroutput on;
DECLARE
ex_query VARCHAR(300);
num NUMBER;
emp_id number;
BEGIN
emp_id := <put your value>;
FOR rec IN
(SELECT table_name
FROM all_tab_columns
WHERE column_name LIKE upper('employee_id')
)
LOOP
num :=0;
ex_query := 'select count(*) from ' || rec.table_name || ' where employee_id = ' || emp_id;
EXECUTE IMMEDIATE ex_query into num;
if (num>0) then
DBMS_OUTPUT.PUT_LINE(rec.table_name);
end if;
END LOOP;
END;
I tried with the xml thing, but I get an error I cannot solve. Something about a zero size result. How difficult is it to solve this instead of raising exception?! Ask Oracle.
Anyway.
What you can do is use the COLS table to know what table has the employee_id column.
1) what table from table TABLE_LIKE_THIS (I assume column with table names is C) has this column?
select *
from COLS, TABLE_LIKE_THIS t
where cols.table_name = t
and cols.column_name = 'EMPLOYEE_ID'
-- think Oracle metadata/ think upper case
2) Which one has the value you are looking for: write a little chunk of Dynamic PL/SQL with EXECUTE IMMEDIATE to count the tables matching above condition
declare
v_id varchar2(10) := 'JP1829'; -- value you are looking for
v_col varchar2(20) := 'EMPLOYEE_ID'; -- column
n_c number := 0;
begin
for x in (
select table_name
from all_tab_columns cols
, TABLE_LIKE_THIS t
where cols.table_name = t.c
and cols.column_name = v_col
) loop
EXECUTE IMMEDIATE
'select count(1) from '||x.table_name
||' where Nvl('||v_col||', ''##'') = ''' ||v_id||'''' -- adding quotes around string is a little specific
INTO n_c;
if n_c > 0 then
dbms_output.put_line(n_C|| ' in ' ||x.table_name||' has '||v_col||'='||v_id);
end if;
-- idem for null values
-- ... ||' where '||v_col||' is null '
-- or
-- ... ||' where Nvl('||v_col||', ''##'') = ''##'' '
end loop;
dbms_output.put_line('done.');
end;
/
Hope this helps

Count the number of null values into an Oracle table?

I need to count the number of null values of all the columns in a table in Oracle.
For instance, I execute the following statements to create a table TEST and insert data.
CREATE TABLE TEST
( A VARCHAR2(20 BYTE),
B VARCHAR2(20 BYTE),
C VARCHAR2(20 BYTE)
);
Insert into TEST (A) values ('a');
Insert into TEST (B) values ('b');
Insert into TEST (C) values ('c');
Now, I write the following code to compute the number of null values in the table TEST:
declare
cnt number :=0;
temp number :=0;
begin
for r in ( select column_name, data_type
from user_tab_columns
where table_name = upper('test')
order by column_id )
loop
if r.data_type <> 'NOT NULL' then
select count(*) into temp FROM TEST where r.column_name IS NULL;
cnt := cnt + temp;
END IF;
end loop;
dbms_output.put_line('Total: '||cnt);
end;
/
It returns 0, when the expected value is 6.
Where is the error?
Thanks in advance.
Counting NULLs for each column
In order to count NULL values for all columns of a table T you could run
SELECT COUNT(*) - COUNT(col1) col1_nulls
, COUNT(*) - COUNT(col2) col2_nulls
,..
, COUNT(*) - COUNT(colN) colN_nulls
, COUNT(*) total_rows
FROM T
/
Where col1, col2, .., colN should be replaced with actual names of columns of T table.
Aggregate functions -like COUNT()- ignore NULL values, so COUNT(*) - COUNT(col) will give you how many nulls for each column.
Summarize all NULLs of a table
If you want to know how many fields are NULL, I mean every NULL of every record you can
WITH d as (
SELECT COUNT(*) - COUNT(col1) col1_nulls
, COUNT(*) - COUNT(col2) col2_nulls
,..
, COUNT(*) - COUNT(colN) colN_nulls
, COUNT(*) total_rows
FROM T
) SELECT col1_nulls + col1_nulls +..+ colN_null
FROM d
/
Summarize all NULLs of a table (using Oracle dictionary tables)
Following is an improvement in which you need to now nothing but table name and it is very easy to code a function based on it
DECLARE
T VARCHAR2(64) := '<YOUR TABLE NAME>';
expr VARCHAR2(32767);
q INTEGER;
BEGIN
SELECT 'SELECT /*+FULL(T) PARALLEL(T)*/' || COUNT(*) || ' * COUNT(*) OVER () - ' || LISTAGG('COUNT(' || COLUMN_NAME || ')', ' + ') WITHIN GROUP (ORDER BY COLUMN_ID) || ' FROM ' || T
INTO expr
FROM USER_TAB_COLUMNS
WHERE TABLE_NAME = T;
-- This line is for debugging purposes only
DBMS_OUTPUT.PUT_LINE(expr);
EXECUTE IMMEDIATE expr INTO q;
DBMS_OUTPUT.PUT_LINE(q);
END;
/
Due to calculation implies a full table scan, code produced in expr variable was optimized for parallel running.
User defined function null_fields
Function version, also includes an optional parameter to be able to run on other schemas.
CREATE OR REPLACE FUNCTION null_fields(table_name IN VARCHAR2, owner IN VARCHAR2 DEFAULT USER)
RETURN INTEGER IS
T VARCHAR2(64) := UPPER(table_name);
o VARCHAR2(64) := UPPER(owner);
expr VARCHAR2(32767);
q INTEGER;
BEGIN
SELECT 'SELECT /*+FULL(T) PARALLEL(T)*/' || COUNT(*) || ' * COUNT(*) OVER () - ' || listagg('COUNT(' || column_name || ')', ' + ') WITHIN GROUP (ORDER BY column_id) || ' FROM ' || o || '.' || T || ' t'
INTO expr
FROM all_tab_columns
WHERE table_name = T;
EXECUTE IMMEDIATE expr INTO q;
RETURN q;
END;
/
-- Usage 1
SELECT null_fields('<your table name>') FROM dual
/
-- Usage 2
SELECT null_fields('<your table name>', '<table owner>') FROM dual
/
Thank you #Lord Peter :
The below PL/SQL script works
declare
cnt number :=0;
temp number :=0;
begin
for r in ( select column_name, nullable
from user_tab_columns
where table_name = upper('test')
order by column_id )
loop
if r.nullable = 'Y' then
EXECUTE IMMEDIATE 'SELECT count(*) FROM test where '|| r.column_name ||' IS NULL' into temp ;
cnt := cnt + temp;
END IF;
end loop;
dbms_output.put_line('Total: '||cnt);
end;
/
The table name test may be replaced the name of table of your interest.
I hope this solution is useful!
The dynamic SQL you execute (this is the string used in EXECUTE IMMEDIATE) should be
select sum(
decode(a,null,1,0)
+decode(b,null,1,0)
+decode(c,null,1,0)
) nullcols
from test;
Where each summand corresponds to a NOT NULL column.
Here only one table scan is necessary to get the result.
Use the data dictionary to find the number of NULL values almost instantly:
select sum(num_nulls) sum_num_nulls
from all_tab_columns
where owner = user
and table_name = 'TEST';
SUM_NUM_NULLS
-------------
6
The values will only be correct if optimizer statistics were gathered recently and if they were gathered with the default value for the sample size.
Those may seem like large caveats but it's worth becoming familiar with your database's statistics gathering process anyway. If your database is not automatically gathering statistics or if your database is not using the default sample size those are likely huge problems you need to be aware of.
To manually gather stats for a specific table a statement like this will work:
begin
dbms_stats.gather_table_stats(user, 'TEST');
end;
/
select COUNT(1) TOTAL from table where COLUMN is NULL;

Find all tables updated on a specific date

I'm using an Oracle DB, and I'm trying to find all tables that were updated on a certain date. All of the tables that track updates have a column called DT_UPDATE. I've been trying this:
SELECT * FROM
(SELECT TABLE_NAME FROM ALL_TAB_COLUMNS WHERE COLUMN_NAME = 'DT_UPDATE')
WHERE DT_UPDATE = <date>
But get this error:
ORA-00904: "DT_UPDATE": invalid identifier
00904. 00000 - "%s: invalid identifier"
*Cause:
*Action:
Error at Line: 3 Column: 7
I've also tried aliasing the nested Select clause.
As #zaratustra said, you have to use dynamic SQL. You can do something like this:
set serveroutput on
declare
counter number;
begin
for r in (
select owner, table_name
from all_tab_columns
where column_name = 'DT_UPDATE'
) loop
execute immediate 'select count(*) from "'
|| r.owner || '"."' || r.table_name
|| '" where dt_update = :dt and rownum = 1'
into counter
using date '2014-07-07';
if counter = 1 then
dbms_output.put_line(r.table_name);
end if;
end loop;
end;
/
For each table_name (and owner, for completeness) identified in all_tab_columns as having a column called dt_update, a new dynamic select is generated, in the form:
select count(*) from "<owner>"."<table_name>"
where dt_update = date '2014-07-07'
and rownum = 1;
The rownum = 1 filter lets the query execution stop as soon as a matching row is found; since you said you want to know which tables were updated, not how many rows or exactly which rows, if one row matches then that is all you really need to know. So for every table the dynamic query gets either 0 or 1.
For any tables that have at least one row matching the date, this printd the table name using dbms_output, so you have to have that enabled - with set serveroutput on, or with the DBMS_OUTPUT panel in SQL Developer, or your favourite client's equivalent.
If I create some tables with that column, but only populate one with the date I'm looking for:
create table tab1 (dt_update date);
create table tab2 (dt_update date);
create table tab3 (dt_update date);
insert into tab1 values (trunc(sysdate) - 1);
insert into tab2 values (trunc(sysdate));
... then running my anonymous block produces:
anonymous block completed
TAB1
Use your own target date, obviously. This assumes your date field doesn't contain a time component. If it does then you'd need to turn that into a range to cover the whole day.
You could also turn this into a pipelined function that takes a date as an argument; this also handles date fields with time elements:
create or replace function get_updated_tables(p_date date)
return sys.odcivarchar2list pipelined as
counter number;
begin
for r in (
select owner, table_name
from all_tab_columns
where column_name = 'DT_UPDATE'
) loop
execute immediate 'select count(*) from "'
|| r.owner || '"."' || r.table_name
|| '" where dt_update >= :dt1 and dt_update < :dt2'
|| ' and rownum = 1'
into counter
using p_date, p_date + interval '1' day;
if counter = 1 then
pipe row (r.table_name);
end if;
end loop;
end;
/
Then you can query it with:
select column_value from table(get_updated_tables(date '2014-07-07'));
COLUMN_VALUE
------------------------------
TAB1
Dynamic SQL is interesting, as you said in a comment, but should only be used when necessary. The generated statement can't be parsed until it's executed, so you might not spot syntax or other errors until run-time. Also make sure you use bind variables for values (but not object names) to avoid SQL injection.
Let's assume we have three tables with the field dt_update, and each of them has one record (doesn't matter if more):
create table tt1 (
dt_update date
);
insert into tt1 values (sysdate);
create table tt2 (
dt_update date
);
insert into tt2 values (sysdate - 1);
create table tt3 (
dt_update date
);
insert into tt3 values (sysdate - 2);
This PL/SQL anonym block prints only tables' names that have record with the value of the column dt_update more than or equals today:
declare
type table_names_tp is table of user_tables.table_name%type index by binary_integer;
table_names table_names_tp;
l_res number(1);
l_deadline date := to_date('2014-07-08', 'YYYY-MM-DD');
begin
select table_name
BULK COLLECT INTO table_names
from user_tab_columns
where lower(column_name) = 'dt_update'
;
for i in table_names.first..table_names.last
loop
execute immediate 'select count(*) from dual where exists (select null from ' || table_names(i) || ' where dt_update >= :dead_line)'
into l_res
using l_deadline;
if l_res = 1
then
DBMS_OUTPUT.put_line('Table ' || table_names(i) || ' was updated after ' || l_deadline);
end if;
end loop;
end;
You can use this code as an example to start writing your code. Pay carefully attention to protect yourself from SQL injections, DO NOT(!) use concatenation of your values, always use bind variables instead. It also helps you to store a cached query plan in SGA, the application will read data from the SGA area and perform soft parsing.

Dynamically select the columns to be used in a SELECT statement

I would love to be able to use the system tables (Oracle in this case) to drive which fields are used in a SELECT statement. Something like:
SELECT
(
select column_name
from all_tab_cols
where table_Name='CLARITY_SER'
AND OWNER='CLARITY'
AND data_type='DATE'
)
FROM CLARITY_SER
This syntax doesn't work, as the subquery returns multiple rows, instead of one row with multiple columns.
Is it possible to generate a SQL statement dynamically by querying the table schema information in order to select only certain columns?
** edit **
Do this without using a function or procedure, if possible.
You can do this:
declare
l_sql varchar2(32767);
rc sys_refcursor;
begin
l_sql := 'select ';
for r in
( select column_name
from all_tab_cols
where table_Name='CLARITY_SER'
AND OWNER='CLARITY'
AND data_type='DATE'
)
loop
l_sql := l_sql || r.column_name || ',';
end loop;
l_sql := rtrim(l_sql,',') || ' from clarity_ser';
open rc for l_sql;
...
end;
No, it's not possible to specify a column list dynamically in SQL. You'll need to use a procedural language to run the first query, use that to construct a second query, then run the second query.
You could use dynamic SQL. Create a function that takes the table name, owner, data type, executes the inner query and returns a comma-separated list of column names, or an array table if you prefer. Then construct the outer query and execute it with execute immediate.
CREATE FUNCTION get_column_list(
table_name IN varchar2,
owner_name IN varchar2,
data_type IN varchar2)
RETURN varchar2
IS
BEGIN
...... (get columns and return comma-separated list)
END;
/
If your function returns a comma-separated list you can inline it:
execute immediate 'select ' || get_column_list(table_name, owner_name, datatype) || ' from ' || table_name
Admittedly it's a long time since I played with oracle so I may be a bit off but I'm pretty sure this is quite doable.
In SQLPlus you could do this:
COLUMN cols NEW_VALUE cols
SELECT max( ltrim( sys_connect_by_path( column_name, ',' ), ',' ) ) cols
FROM
(
select rownum rn, column_name
from all_tab_cols
where table_Name='CLARITY_SER'
and OWNER='CLARITY'
AND data_type='DATE'
)
start with rn = 1 connect by rn = prior rn +1
;
select &cols from clarity.clarity_ser;