I have the following script to count all line of every table for a specific owner. It works perfectly.
However, some tables have a specific column called 'OLD' and other don't...
My current script does not take into account if this column exists or not:
DECLARE
val NUMBER;
BEGIN
FOR I IN (SELECT table_name FROM all_tables where owner='myowner') LOOP
EXECUTE IMMEDIATE 'SELECT count(*) FROM myowner.' || i.table_name INTO val;
DBMS_OUTPUT.PUT_LINE(i.table_name || ';' || val );
END LOOP;
END;
So what I would like to add is something like:
if the OLD column exists, take it in account when OLD=0 (where OLD=0), if it does not exist keep doing the 'normal' count without taking this column into account.
Hope I've been clear enough ;)
Thanks a lot!
An example:
let's say I have 2 tables:
Table1 - columns A B C with the following data:
1 "test" "Steve"
2 "test2" "George"
Table2 - columns E F G OLD with the following data:
1 "test3" "Martin" 0
2 "test4" "Lucas" 0
3 "test5" "Marley" 0
4 "test6" "Bob" 55
The result should then be:
Table1;2 -> there was not the 'OLD' column so I made a simples count
which returned 2
Table2;3 -> there was the 'OLD' column so I made a count where OLD=0
and it returned then 3
Try:
DECLARE
val NUMBER;
BEGIN
FOR I IN (
select a.table_name, c.column_name
from all_tables a
left join all_tab_cols c
ON a.owner = c.owner and a.table_name = c.table_name and c.column_name = 'OLD'
where a.owner = 'MYOWNER'
)
LOOP
IF i.column_name IS NULL THEN
EXECUTE IMMEDIATE 'SELECT count(*) FROM MYOWNER.' || i.table_name INTO val;
ELSE
EXECUTE IMMEDIATE 'SELECT count(*) FROM MYOWNER.' || i.table_name
|| ' WHERE old = 0' INTO val;
END IF;
DBMS_OUTPUT.PUT_LINE(i.table_name || ';' || val );
END LOOP;
END;
Related
I need to know what columns of one table have only null values. I understand that I should do a loop in user_tab_columns. But how detect only columns with null value?
Thanks and sorry for my English
To perform a query where you don't know the column identifies in advance, you need to use dynamic SQL. Assuming you already know the table is not empty, you could do something like:
declare
l_count pls_integer;
begin
for r in (
select table_name, column_name
from user_tab_columns
where table_name = 'T42'
and nullable = 'Y'
)
loop
execute immediate 'select count(*) '
|| ' from "' || r.table_name || '"'
|| ' where "' || r.column_name || '" is not null'
into l_count;
if l_count = 0 then
dbms_output.put_line('Table ' || r.table_name
|| ' column ' || r.column_name || ' only has nulls');
end if;
end loop;
end;
/
Remember to set serveroutput on or your client's equivalent before executing.
The cursor gets the columns from the table which are declared as nullable (if they aren't, not much point checking them; though this won't catch explicit check constraints). For each column it builds a query to count the rows where that column is not null. If that count is zero then it didn't find any that are not null, therefore they all are. Again, assuming you know the table isn't empty before you start.
I've included the table name in the cursor select list and references so you only need to change the name in one place to search a different table, or you could use a variable for that name. Or check multiple tables at once by changing that filter.
You may get better performance by selecting a dummy value from any non-null row, with a rownum stop check - which means it will stop as soon as it finds a non-null value, rather than having to check every row to get an actual count:
declare
l_flag pls_integer;
begin
for r in (
select table_name, column_name
from user_tab_columns
where table_name = 'T42'
and nullable = 'Y'
)
loop
begin -- inner block to allow exception trapping within loop
execute immediate 'select 42 '
|| ' from "' || r.table_name || '"'
|| ' where "' || r.column_name || '" is not null'
|| ' and rownum < 2'
into l_flag;
-- if this foudn anything there is a non-null value
exception
when no_data_found then
dbms_output.put_line('Table ' || r.table_name
|| ' column ' || r.column_name || ' only has nulls');
end;
end loop;
end;
/
or you could do something similar with an exists() check.
If you don't know that the table has data then you can do a simple count(*) from the table before the loop to check if it is empty, and report that instead:
...
begin
if l_count = 0 then
dbms_output.put_line('Table is empty');
return;
end if;
...
Or you could combine it with the cursor query, but this would need some work if you wanted to check multiple tables at once as it would stop as soon as it found any empty one (have to leave you something to do... *8-)
declare
l_count_any pls_integer;
l_count_not_null pls_integer;
begin
for r in (
select table_name, column_name
from user_tab_columns
where table_name = 'T42'
and nullable = 'Y'
)
loop
execute immediate 'select count(*),'
|| ' count(case when "' || r.column_name || '" is not null then 1 end)'
|| ' from "' || r.table_name || '"'
into l_count_any, l_count_not_null;
if l_count_any = 0 then
dbms_output.put_line('Table ' || r.table_name || ' is empty');
exit; -- only report once
elsif l_count_not_null = 0 then
dbms_output.put_line('Table ' || r.table_name
|| ' column ' || r.column_name || ' only has nulls');
end if;
end loop;
end;
/
You could of course populate a collection or make it a pipelined function or whatever if you didn't want to reply on dbms_output, but I assume this is a one-off check so it is probably acceptable.
You can loop though your columns and count null rows. If it is same with your table count, then that column has only null values.
The first question is: one column with zero row could be regarded as only (null) value containing column. But it can remain your decision: the scripts below provide solutions to both ways. (In my opinion: no. The empty columns is not a column with only (null) value)
If you want to know the (null) values about one table you can it with count(column):
select count(column) from table
and when the count(column) = 0 then the column has only (null) value or has no value. (So, you cannot make a correct decision).
E.g. The following three tables (x, y and z) has the following columns:
select * from x;
N_X M_X
---------------
100 (null)
200 (null)
300 (null)
select * from y;
N_Y M_Y
---------------
101 (null)
202 (null)
303 apple
select * from z;
N_Z M_Z
---------------
The count() selects:
select count(n_x), count(m_x) from x;
COUNT(N_X) COUNT(M_X)
-----------------------
3 0
select count(n_y), count(m_y) from y;
COUNT(N_Y) COUNT(M_Y)
-----------------------
3 1
select count(n_z), count(m_Z) from z;
COUNT(N_Z) COUNT(M_Z)
-----------------------
0 0
As you can see, the difference between x and y is appears but you cannot decide that the table z has no rows or only full of (null) values.
The general solution:
I have separeted the schema and the db level but the basic idea is common:
Schema level: the current user’s table
DB level: all users or a chosen schema
The number of (null) in one columns:
all_tab_columns.num_nulls
(Or: user_tab_columns, num_nulls).
And we need the num_rows of the table:
all_all_tables.num_rows
(Or: user_all_tables.num_rows)
Where the num_nulls equals to num_rows there are only (null) values.
Firstly, you need to run the DBMS_STATS for refresh the statistics.
on database level:
exec DBMS_STATS.GATHER_DATABASE_STATS;
(it can use a lot of resources)
on schema level:
EXEC DBMS_STATS.gather_schema_stats('TRANEE',DBMS_STATS.AUTO_SAMPLE_SIZE); (owner = tranee)
-- column with zero row = column has only (null) values -> exclude num_nulls > 0 condition
-- column with zero row <> column has only (null) values -> include num_nulls > 0 condition
the scripts:
-- 1. current user
select
a.table_name,
a.column_name,
a.num_nulls,
b.num_rows
from user_tab_columns a, user_all_tables b
where a.table_name = b.table_name
and num_nulls = num_rows
and num_nulls > 0;
-- 2. chosen user / all user -> exclude the a.owner = 'TRANEE' condition
select
a.owner,
a.table_name,
a.column_name,
a.num_nulls,
b.num_rows
from all_tab_columns a, all_all_tables b
where a.owner = b.owner
and a.table_name = b.table_name
and a.owner = 'TRANEE'
and num_nulls = num_rows
and num_nulls > 0;
TABLE_NAME COLUMN_NAME NUM_NULLS NUM_ROWS
----------------------------------------------------
LEADERS COMM 4 4
EMP_ACTION ACTION 12 12
X M_X 3 3
These tables and columns have only (null) values in tranee schema.
I'm trying to select table columns that are same in myTable1 and myTable2 and than do a select statement upon those columns in myTable1.
So this works, getting same columns into list in variable:
set columnNames = (select listagg(a.*,',') from
(SELECT column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'myTable1') a
inner join
(SELECT column_name
FROM INFORMATION_SCHEMA.COLUMNS b
WHERE TABLE_NAME = 'myTable2') b
on a.column_name = b.column_name);
but this fails:
SET sqlText = 'SELECT ' + $columnNames + ' from myDB.MySchema.myTable
where sold > 0 and hour = ''2022-08-01 13:00:00.000''
and id = ''zzzseee2323''
limit 10';
with error:
000002 (0A000): Unsupported feature 'assignment from non-constant
source expression'.
So how to concatenate that variable with text and execute statement?
I would on the end execute just with
EXECUTE IMMEDIATE $sqlText
The size of session variableSET is 256bytes and most likely will not be usable in this context.
This case could be resolved using Snowflake Scripting:
DECLARE
res RESULTSET;
query TEXT;
BEGIN
LET columnNames := (SELECT listagg(a.*,',') FROM
(SELECT column_name
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'myTable1') a
inner join
(SELECT column_name
FROM INFORMATION_SCHEMA.COLUMNS b
WHERE TABLE_NAME = 'myTable2') b
on a.column_name = b.column_name);
query := 'SELECT ' + :columnNames + '
from myDB.MySchema.myTable
where sold > 0 and hour = ''2022-08-01 13:00:00.000''
and id = ''zzzseee2323''
limit 10';
res := (EXECUTE IMMEDIATE :query);
RETURN TABLE (res);
END;
For example I have a table with data:
Column1 Column2 Column3 ... Column1xx
1 a 10 x
2 b 20 y
3 c 30 z
What SQL query would get me only the last column?
Column1xx
x
y
z
Note: Column Name and Index of the last column is unknown
Fill your_table_schema and your_table_name. Originated from the post https://stackoverflow.com/a/38993544/18396231
SELECT COLUMN_NAME, ORDINAL_POSITION
FROM information_schema.COLUMNS
WHERE TABLE_SCHEMA = '{your_table_schema}'
AND TABLE_NAME ='{your_table_name}'
ORDER BY ORDINAL_POSITION DESC
LIMIT 1;
create so test data:
CREATE TABLE test.test.testo(aa number, bb number, cc number);
INSERT INTO test.test.testo values (1,10,100),(2,20,200),(3,30,300);
use tasozgurcem11 SQL to show we can get the column name:
SELECT COLUMN_NAME
FROM test.information_schema.COLUMNS
WHERE TABLE_SCHEMA = 'TEST'
AND TABLE_NAME ='TESTO'
ORDER BY ORDINAL_POSITION DESC
LIMIT 1;
and the put that into Snowflake Scripting to having it run dynamically:
DECLARE
query STRING;
col_name STRING;
res RESULTSET;
BEGIN
LET _database := 'TEST';
LET _table := 'TESTO';
LET _schema := 'TEST';
SELECT COLUMN_NAME INTO :col_name
FROM test.information_schema.COLUMNS
WHERE TABLE_SCHEMA = :_schema
AND TABLE_NAME = :_table
ORDER BY ORDINAL_POSITION DESC
LIMIT 1;
QUERY := 'SELECT '|| :col_name ||' FROM '|| _database ||'.'|| _schema ||'.'|| _table ||';';
res := (EXECUTE IMMEDIATE :QUERY);
return table(res);
END;
gives:
CC
100
200
300
this may help you where a function returns the last column name.
create or replace TABLE TEST_TABLE (
ID NUMBER(38,0),
C1 NUMBER(38,0),
C2 VARCHAR(16777216) MASKING POLICY TEST_HK.PUBLIC.MASK_DATA,
C3 DATE
);
CREATE OR REPLACE function LST_COLUMN(tab_nam varchar)
RETURNS VARCHAR
AS 'select column_name from information_schema.columns where table_name = tab_nam and ORDINAL_POSITION
= (SELECT max(ORDINAL_POSITION) from information_schema.columns where table_name = tab_nam)';
select LST_COLUMN('TEST_TABLE') from test_table;
I need to 'recreate' over 50 tables (in Oracle) with CREATE AS SELECT statements. However, all this tables will have one column modified using data from another table. Is there a way to achieve this without declaring each column in the SELECT statement?
Something like:
CREATE TABLE table_name_copy AS SELECT *, (SELECT col_name FROM other_table WHERE other_table.col_id = table_name.col_id) AS col_name FROM table_name`
Basically on all tables I have a column which needs to be replaced with the data in the other_table column.
Generate the SQL string as such:
SELECT 'CREATE TABLE table_name_copy AS SELECT '
|| LISTAGG (column_name, ', ') WITHIN GROUP (ORDER BY column_name)
|| ', (SELECT col_name FROM other_table
WHERE other_table.col_id = table_name.col_id) AS col_name'
|| ' FROM table_name'
FROM all_tab_cols
WHERE owner = 'OWNER'
AND table_name = 'TABLE_NAME'
AND column_name != 'COL_NAME'
If you want to run the above statement, you could use EXECUTE IMMEDIATE:
DECLARE
v_sql VARCHAR2(10000);
BEGIN
SELECT 'CREATE TABLE table_name_copy AS SELECT '
|| LISTAGG (column_name, ', ') WITHIN GROUP (ORDER BY column_name)
|| ', (SELECT col_name FROM other_table
WHERE other_table.col_id = table_name.col_id) AS col_name'
|| ' FROM table_name'
INTO v_sql
FROM all_tab_cols
WHERE owner = 'OWNER'
AND table_name = 'TABLE_NAME'
AND column_name != 'COL_NAME';
EXECUTE IMMEDIATE v_sql;
END;
/
If col_id column is fixed for both of the joined tables,
you may use user_tab_columns and user_tables dictionary views through the schema to produce new tables named as "table_name_copy" by using the following mechanism :
declare
v_ddl varchar2(4000);
v_cln varchar2(400);
begin
for c in ( select *
from user_tables t
where t.table_name in
( select c.table_name
from user_tab_columns c
where c.column_name = 'COL_ID' )
order by t.table_name )
loop
v_ddl := 'create table '||c.table_name||'_copy as
select ';
for d in ( select listagg('t1.'||column_name, ',') within group ( order by column_name ) cln
from user_tab_columns
where table_name = c.table_name
and column_name != 'COL_ID' )
loop
v_cln := v_cln||d.cln;
end loop;
v_ddl := v_ddl||v_cln;
v_ddl := v_ddl||', t2.col_id t2_id
from '||c.table_name||' t1
left outer join other_table t2 on ( t1.col_id = t2.col_id )';
execute immediate v_ddl;
v_ddl := null;
v_cln := null;
end loop;
end;
Maybe you can use a simple join and an asterisk to return all columns from the first table, like that:
CREATE TABLE table_name_copy AS
SELECT * FROM (
SELECT tab1.*, tab2.column_name
FROM table_name tab1 LEFT JOIN other_table tab2 ON tab1.col_id = tab2.col_id
);
I would try this (but I don't have Oracle SQL to test on so please leave me the benefit of the doubt)
CREATE TABLE table_name_copy AS
SELECT * FROM (
SELECT *, (SELECT col_name FROM other_table WHERE other_table.col_id = table_name.col_id) as col_name
FROM table_name`
)
edit:
then run
ALTER TABLE table_name_copy DROP COLUMN <old column>
to remove the column you don't need any more
I have a table containing hundreds of columns many of which are null, and I would like have my select statement so that only those columns containing a value are returned. It would help me analyze data better. Something like:
Select (non null columns) from tablename;
I want to select all columns which have at least one non-null value.
Can this be done?
Have a look as statistics information, it may be useful for you:
SQL> exec dbms_stats.gather_table_stats('SCOTT','EMP');
PL/SQL procedure successfully completed.
SQL> select num_rows from all_tables where owner='SCOTT' and table_name='EMP';
NUM_ROWS
----------
14
SQL> select column_name,nullable,num_distinct,num_nulls from all_tab_columns
2 where owner='SCOTT' and table_name='EMP' order by column_id;
COLUMN_NAME N NUM_DISTINCT NUM_NULLS
------------------------------ - ------------ ----------
EMPNO N 14 0
ENAME Y 14 0
JOB Y 5 0
MGR Y 6 1
HIREDATE Y 13 0
SAL Y 12 0
COMM Y 4 10
DEPTNO Y 3 0
8 rows selected.
For example you can check if NUM_NULLS = NUM_ROWS to identify "empty" columns.
Reference: ALL_TAB_COLUMNS, ALL_TABLES.
Use the below:
SELECT *
FROM information_schema.columns
WHERE table_name = 'Table_Name' and is_nullable = 'NO'
Table_Name has to be replaced accordingly...
select column_name
from user_tab_columns
where table_name='Table_name' and num_nulls=0;
Here is simple code to get non null columns..
I don't think this can be done in a single query. You may need some plsql to first test what columns contain data and put together a statement based on that information. Of course, if the data in your table changes you have to recreate the statement.
declare
l_table varchar2(30) := 'YOUR_TABLE';
l_statement varchar2(32767);
l_test_statement varchar2(32767);
l_contains_value pls_integer;
-- select column_names from your table
cursor c is
select column_name
,nullable
from user_tab_columns
where table_name = l_table;
begin
l_statement := 'select ';
for r in c
loop
-- If column is not nullable it will always contain a value
if r.nullable = 'N'
then
-- add column to select list.
l_statement := l_statement || r.column_name || ',';
else
-- check if there is a row that has a value for this column
begin
l_test_statement := 'select 1 from dual where exists (select 1 from ' || l_table || ' where ' ||
r.column_name || ' is not null)';
dbms_output.put_line(l_test_statement);
execute immediate l_test_statement
into l_contains_value;
-- Yes, add column to select list
l_statement := l_statement || r.column_name || ',';
exception
when no_data_found then
null;
end;
end if;
end loop;
-- create a select statement
l_statement := substr(l_statement, 1, length(l_statement) - 1) || ' from ' || l_table;
end;
select rtrim (xmlagg (xmlelement (e, column_name || ',')).extract ('//text()'), ',') col
from (select column_name
from user_tab_columns
where table_name='<table_name>' and low_value is not null)
This block determines all columns in the table, loops through them in dynamic SQL and checks if they are null, then constructs a DBMS output query of the non-null query.
All you have to do is run the returned query.
I've included the exclusion of PKs and BLOB columns.
Obviously, this is quite slow as going through columns one by one, and it's not going to be great for very hot tables, as data may change too quickly, but this works for me as I control traffic in dev env.
DECLARE
l_table_name VARCHAR2(255) := 'XXXX';
l_counter NUMBER;
l_sql CLOB;
BEGIN
FOR r_col IN (SELECT *
FROM user_tab_columns tab_col
WHERE table_name = l_table_name
AND data_type NOT IN ('BLOB')
AND column_name NOT IN (SELECT column_name
FROM user_cons_columns con_col
JOIN user_constraints cons ON con_col.constraint_name = cons.constraint_name AND con_col.table_name = cons.table_name
WHERE con_col.table_name = tab_col.table_name
AND constraint_type = 'P')
ORDER BY column_id)
LOOP
EXECUTE IMMEDIATE 'SELECT COUNT(1) FROM '||l_table_name||' WHERE '||r_col.column_name||' IS NOT NULL'
INTO l_counter;
IF l_counter > 0 THEN
IF l_sql IS NULL THEN
l_sql := r_col.column_name;
ELSE
l_sql := l_sql||','||r_col.column_name;
END IF;
END IF;
END LOOP;
l_sql := 'SELECT '||l_sql||CHR(10)
||'FROM '||l_table_name;
----------
DBMS_OUTPUT.put_line(l_sql);
END;
What you're asking to do is establish a dependency on each row in the whole result. This is in fact not ever what you want. Just think of the ramifications if in one row every column had a value of '0' -- suddenly the schema of your result set grows to include all of those previously "empty" columns. You're effectively growing the badness of '*' exponentially, now your result set is not dependent on just the table's meta-data -- but your whole result set is dependent on the plain data.
What you want to do is just select the fields that have what you want, and not deviate from this simple plan.