I am generating sql statements using this sql:
select 'select ' || listagg(t.column_name, ', ') within group (order by column_id, table_name) ||' from &#source_table# a' query from user_tab_cols t where t.table_name = '&#target_table#';
In some cases the columns I in the statement I generate are CLOB and I want to convert these to varchar2 using the following statement:
dbms_lob.substr (column, 4000, 1) as column and in other cases I want to remove the CLOB column, this I can control with a parameter.
How do I write the sql query?
I managed to create it using a case and a decode in the listagg:
select ' || listagg(
case t.data_type
when 'CLOB' then
decode('&#ignore_clob#',
'no', 'dbms_lob.substr ('||t.column_name||', 4000, 1) as '|| t.column_name,
'yes','')
else t.column_name
end
, ', ') within group (order by column_id, table_name) ||
' from #source_table a' query
from user_tab_cols t where t.table_name = '&#source_table#';
Related
I need a query to count the total number of non-null values for each column in a table. Since my table has hundreds of columns I'm looking for a solution that only requires me to input the table name.
Perhaps using the result of:
select COLUMN_NAME from ALL_TAB_COLUMNS where TABLE_NAME='ORDERS';
to get the column names and then a subquery to put counts against each column name? The additional complication is that I only have read-only access to the DB so I can't create any temp tables.
Slightly out of my league with this one so any help is appreciated.
Construct the query in SQL or using a spreadsheet. Then run the query.
For instance, assuming that your column names are simple and don't have special characters:
select replace('select ''[col]'', count([col]) from orders union all ',
'[col]', COLUMN_NAME
) as sql
from ALL_TAB_COLUMNS
where TABLE_NAME = 'ORDERS';
(Of course, this can be adapted for more complex column names, but I'm trying to show the idea.)
Then copy the code, remove the final union all and run it.
You can put this in one string if there are not too many columns:
select listagg(replace('select ''[col]'', count([col]) from orders',
'[col]', COLUMN_NAME
), ' union all '
) within group (order by column_name) as sql
from ALL_TAB_COLUMNS
where TABLE_NAME = 'ORDERS';
You can also use execute immediate using the same query, but that seems like overkill.
If you're happy with the results row-ar rather than column-ar:
SELECT 'SELECT ''dummy'', 0 FROM DUAL' FROM DUAL
UNION ALL
SELECT
' UNION ALL SELECT ''' ||
column_name ||
''', COUNT(' ||
column_name ||
') FROM ' ||
TABLE_NAME
FROM
all_tab_columns
WHERE
table_name = 'ORDERS'
This is an "SQL that writes an SQL" that you can then copy and run to get your answers. Should make a resultset that looks like:
SELECT 'dummy', 0 FROM dual
UNION ALL SELECT 'col1', COUNT(col1) FROM ORDERS
UNION ALL SELECT 'col2', COUNT(col2) FROM ORDERS
...
If you want your results column-ar:
SELECT 'SELECT '
UNION ALL
SELECT
'COUNT(' ||
column_name ||
') as count_' ||
column_name ||
', ' ||
TABLE_NAME
FROM
all_tab_columns
WHERE
table_name = 'ORDERS'
UNION ALL
SELECT 'null as dummy_column FROM ORDERS'
Should make a resultset that looks like:
SELECT
COUNT(col1) as count_col1,
COUNT(col2) as count_col2,
...
null as dummycoll FROM orders
Caveat: I don't have oracle installed anywhere I can test these, it's written from memory and may need some debugging
This will generate the SQL to get the counts in columns and will handle case sensitive column names and column names with non-alpha-numeric characters:
SELECT 'SELECT '
|| LISTAGG(
'COUNT("' || column_name || '") AS "' || column_name || '"',
', '
) WITHIN GROUP ( ORDER BY column_id )
|| ' FROM "' || table_name || '"' AS sql
FROM ALL_TAB_COLUMNS
WHERE TABLE_NAME = 'ORDERS'
GROUP BY TABLE_NAME;
or, if you have a large number of columns that is generating a string longer than 4000 characters you can use a custom aggregation function to aggregate VARCHAR2s into a CLOB and then do:
SELECT 'SELECT '
|| CLOBAgg( 'COUNT("' || column_name || '") AS "' || column_name || '"' )
|| ' FROM "' || table_name || '"' AS sql
FROM ALL_TAB_COLUMNS
WHERE TABLE_NAME = 'ORDERS'
GROUP BY TABLE_NAME;
In Oracle 19 (I used similar code in Ora 12, maybe that works too), this works without generating another select to execute:
select * from
(
select table_name, column_name,
to_number( extractvalue( xmltype(dbms_xmlgen.getxml('select count(to_char(substr('||column_name||',1,1))) c from '||table_name)) ,'/ROWSET/ROW/C')) count
from all_tab_columns where owner = user
)
--where table_name = 'MY_TABLE'
;
It will create XML with count, from which it extracts the current count. The substr and to_char functions here are used to extract first character, so it will works with CLOB columns also
I'd like to find all column of a set of table with null values in them.
I can find the table and column names
SELECT TABLE_NAME, COLUMN_NAME
FROM user_tab_cols
where nullable='Y'
and table_name in ('myTalbe', 'myTable2');
And also check if the are nulls
select count(*) from myTable where myColumn is null;
but how can I put this toghether to have as result
table_name column_name
myTable myColumn
myTable myCol2
myTable2 col4
An approach could be with some dynamic SQL, but this would require some PL/SQL code; the following only uses SQL to get the result you need:
select *
from (
select table_name,
column_name,
to_number(extractvalue(xmltype(dbms_xmlgen.getxml('select count(*) c from '||table_name || ' where ' || column_name || ' is null')),'/ROWSET/ROW/C')) as rowcount
from user_tab_columns
where nullable='Y'
and table_name in ('myTalbe', 'myTable2')
)
where rowcount > 0
This could be an approach with dynamic SQL:
declare
type tableOfNumber is table of number;
type tableOfChar is table of varchar2(30);
--
vSQl varchar2(4000);
vListNumbers tableOfNumber;
vListTables tableOfChar;
vListColumns tableOfChar;
begin
select listagg( 'select ''' ||
table_name || ''' as table_name, ''' ||
column_name || ''' as column_name, count(*) as rowCount from ' ||
table_name ||
' where ' ||
column_name ||
' is null having count(*) > 0' ,
' UNION ALL '
) within group ( order by table_name, column_name)
into vSQL
from user_tab_columns
where nullable='Y'
and table_name in ('myTalbe', 'myTable2');
--
dbms_output.put_line(vSQL);
/* whatever you may want to do with the query */
/* for example, fetch into some variables and print the result */
execute immediate vSQL
bulk collect into vListTables, vListColumns, vListNumbers;
--
if vListTables.count() > 0 then
for i in vListTables.first .. vListTables.last loop
dbms_output.put_line('Table ' || vListTables(i) ||
', column ' || vListColumns(i) ||
', number of nulls: ' || vListNumbers(i)
);
end loop;
end if;
end;
Here's a routine I wrote a while back to do exactly that. It will output the required DDL to make those columns that are nullable (but do not contain any nulls) not nullable.
https://connormcdonald.wordpress.com/2016/03/11/tightening-up-your-data-model/
It can do the task for a schema or a single table.
Seems like this is impossible, but I'm so close - maybe someone can take me the last step...
I have a bunch of dynamic code and I don't always know the tables and columns I'm going to be dealing with, but I do know that VARCHAR2 columns with data_lengths of 2000 result in errors. I'd love to be able to identify these 'bad' columns dynamically, and remove them from my results in 1 shot.
This code:
SELECT LISTAGG(probs.column_name, ', ')
WITHIN GROUP (ORDER BY column_name) FROM
(select 1 grp, column_name
from all_tab_columns
where TABLE_NAME = 'MYTABLE' AND
DATA_TYPE <> 'VARCHAR2' AND
DATA_LENGTH < 2000
) probs
GROUP BY GRP
Gives me a nice comma, separated list of all of my acceptable column names like this:
FIELD1, FIELD2, FIELD3, FIELD4...
And I am hopeful that there's a way a can simply do something to drop that list of field names into a select statement like this:
SELECT (<my subquery, above>)
FROM MYTABLE;
Is this possible?
Assuming this situation
create table mytable ( a number, b number, c number)
insert into mytable values (10, 20, 30)
insert into mytable values (1, 2, 3)
and that only exists one table with that name (otherwise you should specify the owner in the query from all_tab_columns), your query could be simplified this way:
SELECT 'select ' || LISTAGG(column_name, ', ') WITHIN GROUP (ORDER BY column_name) || ' from ' || table_name
FROM all_tab_columns
WHERE TABLE_NAME = 'MYTABLE'
AND DATA_TYPE <> 'VARCHAR2'
AND DATA_LENGTH < 2000
GROUP BY table_name
this would give: select A, B, C from MYTABLE.
The problem here is that you can not simply run a statement that returns a variable number of columns; one way to use this could be building an xml:
SELECT xmltype(
DBMS_XMLGEN.getxml(
( SELECT 'select ' || LISTAGG(column_name, ', ') WITHIN GROUP (ORDER BY column_name) || ' from ' || table_name
FROM all_tab_columns
WHERE TABLE_NAME = 'MYTABLE'
AND DATA_TYPE <> 'VARCHAR2'
AND DATA_LENGTH < 2000
GROUP BY table_name)
)
)
FROM DUAL
<?xml version="1.0"?>
<ROWSET>
<ROW>
<A>10</A>
<B>20</B>
<C>30</C>
</ROW>
<ROW>
<A>1</A>
<B>2</B>
<C>3</C>
</ROW>
</ROWSET>
Another way could be using some PLSQL and dynamic SQL, with a little modification of yur query to concatenate the fields, to build the result in a unique string:
declare
type tTabResults is table of varchar2(1000);
vSQL varchar2(1000);
vTabResults tTabResults;
begin
SELECT 'select ' || LISTAGG( column_name, '|| '', '' ||') WITHIN GROUP (ORDER BY column_name) || ' from ' || table_name
into vSQL
FROM all_tab_columns
WHERE TABLE_NAME = 'MYTABLE'
AND DATA_TYPE <> 'VARCHAR2'
AND DATA_LENGTH < 2000
GROUP BY table_name;
--
execute immediate vSQL bulk collect into vTabResults;
--
for i in vTabResults.first .. vTabResults.last loop
dbms_output.put_line(vTabResults(i));
end loop;
end;
10, 20, 30
1, 2, 3
Notice that I oversimplified the problem, treating numbers as strings and not using any conversion, by simply printing the values in your table, no matter their type; in a real solution you should handle the possible types of your columns and modify the initial query to add some type conversions.
Assume that the DBA_TAB_COLUMNS looks like this:
I'd like to write a SQL or PL/SQL script to generate following text:
select 'NULL' as A1, B1, QUERY, RECORD_KEY from SMHIST.probsummarym1
union all
select 'NULL' as A1, 'NULL' as B1, QUERY, RECORD_KEY from SMHIST_EIT200.probsummarym1
union all
select A1, 'NULL' as B1, QUERY, RECORD_KEY from SMHIST_EIT300.probsummarym1
the requirements are:
If the table under any of the SMHIST% schemas do not have that column, then insert a default NULL alias for that columns.
the column list is in alphabetical order.
so can anybody tell me how to write this script?
EDIT: Added better alias names and en explicit CROSS JOIN. Added XMLAGG version.
NB: LISTAGG exists from Oracle version 11.2 and onwards and returns VARCHAR2. If the output string is larger than 4000K or if on a prior version you can use XMLAGG which is a bit more cumbersome to work with (eg. http://psoug.org/definition/xmlagg.htm).
With LISTAGG (returning VARCHAR2):
SELECT LISTAGG (line,
CHR (13) || CHR (10) || 'union all' || CHR (13) || CHR (10))
WITHIN GROUP (ORDER BY sortorder)
script
FROM (SELECT line, ROWNUM sortorder
FROM ( SELECT 'select '
|| LISTAGG (
CASE
WHEN tc.column_name IS NULL
THEN
'''NULL'' as '
END
|| col_join.column_name,
', ')
WITHIN GROUP (ORDER BY col_join.column_name)
|| ' from '
|| col_join.owner
|| '.'
|| col_join.table_name
line
FROM dba_tab_columns tc
RIGHT OUTER JOIN
(SELECT DISTINCT
owner, table_name, col_list.column_name
FROM dba_tab_columns
CROSS JOIN
(SELECT DISTINCT column_name
FROM dba_tab_columns
WHERE owner LIKE 'SMHIST%') col_list
WHERE owner LIKE 'SMHIST%') col_join
ON tc.owner = col_join.owner
AND tc.table_name = col_join.table_name
AND tc.column_name = col_join.column_name
GROUP BY col_join.owner, col_join.table_name
ORDER BY col_join.owner, col_join.table_name))
With XMLAGG (returning CLOB by adding .getclobval (), note: RTRIM works here because table names cannot include ',' and ' ' (space)):
SELECT REPLACE (SUBSTR (script, 1, LENGTH (script) - 12),
'&' || 'apos;',
'''')
FROM (SELECT XMLAGG (
XMLELEMENT (
e,
line,
CHR (13)
|| CHR (10)
|| 'union all'
|| CHR (13)
|| CHR (10))).EXTRACT ('//text()').getclobval ()
script
FROM (SELECT line, ROWNUM sortorder
FROM ( SELECT 'select '
|| RTRIM (
REPLACE (
XMLAGG (XMLELEMENT (
e,
CASE
WHEN tc.column_name
IS NULL
THEN
'''NULL'' as '
END
|| col_join.column_name,
', ') ORDER BY
col_join.column_name).EXTRACT (
'//text()').getclobval (),
'&' || 'apos;',
''''),
', ')
|| ' from '
|| col_join.owner
|| '.'
|| col_join.table_name
line
FROM dba_tab_columns tc
RIGHT OUTER JOIN
(SELECT DISTINCT
owner,
table_name,
col_list.column_name
FROM dba_tab_columns
CROSS JOIN
(SELECT DISTINCT column_name
FROM dba_tab_columns
WHERE owner LIKE 'SMHIST%') col_list
WHERE owner LIKE 'SMHIST%') col_join
ON tc.owner = col_join.owner
AND tc.table_name = col_join.table_name
AND tc.column_name = col_join.column_name
GROUP BY col_join.owner, col_join.table_name
ORDER BY col_join.owner, col_join.table_name)))
I need to run a query on generated generated column names.
Here's the query:
select 'col_'||4 from MY_TABLE
Note:
"4" is a variable that is passed to this query from within the Java code
MY_TABLE is a table that contain columns with names (col_4, col_5, etc..)
Inside Oracle you need use dynamic SQL. (YourVariable value is 4 for your example)
EXECUTE IMMEDIATE ' select col_' || YourVariable || ' from MY_TABLE ';
From Java you can build any SQL and execute them
To run a dynamic SELECT statement, you have two choices:
For single row selects, you use EXECUTE IMMEDIATE ... INTO:
EXECUTE IMMEDIATE 'select col_' || l_num || ' from MY_TABLE WHERE id = 37' INTO l_result;
For selecting multiple rows, you can use a dynamic cursor:
DECLARE
TYPE MyCurType IS REF CURSOR;
my_cv MyCurType;
BEGIN
OPEN emp_cv FOR 'select col_' || l_num || ' from MY_TABLE';
...
END;
This code generates a SELECT that returns the tables with their column name:
SELECT
'SELECT ' ||(
SELECT
LISTAGG(
c.TABLE_NAME || '.' || c.COLUMN_NAME || ' AS "' || c.TABLE_NAME || '.' || c.COLUMN_NAME || '"',
', '
) WITHIN GROUP(
ORDER BY
c.TABLE_NAME
) "TABLE_NAMES"
FROM
USER_TAB_COLS c
WHERE
TABLE_NAME IN(
'PESSOA',
'PESSOA_FISICA',
'PESSOA_JURIDICA'
)
)|| 'FROM PERSON;'
FROM
DUAL;