Assume that the DBA_TAB_COLUMNS looks like this:
I'd like to write a SQL or PL/SQL script to generate following text:
select 'NULL' as A1, B1, QUERY, RECORD_KEY from SMHIST.probsummarym1
union all
select 'NULL' as A1, 'NULL' as B1, QUERY, RECORD_KEY from SMHIST_EIT200.probsummarym1
union all
select A1, 'NULL' as B1, QUERY, RECORD_KEY from SMHIST_EIT300.probsummarym1
the requirements are:
If the table under any of the SMHIST% schemas do not have that column, then insert a default NULL alias for that columns.
the column list is in alphabetical order.
so can anybody tell me how to write this script?
EDIT: Added better alias names and en explicit CROSS JOIN. Added XMLAGG version.
NB: LISTAGG exists from Oracle version 11.2 and onwards and returns VARCHAR2. If the output string is larger than 4000K or if on a prior version you can use XMLAGG which is a bit more cumbersome to work with (eg. http://psoug.org/definition/xmlagg.htm).
With LISTAGG (returning VARCHAR2):
SELECT LISTAGG (line,
CHR (13) || CHR (10) || 'union all' || CHR (13) || CHR (10))
WITHIN GROUP (ORDER BY sortorder)
script
FROM (SELECT line, ROWNUM sortorder
FROM ( SELECT 'select '
|| LISTAGG (
CASE
WHEN tc.column_name IS NULL
THEN
'''NULL'' as '
END
|| col_join.column_name,
', ')
WITHIN GROUP (ORDER BY col_join.column_name)
|| ' from '
|| col_join.owner
|| '.'
|| col_join.table_name
line
FROM dba_tab_columns tc
RIGHT OUTER JOIN
(SELECT DISTINCT
owner, table_name, col_list.column_name
FROM dba_tab_columns
CROSS JOIN
(SELECT DISTINCT column_name
FROM dba_tab_columns
WHERE owner LIKE 'SMHIST%') col_list
WHERE owner LIKE 'SMHIST%') col_join
ON tc.owner = col_join.owner
AND tc.table_name = col_join.table_name
AND tc.column_name = col_join.column_name
GROUP BY col_join.owner, col_join.table_name
ORDER BY col_join.owner, col_join.table_name))
With XMLAGG (returning CLOB by adding .getclobval (), note: RTRIM works here because table names cannot include ',' and ' ' (space)):
SELECT REPLACE (SUBSTR (script, 1, LENGTH (script) - 12),
'&' || 'apos;',
'''')
FROM (SELECT XMLAGG (
XMLELEMENT (
e,
line,
CHR (13)
|| CHR (10)
|| 'union all'
|| CHR (13)
|| CHR (10))).EXTRACT ('//text()').getclobval ()
script
FROM (SELECT line, ROWNUM sortorder
FROM ( SELECT 'select '
|| RTRIM (
REPLACE (
XMLAGG (XMLELEMENT (
e,
CASE
WHEN tc.column_name
IS NULL
THEN
'''NULL'' as '
END
|| col_join.column_name,
', ') ORDER BY
col_join.column_name).EXTRACT (
'//text()').getclobval (),
'&' || 'apos;',
''''),
', ')
|| ' from '
|| col_join.owner
|| '.'
|| col_join.table_name
line
FROM dba_tab_columns tc
RIGHT OUTER JOIN
(SELECT DISTINCT
owner,
table_name,
col_list.column_name
FROM dba_tab_columns
CROSS JOIN
(SELECT DISTINCT column_name
FROM dba_tab_columns
WHERE owner LIKE 'SMHIST%') col_list
WHERE owner LIKE 'SMHIST%') col_join
ON tc.owner = col_join.owner
AND tc.table_name = col_join.table_name
AND tc.column_name = col_join.column_name
GROUP BY col_join.owner, col_join.table_name
ORDER BY col_join.owner, col_join.table_name)))
Related
How to convert LISTAGG with case statements to XMLAGG equivalent, so as to avoid the concatenation error.
#ECHO ${cols_2 ||32767||varchar2}$ --Declare variable
SELECT LISTAGG( 'MAX(CASE WHEN CATEGORY = '''||CATEGORY||''' THEN "'||"LEVEL"||'" END) AS "'||"LEVEL"||'_'||CATEGORY||'"' , ',' )
WITHIN GROUP( ORDER BY CATEGORY, "LEVEL" DESC )
INTO cols_2
FROM (
SELECT DISTINCT "LEVEL", CATEGORY
FROM temp
);
I tried this and I'm getting an error saying missing keyword
#ECHO ${cols_2 ||32767||varchar2}$ --Declare variable
select rtrim (
xmlagg (xmlelement (e, 'MAX(CASE WHEN CATEGORY = '''||CATEGORY||''' THEN "'||"LEVEL"||'" END) AS "'||LEVEL||'_'||CATEGORY||'"', ',') order by 1,2 desc).extract (
'//text()'),
', ')
INTO cols_2
FROM (
SELECT DISTINCT "LEVEL", CATEGORY
temp
);
I have tried this an declared the cols_2 as clob type :-
SELECT DBMS_XMLGEN.CONVERT (
RTRIM (
XMLAGG (XMLELEMENT (
e,
'MAX(CASE WHEN CATEGORY = '''
|| CATEGORY
|| ''' THEN "'
|| "LEVEL"
|| '" END) AS "'
|| "LEVEL"
|| '_'
|| CATEGORY
|| '"',
',')
ORDER BY 1, DESC).EXTRACT('//text()').getclobval(),','),1)
', '),
1)
INTO cols_2
FROM (SELECT DISTINCT "LEVEL", CATEGORY
FROM temp);
Yet my issue is not resolved ,Im getting an error while trying to execute it as a procedure like :-
Error in concatenation of `LISTAGG` function[Not a duplicate question]
You are getting the missing keyword error because you are most likely attempting to run the second query as a standalone query instead of in a PL/SQL block. When you are doing that, you have to remove your into cols_2 clause. That is your immediate issue that should resolve your error.
Also, based on your prior question, using the XML functions will escape your ' and " characters so you will want to make sure to unescape them back to their original characters so you can use them in your dynamic sql query like this:
SELECT DBMS_XMLGEN.CONVERT (
RTRIM (
XMLAGG (XMLELEMENT (
e,
'MAX(CASE WHEN CATEGORY = '''
|| CATEGORY
|| ''' THEN "'
|| "LEVEL"
|| '" END) AS "'
|| "LEVEL"
|| '_'
|| CATEGORY
|| '"',
',')
ORDER BY 1, 2 DESC).EXTRACT ('//text()'),
', '),
1)
--INTO cols_2
FROM (SELECT DISTINCT "LEVEL", CATEGORY
FROM temp);
I need a query to count the total number of non-null values for each column in a table. Since my table has hundreds of columns I'm looking for a solution that only requires me to input the table name.
Perhaps using the result of:
select COLUMN_NAME from ALL_TAB_COLUMNS where TABLE_NAME='ORDERS';
to get the column names and then a subquery to put counts against each column name? The additional complication is that I only have read-only access to the DB so I can't create any temp tables.
Slightly out of my league with this one so any help is appreciated.
Construct the query in SQL or using a spreadsheet. Then run the query.
For instance, assuming that your column names are simple and don't have special characters:
select replace('select ''[col]'', count([col]) from orders union all ',
'[col]', COLUMN_NAME
) as sql
from ALL_TAB_COLUMNS
where TABLE_NAME = 'ORDERS';
(Of course, this can be adapted for more complex column names, but I'm trying to show the idea.)
Then copy the code, remove the final union all and run it.
You can put this in one string if there are not too many columns:
select listagg(replace('select ''[col]'', count([col]) from orders',
'[col]', COLUMN_NAME
), ' union all '
) within group (order by column_name) as sql
from ALL_TAB_COLUMNS
where TABLE_NAME = 'ORDERS';
You can also use execute immediate using the same query, but that seems like overkill.
If you're happy with the results row-ar rather than column-ar:
SELECT 'SELECT ''dummy'', 0 FROM DUAL' FROM DUAL
UNION ALL
SELECT
' UNION ALL SELECT ''' ||
column_name ||
''', COUNT(' ||
column_name ||
') FROM ' ||
TABLE_NAME
FROM
all_tab_columns
WHERE
table_name = 'ORDERS'
This is an "SQL that writes an SQL" that you can then copy and run to get your answers. Should make a resultset that looks like:
SELECT 'dummy', 0 FROM dual
UNION ALL SELECT 'col1', COUNT(col1) FROM ORDERS
UNION ALL SELECT 'col2', COUNT(col2) FROM ORDERS
...
If you want your results column-ar:
SELECT 'SELECT '
UNION ALL
SELECT
'COUNT(' ||
column_name ||
') as count_' ||
column_name ||
', ' ||
TABLE_NAME
FROM
all_tab_columns
WHERE
table_name = 'ORDERS'
UNION ALL
SELECT 'null as dummy_column FROM ORDERS'
Should make a resultset that looks like:
SELECT
COUNT(col1) as count_col1,
COUNT(col2) as count_col2,
...
null as dummycoll FROM orders
Caveat: I don't have oracle installed anywhere I can test these, it's written from memory and may need some debugging
This will generate the SQL to get the counts in columns and will handle case sensitive column names and column names with non-alpha-numeric characters:
SELECT 'SELECT '
|| LISTAGG(
'COUNT("' || column_name || '") AS "' || column_name || '"',
', '
) WITHIN GROUP ( ORDER BY column_id )
|| ' FROM "' || table_name || '"' AS sql
FROM ALL_TAB_COLUMNS
WHERE TABLE_NAME = 'ORDERS'
GROUP BY TABLE_NAME;
or, if you have a large number of columns that is generating a string longer than 4000 characters you can use a custom aggregation function to aggregate VARCHAR2s into a CLOB and then do:
SELECT 'SELECT '
|| CLOBAgg( 'COUNT("' || column_name || '") AS "' || column_name || '"' )
|| ' FROM "' || table_name || '"' AS sql
FROM ALL_TAB_COLUMNS
WHERE TABLE_NAME = 'ORDERS'
GROUP BY TABLE_NAME;
In Oracle 19 (I used similar code in Ora 12, maybe that works too), this works without generating another select to execute:
select * from
(
select table_name, column_name,
to_number( extractvalue( xmltype(dbms_xmlgen.getxml('select count(to_char(substr('||column_name||',1,1))) c from '||table_name)) ,'/ROWSET/ROW/C')) count
from all_tab_columns where owner = user
)
--where table_name = 'MY_TABLE'
;
It will create XML with count, from which it extracts the current count. The substr and to_char functions here are used to extract first character, so it will works with CLOB columns also
I need to write a query to compare column by column (ie: find differences) between two rows in the database. For example:
row1: 10 40 sometext 24
row2: 10 25 sometext 24
After the query executed, it should shows only the fields that have difference (ie: the second field)
Here's what I have done so far:
select table1.column1, table1.column2, table1.column3, table1.column4
from table1
where somefield in (field1, field2);
The above query will show me two rows one above another like this:
10 40 sometext 24
10 25 sometext 24
Then I have to manually do the comparison and it takes a lot of time b/c the row contains a lot of column.
So again my question is: How can I write a query that will show me only the columns that have differences??
Thanks
Use UNPIVOT clause (see http://www.oracle-developer.net/display.php?id=506) to turn columns into rows, then filter out the same rows (using GROUP BY HAVING COUNT and finally use PIVOT to get rows with different columns only.
To do this easily you need to query the metadata for the table to get each row. You can use the following code as a script.
Replace the define table_name with your table name and define yes_drop_it = NO. Put your raw WHERE syntax into the where_clause. The comparison logic always compares the first two rows returned for the where clause.
whenever sqlerror exit failure rollback;
set linesize 150
define test_tab_name = tst_cf_cols
define yes_drop_it = YES
define order_by = 1, 2
define where_clause = 1 = 1
define tab_owner = user
<<clearfirst>> begin
for clearout in (
select 'drop table ' || table_name as cmd
from all_tables
where owner = &&tab_owner and table_name = upper('&&test_tab_name')
and '&&yes_drop_it' = 'YES'
) loop
execute immediate clearout.cmd;
execute immediate '
create table &&test_tab_name as
select 10 as column1, 40 as column2, ''sometext'' as column3, 24 as column4 from dual
union all
select 10 as column1, 25 as column2, ''sometext'' as column3, 24 as column4 from dual
';
end loop;
end;
/
column cfsynt format a4000 word_wrap new_value comparison_syntax
with parms as (select 'parmquery' as cte_name, 'row_a' as corr_name_1, 'row_b' as corr_name_2 from dual)
select
'select * from (select ' || LISTAGG(cfcol || ' AS cf_' || trim (to_char (column_id, '000')) || '_' || column_name
, chr(13) || ', ') WITHIN GROUP (order by column_id)
|| chr(13) || ' from (select * from parmquery where row_number = 1) ' || corr_name_1
|| chr(13) || ', (select * from parmquery where row_number = 2) ' || corr_name_2
|| chr(13) || ') where ''DIFFERENT'' IN (' || LISTAGG ('cf_' || trim (to_char (column_id, '000')) || '_' || column_name, chr(13) || ', ') within group (order by column_id) || ')'
as cfsynt
from parms, (
select
'decode (' || corr_name_1 || '.' || column_name || ', ' || corr_name_2
|| '.' || column_name || ', ''SAME'', ''DIFFERENT'')'
as cfcol,
column_name,
column_id
from
parms,
all_tab_columns
where
owner = &&tab_owner and table_name = upper ('&&test_tab_name')
);
with parmquery as (select rownum as row_number, vals.* from (
select * from &&test_tab_name
where &&where_clause
order by &&order_by
) vals
) &&comparison_syntax
;
I am trying to create a SQL code in SQL Server that dynamically selects all the tables in a specific database and then for each column in each table, counts the number of missing values and non-null values. I also want this result inserted into another table.
Is there any way I can do this without manually changing the column names for each:
Table Name - Column selection
I have a teradata code for the same which I tried to convert to SQL Server code. But I am unable to get the dynamic allocation and insertion parts right.
insert into temp
values (select ''CAMP'',
rtrim(''' || tablename || '''),
rtrim(''' || columnname || '''),
rtrim(''' || columnformat || '''),
count(1),
count(rtrim(upper(case when ' || columnname || '='''' then NULL else '|| columnname ||' end))),
(cast (count(rtrim(upper(case when ' || columnname || '='''' then NULL else ' || columnname || ' end))) as float) / (cast (count(1) as float))) * 100,
count(distinct rtrim(upper(case when ' || columnname || '='''' then NULL else '|| columnname ||' end))),
min(rtrim(upper(case when ' || columnname || '='''' then NULL else '|| columnname ||' end))),
max(rtrim(upper(case when ' || columnname || '='''' then NULL else '|| columnname ||' end))),
min(len(rtrim(upper(case when ' || columnname || '='''' then NULL else '|| columnname ||' end)))),
max(len(rtrim(upper(case when ' || columnname || '='''' then NULL else '|| columnname ||' end))))
from ' || tablename ||')
Any help on this front would be great!
Thanks!
Not sure if you need a UNION or a JOIN, but in either case you can just use a three-part name for the object in the other database if you are using multi-database:
USE database1; // Your database name
GO
CREATE VIEW dbo.MyView
AS
SELECT columns FROM dbo.Table1
UNION ALL
SELECT columns FROM database2.dbo.Table2; //second database
GO
select * from dbo.MyView // Getting all data from view
Hope that helps
Does something like this help you:
SELECT [Table] = t.[name]
, [Column] = c.[name]
FROM sys.tables t
INNER JOIN sys.columns c
ON c.[object_id] = t.[object_id]
I have a table with mixed types of data (real, integrer, character ...) but i would only recover columns that have real values.
I can construct this:
SELECT 'SELECT ' || array_to_string(ARRAY(
select 'o' || '.' || c.column_name
from information_schema.columns as c
where table_name = 'final_datas'
and c.data_type = 'real'), ',') || ' FROM final_datas as o' As sqlstmt
that gives that:
"SELECT o.random,o.struct2d_pred2_num,o.pfam_num,o.transmb_num [...] FROM final_datas as o"
The i would like to create a table with these columns. Of course, do this, doesn't work:
create table table2 as (
SELECT 'SELECT ' || array_to_string(ARRAY(
select 'o' || '.' || c.column_name
from information_schema.columns as c
where table_name = 'final_datas'
and c.data_type = 'real'), ',') || ' FROM final_datas as o' As sqlstmt
)
Suggestions?
You need to generate the whole CREATE TABLE statement as dynamic SQL:
SELECT 'CREATE TABLE table2 AS SELECT ' || array_to_string(ARRAY(
select 'o' || '.' || c.column_name
from information_schema.columns as c
where table_name = 'final_datas'
and c.data_type = 'real'), ',') || ' FROM final_datas as o' As sqlstmt
The result can be run with EXECUTE sqlstmt;