I have a query that display another query I need to execute:
So the first part just writes out as text the first part of the query I want to execute
SELECT distinct 'SELECT COUNT(txn_id) FROM '
I then add on all the tables that I want to execute that initial part of the query on
table_name from all_tab_columns WHERE OWNER='RGSWKF_PRGM' AND COLUMN_NAME like '%TXN_ID%';
So my complete query is
SELECT distinct 'SELECT COUNT(txn_id) FROM ' || table_name from all_tab_columns WHERE OWNER='RGSWKF_PRGM' AND COLUMN_NAME like '%TXN_ID%';
This gives me a list of the queries I want to execute like so:
SELECT COUNT(txn_id) FROM MEETING_TXN_LIST
SELECT COUNT(txn_id) FROM TXN_COMMENT
SELECT COUNT(txn_id) FROM TXN_DEAL_FEE
....etc. I was told once I have this result I can auto execute the queries that are created as result of this by adding something to my original query but I can't find anything as of yet?
So basically I want it to execute from one query:
SELECT COUNT(txn_id) FROM MEETING_TXN_LIST
then
SELECT COUNT(txn_id) FROM TXN_COMMENT
then
SELECT COUNT(txn_id) FROM TXN_DEAL_FEE
etc. all in one query.
union
using union all with single quotes gives me the result with the text
SELECT COUNT(txn_id) FROM TXN_COMMENT union all ..etc...
Without the single quotes gives me the following error
ORA-00936: missing expression
00936. 00000 - "missing expression"
I would suggest that you generate a query with the subqueries connected by union all:
SELECT 'MEETING_TXN_LIST' as table_name, COUNT(txn_id) as cnt FROM MEETING_TXN_LIST UNION ALL
SELECT 'TXN_COMMENT', COUNT(txn_id) FROM TXN_COMMENT union all
SELECT 'TXN_DEAL_FEE' COUNT(txn_id) FROM TXN_DEAL_FEE;
The query for this is basically:
SELECT 'SELECT ''' || table_name || ''' as table_name, COUNT(txn_id) as cnt
FROM ' || table_name || ' union all '
from all_tab_columns
WHERE OWNER = 'RGSWKF_PRGM' AND COLUMN_NAME = 'TXN_ID';
Note that you need to remove the final union all from the last row. And, I changed the query to look only for the column TXN_ID, because that is what you are using in the queries.
select txt
|| case
when row_number() over (order by rn desc) = 1 then null
else ul
end
from (select 'SELECT '''
|| table_name
|| ''' as table_name, COUNT(txn_id) as cnt FROM '
|| table_name
as txt
,' union all ' ul
,rownum rn
from all_tab_columns
where OWNER = 'RGSWKF_PRGM' and COLUMN_NAME = 'TXN_ID')
order by rn
Related
I am trying to create a view by joining two tables. These two tables have a few columns with the same name. Which gives an error
SQL Error [42701]: ERROR: column "column_name" specified more than once
I cannot use column names while creating the view as there are 30+ columns and new columns will be added to both tables over the period of time. Hence, I have to use * to get all the columns.
Now, to eliminate columns which exist in both the table, I went ahead and did this:
SELECT 'SELECT ' || STRING_AGG('u2.' || column_name, ', ') || ' FROM schema_name.table_name u2'
FROM information_schema.columns
WHERE table_name = 'table_name'
AND table_schema = 'schema_name'
AND column_name NOT IN ('column_name');
This gives me the query to select data from schema_name.table_name without the column column_name. Great!!
The problem: How do I execute the result of the above query?
I tried PREPARE statement, it is just executing the above query and not the result of the above query.
Also, creating a temporary table with no column "column_name" isn't a viable solution.
You need to prepare a dynamic query and then EXECUTE it. It would be something like this:
DO
$do$
BEGIN
EXECUTE (
SELECT CONCAT('CREATE VIEW temp_view AS SELECT ',
-- SELECT table1 columns here
(SELECT STRING_AGG('u1.' || column_name, ', ')
FROM information_schema.columns
WHERE table_name = 'table1'
AND table_schema = 'schema_name'
-- AND column_name NOT IN ('column_name') -- not omitting for table1
),
', ',
-- SELECT table2 columns here
(SELECT STRING_AGG('u2.' || column_name, ', ')
FROM information_schema.columns
WHERE table_name = 'table2'
AND table_schema = 'schema_name'
AND column_name NOT IN ('column_name')),
-- Dynamically prepare the FROM and JOIN clauses
' FROM table1 u1 JOIN table2 u2 ON u1.id = u2.table1_id'));
END
$do$;
CHECK DEMO
I would like to calculate, for each column in a table, the percent of rows that are null.
For one column, I was using:
SELECT ((SELECT COUNT(Col1)
FROM Table1)
/
(SELECT COUNT(*)
FROM Table1)) AS Table1Stats
Works great and is fast.
However, I want to do this for all ~50 columns of the table, and my environment does not allow me to use dynamic SQL.
Any recommendations? I am using snowflake to connect to AWS, but as an end user I am using the snowflake browser interface.
You can combine this as:
SELECT COUNT(Col1) * 1.0 / COUNT(*)
FROM Table1;
Or, if you prefer:
SELECT AVG( (Col1 IS NOT NULL)::INT )
FROM Table1;
You can use a mix of object_construct() and flatten() to move the column names into rows. Then do the math for the values missing:
create or replace temp table many_cols as
select 1 a, 2 b, 3 c, 4 d
union all select 1, null, 3, 4
union all select 8, 8, null, null
union all select 8, 8, 7, null
union all select null, null, null, null;
select key column_name
, 1-count(*)/(select count(*) from many_cols) ratio_null
from (
select object_construct(a.*) x
from many_cols a
), lateral flatten(x)
group by key
;
You can do this using a SQL generator if you don't mind copying the text and running it once it's done.
-- SQL generator option:
select 'select' || listagg(' ((select count(' || COLUMN_NAME || ') from "SNOWFLAKE_SAMPLE_DATA"."TPCH_SF10000"."ORDERS") / ' ||
'(select count(*) from "SNOWFLAKE_SAMPLE_DATA"."TPCH_SF10000"."ORDERS")) as ' || COLUMN_NAME, ',') as SQL_STATEMENT
from "SNOWFLAKE_SAMPLE_DATA"."INFORMATION_SCHEMA"."COLUMNS"
where TABLE_CATALOG = 'SNOWFLAKE_SAMPLE_DATA' and TABLE_SCHEMA = 'TPCH_SF10000' and TABLE_NAME = 'ORDERS'
;
If the copy and paste is not plausible because you need to script it, you can use the results of the SQL generator in a stored procedure I wrote to execute a single line of dynamic SQL:
call run_dynamic_sql(
select 'select' || listagg(' ((select count(' || COLUMN_NAME || ') from "SNOWFLAKE_SAMPLE_DATA"."TPCH_SF10000"."ORDERS") / ' ||
'(select count(*) from "SNOWFLAKE_SAMPLE_DATA"."TPCH_SF10000"."ORDERS")) as ' || COLUMN_NAME, ',') as SQL_STATEMENT
from "SNOWFLAKE_SAMPLE_DATA"."INFORMATION_SCHEMA"."COLUMNS"
where TABLE_CATALOG = 'SNOWFLAKE_SAMPLE_DATA' and TABLE_SCHEMA = 'TPCH_SF10000' and TABLE_NAME = 'ORDERS'
);
If you want the stored procedure, until it's published on Snowflake's blog it's available here: https://snowflake.pavlik.us/index.php/2021/01/22/running-dynamic-sql-in-snowflake/
I have a requirement to pivot the columns of a particular table specified from a user.
the problem is the number of columns from each table to pivot is dynamic.
so the code below gets the name of the columns from the table.
SELECT DISTINCT
LISTAGG('''' || column_name || '''', ',')
WITHIN GROUP (ORDER BY column_name) AS temp_in_statement
FROM (SELECT DISTINCT column_name FROM all_tab_columns WHERE table_name = 'DIM_XYZ')
the code above returns the columns in the following format:
col1, col2
I have to use a pivot for this requirement and plug the code above in the code below in order to pivot the columns.
SELECT * FROM
(
SELECT table_name, column_name
FROM ALL_TAB_COLUMNS
WHERE
table_name = 'DIM_XYZ'
)
PIVOT
(
MIN(column_name)
FOR column_name IN (
-- values added manually
'col1','col2'
-- values added manually
)
)
ORDER BY table_name;
The code works fine in this case but when replacing 'col1','col2' by the select statement to retrieve columns names the system throws the following error:
ORA-00936: missing expression
00936. 00000 - "missing expression"
*Cause:
*Action:
Error at Line: 39 Column: 40
CODE:
SELECT * FROM
(
SELECT table_name, column_name
FROM ALL_TAB_COLUMNS
WHERE
table_name = 'DIM_XYZ'
)
PIVOT
(
MIN(column_name)
FOR column_name IN (
--code below does not work when plugged in the statement above
SELECT DISTINCT
LISTAGG('''' || column_name || '''', ',')
WITHIN GROUP (ORDER BY column_name) AS temp_in_statement
FROM (SELECT DISTINCT column_name FROM all_tab_columns WHERE table_name = 'DIM_XYZ')
--code above does not work
)
)
ORDER BY table_name;
----------------
do you guys have any idea how to solve this problem?
You cannot directly add dynamic expressions as an input to Pivot table,
You can try something like this where we retrieve all the columns of the table in a variable via a PL/SQL Block and then pass it in a way to the expected by the Oracle Pivot Table Functionality.
SET serveroutput ON;
DECLARE
sqlquery VARCHAR(32767);
cols VARCHAR2(32767);
BEGIN
SELECT listagg('''' || column_name || '''', ',') within
GROUP(
ORDER BY column_name)
INTO cols
FROM
(SELECT DISTINCT column_name
FROM all_tab_columns
WHERE TABLE_NAME = 'TABLE_NAME')
;
sqlquery := '
SELECT * FROM
(
SELECT table_name, column_name
FROM ALL_TAB_COLUMNS
WHERE
table_name = ''TABLE_NAME''
)
PIVOT
(
MIN(column_name)
FOR column_name IN (
''||cols||''
)
)
ORDER BY table_name';
DBMS_OUTPUT.PUT_LINE(sqlquery);
EXECUTE IMMEDIATE sqlquery;
END;
/
I need a query to count the total number of non-null values for each column in a table. Since my table has hundreds of columns I'm looking for a solution that only requires me to input the table name.
Perhaps using the result of:
select COLUMN_NAME from ALL_TAB_COLUMNS where TABLE_NAME='ORDERS';
to get the column names and then a subquery to put counts against each column name? The additional complication is that I only have read-only access to the DB so I can't create any temp tables.
Slightly out of my league with this one so any help is appreciated.
Construct the query in SQL or using a spreadsheet. Then run the query.
For instance, assuming that your column names are simple and don't have special characters:
select replace('select ''[col]'', count([col]) from orders union all ',
'[col]', COLUMN_NAME
) as sql
from ALL_TAB_COLUMNS
where TABLE_NAME = 'ORDERS';
(Of course, this can be adapted for more complex column names, but I'm trying to show the idea.)
Then copy the code, remove the final union all and run it.
You can put this in one string if there are not too many columns:
select listagg(replace('select ''[col]'', count([col]) from orders',
'[col]', COLUMN_NAME
), ' union all '
) within group (order by column_name) as sql
from ALL_TAB_COLUMNS
where TABLE_NAME = 'ORDERS';
You can also use execute immediate using the same query, but that seems like overkill.
If you're happy with the results row-ar rather than column-ar:
SELECT 'SELECT ''dummy'', 0 FROM DUAL' FROM DUAL
UNION ALL
SELECT
' UNION ALL SELECT ''' ||
column_name ||
''', COUNT(' ||
column_name ||
') FROM ' ||
TABLE_NAME
FROM
all_tab_columns
WHERE
table_name = 'ORDERS'
This is an "SQL that writes an SQL" that you can then copy and run to get your answers. Should make a resultset that looks like:
SELECT 'dummy', 0 FROM dual
UNION ALL SELECT 'col1', COUNT(col1) FROM ORDERS
UNION ALL SELECT 'col2', COUNT(col2) FROM ORDERS
...
If you want your results column-ar:
SELECT 'SELECT '
UNION ALL
SELECT
'COUNT(' ||
column_name ||
') as count_' ||
column_name ||
', ' ||
TABLE_NAME
FROM
all_tab_columns
WHERE
table_name = 'ORDERS'
UNION ALL
SELECT 'null as dummy_column FROM ORDERS'
Should make a resultset that looks like:
SELECT
COUNT(col1) as count_col1,
COUNT(col2) as count_col2,
...
null as dummycoll FROM orders
Caveat: I don't have oracle installed anywhere I can test these, it's written from memory and may need some debugging
This will generate the SQL to get the counts in columns and will handle case sensitive column names and column names with non-alpha-numeric characters:
SELECT 'SELECT '
|| LISTAGG(
'COUNT("' || column_name || '") AS "' || column_name || '"',
', '
) WITHIN GROUP ( ORDER BY column_id )
|| ' FROM "' || table_name || '"' AS sql
FROM ALL_TAB_COLUMNS
WHERE TABLE_NAME = 'ORDERS'
GROUP BY TABLE_NAME;
or, if you have a large number of columns that is generating a string longer than 4000 characters you can use a custom aggregation function to aggregate VARCHAR2s into a CLOB and then do:
SELECT 'SELECT '
|| CLOBAgg( 'COUNT("' || column_name || '") AS "' || column_name || '"' )
|| ' FROM "' || table_name || '"' AS sql
FROM ALL_TAB_COLUMNS
WHERE TABLE_NAME = 'ORDERS'
GROUP BY TABLE_NAME;
In Oracle 19 (I used similar code in Ora 12, maybe that works too), this works without generating another select to execute:
select * from
(
select table_name, column_name,
to_number( extractvalue( xmltype(dbms_xmlgen.getxml('select count(to_char(substr('||column_name||',1,1))) c from '||table_name)) ,'/ROWSET/ROW/C')) count
from all_tab_columns where owner = user
)
--where table_name = 'MY_TABLE'
;
It will create XML with count, from which it extracts the current count. The substr and to_char functions here are used to extract first character, so it will works with CLOB columns also
Seems like this is impossible, but I'm so close - maybe someone can take me the last step...
I have a bunch of dynamic code and I don't always know the tables and columns I'm going to be dealing with, but I do know that VARCHAR2 columns with data_lengths of 2000 result in errors. I'd love to be able to identify these 'bad' columns dynamically, and remove them from my results in 1 shot.
This code:
SELECT LISTAGG(probs.column_name, ', ')
WITHIN GROUP (ORDER BY column_name) FROM
(select 1 grp, column_name
from all_tab_columns
where TABLE_NAME = 'MYTABLE' AND
DATA_TYPE <> 'VARCHAR2' AND
DATA_LENGTH < 2000
) probs
GROUP BY GRP
Gives me a nice comma, separated list of all of my acceptable column names like this:
FIELD1, FIELD2, FIELD3, FIELD4...
And I am hopeful that there's a way a can simply do something to drop that list of field names into a select statement like this:
SELECT (<my subquery, above>)
FROM MYTABLE;
Is this possible?
Assuming this situation
create table mytable ( a number, b number, c number)
insert into mytable values (10, 20, 30)
insert into mytable values (1, 2, 3)
and that only exists one table with that name (otherwise you should specify the owner in the query from all_tab_columns), your query could be simplified this way:
SELECT 'select ' || LISTAGG(column_name, ', ') WITHIN GROUP (ORDER BY column_name) || ' from ' || table_name
FROM all_tab_columns
WHERE TABLE_NAME = 'MYTABLE'
AND DATA_TYPE <> 'VARCHAR2'
AND DATA_LENGTH < 2000
GROUP BY table_name
this would give: select A, B, C from MYTABLE.
The problem here is that you can not simply run a statement that returns a variable number of columns; one way to use this could be building an xml:
SELECT xmltype(
DBMS_XMLGEN.getxml(
( SELECT 'select ' || LISTAGG(column_name, ', ') WITHIN GROUP (ORDER BY column_name) || ' from ' || table_name
FROM all_tab_columns
WHERE TABLE_NAME = 'MYTABLE'
AND DATA_TYPE <> 'VARCHAR2'
AND DATA_LENGTH < 2000
GROUP BY table_name)
)
)
FROM DUAL
<?xml version="1.0"?>
<ROWSET>
<ROW>
<A>10</A>
<B>20</B>
<C>30</C>
</ROW>
<ROW>
<A>1</A>
<B>2</B>
<C>3</C>
</ROW>
</ROWSET>
Another way could be using some PLSQL and dynamic SQL, with a little modification of yur query to concatenate the fields, to build the result in a unique string:
declare
type tTabResults is table of varchar2(1000);
vSQL varchar2(1000);
vTabResults tTabResults;
begin
SELECT 'select ' || LISTAGG( column_name, '|| '', '' ||') WITHIN GROUP (ORDER BY column_name) || ' from ' || table_name
into vSQL
FROM all_tab_columns
WHERE TABLE_NAME = 'MYTABLE'
AND DATA_TYPE <> 'VARCHAR2'
AND DATA_LENGTH < 2000
GROUP BY table_name;
--
execute immediate vSQL bulk collect into vTabResults;
--
for i in vTabResults.first .. vTabResults.last loop
dbms_output.put_line(vTabResults(i));
end loop;
end;
10, 20, 30
1, 2, 3
Notice that I oversimplified the problem, treating numbers as strings and not using any conversion, by simply printing the values in your table, no matter their type; in a real solution you should handle the possible types of your columns and modify the initial query to add some type conversions.