Oracle xmlagg get correct output - sql
I'm trying to do a dynamic pivot for some columns in a table. Normally, it goes smoothly with a listagg but in this case, since the resulting variable was greater than 3K characters, I had to go with xmlagg. However, I'm not able to get the output as
'COLUMN' as "COLUMN"
so I can then pivot the columns just I'm used to do with listagg.
SELECT
rtrim(XMLAGG(xmlelement(e, ''''
|| agr_name
|| ''' as "'
|| agr_name
|| '"', ', ').extract('//text()')
ORDER BY
agr_name
).getclobval(), ', ') agr_name
FROM
(
SELECT DISTINCT
agr_name
FROM
dat_skills
);
What I'm getting instead is
'COLUMN; as "COLUMN;,
Use xmlcast - it will solve several of your problems. I don't have your table, but here is how it works on a different one:
SELECT
rtrim(xmlcast(XMLAGG(xmlelement(e, ''''
|| object_type
|| ''' as "'
|| object_type
|| '"', ', ')
ORDER BY
object_type) as clob), ', ')
object_type
FROM
(
SELECT DISTINCT
object_type
FROM
user_objects
);
OBJECT_TYPE
--------------------------------------------------------------------------------
'FUNCTION' as "FUNCTION", 'INDEX' as "INDEX", 'LIBRARY' as "LIBRARY", 'LOB' as "
LOB", 'PROCEDURE' as "PROCEDURE", 'SEQUENCE' as "SEQUENCE", 'TABLE' as "TABLE",
'TYPE' as "TYPE", 'TYPE BODY' as "TYPE BODY", 'VIEW' as "VIEW"
You no longer need to extract text (cast as CLOB does that already), you don't need to worry about escaping and unescaping, and the result is a CLOB as you need it to be.
Include call to UTL_I18N.unescape_reference:
SELECT UTL_I18N.unescape_reference (
XMLAGG (XMLELEMENT (
e,
'''' || agr_name || ''' as "' || agr_name || '"',
', ').EXTRACT ('//text()')
ORDER BY agr_name).getclobval ()) agr_name
FROM (SELECT DISTINCT agr_name
FROM dat_skills);
With:
SQL> SELECT UTL_I18N.unescape_reference (
2 XMLAGG (XMLELEMENT (
3 e,
4 '''' || agr_name || ''' as "' || agr_name || '"',
5 ', ').EXTRACT ('//text()')
6 ORDER BY agr_name).getclobval ()) agr_name
7 FROM (SELECT DISTINCT agr_name
8 FROM dat_skills);
AGR_NAME
--------------------------------------------------------------------------------
'ACCOUNTING' as "ACCOUNTING", 'OPERATIONS' as "OPERATIONS", 'RESEARCH' as "RESEA
RCH", 'SALES' as "SALES",
Without:
SQL> SELECT
2 XMLAGG (XMLELEMENT (
3 e,
4 '''' || agr_name || ''' as "' || agr_name || '"',
5 ', ').EXTRACT ('//text()')
6 ORDER BY agr_name).getclobval () agr_name
7 FROM (SELECT DISTINCT dname agr_name
8 FROM dept);
AGR_NAME
--------------------------------------------------------------------------------
'ACCOUNTING' as "ACCOUNTING", 'OPERATIONS' as &quo
SQL>
Assuming you don't have any other characters that need unescaping, you can replace " and ' them with the actual quotes:
SELECT REPLACE(
REPLACE(
RTRIM(
XMLAGG(
xmlelement(
e,
'''' || agr_name || ''' as "' || agr_name || '"',
', '
).extract('//text()')
ORDER BY agr_name
).getclobval(),
', '
),
'"',
'"'
),
''',
''''
) AS agr_name
FROM (
SELECT DISTINCT
agr_name
FROM dat_skills
);
db<>fiddle here
Related
ORA-01489 : result of string concatenation to long
I am creating a dynamic query from a list of tables which are their in my table called : get_table_names My Query : SELECT listagg('SELECT ' || '''' || tbl_name || '''' || ' AS TBL , COUNT(*) as CNT FROM ' || 'TGT_MB.' || tbl_name || ' union all' || CHAR(10) ) WITH GROUP ( ORDER BY tbl_name) AS sql_str FROM get_table_names; My get_table_names as lots of table , at least 70 table names. The query works fine for 10 tables but more then that it throws the error like below ORA-01489 : result of string concatenation to long Their is some option called EXTEND that option I cannot touch as I have low level privileges. cannot make changes to it until DBA. Any other work around would be much appreciated like using some XMLAGG or CLOB , BLOB
Can I respectively suggest that when posting a code snippet, make it sure it runs so that people can help you more easily. eg your code above took the following debugging before we could even start to see what you want to do SQL> SELECT listagg('SELECT ' 2 || '''' 3 || table_name 4 || '''' 5 || ' AS TBL , COUNT(*) as CNT FROM ' 6 || 'TGT_MB.' 7 || table_name 8 || ' union all' 9 || CHAR(10) ) WITH GROUP ( ORDER BY table_name) AS sql_str 10 FROM dba_tables 11 where owner = 'SCOTT'; || CHAR(10) ) WITH GROUP ( ORDER BY table_name) AS sql_str * ERROR at line 9: ORA-00936: missing expression SQL> SELECT listagg('SELECT ' 2 || '''' 3 || table_name 4 || '''' 5 || ' AS TBL , COUNT(*) as CNT FROM ' 6 || 'TGT_MB.' 7 || table_name 8 || ' union all' 9 || chr(10) ) WITH GROUP ( ORDER BY table_name) AS sql_str 10 FROM dba_tables 11 where owner = 'SCOTT'; || chr(10) ) WITH GROUP ( ORDER BY table_name) AS sql_str * ERROR at line 9: ORA-00923: FROM keyword not found where expected Once those fixes are done, we end up here SQL> SELECT listagg('SELECT ' 2 || '''' 3 || table_name 4 || '''' 5 || ' AS TBL , COUNT(*) as CNT FROM ' 6 || 'TGT_MB.' 7 || table_name 8 || ' union all' 9 || chr(10),'' ) WITHIN GROUP ( ORDER BY table_name) AS sql_str 10 FROM dba_tables 11 where owner = 'SCOTT'; SQL_STR ------------------------------------------------------------------------------------- SELECT 'BONUS' AS TBL , COUNT(*) as CNT FROM TGT_MB.BONUS union all SELECT 'DEPT' AS TBL , COUNT(*) as CNT FROM TGT_MB.DEPT union all SELECT 'EMP' AS TBL , COUNT(*) as CNT FROM TGT_MB.EMP union all SELECT 'EMP2' AS TBL , COUNT(*) as CNT FROM TGT_MB.EMP2 union all SELECT 'SALGRADE' AS TBL , COUNT(*) as CNT FROM TGT_MB.SALGRADE union all which ultimately blows up when we get too many tables SQL> SELECT listagg('SELECT ' 2 || '''' 3 || table_name 4 || '''' 5 || ' AS TBL , COUNT(*) as CNT FROM ' 6 || 'TGT_MB.' 7 || table_name 8 || ' union all' 9 || chr(10),'' ) WITHIN GROUP ( ORDER BY table_name) AS sql_str 10 FROM dba_tables; FROM dba_tables * ERROR at line 10: ORA-01489: result of string concatenation is too long Now even if we use a tool (eg https://github.com/connormcd/listagg_clob) to get the results as a clob, then I imagine you are just going to fire that SQL at the database to get a count of all tables. If that is the case, then why do you need to LISTAGG at all, just build a script to do it, ie SELECT 'SELECT ' || '''' || table_name || '''' || ' AS TBL , COUNT(*) as CNT FROM ' || 'TGT_MB.' || table_name || ' union all' FROM dba_tables Spool that to a file, trim the last union all and run it as a script. Or even better...think about what benefit the output gives you. Why does anyone need the exact row count? Querying NUM_ROWS from USER_TABLES is probably sufficient.
Search and return multiple wildcard values in a single column
All - I'm looking at a large volume of sql query history data. Ultimately I need to create a distinct list of tables used by each from a list of their executed queries. Let's say my simplified table for this example is: create table zwork_example ( username varchar2(50), sql_text clob); insert into zwork_example (username, sql_text) values ('user1', 'schema1.table1, schema1.table2, schema2.table1, schema1.table1'); insert into zwork_example (username, sql_text) Values ('user2', 'schema1.table3, schema1.table2, schema2.table1, schema1.table6'); Does anyone have any ideas on how I can search for schema1* and return N number of table names that belong to schema1? In this example I have a particular schema I'm interested in, so I can explicitly state that schema1 is the only schema I'm interested in returning the table names from. The output I would look for given this example is: User Schema Tables -------- -------- -------- User1 schema1 table1, table2 User2 schema1 table3, table2, table6
CLOB, large data set ... doesn't sound promising. Read: it'll probably take some time to get the result. For such a small data set, see if this helps. SQL> with 2 search_for (schema) as 3 (select 'schema1' from dual), 4 temp as 5 (select distinct 6 username, 7 trim(regexp_substr(dbms_lob.substr(sql_text, 32767), '[^,]+', 1, column_value)) col 8 from zwork_example cross join 9 table(cast(multiset(select level from dual 10 connect by level <= regexp_count(sql_text, ',') + 1 11 ) as sys.odcinumberlist)) 12 ) 13 select 14 t.username, 15 s.schema, 16 listagg(trim(replace(t.col, s.schema ||'.', null)), ', ') within group (order by null) tables 17 from temp t join search_for s on instr(t.col, s.schema) > 0 18 group by t.username, s.schema; USERNAME SCHEMA TABLES ---------- ------- -------------------------------------------------- user1 schema1 table1, table2 user2 schema1 table2, table3, table6 SQL> What does it do? search_for CTE contains a schema you're searching for (schema1, right?) temp CTE splits the sql_text into rows moreover, in order to get distinct list of tables, I applied dbms_lob.substr to the column, hoping that you don't actually have strings longer than that (32767, that is) final query just aggregates tables it found - rows where schemas match
For a large dataset you can use: WITH search_string ( schema_name ) AS ( SELECT 'schema1' FROM DUAL ), matches ( username, sql_text, schema_name, end_pos, matches ) AS ( SELECT username, sql_text, schema_name, CASE WHEN sql_text LIKE schema_name || '%' THEN INSTR( sql_text, ',' ) WHEN sql_text NOT LIKE ', ' || schema_name || '.' THEN 0 ELSE INSTR( sql_text, ', ' || schema_name || '.', INSTR( sql_text, ', ' || schema_name || '.' ) + 3 ) END, CASE WHEN sql_text LIKE schema_name || '%' THEN EMPTY_CLOB() || SUBSTR( sql_text, LENGTH(schema_name || '.') + 1, INSTR( sql_text, ',' ) - LENGTH(schema_name || '.') - 1 ) WHEN INSTR( sql_text, ', ' || schema_name || '.' ) = 0 THEN NULL WHEN INSTR( sql_text, ',', INSTR( sql_text, ', ' || schema_name || '.' ) + 3 ) = 0 THEN EMPTY_CLOB() || SUBSTR( sql_text, INSTR( sql_text, ', ' || schema_name || '.' ) + LENGTH(', ' || schema_name || '.') ) ELSE EMPTY_CLOB() || SUBSTR( sql_text, INSTR( sql_text, ', ' || schema_name || '.' ) + LENGTH(', ' || schema_name || '.'), INSTR( sql_text, ',', INSTR( sql_text, ', ' || schema_name || '.' ) + 3 ) - INSTR( sql_text, ', ' || schema_name || '.' ) - LENGTH(', ' || schema_name || '.') ) END FROM zwork_example CROSS JOIN search_string UNION ALL SELECT username, sql_text, schema_name, INSTR( sql_text, ', ' || schema_name || '.', end_pos + 3 ), matches || ', ' || CASE WHEN INSTR( sql_text, ',', end_pos + 1 ) = 0 THEN SUBSTR( sql_text, end_pos + LENGTH(', ' || schema_name || '.') ) ELSE SUBSTR( sql_text, end_pos + LENGTH(', ' || schema_name || '.'), INSTR( sql_text, ',', end_pos + 1 ) - end_pos - LENGTH(', ' || schema_name || '.') ) END FROM matches WHERE end_pos > 0 ) SELECT username, matches FROM matches WHERE end_pos = 0; Which, for the sample data: create table zwork_example ( username varchar2(50), sql_text clob ); DECLARE v_text CLOB; BEGIN insert into zwork_example (username, sql_text) values ('user1', 'schema1.table1, schema1.table2, schema2.table1, schema1.table1'); insert into zwork_example (username, sql_text) Values ('user2', 'schema1.table3, schema1.table2, schema2.table1, schema1.table6'); insert into zwork_example (username, sql_text) Values ('user3', 'schema2.table3, schema3.table2, schema4.table1, schema2.table6'); insert into zwork_example (username, sql_text) Values ('user4', 'schema2.table3, schema3.table2, schema4.table1, schema1.table6'); v_text := 'schema1.table1'; FOR i IN 2 .. 250 LOOP v_text := v_text || ', schema1.table' || i; END LOOP; insert into zwork_example (username, sql_text) values ( 'user5', v_text ); END; / Outputs: USERNAME MATCHES user3 user4 table6 user1 table1, table2, table1 user2 table3, table2, table6 user5 table1, table2, table3, table4, table5, table6, table7, table8, table9, table10, table11, table12, table13, table14, table15, table16, table17, table18, table19, table20, table21, table22, table23, table24, table25, table26, table27, table28, table29, table30, table31, table32, table33, table34, table35, table36, table37, table38, table39, table40, table41, table42, table43, table44, table45, table46, table47, table48, table49, table50, table51, table52, table53, table54, table55, table56, table57, table58, table59, table60, table61, table62, table63, table64, table65, table66, table67, table68, table69, table70, table71, table72, table73, table74, table75, table76, table77, table78, table79, table80, table81, table82, table83, table84, table85, table86, table87, table88, table89, table90, table91, table92, table93, table94, table95, table96, table97, table98, table99, table100, table101, table102, table103, table104, table105, table106, table107, table108, table109, table110, table111, table112, table113, table114, table115, table116, table117, table118, table119, table120, table121, table122, table123, table124, table125, table126, table127, table128, table129, table130, table131, table132, table133, table134, table135, table136, table137, table138, table139, table140, table141, table142, table143, table144, table145, table146, table147, table148, table149, table150, table151, table152, table153, table154, table155, table156, table157, table158, table159, table160, table161, table162, table163, table164, table165, table166, table167, table168, table169, table170, table171, table172, table173, table174, table175, table176, table177, table178, table179, table180, table181, table182, table183, table184, table185, table186, table187, table188, table189, table190, table191, table192, table193, table194, table195, table196, table197, table198, table199, table200, table201, table202, table203, table204, table205, table206, table207, table208, table209, table210, table211, table212, table213, table214, table215, table216, table217, table218, table219, table220, table221, table222, table223, table224, table225, table226, table227, table228, table229, table230, table231, table232, table233, table234, table235, table236, table237, table238, table239, table240, table241, table242, table243, table244, table245, table246, table247, table248, table249, table250 db<>fiddle here
Convert ` LISTAGG` TO `XMLAGG`
How to convert LISTAGG with case statements to XMLAGG equivalent, so as to avoid the concatenation error. #ECHO ${cols_2 ||32767||varchar2}$ --Declare variable SELECT LISTAGG( 'MAX(CASE WHEN CATEGORY = '''||CATEGORY||''' THEN "'||"LEVEL"||'" END) AS "'||"LEVEL"||'_'||CATEGORY||'"' , ',' ) WITHIN GROUP( ORDER BY CATEGORY, "LEVEL" DESC ) INTO cols_2 FROM ( SELECT DISTINCT "LEVEL", CATEGORY FROM temp ); I tried this and I'm getting an error saying missing keyword #ECHO ${cols_2 ||32767||varchar2}$ --Declare variable select rtrim ( xmlagg (xmlelement (e, 'MAX(CASE WHEN CATEGORY = '''||CATEGORY||''' THEN "'||"LEVEL"||'" END) AS "'||LEVEL||'_'||CATEGORY||'"', ',') order by 1,2 desc).extract ( '//text()'), ', ') INTO cols_2 FROM ( SELECT DISTINCT "LEVEL", CATEGORY temp ); I have tried this an declared the cols_2 as clob type :- SELECT DBMS_XMLGEN.CONVERT ( RTRIM ( XMLAGG (XMLELEMENT ( e, 'MAX(CASE WHEN CATEGORY = ''' || CATEGORY || ''' THEN "' || "LEVEL" || '" END) AS "' || "LEVEL" || '_' || CATEGORY || '"', ',') ORDER BY 1, DESC).EXTRACT('//text()').getclobval(),','),1) ', '), 1) INTO cols_2 FROM (SELECT DISTINCT "LEVEL", CATEGORY FROM temp); Yet my issue is not resolved ,Im getting an error while trying to execute it as a procedure like :- Error in concatenation of `LISTAGG` function[Not a duplicate question]
You are getting the missing keyword error because you are most likely attempting to run the second query as a standalone query instead of in a PL/SQL block. When you are doing that, you have to remove your into cols_2 clause. That is your immediate issue that should resolve your error. Also, based on your prior question, using the XML functions will escape your ' and " characters so you will want to make sure to unescape them back to their original characters so you can use them in your dynamic sql query like this: SELECT DBMS_XMLGEN.CONVERT ( RTRIM ( XMLAGG (XMLELEMENT ( e, 'MAX(CASE WHEN CATEGORY = ''' || CATEGORY || ''' THEN "' || "LEVEL" || '" END) AS "' || "LEVEL" || '_' || CATEGORY || '"', ',') ORDER BY 1, 2 DESC).EXTRACT ('//text()'), ', '), 1) --INTO cols_2 FROM (SELECT DISTINCT "LEVEL", CATEGORY FROM temp);
Convert SQL query to pivot
I can't seem able to get the logic to convert the following query to a pivot SQL. My table has 20 columns with roles on them, I'd like to convert those columns into rows so, when exported to Excel, I can filter on a single column since the values can be the same on the 20 columns. So far what I've done is convert the 20 columns into a single one and then split that single one into rows: select distinct TASKID, regexp_substr(t.roles,'[^|]+', 1, lines.column_value) as role from ( select TASKID, TRIM(ROLE1) || '|' || TRIM(ROLE2) || '|' || TRIM(ROLE3) || '|' || TRIM(ROLE4) || '|' || TRIM(ROLE5) || '|' || TRIM(ROLE6) || '|' || TRIM(ROLE7) || '|' || TRIM(ROLE8) || '|' || TRIM(ROLE9) || '|' || TRIM(ROLE10) || '|' || TRIM(ROLE11) || '|' || TRIM(ROLE12) || '|' || TRIM(ROLE13) || '|' || TRIM(ROLE14) || '|' || TRIM(ROLE15) || '|' || TRIM(ROLE16) || '|' || TRIM(ROLE17) || '|' || TRIM(ROLE18) || '|' || TRIM(ROLE19) || '|' || TRIM(ROLE20) as roles from menu_roles where RLTYPE='58' ) t, TABLE(CAST(MULTISET(select LEVEL from dual connect by instr(t.roles, '|', 1, LEVEL - 1) > 0) as sys.odciNumberList)) lines where regexp_substr(t.roles,'[^|]+', 1, lines.column_value) is not null order by regexp_substr(t.roles,'[^|]+', 1, lines.column_value) I'd understand that using PIVOT would be more efficient vs concatenating and splitting a string. Thank you!
You appear to want UNPIVOT: SELECT task_id, role FROM menu_roles UNPIVOT ( role FOR role_number IN ( ROLE1, ROLE2, ROLE3, ROLE4 /*, ... */ ) ); Or, using UNION ALL: SELECT task_id, role1 AS role FROM menu_roles UNION ALL SELECT task_id, role2 AS role FROM menu_roles UNION ALL SELECT task_id, role3 AS role FROM menu_roles UNION ALL SELECT task_id, role4 AS role FROM menu_roles -- ...
SQL scripts to generate SQL scripts
Assume that the DBA_TAB_COLUMNS looks like this: I'd like to write a SQL or PL/SQL script to generate following text: select 'NULL' as A1, B1, QUERY, RECORD_KEY from SMHIST.probsummarym1 union all select 'NULL' as A1, 'NULL' as B1, QUERY, RECORD_KEY from SMHIST_EIT200.probsummarym1 union all select A1, 'NULL' as B1, QUERY, RECORD_KEY from SMHIST_EIT300.probsummarym1 the requirements are: If the table under any of the SMHIST% schemas do not have that column, then insert a default NULL alias for that columns. the column list is in alphabetical order. so can anybody tell me how to write this script?
EDIT: Added better alias names and en explicit CROSS JOIN. Added XMLAGG version. NB: LISTAGG exists from Oracle version 11.2 and onwards and returns VARCHAR2. If the output string is larger than 4000K or if on a prior version you can use XMLAGG which is a bit more cumbersome to work with (eg. http://psoug.org/definition/xmlagg.htm). With LISTAGG (returning VARCHAR2): SELECT LISTAGG (line, CHR (13) || CHR (10) || 'union all' || CHR (13) || CHR (10)) WITHIN GROUP (ORDER BY sortorder) script FROM (SELECT line, ROWNUM sortorder FROM ( SELECT 'select ' || LISTAGG ( CASE WHEN tc.column_name IS NULL THEN '''NULL'' as ' END || col_join.column_name, ', ') WITHIN GROUP (ORDER BY col_join.column_name) || ' from ' || col_join.owner || '.' || col_join.table_name line FROM dba_tab_columns tc RIGHT OUTER JOIN (SELECT DISTINCT owner, table_name, col_list.column_name FROM dba_tab_columns CROSS JOIN (SELECT DISTINCT column_name FROM dba_tab_columns WHERE owner LIKE 'SMHIST%') col_list WHERE owner LIKE 'SMHIST%') col_join ON tc.owner = col_join.owner AND tc.table_name = col_join.table_name AND tc.column_name = col_join.column_name GROUP BY col_join.owner, col_join.table_name ORDER BY col_join.owner, col_join.table_name)) With XMLAGG (returning CLOB by adding .getclobval (), note: RTRIM works here because table names cannot include ',' and ' ' (space)): SELECT REPLACE (SUBSTR (script, 1, LENGTH (script) - 12), '&' || 'apos;', '''') FROM (SELECT XMLAGG ( XMLELEMENT ( e, line, CHR (13) || CHR (10) || 'union all' || CHR (13) || CHR (10))).EXTRACT ('//text()').getclobval () script FROM (SELECT line, ROWNUM sortorder FROM ( SELECT 'select ' || RTRIM ( REPLACE ( XMLAGG (XMLELEMENT ( e, CASE WHEN tc.column_name IS NULL THEN '''NULL'' as ' END || col_join.column_name, ', ') ORDER BY col_join.column_name).EXTRACT ( '//text()').getclobval (), '&' || 'apos;', ''''), ', ') || ' from ' || col_join.owner || '.' || col_join.table_name line FROM dba_tab_columns tc RIGHT OUTER JOIN (SELECT DISTINCT owner, table_name, col_list.column_name FROM dba_tab_columns CROSS JOIN (SELECT DISTINCT column_name FROM dba_tab_columns WHERE owner LIKE 'SMHIST%') col_list WHERE owner LIKE 'SMHIST%') col_join ON tc.owner = col_join.owner AND tc.table_name = col_join.table_name AND tc.column_name = col_join.column_name GROUP BY col_join.owner, col_join.table_name ORDER BY col_join.owner, col_join.table_name)))