Oracle xmlagg get correct output - sql

I'm trying to do a dynamic pivot for some columns in a table. Normally, it goes smoothly with a listagg but in this case, since the resulting variable was greater than 3K characters, I had to go with xmlagg. However, I'm not able to get the output as
'COLUMN' as "COLUMN"
so I can then pivot the columns just I'm used to do with listagg.
SELECT
rtrim(XMLAGG(xmlelement(e, ''''
|| agr_name
|| ''' as "'
|| agr_name
|| '"', ', ').extract('//text()')
ORDER BY
agr_name
).getclobval(), ', ') agr_name
FROM
(
SELECT DISTINCT
agr_name
FROM
dat_skills
);
What I'm getting instead is
'COLUMN; as "COLUMN;,

Use xmlcast - it will solve several of your problems. I don't have your table, but here is how it works on a different one:
SELECT
rtrim(xmlcast(XMLAGG(xmlelement(e, ''''
|| object_type
|| ''' as "'
|| object_type
|| '"', ', ')
ORDER BY
object_type) as clob), ', ')
object_type
FROM
(
SELECT DISTINCT
object_type
FROM
user_objects
);
OBJECT_TYPE
--------------------------------------------------------------------------------
'FUNCTION' as "FUNCTION", 'INDEX' as "INDEX", 'LIBRARY' as "LIBRARY", 'LOB' as "
LOB", 'PROCEDURE' as "PROCEDURE", 'SEQUENCE' as "SEQUENCE", 'TABLE' as "TABLE",
'TYPE' as "TYPE", 'TYPE BODY' as "TYPE BODY", 'VIEW' as "VIEW"
You no longer need to extract text (cast as CLOB does that already), you don't need to worry about escaping and unescaping, and the result is a CLOB as you need it to be.

Include call to UTL_I18N.unescape_reference:
SELECT UTL_I18N.unescape_reference (
XMLAGG (XMLELEMENT (
e,
'''' || agr_name || ''' as "' || agr_name || '"',
', ').EXTRACT ('//text()')
ORDER BY agr_name).getclobval ()) agr_name
FROM (SELECT DISTINCT agr_name
FROM dat_skills);
With:
SQL> SELECT UTL_I18N.unescape_reference (
2 XMLAGG (XMLELEMENT (
3 e,
4 '''' || agr_name || ''' as "' || agr_name || '"',
5 ', ').EXTRACT ('//text()')
6 ORDER BY agr_name).getclobval ()) agr_name
7 FROM (SELECT DISTINCT agr_name
8 FROM dat_skills);
AGR_NAME
--------------------------------------------------------------------------------
'ACCOUNTING' as "ACCOUNTING", 'OPERATIONS' as "OPERATIONS", 'RESEARCH' as "RESEA
RCH", 'SALES' as "SALES",
Without:
SQL> SELECT
2 XMLAGG (XMLELEMENT (
3 e,
4 '''' || agr_name || ''' as "' || agr_name || '"',
5 ', ').EXTRACT ('//text()')
6 ORDER BY agr_name).getclobval () agr_name
7 FROM (SELECT DISTINCT dname agr_name
8 FROM dept);
AGR_NAME
--------------------------------------------------------------------------------
'ACCOUNTING' as "ACCOUNTING", 'OPERATIONS' as &quo
SQL>

Assuming you don't have any other characters that need unescaping, you can replace " and ' them with the actual quotes:
SELECT REPLACE(
REPLACE(
RTRIM(
XMLAGG(
xmlelement(
e,
'''' || agr_name || ''' as "' || agr_name || '"',
', '
).extract('//text()')
ORDER BY agr_name
).getclobval(),
', '
),
'"',
'"'
),
''',
''''
) AS agr_name
FROM (
SELECT DISTINCT
agr_name
FROM dat_skills
);
db<>fiddle here

Related

ORA-01489 : result of string concatenation to long

I am creating a dynamic query from a list of tables which are their in my table called : get_table_names
My Query :
SELECT listagg('SELECT '
|| ''''
|| tbl_name
|| ''''
|| ' AS TBL , COUNT(*) as CNT FROM '
|| 'TGT_MB.'
|| tbl_name
|| ' union all'
|| CHAR(10) ) WITH GROUP ( ORDER BY tbl_name) AS sql_str
FROM get_table_names;
My get_table_names as lots of table , at least 70 table names.
The query works fine for 10 tables but more then that it throws the error like below
ORA-01489 : result of string concatenation to long
Their is some option called EXTEND that option I cannot touch as I have low level privileges.
cannot make changes to it until DBA.
Any other work around would be much appreciated like using some XMLAGG or CLOB , BLOB
Can I respectively suggest that when posting a code snippet, make it sure it runs so that people can help you more easily. eg your code above took the following debugging before we could even start to see what you want to do
SQL> SELECT listagg('SELECT '
2 || ''''
3 || table_name
4 || ''''
5 || ' AS TBL , COUNT(*) as CNT FROM '
6 || 'TGT_MB.'
7 || table_name
8 || ' union all'
9 || CHAR(10) ) WITH GROUP ( ORDER BY table_name) AS sql_str
10 FROM dba_tables
11 where owner = 'SCOTT';
|| CHAR(10) ) WITH GROUP ( ORDER BY table_name) AS sql_str
*
ERROR at line 9:
ORA-00936: missing expression
SQL> SELECT listagg('SELECT '
2 || ''''
3 || table_name
4 || ''''
5 || ' AS TBL , COUNT(*) as CNT FROM '
6 || 'TGT_MB.'
7 || table_name
8 || ' union all'
9 || chr(10) ) WITH GROUP ( ORDER BY table_name) AS sql_str
10 FROM dba_tables
11 where owner = 'SCOTT';
|| chr(10) ) WITH GROUP ( ORDER BY table_name) AS sql_str
*
ERROR at line 9:
ORA-00923: FROM keyword not found where expected
Once those fixes are done, we end up here
SQL> SELECT listagg('SELECT '
2 || ''''
3 || table_name
4 || ''''
5 || ' AS TBL , COUNT(*) as CNT FROM '
6 || 'TGT_MB.'
7 || table_name
8 || ' union all'
9 || chr(10),'' ) WITHIN GROUP ( ORDER BY table_name) AS sql_str
10 FROM dba_tables
11 where owner = 'SCOTT';
SQL_STR
-------------------------------------------------------------------------------------
SELECT 'BONUS' AS TBL , COUNT(*) as CNT FROM TGT_MB.BONUS union all
SELECT 'DEPT' AS TBL , COUNT(*) as CNT FROM TGT_MB.DEPT union all
SELECT 'EMP' AS TBL , COUNT(*) as CNT FROM TGT_MB.EMP union all
SELECT 'EMP2' AS TBL , COUNT(*) as CNT FROM TGT_MB.EMP2 union all
SELECT 'SALGRADE' AS TBL , COUNT(*) as CNT FROM TGT_MB.SALGRADE union all
which ultimately blows up when we get too many tables
SQL> SELECT listagg('SELECT '
2 || ''''
3 || table_name
4 || ''''
5 || ' AS TBL , COUNT(*) as CNT FROM '
6 || 'TGT_MB.'
7 || table_name
8 || ' union all'
9 || chr(10),'' ) WITHIN GROUP ( ORDER BY table_name) AS sql_str
10 FROM dba_tables;
FROM dba_tables
*
ERROR at line 10:
ORA-01489: result of string concatenation is too long
Now even if we use a tool (eg
https://github.com/connormcd/listagg_clob) to get the results as a clob, then I imagine you are just going to fire that SQL at the database to get a count of all tables. If that is the case, then why do you need to LISTAGG at all, just build a script to do it, ie
SELECT 'SELECT '
|| ''''
|| table_name
|| ''''
|| ' AS TBL , COUNT(*) as CNT FROM '
|| 'TGT_MB.'
|| table_name
|| ' union all'
FROM dba_tables
Spool that to a file, trim the last union all and run it as a script.
Or even better...think about what benefit the output gives you. Why does anyone need the exact row count? Querying NUM_ROWS from USER_TABLES is probably sufficient.

Search and return multiple wildcard values in a single column

All -
I'm looking at a large volume of sql query history data. Ultimately I need to create a distinct list of tables used by each from a list of their executed queries. Let's say my simplified table for this example is:
create table zwork_example (
username varchar2(50),
sql_text clob);
insert into zwork_example (username, sql_text)
values ('user1', 'schema1.table1, schema1.table2, schema2.table1, schema1.table1');
insert into zwork_example (username, sql_text)
Values ('user2', 'schema1.table3, schema1.table2, schema2.table1, schema1.table6');
Does anyone have any ideas on how I can search for schema1* and return N number of table names that belong to schema1? In this example I have a particular schema I'm interested in, so I can explicitly state that schema1 is the only schema I'm interested in returning the table names from.
The output I would look for given this example is:
User Schema Tables
-------- -------- --------
User1 schema1 table1, table2
User2 schema1 table3, table2, table6
CLOB, large data set ... doesn't sound promising. Read: it'll probably take some time to get the result. For such a small data set, see if this helps.
SQL> with
2 search_for (schema) as
3 (select 'schema1' from dual),
4 temp as
5 (select distinct
6 username,
7 trim(regexp_substr(dbms_lob.substr(sql_text, 32767), '[^,]+', 1, column_value)) col
8 from zwork_example cross join
9 table(cast(multiset(select level from dual
10 connect by level <= regexp_count(sql_text, ',') + 1
11 ) as sys.odcinumberlist))
12 )
13 select
14 t.username,
15 s.schema,
16 listagg(trim(replace(t.col, s.schema ||'.', null)), ', ') within group (order by null) tables
17 from temp t join search_for s on instr(t.col, s.schema) > 0
18 group by t.username, s.schema;
USERNAME SCHEMA TABLES
---------- ------- --------------------------------------------------
user1 schema1 table1, table2
user2 schema1 table2, table3, table6
SQL>
What does it do?
search_for CTE contains a schema you're searching for (schema1, right?)
temp CTE splits the sql_text into rows
moreover, in order to get distinct list of tables, I applied dbms_lob.substr to the column, hoping that you don't actually have strings longer than that (32767, that is)
final query just aggregates tables it found - rows where schemas match
For a large dataset you can use:
WITH search_string ( schema_name ) AS (
SELECT 'schema1' FROM DUAL
),
matches ( username, sql_text, schema_name, end_pos, matches ) AS (
SELECT username,
sql_text,
schema_name,
CASE
WHEN sql_text LIKE schema_name || '%'
THEN INSTR( sql_text, ',' )
WHEN sql_text NOT LIKE ', ' || schema_name || '.'
THEN 0
ELSE INSTR(
sql_text,
', ' || schema_name || '.',
INSTR( sql_text, ', ' || schema_name || '.' ) + 3
)
END,
CASE
WHEN sql_text LIKE schema_name || '%'
THEN EMPTY_CLOB() || SUBSTR(
sql_text,
LENGTH(schema_name || '.') + 1,
INSTR( sql_text, ',' ) - LENGTH(schema_name || '.') - 1
)
WHEN INSTR( sql_text, ', ' || schema_name || '.' ) = 0
THEN NULL
WHEN INSTR( sql_text, ',', INSTR( sql_text, ', ' || schema_name || '.' ) + 3 ) = 0
THEN EMPTY_CLOB() || SUBSTR(
sql_text,
INSTR( sql_text, ', ' || schema_name || '.' )
+ LENGTH(', ' || schema_name || '.')
)
ELSE EMPTY_CLOB() || SUBSTR(
sql_text,
INSTR( sql_text, ', ' || schema_name || '.' )
+ LENGTH(', ' || schema_name || '.'),
INSTR( sql_text, ',', INSTR( sql_text, ', ' || schema_name || '.' ) + 3 )
- INSTR( sql_text, ', ' || schema_name || '.' )
- LENGTH(', ' || schema_name || '.')
)
END
FROM zwork_example
CROSS JOIN search_string
UNION ALL
SELECT username,
sql_text,
schema_name,
INSTR(
sql_text,
', ' || schema_name || '.',
end_pos + 3
),
matches
|| ', '
|| CASE
WHEN INSTR( sql_text, ',', end_pos + 1 ) = 0
THEN SUBSTR(
sql_text,
end_pos + LENGTH(', ' || schema_name || '.')
)
ELSE SUBSTR(
sql_text,
end_pos + LENGTH(', ' || schema_name || '.'),
INSTR( sql_text, ',', end_pos + 1 )
- end_pos - LENGTH(', ' || schema_name || '.')
)
END
FROM matches
WHERE end_pos > 0
)
SELECT username,
matches
FROM matches
WHERE end_pos = 0;
Which, for the sample data:
create table zwork_example (
username varchar2(50),
sql_text clob
);
DECLARE
v_text CLOB;
BEGIN
insert into zwork_example (username, sql_text)
values ('user1', 'schema1.table1, schema1.table2, schema2.table1, schema1.table1');
insert into zwork_example (username, sql_text)
Values ('user2', 'schema1.table3, schema1.table2, schema2.table1, schema1.table6');
insert into zwork_example (username, sql_text)
Values ('user3', 'schema2.table3, schema3.table2, schema4.table1, schema2.table6');
insert into zwork_example (username, sql_text)
Values ('user4', 'schema2.table3, schema3.table2, schema4.table1, schema1.table6');
v_text := 'schema1.table1';
FOR i IN 2 .. 250 LOOP
v_text := v_text || ', schema1.table' || i;
END LOOP;
insert into zwork_example (username, sql_text) values ( 'user5', v_text );
END;
/
Outputs:
USERNAME
MATCHES
user3
user4
table6
user1
table1, table2, table1
user2
table3, table2, table6
user5
table1, table2, table3, table4, table5, table6, table7, table8, table9, table10, table11, table12, table13, table14, table15, table16, table17, table18, table19, table20, table21, table22, table23, table24, table25, table26, table27, table28, table29, table30, table31, table32, table33, table34, table35, table36, table37, table38, table39, table40, table41, table42, table43, table44, table45, table46, table47, table48, table49, table50, table51, table52, table53, table54, table55, table56, table57, table58, table59, table60, table61, table62, table63, table64, table65, table66, table67, table68, table69, table70, table71, table72, table73, table74, table75, table76, table77, table78, table79, table80, table81, table82, table83, table84, table85, table86, table87, table88, table89, table90, table91, table92, table93, table94, table95, table96, table97, table98, table99, table100, table101, table102, table103, table104, table105, table106, table107, table108, table109, table110, table111, table112, table113, table114, table115, table116, table117, table118, table119, table120, table121, table122, table123, table124, table125, table126, table127, table128, table129, table130, table131, table132, table133, table134, table135, table136, table137, table138, table139, table140, table141, table142, table143, table144, table145, table146, table147, table148, table149, table150, table151, table152, table153, table154, table155, table156, table157, table158, table159, table160, table161, table162, table163, table164, table165, table166, table167, table168, table169, table170, table171, table172, table173, table174, table175, table176, table177, table178, table179, table180, table181, table182, table183, table184, table185, table186, table187, table188, table189, table190, table191, table192, table193, table194, table195, table196, table197, table198, table199, table200, table201, table202, table203, table204, table205, table206, table207, table208, table209, table210, table211, table212, table213, table214, table215, table216, table217, table218, table219, table220, table221, table222, table223, table224, table225, table226, table227, table228, table229, table230, table231, table232, table233, table234, table235, table236, table237, table238, table239, table240, table241, table242, table243, table244, table245, table246, table247, table248, table249, table250
db<>fiddle here

Convert ` LISTAGG` TO `XMLAGG`

How to convert LISTAGG with case statements to XMLAGG equivalent, so as to avoid the concatenation error.
#ECHO ${cols_2 ||32767||varchar2}$ --Declare variable
SELECT LISTAGG( 'MAX(CASE WHEN CATEGORY = '''||CATEGORY||''' THEN "'||"LEVEL"||'" END) AS "'||"LEVEL"||'_'||CATEGORY||'"' , ',' )
WITHIN GROUP( ORDER BY CATEGORY, "LEVEL" DESC )
INTO cols_2
FROM (
SELECT DISTINCT "LEVEL", CATEGORY
FROM temp
);
I tried this and I'm getting an error saying missing keyword
#ECHO ${cols_2 ||32767||varchar2}$ --Declare variable
select rtrim (
xmlagg (xmlelement (e, 'MAX(CASE WHEN CATEGORY = '''||CATEGORY||''' THEN "'||"LEVEL"||'" END) AS "'||LEVEL||'_'||CATEGORY||'"', ',') order by 1,2 desc).extract (
'//text()'),
', ')
INTO cols_2
FROM (
SELECT DISTINCT "LEVEL", CATEGORY
temp
);
I have tried this an declared the cols_2 as clob type :-
SELECT DBMS_XMLGEN.CONVERT (
RTRIM (
XMLAGG (XMLELEMENT (
e,
'MAX(CASE WHEN CATEGORY = '''
|| CATEGORY
|| ''' THEN "'
|| "LEVEL"
|| '" END) AS "'
|| "LEVEL"
|| '_'
|| CATEGORY
|| '"',
',')
ORDER BY 1, DESC).EXTRACT('//text()').getclobval(),','),1)
', '),
1)
INTO cols_2
FROM (SELECT DISTINCT "LEVEL", CATEGORY
FROM temp);
Yet my issue is not resolved ,Im getting an error while trying to execute it as a procedure like :-
Error in concatenation of `LISTAGG` function[Not a duplicate question]
You are getting the missing keyword error because you are most likely attempting to run the second query as a standalone query instead of in a PL/SQL block. When you are doing that, you have to remove your into cols_2 clause. That is your immediate issue that should resolve your error.
Also, based on your prior question, using the XML functions will escape your ' and " characters so you will want to make sure to unescape them back to their original characters so you can use them in your dynamic sql query like this:
SELECT DBMS_XMLGEN.CONVERT (
RTRIM (
XMLAGG (XMLELEMENT (
e,
'MAX(CASE WHEN CATEGORY = '''
|| CATEGORY
|| ''' THEN "'
|| "LEVEL"
|| '" END) AS "'
|| "LEVEL"
|| '_'
|| CATEGORY
|| '"',
',')
ORDER BY 1, 2 DESC).EXTRACT ('//text()'),
', '),
1)
--INTO cols_2
FROM (SELECT DISTINCT "LEVEL", CATEGORY
FROM temp);

Convert SQL query to pivot

I can't seem able to get the logic to convert the following query to a pivot SQL. My table has 20 columns with roles on them, I'd like to convert those columns into rows so, when exported to Excel, I can filter on a single column since the values can be the same on the 20 columns. So far what I've done is convert the 20 columns into a single one and then split that single one into rows:
select distinct TASKID,
regexp_substr(t.roles,'[^|]+', 1, lines.column_value) as role
from (
select TASKID,
TRIM(ROLE1) || '|' ||
TRIM(ROLE2) || '|' ||
TRIM(ROLE3) || '|' ||
TRIM(ROLE4) || '|' ||
TRIM(ROLE5) || '|' ||
TRIM(ROLE6) || '|' ||
TRIM(ROLE7) || '|' ||
TRIM(ROLE8) || '|' ||
TRIM(ROLE9) || '|' ||
TRIM(ROLE10) || '|' ||
TRIM(ROLE11) || '|' ||
TRIM(ROLE12) || '|' ||
TRIM(ROLE13) || '|' ||
TRIM(ROLE14) || '|' ||
TRIM(ROLE15) || '|' ||
TRIM(ROLE16) || '|' ||
TRIM(ROLE17) || '|' ||
TRIM(ROLE18) || '|' ||
TRIM(ROLE19) || '|' ||
TRIM(ROLE20) as roles
from menu_roles
where RLTYPE='58'
) t,
TABLE(CAST(MULTISET(select LEVEL from dual connect by instr(t.roles, '|', 1, LEVEL - 1) > 0) as sys.odciNumberList)) lines
where regexp_substr(t.roles,'[^|]+', 1, lines.column_value) is not null
order by regexp_substr(t.roles,'[^|]+', 1, lines.column_value)
I'd understand that using PIVOT would be more efficient vs concatenating and splitting a string.
Thank you!
You appear to want UNPIVOT:
SELECT task_id,
role
FROM menu_roles
UNPIVOT ( role FOR role_number IN ( ROLE1, ROLE2, ROLE3, ROLE4 /*, ... */ ) );
Or, using UNION ALL:
SELECT task_id, role1 AS role FROM menu_roles
UNION ALL SELECT task_id, role2 AS role FROM menu_roles
UNION ALL SELECT task_id, role3 AS role FROM menu_roles
UNION ALL SELECT task_id, role4 AS role FROM menu_roles
-- ...

SQL scripts to generate SQL scripts

Assume that the DBA_TAB_COLUMNS looks like this:
I'd like to write a SQL or PL/SQL script to generate following text:
select 'NULL' as A1, B1, QUERY, RECORD_KEY from SMHIST.probsummarym1
union all
select 'NULL' as A1, 'NULL' as B1, QUERY, RECORD_KEY from SMHIST_EIT200.probsummarym1
union all
select A1, 'NULL' as B1, QUERY, RECORD_KEY from SMHIST_EIT300.probsummarym1
the requirements are:
If the table under any of the SMHIST% schemas do not have that column, then insert a default NULL alias for that columns.
the column list is in alphabetical order.
so can anybody tell me how to write this script?
EDIT: Added better alias names and en explicit CROSS JOIN. Added XMLAGG version.
NB: LISTAGG exists from Oracle version 11.2 and onwards and returns VARCHAR2. If the output string is larger than 4000K or if on a prior version you can use XMLAGG which is a bit more cumbersome to work with (eg. http://psoug.org/definition/xmlagg.htm).
With LISTAGG (returning VARCHAR2):
SELECT LISTAGG (line,
CHR (13) || CHR (10) || 'union all' || CHR (13) || CHR (10))
WITHIN GROUP (ORDER BY sortorder)
script
FROM (SELECT line, ROWNUM sortorder
FROM ( SELECT 'select '
|| LISTAGG (
CASE
WHEN tc.column_name IS NULL
THEN
'''NULL'' as '
END
|| col_join.column_name,
', ')
WITHIN GROUP (ORDER BY col_join.column_name)
|| ' from '
|| col_join.owner
|| '.'
|| col_join.table_name
line
FROM dba_tab_columns tc
RIGHT OUTER JOIN
(SELECT DISTINCT
owner, table_name, col_list.column_name
FROM dba_tab_columns
CROSS JOIN
(SELECT DISTINCT column_name
FROM dba_tab_columns
WHERE owner LIKE 'SMHIST%') col_list
WHERE owner LIKE 'SMHIST%') col_join
ON tc.owner = col_join.owner
AND tc.table_name = col_join.table_name
AND tc.column_name = col_join.column_name
GROUP BY col_join.owner, col_join.table_name
ORDER BY col_join.owner, col_join.table_name))
With XMLAGG (returning CLOB by adding .getclobval (), note: RTRIM works here because table names cannot include ',' and ' ' (space)):
SELECT REPLACE (SUBSTR (script, 1, LENGTH (script) - 12),
'&' || 'apos;',
'''')
FROM (SELECT XMLAGG (
XMLELEMENT (
e,
line,
CHR (13)
|| CHR (10)
|| 'union all'
|| CHR (13)
|| CHR (10))).EXTRACT ('//text()').getclobval ()
script
FROM (SELECT line, ROWNUM sortorder
FROM ( SELECT 'select '
|| RTRIM (
REPLACE (
XMLAGG (XMLELEMENT (
e,
CASE
WHEN tc.column_name
IS NULL
THEN
'''NULL'' as '
END
|| col_join.column_name,
', ') ORDER BY
col_join.column_name).EXTRACT (
'//text()').getclobval (),
'&' || 'apos;',
''''),
', ')
|| ' from '
|| col_join.owner
|| '.'
|| col_join.table_name
line
FROM dba_tab_columns tc
RIGHT OUTER JOIN
(SELECT DISTINCT
owner,
table_name,
col_list.column_name
FROM dba_tab_columns
CROSS JOIN
(SELECT DISTINCT column_name
FROM dba_tab_columns
WHERE owner LIKE 'SMHIST%') col_list
WHERE owner LIKE 'SMHIST%') col_join
ON tc.owner = col_join.owner
AND tc.table_name = col_join.table_name
AND tc.column_name = col_join.column_name
GROUP BY col_join.owner, col_join.table_name
ORDER BY col_join.owner, col_join.table_name)))