Spool space issue using XMLAGG in a single table - sql

Question

I need to aggregate all the NOTI_TEXT corresponding to NOTI_IDs. One NOTI_ID can have multiple NOTI_TEXT. I am using XMLAGG but it is running out of spool .
Below is the query:
select
NOTI_ID,
cast(XMLAGG(NOTI_TEXT order by NOTI_TEXT_LINE_ID) as varchar(32000)) as NOTI_TEXT,
NOTI_COUNTRY_ID,
NOTI_MAT_DIVISION_ID,
NOTI_MAT_DIVISION_TEXT,
NOTI_SOURCESYSTEM_ID,
CURRENT_DATE as TABLE_LOAD_DT
from
HC_PRD_D_RDDL_SDTB_0_1_0_0_0_0_0_0.SDTB_DM_SEV_111_NOTI_TXT_LINES_TEST_1
group by
NOTI_ID,
NOTI_COUNTRY_ID,
NOTI_MAT_DIVISION_ID,
NOTI_MAT_DIVISION_TEXT,
NOTI_SOURCESYSTEM_ID
All relevant stats have been collected. The skew factor of the source table is 1.5
Below is the EXPLAIN plan :
Explain select
NOTI_ID,
cast(XMLAGG(NOTI_TEXT order by NOTI_TEXT_LINE_ID) as varchar(32000)) as NOTI_TEXT,
NOTI_COUNTRY_ID,
NOTI_MAT_DIVISION_ID,
NOTI_MAT_DIVISION_TEXT,
NOTI_SOURCESYSTEM_ID,
CURRENT_DATE as TABLE_LOAD_DT
from
HC_PRD_D_RDDL_SDTB_0_1_0_0_0_0_0_0.SDTB_DM_SEV_111_NOTI_TXT_LINES_TEST_1
group by
NOTI_ID,
NOTI_COUNTRY_ID,
NOTI_MAT_DIVISION_ID,
NOTI_MAT_DIVISION_TEXT,
NOTI_SOURCESYSTEM_ID;
1) First, we lock
HC_PRD_D_RDDL_SDTB_0_1_0_0_0_0_0_0.SDTB_DM_SEV_111_NOTI_TXT_LINES_TEST
_1 for read on a reserved RowHash in all partitions to prevent
global deadlock.
2) Next, we lock
HC_PRD_D_RDDL_SDTB_0_1_0_0_0_0_0_0.SDTB_DM_SEV_111_NOTI_TXT_LINES_TEST
_1 for read.
3) We do an all-AMPs SUM step to aggregate from
HC_PRD_D_RDDL_SDTB_0_1_0_0_0_0_0_0.SDTB_DM_SEV_111_NOTI_TXT_LINES_TEST
_1 by way of an all-rows scan with no residual conditions, and the
grouping identifier in field 1. Aggregate Intermediate Results
are computed globally, then placed in Spool 3. The input table
will not be cached in memory, but it is eligible for synchronized
scanning. The aggregate spool file will not be cached in memory.
The size of Spool 3 is estimated with high confidence to be
13,749,188 rows (64,456,193,344 bytes). The estimated time for
this step is 15 hours and 20 minutes.
4) We do an all-AMPs RETRIEVE step from Spool 3 (Last Use) by way of
an all-rows scan into Spool 1 (group_amps), which is built locally
on the AMPs. The result spool file will not be cached in memory.
The size of Spool 1 is estimated with high confidence to be
13,749,188 rows (148,092,503,948 bytes). The estimated time for
this step is 4 minutes and 10 seconds.
5) Finally, we send out an END TRANSACTION step to all AMPs involved
in processing the request.
-> The contents of Spool 1 are sent back to the user as the result of
statement 1. The total estimated time is 15 hours and 24 minutes.
Having used this table for other queries we never found any anomaly. I want to check if it can be further optimized or any alternative to achieve the same .

Answer 1

It's a large base table and a large aggregation and your system is probably not that big (spools are not cached).
Try to aggregate seperatly, .i.e. only based on the key column(s) (probably NOTI_ID) and then join back, this will remove those additional Group By columns from spool (NOTI_MAT_DIVISION_TEXT might cause this issue if it's a large VarChar):
select
t1.NOTI_ID,
t1.NOTI_TEXT,
t2.NOTI_COUNTRY_ID,
t2.NOTI_MAT_DIVISION_ID,
t2.NOTI_MAT_DIVISION_TEXT,
t2.NOTI_SOURCESYSTEM_ID,
CURRENT_DATE as TABLE_LOAD_DT
from
( select
NOTI_ID,
cast(XMLAGG(NOTI_TEXT order by NOTI_TEXT_LINE_ID) as varchar(32000)) as NOTI_TEXT
from
HC_PRD_D_RDDL_SDTB_0_1_0_0_0_0_0_0.SDTB_DM_SEV_111_NOTI_TXT_LINES_TEST_1
group by
NOTI_ID,
) as t1
join
( select distinct
NOTI_ID,
NOTI_COUNTRY_ID,
NOTI_MAT_DIVISION_ID,
NOTI_MAT_DIVISION_TEXT,
NOTI_SOURCESYSTEM_ID
from
HC_PRD_D_RDDL_SDTB_0_1_0_0_0_0_0_0.SDTB_DM_SEV_111_NOTI_TXT_LINES_TEST_1
) as t2
on t1.NOTI_ID = t2.NOTI_ID