I have a table of record tried to concatenate multiple rows on group wise and i use XMLAGG function but when i try to run the query for particular group which has 2000 records, getting error message:
Select failed 9134 : Intermediate aggregate storage limit for aggregation has been exceeded during computation
SELECT
H.GROUP_id,
H.Group_name,
TRIM(
TRAILING ',' FROM (
XMLAGG(TRIM(COALESCE(H.Group_desc, -1) || '') ORDER BY H.LINE_NBR) (VARCHAR(7000))
)
) AS Group_detail
even increased the varchar value but still having same issue
XMLAGG() adds overhead. However, you can get a sense for how large the result set is by using:
SELECT H.GROUP_id, H.Group_name,
SUM(LENGTH(COALESCE(H.Group_Desc, '-1'))) as total_string_length,
COUNT(*) as cnt
FROM . . .
GROUP BY H.GROUP_id, H.Group_name
ORDER BY total_string_length DESC
You will probably find that some of the groups have total string close to or more than 7000 characters.
I'm not sure if you want to fix the data or do something else. But this should at least identify the problem.
The problem is that the concatenation would be repeated for every row in the dataset, you need to get the distinct Group_desc first, try this:
WITH BASE AS(
SEL
H.GROUP_id,
H.Group_name,
H.Group_desc,
MAX(H.LINE_NBR) AS LINE_NBR
FROM TABLE_NAME
GROUP BY 1,2,3
)
SELECT
BASE.GROUP_id,
BASE.Group_name,
TRIM(
TRAILING ',' FROM (
XMLAGG(TRIM(COALESCE(BASE.Group_desc, -1) || '') ORDER BY BASE.LINE_NBR) (VARCHAR(7000)) -- You probably won't need the varchar to be that large.
)
) AS Group_detail
FROM BASE
Related
I am using LISTAGG() function in my select statement in a procedure such as
SELECT DISTINCT REPLACE(LISTAGG(a.student1||':'||a.score||'+')
WITHIN GROUP ( ORDER BY a.roll_no) OVER (PARTITION BY a.class)
If my select statement has not null values then i get
for eg: ABC:100
But if the select is empty then i get
for eg: :
I wanted it to be null if there are no rows selected.
The problem is that the expression you're aggregating returns a non-NULL value whether or not the values you're concatenating are NULL. My guess is you want something like
listagg(case when a.student1 is not null
then a.student1||':'||a.score||'+'
else null
end)
If you can have a null value for student1 but a non-NULL value for score, you'd need to adjust the case statement to specify what you want to happen in that case (do you want the ":" and the "+"? Just the "+"?)
Well, your REPLACE is incomplete. Also, is '+' really supposed to be concatenated? What string separates values, then?
REPLACE (
LISTAGG (a.student1 || ':' || a.score, '+') --> removed || for the + sign
WITHIN GROUP (ORDER BY a.roll_no)
OVER (PARTITION BY a.class),
':', '') --> missing arguments for REPLACE
If that's still not what you asked, please, provide sample test data and desired output.
I have the following data in a table:
GROUP1|FIELD
Z_12TXT|111
Z_2TXT|222
Z_31TBT|333
Z_4TXT|444
Z_52TNT|555
Z_6TNT|666
And I engineer in a field that removes the leading numbers after the '_'
GROUP1|GROUP_ALIAS|FIELD
Z_12TXT|Z_TXT|111
Z_2TXT|Z_TXT|222
Z_31TBT|Z_TBT|333 <- to be removed
Z_4TXT|Z_TXT|444
Z_52TNT|Z_TNT|555
Z_6TNT|Z_TNT|666
How can I easily query the original table for only GROUP's that correspond to GROUP_ALIASES with only one Distinct FIELD in it?
Desired result:
GROUP1|GROUP_ALIAS|FIELD
Z_12TXT|Z_TXT|111
Z_2TXT|Z_TXT|222
Z_4TXT|Z_TXT|444
Z_52TNT|Z_TNT|555
Z_6TNT|Z_TNT|666
This is how I get all the GROUP_ALIAS's I don't want:
SELECT GROUP_ALIAS
FROM
(SELECT
GROUP1,FIELD,
case when instr(GROUP1, '_') = 2
then
substr(GROUP1, 1, 2) ||
ltrim(substr(GROUP1, 3), '0123456789')
else
substr(GROUP1 , 1, 1) ||
ltrim(substr(GROUP1, 2), '0123456789')
end GROUP_ALIAS
FROM MY_TABLE
GROUP BY GROUP_ALIAS
HAVING COUNT(FIELD)=1
Probably I could make the engineered field a second time simply on the original table and check that it isn't in the result from the latter, but want to avoid so much nesting. I don't know how to partition or do anything more sophisticated on my case statement making this engineered field, though.
UPDATE
Thanks for all the great replies below. Something about the SQL used must differ from what I thought because I'm getting info like:
GROUP1|GROUP_ALIAS|FIELD
111,222|,111|111
111,222|,222|222
etc.
Not sure why since the solutions work on my unabstracted data in db-fiddle. If anyone can spot what db it's actually using that would help but I'll also check on my end.
Here is one way, using analytic count. If you are not familiar with the with clause, read up on it - it's a very neat way to make your code readable. The way I declare column names in the with clause works since Oracle 11.2; if your version is older than that, the code needs to be re-written just slightly.
I also computed the "engineered field" in a more compact way. Use whatever you need to.
I used sample_data for the table name; adapt as needed.
with
add_alias (group1, group_alias, field) as (
select group1,
substr(group1, 1, instr(group1, '_')) ||
ltrim(substr(group1, instr(group1, '_') + 1), '0123456789'),
field
from sample_data
)
, add_counts (group1, group_alias, field, ct) as (
select group1, group_alias, field, count(*) over (partition by group_alias)
from add_alias
)
select group1, group_alias, field
from add_counts
where ct > 1
;
With Oracle you can use REGEXP_REPLACE and analytic functions:
select Group1, group_alias, field
from (select group1, REGEXP_REPLACE(group1,'_\d+','_') group_alias, field,
count(*) over (PARTITION BY REGEXP_REPLACE(group1,'_\d+','_')) as count from test) a
where count > 1
db-fiddle
I am doing concatenation of values where there is grouping value greater than 1. This works fine, but now I am trying to figure out how to utilize an sequence/order number in which to concatenate the values in that order. Is there a way I can do this?
So for example, I have a table which has the following:
I need the ability to concatenate Field1 and Field4 since the StagingCol is the same name and I also need to be able to concatenate in the order provided in the ConcatenateOrder column. I can't have it out of sequence i.e. Field4 + Field1
This is a snippet of the code I have so far which is working fine, it concatenates the two LandingZoneCol values...
--DECLARATION OF LANDING ZONE FIELD NAMES CONCATENATED TOGETHER AND DELMITED BY A COMMA WHERE VALUE NEEDS TO BE CONCATENATED (I.E. SUBSCRIBER + SEQ# = MEMBER_ID)
SELECT #ConcatLandingZoneFieldNames = ISNULL(#ConcatLandingZoneFieldNames,'') + ISNULL(LandZoneMapping.LandingZoneFieldName,'') + ', '
FROM #LandingZoneMapping AS LandZoneMapping
WHERE LandZoneMapping.StagingColumnName <> 'BreadCrumb'
AND LandZoneMapping.StagingColumnName IN (SELECT StagingColumnName
FROM #TEST
WHERE Total > 1)
--DECLARATION OF VARIABLES
SET #ConcatLandingZoneFieldNames = CONCAT('CONCAT(',SUBSTRING(#ConcatLandingZoneFieldNames,1,LEN(#ConcatLandingZoneFieldNames)-1),')')
Current Results
CONCAT(Field1, Field4)
Expected Results
CONCAT(Field1, Field4)
Although both Current and Expected are the same right now, I want to ensure that the concatenation of the values are concatenated in the correct order. If I was to flip the ConcatenateOrder numbers in the above table then the outcome would be different. The "Current" Results would end up being CONCAT(Field1, Field4) but the "Expected Results" should be CONCAT(Field4, Field1)
Any help would be greatly appreciated.
Your code looks like SQL Server. You an use string_agg():
select string_agg(lzm.landingzonecol, ',') within group (order by lzm.concatenateorder)
from #LandingZoneMapping lzm
where lzm.stagingcol = 'ID';
You can control the ordering with the order by clause.
As mention by Gordon you can use string_agg (Doc) function if you are using SQL Server version 2017 or above.
In case if you need same functionality for SQL Server version below 2017, use this:
SELECT STUFF((
SELECT CONCAT (
','
,LandingZoneCol
)
FROM #LandingZoneMapping LZM
WHERE StagingCol = 'ID'
ORDER BY ConcatenateOrder
FOR XML PATH('')
), 1, 1, '') AS Result
I have a table of Matricies (JobMatricies) ID, Desc, DeptIDs
1 Admin (PM) 6,7,138,131,11,9,10,134,135,14,105,129
5 Sales Processing (PM) 92,16,153,17,91,32,26,93,99,18,89,90,155,19
6 Construction Processing (PM) 100,36,20,136,22,88,23,25,34,106,38,39,132,41,42,43,154,152,84
DeptIDs are a Comma Delimited list of departments that I want to use to count how many records are represented by the Matrix.
Normally I would do something like....
select Matrix_ID,
Matrix_Desc,
JobCount = (select count(sched_ID) from JobSchedule where dept_ID in (**92,16,153,17,91,32,26,93,99,18,89,90,155,19**))
from jobMatrices
How do I replace the hard coded delimited string with the ID's stored with each matrix, so that I can produce a list of matricies with their own unique count based on the comma delimited string that is stored with each matrix.
Thanks
I just answered a similar question where the poster wanted to sum the delimited list of numbers in a similar table layout. My solution used CTE's in Oracle to turn that list into a CTE table which would allow you to join against it. I believe that technique would be of use here if your RDBMS supports that. Please have a look: https://stackoverflow.com/a/38231838/2543416
select s.ID
, m."Desc"
from JobSchedule s
, jobMatrices m
where position(',' || s.ID || ',' in ',' || m.DeptIDs || ',')>0;
select m."Desc"
, count(s.ID) JobCount
from JobSchedule s
, jobMatrices m
where position(',' || s.ID || ',' in ',' || m.DeptIDs || ',')>0
group by m."Desc";
In MySQL, you can treat a comma-separated string as a SET, and there's a builtin function FIND_IN_SET() that helps:
select m.Matrix_ID,
m.Matrix_Desc,
(select count(sched_ID) from JobSchedule
where FIND_IN_SET(dept_ID, m.DeptIds)) AS JobCount
from jobMatrices AS m;
This will have terrible performance, however.
If you use some RDBMS other than MySQL, they may have a different solution that works similarly.
You should be specific in your question and tag your question appropriately. You only tagged your question sql, but this is a language used by many RDBMS vendors.
I wasn't sure how to search for the answer:
select orderid, REGEXP_REPLACE (
orderid,
'^0+(.)',
'\1'
) as new_order_id
from orders
where length('new_order_id') < 6
This returns nothing. But I know the data is there. If I do:
select orderid, REGEXP_REPLACE (
orderid,
'^0+(.)',
'\1'
) as new_order_id
from orders
order by order_id asc
I get order ids like 1, 2, 3...
So how can I get back the ones that are less than six? Does the where not operation on my returned regexp_replace data after the dataset is returned. Oracle if it matters.
Also, I believe my query is knocking out all leading zeros and replacing it with nothing. Not sure what the \1 means. Yes, I copied it. I thought it was putting nothing there, which is what I want. Just truncate leading zeros.
Thanks.
In your query,
where length('new_order_id') < 6
compares the length of the literal string 'new_order_id', not the value of the field new_order_id.
Try removing the quotes:
where length(new_order_id) < 6
Try this:
select * from
(select orderid
, regexp_replace(orderid,'^0+(.)','\1') new_order_id
from orders)
where length(new_order_id) < 6;
You can avoid using regexp:
select orderid
, ltrim(orderid,'0') new_order_id
from orders
where length(ltrim(orderid,'0'))<6
order by 1;
The length of the string 'new_order_id' is never less than 6. Probably you will have to do the length(regexp_replace(...)) < 6 instead if oracle doesn't support using the output column name without quotes (I have no idea).