Aggregate data from for-each loop - mule

Question

Scenario - Converting a csv file to json format, taking each json element and making a get request api call. I am doing this in a for-each loop sequence. I am getting a json response (extracting eventId and cost from each). Now I wish to club all these responses together under the main header listings and make a bigger json payload.
For example:
{
"listings": [
{
"eventId":"8993478",
"cost":34
},
{
"eventId":"xxxxxyyyy",
"cost":zz
},
]
}
How would I do this for all iteration entries. I can do it for a single entry(using groovy script).

Answer 1

You could define a variable before the for-each loop as an empty list with:
<set-variable variableName="listings" value="#[[]]" />
Then, on each iteration inside the for-each loop add an element to the previous variable with:
<expression-transformer expression="#[flowVars.listings.add(flowVars.iterationMap)]" />
In the previous code fragment I used the variable flowVars.iterationMap to denote the map generated on each iteration.
Finally, if needed, you can add a set-payload transformer after the for-each loop:
<set-payload value="#[flowVars.listings]" />
HTH, Marcos

Answer 2

You can use the Batch module but you would have to rewrite this logic a little bit different. For example, you will no longer be able to use an aggregation flowVar like Marcos suggested. Instead, you would need to use a fixed size batch:commit block (which would actually be better in many ways, for example you could start sending bulks to the remote API while still processing some of the other records in the background).

Answer 3

...I like Marco's answer and it worked perfectly for my use case.
Simply creating an array in a flow variable and using the add() method on the array in a ForEach scope did the trick.
The OP follow up question was a good one. It prompted me to do an alternate test using the approach suggested. See both of my flows here:
<flow name="sampleAggregatorFlow" doc:description="this is a simple demo that shows how to aggregate results into an accumulator array">
<http:listener config-ref="manage-s3-api-httpListenerConfig" path="/aggregate" allowedMethods="GET" doc:name="HTTP"/>
<set-payload value="#[['red','blue','gold']]" doc:name="Set Payload"/>
<set-variable variableName="accumulator" value="#[[]]" doc:name="accumulator"/>
<foreach doc:name="For Each">
<expression-transformer expression="#[flowVars.accumulator.add(payload)]" doc:name="addEm"/>
</foreach>
<set-payload value="#[flowVars.accumulator]" doc:name="Set Payload"/>
<json:object-to-json-transformer doc:name="Object to JSON"/>
</flow>
<flow name="Copy_of_sampleAggregatorFlow" doc:description="this is a simple demo that shows how to aggregate results into an accumulator array">
<http:listener config-ref="manage-s3-api-httpListenerConfig" path="/aggregate2" allowedMethods="GET" doc:name="Copy_of_HTTP"/>
<set-payload value="#[['red','blue','gold']]" doc:name="Copy_of_Set Payload"/>
<set-variable variableName="accumulator" value="#[new java.util.ArrayList()]" doc:name="Copy_of_accumulator"/>
<foreach doc:name="Copy_of_For Each">
<expression-transformer expression="#[flowVars.accumulator.add(payload)]" doc:name="Copy_of_addEm"/>
</foreach>
<set-payload value="#[flowVars.accumulator]" doc:name="Copy_of_Set Payload"/>
<json:object-to-json-transformer doc:name="Copy_of_Object to JSON"/>
</flow>
Both flows produced the same outcome:
[
"red",
"blue",
"gold"
]
Tests conducted 12/26/2017 with Anypoint Studio 6.4.1 and wth Mule Runtime 3.9