Aggregation in ForEach in Mule 4 - mule

I want to aggregate the responses of each iteration of For Each. In each iteration I am getting Json data and I want to aggregate that Json data into Json Array/List. Please let me know how I can do that by using data weave in Transform or by any other way?
Below is sample Json data that I am getting in each iteration of For Each.
1st iteration:
{
"accountId": "12345",
"accountNumber": "999",
"accountTitle": "ABC"
}
2nd iteration:
{
"accountId": "98765",
"accountNumber": "888",
"accountTitle": "XYZ"
}
I want final aggregated output as below.
{
accountList: [
{
"accountId": "12345",
"accountNumber": "999",
"accountTitle": "ABC"
},
{
"accountId": "98765",
"accountNumber": "888",
"accountTitle": "XYZ"
}
]
}

You can follow something like
<!-- Define blank arraylist for aggregation -->
<dw:transform-message doc:name="Transform Message">
<dw:set-variable variableName="result"><![CDATA[%dw 1.0
%output application/java
---
[]]]></dw:set-variable>
</dw:transform-message>
<foreach doc:name="For Each">
<!-- generate response using flow logic and add following step for updating result array -->
<dw:transform-message doc:name="Transform Message">
<dw:set-variable variableName="result"><![CDATA[%dw 1.0
%output application/java
---
flowVars.result ++ [payload]]]></dw:set-variable>
</dw:transform-message>
</foreach>
<!-- Transform aggregated result to any format. I used JSON -->
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
accountList : flowVars.result]]></dw:set-payload>
</dw:transform-message>
I have used Mule 3.x syntax. For mule 4 you can use attribute instead of variable for storing records. Dataweave syntax will remain same except for changing Dataweave language version from 1.0 to 2.0.
Hope this helps.

Related

Dataweave is not identifying headers in Excel file

I am using Mule 4 and Anypoint Studio 7.
I want to extract data from an Excel spreadsheet but the table I want to extract data from starts at line 4 in the worksheet and the output from Transform message is null which I assume is because it doesn't detect the column names as they are at line 4 and not line 1.
How can I fix this?
Dataweave XML
<ee:transform doc:name="Transform Message" doc:id="1bdda7fe-2abe-48d3-8bc5-42a94c12b6b9" >
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
input payload application/xlsx header=true
output application/json
---
{
"Customers": payload."CUSTOMERS" filter $$ > 2 map ( cUSTOMER , indexOfCUSTOMER ) -> {
"Type": cUSTOMER.type,
"Category": cUSTOMER.category
}
}]]></ee:set-payload>
</ee:message>
</ee:transform>
Dataweave code:
%dw 2.0
input payload application/xlsx header=true
output application/json
---
{
"Customers": payload."CUSTOMERS" filter $$ > 2 map ( cUSTOMER , indexOfCUSTOMER ) -> {
"Type": cUSTOMER.type,
"Category": cUSTOMER.category
}
}
Excel
Thanks for any help
You can set the property tableOffset to the column where the data starts for xlsx format.
For Mule 3:
In GUI:
Right click on the "Payload" label (left panel).
Click on the "Reader Configuration" option.
GUI opens with multiple options to be set.
Configure the "tableOffset" to the starting cell of the table. in your example A4.
Code example:
<dw:transform-message doc:name="Transform Message" metadata:id="9abf7128-71b8-4610-8fca-7ceda17f852e">
<dw:input-payload mimeType="application/xlsx">
<dw:reader-property name="tableOffset" value="A4"/>
</dw:input-payload>
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
payload]]></dw:set-payload>
</dw:transform-message>
For Mule 4, you need to set the reader properties on the event source:
https://docs.mulesoft.com/mule-runtime/4.1/dataweave-formats#reader_writer_properties
For example, if you are reading the xlsx from a file:
<file:listener doc:name="On New File" config-ref="File_Config" outputMimeType='application/xlsx tableOffset="A4"'>
<scheduling-strategy >
<fixed-frequency frequency="45" timeUnit="SECONDS"/>
</scheduling-strategy>
<file:matcher filenamePattern="myfile.xlsx" />
</file:listener>
Before your transform you could also try using a simple transform to add the new reader properties:
<set-payload value="#[output application/xlsx tableOffset='A4' --- payload]" />
<!-- Then your normal transformer -->
<ee:transform xsi:schemaLocation="http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd"
doc:id="bed303c7-1549-45da-af58-10c4ad937926">
<ee:message>
<ee:set-payload><![CDATA[%dw 2.0
output application/json --- payload]]></ee:set-payload>
</ee:message>
</ee:transform>

Mule Expression to get individual object from a Json Array

Want to extract each object from my json payload which looks like this: Need to pass each object from this map into splitter or collection splitter. I cannot use foreach scope here.
[
{
"Name": null,
"Key": "4",
"Header": {
"Id": "14"
}
},
{
"Name": null,
"Key": "5",
"Header": {
"Id": "15"
}
}
]
To get the entire object from the array we can use simple #[payload] and this will take care just like for each scope.
If you want to parse and access the JSON elements then
**<json:json-to-object-transformer/>** is the way to do the trick.
java.util.HashMap or java.util.List or java.util.Map
depends on the type of your JSON data.
You can also use java.lang.Object
There are multiple ways to do this :
1) As mentioned by Vijay, use json-to-object transformer to map input json to any Collection type e.g.
<json:json-to-object-transformer returnClass="java.util.List" doc:name="JSON to Object"/>
<collection-splitter doc:name="Collection Splitter"/>
2) Use dataweave to get a Collection from input json data :
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
payload
]]></dw:set-payload>
</dw:transform-message>
<collection-splitter doc:name="Collection Splitter"/>

How to convert HashMap to XML using dataweave in mule

I want to convert DB records into XML. Select query from DB records is giving LinkedHashMap as payload. Now I want to use dataweave to transform this into XML. I am pretty new to mule so a bit lost. What all steps do I need to follow to do this?
Thank you.
As informed by Victor please go through Dataweave Documentation which is the Data transformation component in Mule.
For you question the below methodology will help you to transform. The below script is an example that you can refer to be added to the data weave component in the mule code.
%dw 1.0
%output application/xml
---
payload map {
element: $.columnName_Or_aliasName
}
Map function will iterate over an array or complex xml or array in JSON.
Here for your DB response you can consider as every row in your DB resulset will be iterated
Transform Java Map to XML using DataWeave :
You can try below POC :
<sub-flow name="testdataweaveSub_Flow16_MapToXml" doc:description="http://blogs.mulesoft.com/dev/getting-started-with-dataweave-part-1/ Literal Expressions Variable Reference Expressions">
<set-payload value="{ "item_id": "B0002345W45", "item_type": "Item Type 1", "item_type_name": "Item Type 1 Name", "item_name": "item 1 name", "item_summary": "item 1 summary", "item_brand": "Brand 1", "image_type_name": "SmallImage", "url": "http://a/b/c" }" mimeType="application/json" doc:name="Set JSON Payload"/>
<json:json-to-object-transformer returnClass="java.util.Map" doc:name="JSON to Object"/>
<dw:transform-message metadata:id="3dad12b9-422d-4c59-9ec6-dd220ebca9ef" doc:name="Transform Message">
<dw:input-payload doc:sample="sample_data\flow16_HashMap_1.dwl" mimeType="application/java"/>
<dw:set-payload><![CDATA[%dw 1.0
%namespace mes http://www.namespace1.com/test/message/1.0
%namespace mod htto://www.namespace2.com/test/model/1.0
%output application/xml
---
mes#getItemResponse: {
mod#item : {
(payload)
}
}]]></dw:set-payload>
</dw:transform-message>
<byte-array-to-string-transformer doc:name="Byte Array to String"/>
<logger message="Test1 Transformer output : #[payload]" level="INFO" doc:name="Logger"/>
</sub-flow>
POC contain DataWeave script to transform Java Map to SOAP XML response
%dw 1.0
%namespace mes http://www.namespace1.com/test/message/1.0
%namespace mod htto://www.namespace2.com/test/model/1.0
%output application/xml
---
mes#getItemResponse: {
mod#item : {
(payload)
}
}
Hardcoded Input Java Map :
{
"item_id": "B0002345W45",
"item_type": "Item Type 1",
"item_type_name": "Item Type 1 Name",
"item_name": "item 1 name",
"item_summary": "item 1 summary",
"item_brand": "Brand 1",
"image_type_name": "SmallImage",
"url": "http://a/b/c"
}
Output :
<mes:getItemResponse xmlns:mes="http://www.namespace1.com/test/message/1.0">
<mod:item xmlns:mod="htto://www.namespace2.com/test/model/1.0">
<item_id>B0002345W45</item_id>
<item_type>Item Type 1</item_type>
<item_type_name>Item Type 1 Name</item_type_name>
<item_name>item 1 name</item_name>
<item_summary>item 1 summary</item_summary>
<item_brand>Brand 1</item_brand>
<image_type_name>SmallImage</image_type_name>
<url>http://a/b/c</url>
</mod:item>
</mes:getItemResponse>
You can try using
{`%dw 1.0
%output application/xml
{
xmlRootElement :
tagName1 : payload.DBEntryName1,
tagName2 : payload.DBEntryName2
}
} `

Dataweave equivalent of SQL join

How can I do the equivalent of a SQL join in Dataweave? I'm basically implementing a lookup. I have CSV data and JSON data in flow variables. I'm able to use both of them, but I can't figure out how connect them. Let's say the CSV has two columns:
Name,ExternalId
Foo,1
Bar,2
Baz,2
The JSON data is:
{
ExternalEntities: [
{ "Id": 1, "Name": "One", "Description": "Thing One" }
, { "Id": 2, "Name": "Two", "Description": "Thing Two" }
]
}
In the end, I'd like a List<Hashmap> with the following (expressed here in JSON format for convenience.)
[
{ "Name": "Foo", "ExternalName": "One", "ExternalDescription": "Thing One" }
, { "Name": "Bar", "ExternalName": "Two", "ExternalDescription": "Thing Two" }
, { "Name": "Baz", "ExternalName": "Two", "ExternalDescription": "Thing Two" }
]
So here is a sample using the json as the payload and the csv as a lookup table. The first flows parses the json and does a flow lookup passing the 'Id' as the payload. The second flow loads the csv as an array and does a search by the Id passed in. This returns a map of the CSV record found where you can then extract the 'Name' field.
It is possible to do this in one transformer, but for demo sake this was easier. You can also reverse the lookup with some tinkering if you want the csv as your payload and the json as your lookup.
<flow name="lookuptest" processingStrategy="synchronous">
<poll doc:name="Poll">
<logger level="INFO" doc:name="Logger" />
</poll>
<set-payload
value="{ "ExternalEntities": [ { "Id": 1, "Name": "One", "Description": "Thing One" } , { "Id": 2, "Name": "Two", "Description": "Thing Two" } ] }"
doc:name="Set Payload" mimeType="application/json"></set-payload>
<dw:transform-message doc:name="Transform Message">
<dw:input-payload doc:sample="string_2.dwl" />
<dw:set-payload><![CDATA[%dw 1.0
%input payload application/json
%output application/json
---
payload.ExternalEntities map ((value , index) -> {
Name: lookup("NameLookup", [value.Id as :string]).Name
})]]></dw:set-payload>
</dw:transform-message>
<object-to-string-transformer doc:name="Object to String"/>
<logger level="ERROR" message="#[payload]" doc:name="Logger"/>
</flow>
<flow name="NameLookup" processingStrategy="synchronous">
<set-variable value="#[payload[0]]" variableName="ExternalId"
doc:name="Variable" />
<set-payload
value="#[Thread.currentThread().getContextClassLoader().getResourceAsStream('lookuptables/namelookup.csv')]"
mimeType="application/csv" doc:name="Variable" />
<dw:transform-message doc:name="Transform Message">
<dw:input-payload doc:sample="string_2.dwl" />
<dw:set-payload><![CDATA[%dw 1.0
%input payload application/csv
%output application/java
---
(payload[?($.ExternalId == flowVars['ExternalId'])])[0]
]]></dw:set-payload>
</dw:transform-message>
</flow>
Here's a second solution that works, though I accepted #Ryan Carter's answer.
%dw 1.0
%output application/json
---
flowVars.myCsv map (
(row0, i) -> using (lookupElement = (flowVars.jsonLookup.ExternalEntities filter ((obj1) -> row0.ExternalId ~= obj1.Id))[0]) {
Name: row0.Name
, ExternalName: lookupElement.Name
, ExternalDescription: lookupElement.Description
} mapObject ({"$$": $, (StartsWithB:$[0] == "B") when $$ ~= "Name"})
)

Generating random numbers in mule dataweave

I have to concatenate randomly generated number with the field from request in dataweave.
NUMBR: "AA" ++ $.Load.Reference.*Reference ++ RandomNumber
How to achieve this in Mule Dataweave
Not sure what you can do in Datawevae to do this, but you can you set a Random number in a flowVariable and invoke it from your Dataweave script like so:
<set-variable variableName="random"
value="#[new java.util.Random().nextInt(100)]" doc:name="Variable" />
<dw:transform-message doc:name="Transform Message">
<dw:input-variable doc:sample="unknown.dwl" variableName="random" />
<dw:set-payload>
<![CDATA[%dw 1.0
%output application/dw
---
{
"data": ("22" as :number + flowVars.random)
} ]]>
</dw:set-payload>
</dw:transform-message>
You can also use Expression Component to assign it to Payload or variables and then concatenate
<flow name="random-numbersFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/random" allowedMethods="GET" doc:name="HTTP"/>
<expression-component doc:name="Expression"><![CDATA[payload = new java.util.Random().nextInt(100)]]></expression-component>
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
{
data: payload
}]]></dw:set-payload>
</dw:transform-message>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
</flow>
You can't do this in pure DataWeave but you could use two methods to generate the number elsewhere in the application:
You could call a global MEL function from DataWeave: https://docs.mulesoft.com/mule-user-guide/v/3.7/dataweave-reference-documentation#global-mel-functions
You could call a flow that returns the value: https://docs.mulesoft.com/mule-user-guide/v/3.7/dataweave-reference-documentation#expressions-that-call-external-flows
Just set a random value generated by Java into a Flow Variable
<set-variable variableName="Random_Variable" value="#[java.util.Random().nextInt(10)]" doc:name="Random Variable"/>
Then Use that Flow Variable in your Dataweave Transform.
<dw:transform-message doc:name="Transform Message" metadata:id="8098b24c-30c1-4e9e-a3ce-9e8aaaec7bd1">
<dw:input-variable mimeType="application/java" variableName="Random_Variable"/>
<dw:set-payload><![CDATA[%dw 2.0
%output application/json
---
{
NUMBR: "AA" ++ $.Load.Reference.*Reference ++ flowVars.Random_Variable
}]]></dw:set-payload>
</dw:transform-message>
In Mule 4 Dataweave 2 function random()
Returns a pseudo-random number greater than or equal to 0.0 and less than 1.0
MULE 4 DOC:
https://docs.mulesoft.com/dataweave/2.3/dw-core-functions-random
Example:
%dw 2.0
output application/json
{ price: random() * 1000 }