Dataweave is not identifying headers in Excel file - mule

I am using Mule 4 and Anypoint Studio 7.
I want to extract data from an Excel spreadsheet but the table I want to extract data from starts at line 4 in the worksheet and the output from Transform message is null which I assume is because it doesn't detect the column names as they are at line 4 and not line 1.
How can I fix this?
Dataweave XML
<ee:transform doc:name="Transform Message" doc:id="1bdda7fe-2abe-48d3-8bc5-42a94c12b6b9" >
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
input payload application/xlsx header=true
output application/json
---
{
"Customers": payload."CUSTOMERS" filter $$ > 2 map ( cUSTOMER , indexOfCUSTOMER ) -> {
"Type": cUSTOMER.type,
"Category": cUSTOMER.category
}
}]]></ee:set-payload>
</ee:message>
</ee:transform>
Dataweave code:
%dw 2.0
input payload application/xlsx header=true
output application/json
---
{
"Customers": payload."CUSTOMERS" filter $$ > 2 map ( cUSTOMER , indexOfCUSTOMER ) -> {
"Type": cUSTOMER.type,
"Category": cUSTOMER.category
}
}
Excel
Thanks for any help

You can set the property tableOffset to the column where the data starts for xlsx format.
For Mule 3:
In GUI:
Right click on the "Payload" label (left panel).
Click on the "Reader Configuration" option.
GUI opens with multiple options to be set.
Configure the "tableOffset" to the starting cell of the table. in your example A4.
Code example:
<dw:transform-message doc:name="Transform Message" metadata:id="9abf7128-71b8-4610-8fca-7ceda17f852e">
<dw:input-payload mimeType="application/xlsx">
<dw:reader-property name="tableOffset" value="A4"/>
</dw:input-payload>
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
payload]]></dw:set-payload>
</dw:transform-message>
For Mule 4, you need to set the reader properties on the event source:
https://docs.mulesoft.com/mule-runtime/4.1/dataweave-formats#reader_writer_properties
For example, if you are reading the xlsx from a file:
<file:listener doc:name="On New File" config-ref="File_Config" outputMimeType='application/xlsx tableOffset="A4"'>
<scheduling-strategy >
<fixed-frequency frequency="45" timeUnit="SECONDS"/>
</scheduling-strategy>
<file:matcher filenamePattern="myfile.xlsx" />
</file:listener>
Before your transform you could also try using a simple transform to add the new reader properties:
<set-payload value="#[output application/xlsx tableOffset='A4' --- payload]" />
<!-- Then your normal transformer -->
<ee:transform xsi:schemaLocation="http://www.mulesoft.org/schema/mule/ee/core http://www.mulesoft.org/schema/mule/ee/core/current/mule-ee.xsd"
doc:id="bed303c7-1549-45da-af58-10c4ad937926">
<ee:message>
<ee:set-payload><![CDATA[%dw 2.0
output application/json --- payload]]></ee:set-payload>
</ee:message>
</ee:transform>

Related

Cannot open a new cursor on a closed stream, what does it mean in Mule 4?

I am new to Mulesfot and Mule4 and I am still trying to understand how things works here. Check the following image:
On the first processor I am calling a Salesforce URL and getting an XML back which I am transforming to a JSON in the second processor, there I am storing/outputting the result to a variable SalesforceResponse and later on the third one I am calling a subflow which stores some values on the database. Until that point everything works as expected. The problem comes once I try to access the variable SalesforceResponse in the last processor which I need to get some values as the serverURL for example. Currently I am getting the following error:
Message : Cannot open a new cursor on a closed stream.
Element : SalesforceFlow/processors/3 # salesforcefulfillment:salesforcefulfillment.xml:340 (Transform Message)
Element XML : <ee:transform doc:name="Transform Message" doc:id="bea5898f-5aa8-4835-8e9d-fa1fd165d6d2">
<ee:message></ee:message>
<ee:variables>
<ee:set-variable variableName="SalesforceServerUrl">%dw 2.0
import * from dw::core::URL
var urlComposition = parseURI(vars.SalesforceResponse.serverUrl)
output text/plain
---
log("Houston, we have a problem.")</ee:set-variable>
</ee:variables>
</ee:transform>
Error type : MULE:UNKNOWN
What does error even means? I would expect to be able to access a "session" variable all over the places along the flow without having issues or I can't for some reason that I am not aware of?
Here is the XML definition for the image above:
<sub-flow name="SalesforceFlow" doc:id="237910c7-38c7-42bb-969b-63d6ff03bdaa">
<http:request method="POST" doc:name="Get SF SessionId" doc:id="37908152-700d-4c0e-9b52-b0de25f50fdf" config-ref="HTTP_Request_Salesforce" path="/services/Soap/u/44.0" sendCorrelationId="ALWAYS" outputMimeType="application/xml">
<http:body><![CDATA[#[%dw 2.0
ns soapenv http://schemas.xmlsoap.org/soap/envelope/
ns urn urn:partner.soap.sforce.com
output application/xml
---
soapenv#Envelope: {
soapenv#Body: {
urn#login: {
urn#username: p('SalesforceUsername'),
urn#password: p('SalesforcePassword')
}
}
}]]]>
</http:body>
</http:request>
<ee:transform doc:name="Transform Message" doc:id="b58ed3bd-7d74-4ad1-961f-8bab7a2ff128">
<ee:message>
</ee:message>
<ee:variables>
<ee:set-variable variableName="SalesforceResponse" ><![CDATA[ %dw 2.0
output application/json
ns soap http://schemas.xmlsoap.org/soap/envelope/
---
payload.soap#Envelope.soap#Body.loginResponse.result]]></ee:set-variable>
</ee:variables>
</ee:transform>
<flow-ref doc:name="Log Salesforce Response" doc:id="48326f18-a89b-49b2-ab96-ef2fe7214aaf" name="IntegrationFabricLogsFlow"/>
<ee:transform doc:name="Transform Message" doc:id="bea5898f-5aa8-4835-8e9d-fa1fd165d6d2" >
<ee:message >
</ee:message>
<ee:variables >
<ee:set-variable variableName="SalesforceServerUrl" ><![CDATA[%dw 2.0
import * from dw::core::URL
var urlComposition = parseURI(vars.SalesforceResponse.serverUrl)
output text/plain
---
log("Houston, we have a problem.")]]></ee:set-variable>
</ee:variables>
</ee:transform>
</sub-flow>
Can I get some help to understand how variable definition/access does work in Mule4?

Can I have a dynamic output media type in a basic DataWeave transform?

I have a reasonably straight forward RESTful API written in Mule 4.1.5 which can produce XML or JSON based on the Accept header of the GET request.
For happy days scenarios where the data is retrieved and transformed from source, the transformations are different enough to justify separate dataweave scripts for each media type. But for the standard error situations (HTTP 400, 404, 405, 406 etc) in the API Kit error handler it is mostly just a canned response being returned, which can be created for either media type by the same script.
Is it possible to have one script which has a dynamic output media type based on the Accept Header?
My current approach (which does work) is to have a choice based on the accept header (which has been saved to a variable). But this is a bit clunky, and if there is a smarter way of doing it that would be great.
<choice doc:name="Route by Accept Header">
<when expression="#[lower(vars.outputType) == 'application/xml']" >
<ee:transform doc:name="XML">
<ee:message>
<ee:set-payload><![CDATA[%dw 2.0
output application/xml
---
{message: "Bad request"}]]></ee:set-payload>
</ee:message>
<ee:variables>
<ee:set-variable variableName="httpStatus"><![CDATA[400]]></ee:set-variable>
</ee:variables>
</ee:transform>
</when>
<otherwise>
<ee:transform doc:name="JSON">
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
{message: "Bad request"}]]></ee:set-payload>
</ee:message>
<ee:variables >
<ee:set-variable variableName="httpStatus" ><![CDATA[400]]></ee:set-variable>
</ee:variables>
</ee:transform>
</otherwise>
</choice>
There are a few options to achieve this.
First you can use the dynamic-evaluate component. Here are the docs: https://docs.mulesoft.com/mule-runtime/4.1/dynamic-evaluate-component-reference
In this example they use it to decide the output media type. But it still requires multiple scripts loaded from a db. You can tailor it to load from files or do some concatenation etc. For example here is a ROUGH example of building up the script using manual concatenation:
<set-variable value="application/xml" variableName="outputType" />
<set-variable value="#['output ' ++ lower(vars.outputType) ++ ' --- ' ++ '{message: \'Bad request\'}']" variableName="script" />
<ee:dynamic-evaluate expression="#[vars.script]" />
Second you can have all your transforms output 1 common media-type. Say application/java.
And then at the end call a flow-ref that does just the media-type mapping:
<flow name="change-media-type">
<choice>
<when expression="#[lower(vars.outputType) == 'application/xml']">
<set-payload value="#[output application/xml --- payload]" />
</when>
<otherwise>
<set-payload value="#[output application/json --- payload]" />
</otherwise>
</choice>
</flow>
...
<ee:transform>
<ee:message>
<ee:set-payload><![CDATA[%dw 2.0
output application/java
---
{message: "Bad request"}]]></ee:set-payload>
</ee:message>
<ee:variables>
<ee:set-variable variableName="httpStatus"><![CDATA[400]] />
</ee:variables>
</ee:transform>
<flow-ref name="change-media-type" />
Also note, using dynamic media type mapping you will need to make sure your body is always compatible with XML and JSON. Not all JSON payloads are compatible due to root elements etc in XML.

How to convert data from object to string in mule 4?

I used to convert payload to string whenever i need to log the payload on console using the syntax in mule 3 [message.payloadAs(java.lang.String)] and some times used to use object to string converter or object to array converter.
But in mule 4 i don't see any converter as in mule 3 and [message.payloadAs(java.lang.String)] is alos not working.
Please assist me any new syntax of component or consept has come in mule 4 ?
Input
[
{
"firstName": "Ram",
"lastName": "Ram1",
"address": [
{
"street": "a",
"city": "b",
"state": "c"
},
{
"street": "a1",
"city": "b1",
"state": "c1"
}
]
}
Batch Image
Code
<batch:job jobName="Batch_ProcessingBatch_Job" doc:id="2582eccb-720f-499f-99fe-424cab3c9a33" >
<batch:process-records >
<batch:step name="Batch_Step1" doc:id="cb045f6b-edd3-426f-8f28-0d20b301fb6d" >
<logger level="INFO" doc:name="Logger" doc:id="2491184f-e8d7-47b7-bef0-29bafd43247f" message="In Batch_step1........... #[payload]"/>
<batch:aggregator doc:name="Batch Aggregator" doc:id="aa9c1b60-395f-4376-804d-bced200dfb43" size="2">
<set-payload value='#[write(payload, "application/json")]' doc:name="Set Payload" doc:id="9d4b27ee-5692-4a99-a93e-bd7c694f398d" />
<ee:transform doc:name="Transform Message" doc:id="06064422-1352-4ff9-8817-3e87eb1f4264" >
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
output application/json
---
payload map(item,index) -> {
name: item.firstName,
surname: item.lastName
}]]></ee:set-payload>
</ee:message>
</ee:transform>
<logger level="INFO" doc:name="Logger" doc:id="c955490e-fab1-436e-8265-f706a577c9d2" message="In Batch_step1 Aggrigator........... #[payload]" />
</batch:aggregator>
</batch:step>
</batch:process-records>
<batch:on-complete >
<logger level="INFO" doc:name="Logger" doc:id="a3923c90-557b-4955-9c40-afd748719abd" message="#['#####It is oncomplete result####': write(payload, "application/json")]"/>
</batch:on-complete>
</batch:job>
Getting Error
Message : "You called the function 'map' with these arguments:
1: String ("[\n \"{\\n \\\"firstName\\\": \\\"Ram\\\",\\n \\\"lastName\\\": \\\"Ram1\...)
2: Function ((item:Any, index:Any) -> ???)
But it expects arguments of these types:
1: Array
2: Function
4| payload map(item,index) -> {
| ...
7| }
Trace:
at map (line: 4, column: 1)
at main (line: 4, column: 9)" evaluating expression: "%dw 2.0
output application/json
---
payload map(item,index) -> {
name: item.firstName,
surname: item.lastName
}".
Error type : MULE:EXPRESSION
Element : Batch_ProcessingFlow/processors/0/route/0/route/0/aggregator/processors/1 # training-1:Batch Processing.xml:27 (Transform Message)
Element XML : <ee:transform doc:name="Transform Message" doc:id="06064422-1352-4ff9-8817-3e87eb1f4264">
<ee:message>
<ee:set-payload>%dw 2.0
output application/json
---
payload map(item,index) -> {
name: item.firstName,
surname: item.lastName
}</ee:set-payload>
</ee:message>
</ee:transform>
(set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
After quick googling the new way is to use #[message.payload as String] as shown in link.
You can use the payload.^raw syntax for the loggers. It was introduced in mule 4 and can handles cases where string conversion is needed. Typically pairs well with output text/plain format in a logger.
Ref: https://docs.mulesoft.com/mule-runtime/4.3/dataweave-cookbook-extract-data#metadata_selector
NOTE its never a good idea to log entire payloads (that could contain PII etc..) with maybe an exception where you are building a report that you need to log at the end of your batch process which possibly could be the case here...

Aggregation in ForEach in Mule 4

I want to aggregate the responses of each iteration of For Each. In each iteration I am getting Json data and I want to aggregate that Json data into Json Array/List. Please let me know how I can do that by using data weave in Transform or by any other way?
Below is sample Json data that I am getting in each iteration of For Each.
1st iteration:
{
"accountId": "12345",
"accountNumber": "999",
"accountTitle": "ABC"
}
2nd iteration:
{
"accountId": "98765",
"accountNumber": "888",
"accountTitle": "XYZ"
}
I want final aggregated output as below.
{
accountList: [
{
"accountId": "12345",
"accountNumber": "999",
"accountTitle": "ABC"
},
{
"accountId": "98765",
"accountNumber": "888",
"accountTitle": "XYZ"
}
]
}
You can follow something like
<!-- Define blank arraylist for aggregation -->
<dw:transform-message doc:name="Transform Message">
<dw:set-variable variableName="result"><![CDATA[%dw 1.0
%output application/java
---
[]]]></dw:set-variable>
</dw:transform-message>
<foreach doc:name="For Each">
<!-- generate response using flow logic and add following step for updating result array -->
<dw:transform-message doc:name="Transform Message">
<dw:set-variable variableName="result"><![CDATA[%dw 1.0
%output application/java
---
flowVars.result ++ [payload]]]></dw:set-variable>
</dw:transform-message>
</foreach>
<!-- Transform aggregated result to any format. I used JSON -->
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
accountList : flowVars.result]]></dw:set-payload>
</dw:transform-message>
I have used Mule 3.x syntax. For mule 4 you can use attribute instead of variable for storing records. Dataweave syntax will remain same except for changing Dataweave language version from 1.0 to 2.0.
Hope this helps.

How to convert HashMap to XML using dataweave in mule

I want to convert DB records into XML. Select query from DB records is giving LinkedHashMap as payload. Now I want to use dataweave to transform this into XML. I am pretty new to mule so a bit lost. What all steps do I need to follow to do this?
Thank you.
As informed by Victor please go through Dataweave Documentation which is the Data transformation component in Mule.
For you question the below methodology will help you to transform. The below script is an example that you can refer to be added to the data weave component in the mule code.
%dw 1.0
%output application/xml
---
payload map {
element: $.columnName_Or_aliasName
}
Map function will iterate over an array or complex xml or array in JSON.
Here for your DB response you can consider as every row in your DB resulset will be iterated
Transform Java Map to XML using DataWeave :
You can try below POC :
<sub-flow name="testdataweaveSub_Flow16_MapToXml" doc:description="http://blogs.mulesoft.com/dev/getting-started-with-dataweave-part-1/ Literal Expressions Variable Reference Expressions">
<set-payload value="{ "item_id": "B0002345W45", "item_type": "Item Type 1", "item_type_name": "Item Type 1 Name", "item_name": "item 1 name", "item_summary": "item 1 summary", "item_brand": "Brand 1", "image_type_name": "SmallImage", "url": "http://a/b/c" }" mimeType="application/json" doc:name="Set JSON Payload"/>
<json:json-to-object-transformer returnClass="java.util.Map" doc:name="JSON to Object"/>
<dw:transform-message metadata:id="3dad12b9-422d-4c59-9ec6-dd220ebca9ef" doc:name="Transform Message">
<dw:input-payload doc:sample="sample_data\flow16_HashMap_1.dwl" mimeType="application/java"/>
<dw:set-payload><![CDATA[%dw 1.0
%namespace mes http://www.namespace1.com/test/message/1.0
%namespace mod htto://www.namespace2.com/test/model/1.0
%output application/xml
---
mes#getItemResponse: {
mod#item : {
(payload)
}
}]]></dw:set-payload>
</dw:transform-message>
<byte-array-to-string-transformer doc:name="Byte Array to String"/>
<logger message="Test1 Transformer output : #[payload]" level="INFO" doc:name="Logger"/>
</sub-flow>
POC contain DataWeave script to transform Java Map to SOAP XML response
%dw 1.0
%namespace mes http://www.namespace1.com/test/message/1.0
%namespace mod htto://www.namespace2.com/test/model/1.0
%output application/xml
---
mes#getItemResponse: {
mod#item : {
(payload)
}
}
Hardcoded Input Java Map :
{
"item_id": "B0002345W45",
"item_type": "Item Type 1",
"item_type_name": "Item Type 1 Name",
"item_name": "item 1 name",
"item_summary": "item 1 summary",
"item_brand": "Brand 1",
"image_type_name": "SmallImage",
"url": "http://a/b/c"
}
Output :
<mes:getItemResponse xmlns:mes="http://www.namespace1.com/test/message/1.0">
<mod:item xmlns:mod="htto://www.namespace2.com/test/model/1.0">
<item_id>B0002345W45</item_id>
<item_type>Item Type 1</item_type>
<item_type_name>Item Type 1 Name</item_type_name>
<item_name>item 1 name</item_name>
<item_summary>item 1 summary</item_summary>
<item_brand>Brand 1</item_brand>
<image_type_name>SmallImage</image_type_name>
<url>http://a/b/c</url>
</mod:item>
</mes:getItemResponse>
You can try using
{`%dw 1.0
%output application/xml
{
xmlRootElement :
tagName1 : payload.DBEntryName1,
tagName2 : payload.DBEntryName2
}
} `