Mule if flowVar used in Dataweave Transformer payload converted to Map - mule

I have an issue with a Mule Dataweave Transformer getting an error, this only occurs if :-
An enricher is present wrapping a json:json-to-object-transformer
<enricher target="#[flowVars.myVar]" doc:name="Message Enricher">
<json:json-to-object-transformer returnClass="java.util.HashMap" doc:name="JSON to Object"/>
</enricher>
A flowVar is involved in the Dataweave Transformer
The exception occurs on the 2nd dataweave :-
ERROR 2017-08-15 09:09:52,239 [amqpReceiver.02] org.mule.exception.RollbackMessagingExceptionStrategy:
********************************************************************************
Message : Exception while executing:
{bar=1, foo=0}
^
Unexpected character 'b' at index 1 (line 1, position 2), expected '"'
Payload : com.mulesoft.weave.reader.ByteArraySeekableStream#39e7a0d3
Element : /subscribe-to-changeFlow4/processors/4 # voa-009-mule-authorisation-search-api:test3.xml:58 (Transform Message)
Element XML : <dw:transform-message doc:name="Transform Message">
<dw:input-payload mimeType="application/json"></dw:input-payload>
<dw:set-payload>%dw 1.0%output application/json---{field1: payload.foo,field2: payload.bar,field3: flowVars.myFlowVar}</dw:set-payload>
</dw:transform-message>
--------------------------------------------------------------------------------
Root Exception stack trace:
com.mulesoft.weave.reader.json.JsonReaderException: Unexpected character 'b' at index 1 (line 1, position 2), expected '"'
at com.mulesoft.weave.reader.json.JsonTokenizer.fail(JsonTokenizer.scala:193)
at com.mulesoft.weave.reader.json.JsonTokenizer.require(JsonTokenizer.scala:190)
at com.mulesoft.weave.reader.json.JsonTokenizer.readString(JsonTokenizer.scala:74)
at com.mulesoft.weave.reader.json.JsonTokenizer.readObjectMembers(JsonTokenizer.scala:146)
at com.mulesoft.weave.reader.json.JsonTokenizer.com$mulesoft$weave$reader$json$JsonTokenizer$$readObject(JsonTokenizer.scala:140)
at com.mulesoft.weave.reader.json.JsonTokenizer$$anonfun$readValue$1.apply$mcV$sp(JsonTokenizer.scala:37)
What happens is as soon as a flowVar is added into the 2nd data weave, then the payload seems to show as a Map rather than JSON.
The enricher with nested json-to-object-transformer is required to take a payload and allow the JSON to be accessed. If the enricher is removed it works fine
The seek(0) is required to reset the stream after the json-to-object-transformer as Answered in my last question
if the flowVar is removed from the 2nd dataweave it works fine
Test Case
<flow name="subscribe-to-changeFlow4" processingStrategy="synchronous">
<amqp:inbound-endpoint numberOfChannels="1" ref="authorisationChangeQueueDef4" responseTimeout="10000" doc:name="authorisation-change-consumer" />
<dw:transform-message doc:name="Transform Message">
<dw:set-payload>
<![CDATA[%dw 1.0
%output application/json
---
{
foo:0,
bar:1
}]]>
</dw:set-payload>
</dw:transform-message>
<enricher target="#[flowVars.myVar]" doc:name="Message Enricher">
<json:json-to-object-transformer returnClass="java.util.HashMap" doc:name="JSON to Object"/>
</enricher>
<expression-component doc:name="Expression">
<![CDATA[payload.seek(0);]]>
</expression-component>
<set-variable variableName="myFlowVar" value="1234" doc:name="Variable" />
<dw:transform-message doc:name="Transform Message">
<dw:input-payload mimeType="application/json"/>
<dw:set-payload>
<![CDATA[%dw 1.0
%output application/json
---
{
field1: payload.foo,
field2: payload.bar,
field3: flowVars.myFlowVar
}
]]>
</dw:set-payload>
</dw:transform-message>
<logger level="INFO" message="#[message.payloadAs(java.lang.String)]" doc:name="Logger all "/>
</flow>

This is probably a bug in Mule and you can raise it in Mule jira
Which Mule runtime version are you using?
I tested locally and found this issue with Mule runtime 3.8.0 and Mule runtime 3.8.1:
but worked fine with Mule runtime 3.8.2 and Mule 3.8.5
Also tested with Mule 3.7.3 locally and it worked fine as below:-

I had similar problem, the DW processor messes up the payload. So I added an Object-to-String before DW transformer and that worked fine.

Related

Mule - Type mismatch for 'map' operator

Requirement is to bring CXF service in mule.
As per my knowledge i tried creating the configuration in mule but getting an error as shown below.
Exception while executing:
(payload.ns0#ValidateVIN.*ns0#VIN map ((vIN , indexOfVIN) -> {
^
Type mismatch for 'map' operator
found :null, :function
required :array, :function.
Attaching the code below.
<?xml version="1.0" encoding="UTF-8"?>
<mule ... >
<http:listener-config name="HTTP_Listener_Configuration" host="0.0.0.0" port="9009" basePath="Mule" doc:name="HTTP Listener Configuration"/>
<cxf:configuration name="Autoprefill_CXF_Configuration" enableMuleSoapHeaders="true" initializeStaticBusInstance="true" doc:name="CXF Configuration"/>
<flow name="autoprefillFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/VINService" allowedMethods="POST" doc:name="HTTP"/>
<logger message="#[message.payloadAs(String)]" level="INFO" doc:name="Logger"/>
<dw:transform-message doc:name="Transform Message" metadata:id="566ab616-f256-4763-af42-82d4cbbcc277">
<dw:input-payload mimeType="application/xml"/>
<dw:set-payload><![CDATA[%dw 1.0
%output application/xml
%namespace ns0 http://www.pwc.com/vin
---
{
ns0#ValidateVIN: {
(payload.ns0#ValidateVIN.*ns0#VIN map ((vIN , indexOfVIN) -> {
ns0#VIN: vIN
}))
}
}]]></dw:set-payload>
</dw:transform-message>
<cxf:simple-client operation="validateVIN" serviceClass="autoprefill.VINPort" doc:name="CXF"/>
<logger message="#[message.payloadAs(String)]" level="INFO" doc:name="Logger"/>
</flow>
</mule>
I tried removing the message transform property, but that time i got the below error
java.lang.String cannot be cast to java.util.Collection. Failed to route event via endpoint: org.mule.module.cxf.CxfOutboundMessageProcessor
The service is expecting a list of string. Im not sure how that can be done in mule. Any help will be much appreciated.
The problem is in the Transform Message component.
The map function is expecting an array of data (collection of data), but it did not receive any value.
I suggest to check the payload before Transform Message component. Perhaps payload is not in expected format. If you can share the payload (or full logs at least) here, I can help you with the transformation.

MULE example: Sending a CSV file through Email using SMTP

Hej,
I'm new in MULE, I try to learn "how to" following examples from Mule's sida: https://www.mulesoft.com/exchange#!/sending-csv-email-smtp?searchTerm=email
I think I did everything well, project is deploing good, but when file is taken exception is thrown and I'v got communicate:
Exception stack is:
1. Cannot coerce a :binary to a :array (com.mulesoft.weave.model.values.coercion.exception.UnsupportedTypeCoercionException)
com.mulesoft.weave.model.values.formatting.ArrayFormatTypeCoercionValue:31 (null)
2. Exception while executing:
payload map {
^
Cannot coerce a :binary to a :array (com.mulesoft.weave.mule.exception.WeaveExecutionException)
com.mulesoft.weave.mule.WeaveMessageProcessor:124 (null).
Abybody have idea what's happened. Anybody had same problem with that example?
Use the following mimeType in your DataWeave :- <dw:input-payload doc:sample="list_csv.csv" mimeType="application/csv"/>
So, the full code will be some thing like as follows:-
<smtp:gmail-connector doc:name="Gmail" name="Gmail" validateConnections="true"/>
<flow name="csv-to-smtpFlow">
<file:inbound-endpoint doc:name="Poll files" metadata:id="a75d5954-7fb4-4721-85f6-7ee66b793750" path="E:\test" responseTimeout="10000"/>
<dw:transform-message doc:name="Transform CSV to Map" metadata:id="84d175de-b3c6-4480-af9d-150e736f051e">
<dw:input-payload doc:sample="list_csv.csv" mimeType="application/csv"/>
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
payload map {
name: $.name,
orderId: $.orderId,
pricePerUnit: $.pricePerUnit as :number,
units: $.units as :number,
totalPrice: ($.units * $.pricePerUnit ) as :number
}]]></dw:set-payload>
</dw:transform-message>
<logger doc:name="Log the processed objects" level="INFO" message="Processing objects: #[payload]"/>
<smtp:outbound-endpoint connector-ref="Gmail" doc:name="Send an email via SMTP" from="${mail.from}" host="${smtp.host}" password="${smtp.password}" port="587" responseTimeout="10000" subject="${mail.subject}" to="${mail.to}" user="${smtp.user}"/>
</flow>
This will solve your issue

DataWeave Transform Message transforms 8192 bytes when deployed to Cloudhub

I am receiving a JSON request with 3 attribute and one of the attribute is content of image which is base64 in string format. After receiving this request I am creating a Salesforce attachment using Salesforce connector.
Now when I test this via my local Anypoint Studio everything is fine, however after deploying to Cloudhub the fileContent (Image content) is getting truncated to 8192 bytes and creating the attachment as corrupted.
I can see API is receiving the content since message content length is 38820 (both local and Cloudhub), and it looks like message is getting lost after the Message Transformation, and code is as follows:
<processor-chain doc:name="Processor Chain">
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
{
accountId : payload.accountId,
name : 'MyProfilePicture',
ContentType: payload.contentType,
fileContent : payload.fileContent
}]]></dw:set-payload>
</dw:transform-message>
<set-session-variable variableName="accountId" value="#[payload.accountId]" doc:name="Session Variable - accountId"/>
<set-session-variable variableName="contentType" value="#[payload.ContentType]" doc:name="Session Variable - contentType"/>
<logger message="payload.fileContent Before Conversion size #[payload.fileContent.length()], data:#[payload.fileContent] " level="INFO" doc:name="Logger"/>
<set-session-variable variableName="fileContent" value="#[(org.apache.commons.codec.binary.Base64.decodeBase64(payload.fileContent))]" mimeType="binary/octet-stream" doc:name="Session Variable - fileContent"/>
<dw:transform-message metadata:id="1676949a-75ba-4e51-82ac-b5788a8e87dc" doc:name="Transform Message">
<dw:input-payload/>
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
[{
Name: 'MyProfilePicture',
ParentId: sessionVars.accountId,
ContentType: sessionVars.contentType,
Body: sessionVars.fileContent
}]]]></dw:set-payload>
</dw:transform-message>
<logger message="Payload after Transform Message: #[payload]" level="INFO" doc:name="Logger"/>
<sfdc:create config-ref="Salesforce__Cfg" type="Attachment" doc:name="Salesforce">
<sfdc:objects ref="#[payload]"/>
</sfdc:create>
</processor-chain>
Could anyone please help on this? why when I deploy to Cloudhub it is only processing 8192 bytes?
Thanks,
Thomas
This can be fixed by switching to the deprecated DataMapper for the huge base64 transforms.

How to read plain text file in mule 3.8

In mule 3.8, when I use file connector to process text file , I am getting exception as below, please help.
Here is flow xml
file:connector name="File" autoDelete="true" streaming="true" validateConnections="true" doc:name="File"/>
flow name="DW-FixedWidth-Processing">
file:inbound-endpoint path="D:/mule/input" connector-ref="File" responseTimeout="10000" doc:name="File"/>
file:file-to-string-transformer doc:name="File to String"/>
dw:transform-message doc:name="Transform Message">
dw:input-payload />
dw:set-payload><![CDATA[%dw 1.0
%output application/csv header=false
---
((payload splitBy /\n/)[0..8]) map {
location:trim $[0..14],
sku:trim $[15..39],
dtc:trim $[40..42],
tt:trim $[43..44],
txnqty:trim $[45..54],
um:trim $[55..56],
rcd:trim $[57..59],
te:trim $[60..89],
ul:trim $[90..104],
date:trim $[105..114],
time:trim $[115..120]
} ]]>
dw:set-payload>
dw:transform-message>
logger message="#[message.payloadAs(java.lang.String)]" level="INFO" doc:name="Logger"/>
file:outbound-endpoint path="D:/mule/output" responseTimeout="10000" doc:name="File"/>
Exception
Root Exception stack trace:
java.lang.IllegalStateException: No schema set at
com.mulesoft.weave.module.flatfile.FlatFileSettings.loadSchema(FlatFileSettings.scala:45)
at
com.mulesoft.weave.module.flatfile.FlatFileReader.ffParser(FlatFileReader.scala:42)
When you disable streaming in the file connector, you no longer need the
<object-to-string/>
You can configure DataWeave to use a flat file schema, as already said in the other answer.
When you want to use it the way, you have to tell the correct mime type, in this case it is
application/java
You can do this on any message processor before the DataWeave, in this case the file:inbound-endpoint. Or you can do this in the DataWeave itself (unfortunately this is only visible when you switch Studio to XML).
For me everything worked with the following flow:
<file:connector name="File" autoDelete="false" streaming="false" validateConnections="true" doc:name="File"/>
<flow name="dw-testFlow">
<file:inbound-endpoint path="in" moveToDirectory="processed" connector-ref="File" responseTimeout="10000" mimeType="application/java" doc:name="File">
<file:filename-regex-filter pattern=".*\.txt" caseSensitive="true"/>
</file:inbound-endpoint>
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
((payload splitBy /\n/)) map {
location:trim $[0..14],
sku:trim $[15..39],
dtc:trim $[40..42],
tt:trim $[43..44],
txnqty:trim $[45..54],
um:trim $[55..56],
rcd:trim $[57..59],
te:trim $[60..89],
ul:trim $[90..104],
date:trim $[105..114],
time:trim $[115..120]
}
]]></dw:set-payload>
</dw:transform-message>
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
</flow>
The result of the DataWeave is a List of HashMap.
DW does not currently support mapping from an unstructured string, even if you declare MIME Type = text/plain upstream and you set the input payload metadata to String in the Transform Message processor.
The data formats supported (as per Mule documentation) are Java, XML, JSON, CSV, Flat File, Excel, Fixed Width, and Cobol copybook.
The case here is clearly a Fixed Width format.
#Prasad, generally the error No Schema Set at throws when the input mime type is no set. In your case, it seems like you are dealing with fixed width data hence you either define a fixed width schema. this article might help.
Instead of going to dataweave just use an groovy component to split payload by new line('\n').
Don't use file to string transformer it kills the Java-VM.
Also kindly brief what you need to achieve finally in your question.
Cheers!

How to retrieve statusCode from ApiKit mapping in Mule

We want to process all of the exceptions we catch with the same process in our exception strategy. It is the one generated by apikit. We first send a lot with a custom component and then we generate json response. In the response with DataWeave we want to set the same statusCode. However, the statusCode do not seem to be a variable that is retrievable. Am I correct, or there is a good way to retrieve it?
<apikit:mapping-exception-strategy xmlns:apikit="http://www.mulesoft.org/schema/mule/apikit" name="svc0031_hotel-apiKitGlobalExceptionMapping">
<apikit:mapping statusCode="504">
<apikit:exception value="org.mule.api.transformer.TransformerMessagingException"/>
<flow-ref name="svc0031_manageErrors" doc:name="Manage Error"/>
</apikit:mapping>
<apikit:mapping statusCode="404">
<apikit:exception value="org.mule.api.transformer.TransformerMessagingException"/>
<flow-ref name="svc0031_manageErrors" doc:name="Manage Error"/>
</apikit:mapping>
</apikit:mapping-exception-strategy>
<sub-flow name="svc0031_manageErrors">
<set-payload value="#[groovy:message.exceptionPayload.rootException.message]" doc:name="Set BIP Payload"/>
<custom-transformer class="se.zystems.baseline.BaselineLogging" doc:name="Log BIP Outgoing">
<spring:property name="Level" value="ERROR"/>
<spring:property name="ObjectId" value="CatchErrors"/>
<spring:property name="TransactionStatus" value="failed"/>
</custom-transformer>
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
{
status : 400,
message : payload,
code : 42,
more_info :"https://anypoint.mulesoft.com/apiplatform/nordic-choice-hotels"
}]]></dw:set-payload>
</dw:transform-message>
<logger level="INFO" doc:name="Logger"/>
</sub-flow>
Good,
If you create a variable with the http.status then you can use it in the mapper:
<flow name="test">
<set-variable variableName="httpStatus" value="#[message.inboundProperties['http.status']" doc:name="Variable"/>
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
{
status : flowVars.httpStatus,
message : payload,
code : 42,
more_info :"https://anypoint.mulesoft.com/apiplatform/nordic-choice-hotels"
}]]></dw:set-payload>
</dw:transform-message>
</flow>
I found out where the statusCode value is stored thanks to the answer by Jesús Pablo Fernández
He was almost right in his answer, however, the statusCode is stored not in message.inboundProperties['http.status'], but in message.outboundProperties['http.status']. No need to even extract a variable, one can just access it in data transformer directly like this:
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
{
status : outboundProperties['http.status'],
message : payload,
code : 42,
more_info :"https://anypoint.mulesoft.com/apiplatform/nordic-choice-hotels"
}]]></dw:set-payload>
</dw:transform-message>
Status codes defined in the APIKitExceptionStrategy will be those sent back in the response header as http.status given the corresponding exception to be raised.
For instance,
<apikit:mapping statusCode="504">
<apikit:exception value="org.mule.api.transformer.TransformerMessagingException"/>
<flow-ref name="svc0031_manageErrors" doc:name="Manage Error"/>
</apikit:mapping>
means that when a org.mule.api.transformer.TransformerMessagingException exception is thrown and caught by this exception block, then a 504 status code is sent back as a response header.
If you want to bypass the statusCode attribute in the APIKit exception block, you should have another exceptionStrategy block to catch your desired exception, and then you could set your status code like that
<set-property name="http.status" value="<the_desired_status, e.g. 401>" />
Hope it helps.
/Tony
Good Stanislav Ivanov,
I have a flow implemented with the APIKit, which is the code that I attach and whether there is property in the inboundProperties http.status, review it because if it should contain the value, if you do not have your flow may modify the value in a previous step.
I pointed him to directly access the outboundProperties in the mapper, I was not aware, thank you very much.