I am receiving a JSON request with 3 attribute and one of the attribute is content of image which is base64 in string format. After receiving this request I am creating a Salesforce attachment using Salesforce connector.
Now when I test this via my local Anypoint Studio everything is fine, however after deploying to Cloudhub the fileContent (Image content) is getting truncated to 8192 bytes and creating the attachment as corrupted.
I can see API is receiving the content since message content length is 38820 (both local and Cloudhub), and it looks like message is getting lost after the Message Transformation, and code is as follows:
<processor-chain doc:name="Processor Chain">
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
{
accountId : payload.accountId,
name : 'MyProfilePicture',
ContentType: payload.contentType,
fileContent : payload.fileContent
}]]></dw:set-payload>
</dw:transform-message>
<set-session-variable variableName="accountId" value="#[payload.accountId]" doc:name="Session Variable - accountId"/>
<set-session-variable variableName="contentType" value="#[payload.ContentType]" doc:name="Session Variable - contentType"/>
<logger message="payload.fileContent Before Conversion size #[payload.fileContent.length()], data:#[payload.fileContent] " level="INFO" doc:name="Logger"/>
<set-session-variable variableName="fileContent" value="#[(org.apache.commons.codec.binary.Base64.decodeBase64(payload.fileContent))]" mimeType="binary/octet-stream" doc:name="Session Variable - fileContent"/>
<dw:transform-message metadata:id="1676949a-75ba-4e51-82ac-b5788a8e87dc" doc:name="Transform Message">
<dw:input-payload/>
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
[{
Name: 'MyProfilePicture',
ParentId: sessionVars.accountId,
ContentType: sessionVars.contentType,
Body: sessionVars.fileContent
}]]]></dw:set-payload>
</dw:transform-message>
<logger message="Payload after Transform Message: #[payload]" level="INFO" doc:name="Logger"/>
<sfdc:create config-ref="Salesforce__Cfg" type="Attachment" doc:name="Salesforce">
<sfdc:objects ref="#[payload]"/>
</sfdc:create>
</processor-chain>
Could anyone please help on this? why when I deploy to Cloudhub it is only processing 8192 bytes?
Thanks,
Thomas
This can be fixed by switching to the deprecated DataMapper for the huge base64 transforms.
Related
I have a csv file which looks like this :
I am iterating the CSV file using For each after converting it to Java ArrayList using Dataweave.
Now, I need to convert one of the elements which is Date from String to Oracle Timestamp. Please help with your thoughts. The source code looks like this :
<flow name="testFlow">
<file:inbound-endpoint path="src/main/resources/input"
moveToDirectory="src/main/resources/output" connector-ref="File"
responseTimeout="10000" doc:name="File"/>
<dw:transform-message metadata:id="4e33c7ff-c33b-4c78-be34-79a154aa16df"
doc:name="Transform Message">
<dw:input-payload doc:sample="sample_data\list_csv.csv"/>
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
payload]]></dw:set-payload>
</dw:transform-message>
<logger level="INFO" doc:name="Logger"/>
<foreach collection="#[payload]" doc:name="For Each">
<logger message="#[payload]" level="INFO" doc:name="Logger"/>
</foreach>
</flow>
Flow design looks like this :
Basically what you need to do is to convert the date field which is a string in csv to date format. To do it just provide the format of the incoming data field and it will work for the below input example(csv is comma separated) :-
foo,bar,Name,Date
foo1,baar1,Name1,15-01-2016 12:08
foo2,baar2,Name2,15-01-2016 12:08
you can try below dw :-
%dw 1.0
%output application/java
---
payload map {
foo : $.foo,
bar : $.bar,
Name : $.Name,
Date : $.Date as :date {format: "dd-MM-yyyy hh:mm" }
}
I have an issue with a Mule Dataweave Transformer getting an error, this only occurs if :-
An enricher is present wrapping a json:json-to-object-transformer
<enricher target="#[flowVars.myVar]" doc:name="Message Enricher">
<json:json-to-object-transformer returnClass="java.util.HashMap" doc:name="JSON to Object"/>
</enricher>
A flowVar is involved in the Dataweave Transformer
The exception occurs on the 2nd dataweave :-
ERROR 2017-08-15 09:09:52,239 [amqpReceiver.02] org.mule.exception.RollbackMessagingExceptionStrategy:
********************************************************************************
Message : Exception while executing:
{bar=1, foo=0}
^
Unexpected character 'b' at index 1 (line 1, position 2), expected '"'
Payload : com.mulesoft.weave.reader.ByteArraySeekableStream#39e7a0d3
Element : /subscribe-to-changeFlow4/processors/4 # voa-009-mule-authorisation-search-api:test3.xml:58 (Transform Message)
Element XML : <dw:transform-message doc:name="Transform Message">
<dw:input-payload mimeType="application/json"></dw:input-payload>
<dw:set-payload>%dw 1.0%output application/json---{field1: payload.foo,field2: payload.bar,field3: flowVars.myFlowVar}</dw:set-payload>
</dw:transform-message>
--------------------------------------------------------------------------------
Root Exception stack trace:
com.mulesoft.weave.reader.json.JsonReaderException: Unexpected character 'b' at index 1 (line 1, position 2), expected '"'
at com.mulesoft.weave.reader.json.JsonTokenizer.fail(JsonTokenizer.scala:193)
at com.mulesoft.weave.reader.json.JsonTokenizer.require(JsonTokenizer.scala:190)
at com.mulesoft.weave.reader.json.JsonTokenizer.readString(JsonTokenizer.scala:74)
at com.mulesoft.weave.reader.json.JsonTokenizer.readObjectMembers(JsonTokenizer.scala:146)
at com.mulesoft.weave.reader.json.JsonTokenizer.com$mulesoft$weave$reader$json$JsonTokenizer$$readObject(JsonTokenizer.scala:140)
at com.mulesoft.weave.reader.json.JsonTokenizer$$anonfun$readValue$1.apply$mcV$sp(JsonTokenizer.scala:37)
What happens is as soon as a flowVar is added into the 2nd data weave, then the payload seems to show as a Map rather than JSON.
The enricher with nested json-to-object-transformer is required to take a payload and allow the JSON to be accessed. If the enricher is removed it works fine
The seek(0) is required to reset the stream after the json-to-object-transformer as Answered in my last question
if the flowVar is removed from the 2nd dataweave it works fine
Test Case
<flow name="subscribe-to-changeFlow4" processingStrategy="synchronous">
<amqp:inbound-endpoint numberOfChannels="1" ref="authorisationChangeQueueDef4" responseTimeout="10000" doc:name="authorisation-change-consumer" />
<dw:transform-message doc:name="Transform Message">
<dw:set-payload>
<![CDATA[%dw 1.0
%output application/json
---
{
foo:0,
bar:1
}]]>
</dw:set-payload>
</dw:transform-message>
<enricher target="#[flowVars.myVar]" doc:name="Message Enricher">
<json:json-to-object-transformer returnClass="java.util.HashMap" doc:name="JSON to Object"/>
</enricher>
<expression-component doc:name="Expression">
<![CDATA[payload.seek(0);]]>
</expression-component>
<set-variable variableName="myFlowVar" value="1234" doc:name="Variable" />
<dw:transform-message doc:name="Transform Message">
<dw:input-payload mimeType="application/json"/>
<dw:set-payload>
<![CDATA[%dw 1.0
%output application/json
---
{
field1: payload.foo,
field2: payload.bar,
field3: flowVars.myFlowVar
}
]]>
</dw:set-payload>
</dw:transform-message>
<logger level="INFO" message="#[message.payloadAs(java.lang.String)]" doc:name="Logger all "/>
</flow>
This is probably a bug in Mule and you can raise it in Mule jira
Which Mule runtime version are you using?
I tested locally and found this issue with Mule runtime 3.8.0 and Mule runtime 3.8.1:
but worked fine with Mule runtime 3.8.2 and Mule 3.8.5
Also tested with Mule 3.7.3 locally and it worked fine as below:-
I had similar problem, the DW processor messes up the payload. So I added an Object-to-String before DW transformer and that worked fine.
Hej,
I'm new in MULE, I try to learn "how to" following examples from Mule's sida: https://www.mulesoft.com/exchange#!/sending-csv-email-smtp?searchTerm=email
I think I did everything well, project is deploing good, but when file is taken exception is thrown and I'v got communicate:
Exception stack is:
1. Cannot coerce a :binary to a :array (com.mulesoft.weave.model.values.coercion.exception.UnsupportedTypeCoercionException)
com.mulesoft.weave.model.values.formatting.ArrayFormatTypeCoercionValue:31 (null)
2. Exception while executing:
payload map {
^
Cannot coerce a :binary to a :array (com.mulesoft.weave.mule.exception.WeaveExecutionException)
com.mulesoft.weave.mule.WeaveMessageProcessor:124 (null).
Abybody have idea what's happened. Anybody had same problem with that example?
Use the following mimeType in your DataWeave :- <dw:input-payload doc:sample="list_csv.csv" mimeType="application/csv"/>
So, the full code will be some thing like as follows:-
<smtp:gmail-connector doc:name="Gmail" name="Gmail" validateConnections="true"/>
<flow name="csv-to-smtpFlow">
<file:inbound-endpoint doc:name="Poll files" metadata:id="a75d5954-7fb4-4721-85f6-7ee66b793750" path="E:\test" responseTimeout="10000"/>
<dw:transform-message doc:name="Transform CSV to Map" metadata:id="84d175de-b3c6-4480-af9d-150e736f051e">
<dw:input-payload doc:sample="list_csv.csv" mimeType="application/csv"/>
<dw:set-payload><![CDATA[%dw 1.0
%output application/java
---
payload map {
name: $.name,
orderId: $.orderId,
pricePerUnit: $.pricePerUnit as :number,
units: $.units as :number,
totalPrice: ($.units * $.pricePerUnit ) as :number
}]]></dw:set-payload>
</dw:transform-message>
<logger doc:name="Log the processed objects" level="INFO" message="Processing objects: #[payload]"/>
<smtp:outbound-endpoint connector-ref="Gmail" doc:name="Send an email via SMTP" from="${mail.from}" host="${smtp.host}" password="${smtp.password}" port="587" responseTimeout="10000" subject="${mail.subject}" to="${mail.to}" user="${smtp.user}"/>
</flow>
This will solve your issue
We want to process all of the exceptions we catch with the same process in our exception strategy. It is the one generated by apikit. We first send a lot with a custom component and then we generate json response. In the response with DataWeave we want to set the same statusCode. However, the statusCode do not seem to be a variable that is retrievable. Am I correct, or there is a good way to retrieve it?
<apikit:mapping-exception-strategy xmlns:apikit="http://www.mulesoft.org/schema/mule/apikit" name="svc0031_hotel-apiKitGlobalExceptionMapping">
<apikit:mapping statusCode="504">
<apikit:exception value="org.mule.api.transformer.TransformerMessagingException"/>
<flow-ref name="svc0031_manageErrors" doc:name="Manage Error"/>
</apikit:mapping>
<apikit:mapping statusCode="404">
<apikit:exception value="org.mule.api.transformer.TransformerMessagingException"/>
<flow-ref name="svc0031_manageErrors" doc:name="Manage Error"/>
</apikit:mapping>
</apikit:mapping-exception-strategy>
<sub-flow name="svc0031_manageErrors">
<set-payload value="#[groovy:message.exceptionPayload.rootException.message]" doc:name="Set BIP Payload"/>
<custom-transformer class="se.zystems.baseline.BaselineLogging" doc:name="Log BIP Outgoing">
<spring:property name="Level" value="ERROR"/>
<spring:property name="ObjectId" value="CatchErrors"/>
<spring:property name="TransactionStatus" value="failed"/>
</custom-transformer>
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
{
status : 400,
message : payload,
code : 42,
more_info :"https://anypoint.mulesoft.com/apiplatform/nordic-choice-hotels"
}]]></dw:set-payload>
</dw:transform-message>
<logger level="INFO" doc:name="Logger"/>
</sub-flow>
Good,
If you create a variable with the http.status then you can use it in the mapper:
<flow name="test">
<set-variable variableName="httpStatus" value="#[message.inboundProperties['http.status']" doc:name="Variable"/>
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
{
status : flowVars.httpStatus,
message : payload,
code : 42,
more_info :"https://anypoint.mulesoft.com/apiplatform/nordic-choice-hotels"
}]]></dw:set-payload>
</dw:transform-message>
</flow>
I found out where the statusCode value is stored thanks to the answer by Jesús Pablo Fernández
He was almost right in his answer, however, the statusCode is stored not in message.inboundProperties['http.status'], but in message.outboundProperties['http.status']. No need to even extract a variable, one can just access it in data transformer directly like this:
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
{
status : outboundProperties['http.status'],
message : payload,
code : 42,
more_info :"https://anypoint.mulesoft.com/apiplatform/nordic-choice-hotels"
}]]></dw:set-payload>
</dw:transform-message>
Status codes defined in the APIKitExceptionStrategy will be those sent back in the response header as http.status given the corresponding exception to be raised.
For instance,
<apikit:mapping statusCode="504">
<apikit:exception value="org.mule.api.transformer.TransformerMessagingException"/>
<flow-ref name="svc0031_manageErrors" doc:name="Manage Error"/>
</apikit:mapping>
means that when a org.mule.api.transformer.TransformerMessagingException exception is thrown and caught by this exception block, then a 504 status code is sent back as a response header.
If you want to bypass the statusCode attribute in the APIKit exception block, you should have another exceptionStrategy block to catch your desired exception, and then you could set your status code like that
<set-property name="http.status" value="<the_desired_status, e.g. 401>" />
Hope it helps.
/Tony
Good Stanislav Ivanov,
I have a flow implemented with the APIKit, which is the code that I attach and whether there is property in the inboundProperties http.status, review it because if it should contain the value, if you do not have your flow may modify the value in a previous step.
I pointed him to directly access the outboundProperties in the mapper, I was not aware, thank you very much.
I am trying to pass the http header value as the web service consumer,
httpHeader
I use this property as web service input,
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/xml
%namespace ns0 namespace
---
{
ns0#GetProject: {
ns0#projectId: inboundProperties."http.query.params".projectId,
ns0#upi: inboundProperties.sm_user
}
}]]></dw:set-payload>
</dw:transform-message>
I have also tried setting the sm_user to a variable and trying to access the variable in TransformMessage as below, but the same error,
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/xml
%namespace ns0 namespace
---
{
ns0#GetProject: {
ns0#projectId: inboundProperties."http.query.params".projectId,
ns0#upi: flowVars.setUPI
}
}]]></dw:set-payload>
</dw:transform-message>
Error:
Server was unable to read request. ---> There is an error in XML document (3, 32). ---> Input string was not in a correct format.. Message payload is of type: ElementNSImpl
Transform XML using the SM_USER header:
<?xml version='1.0' encoding='windows-1252'?>
<ns0:GetProject xmlns:ns0="namespace">
<ns0:projectId xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>
<ns0:upi xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:nil="true"/>
</ns0:GetProject>
Flow XML:
<flow name="ProjectEC">
<dw:transform-message doc:name="Transform Message">
<dw:set-payload><![CDATA[%dw 1.0
%output application/xml
%namespace ns0 namespace
---
{
ns0#GetProject: {
ns0#projectId: inboundProperties."http.query.params".projectId ,
ns0#upi: inboundProperties.SM_USER
}
}]]></dw:set-payload>
</dw:transform-message>
<ws:consumer config-ref="ECDataService_Consumer" operation="GetProject" doc:name="ECWebServiceConsumer"/>
<mulexml:dom-to-xml-transformer doc:name="DOM to XML"/>
<json:xml-to-json-transformer doc:name="XML to JSON"/>
<set-variable variableName="extractJsondata" value="#[json:GetProjectResponse/GetProjectResult]" mimeType="application/json" doc:name="Variable"/>
<set-payload value="#[flowVars.extractJsondata]" doc:name="Set Payload"/>
<json:xml-to-json-transformer doc:name="XML to JSON"/>
</flow>
Can someone tell me if I am missing something.
Thanks
Few checks
Check whether your xml is the exact format which your webservice is really looking for. or
Change the encoding from encoding='windows-1252' to UTF-8. Follow the url Output Encoding issue in mule esb. or
As trial and error version directly use the xml in postman to hit webservice ( without making dataweave to do the transformation). or
Make sure you have no space <?xml version='1.0' encoding="UTF-8"?> before starting of xml greater sysmbol tag.
Check the above 4 steps anyone of this might cause issue. Step 3 will be helpful for testing.