Mule 4.4 convert Binary payload into JSON - mule

So I have gone through a number of questions related to this topic and tried the various options but none of them seem to work .
Environment - Mule runtime 4.4 , Anypoint Studio 7.11.1
Our limitation is we are not using Enterprise edition only community edition
So cannot use Transform component . However can use dataweave in setPayload / set Variable
I am receiving payload as Binary : see below :
Data appears as JSON but is Binary
I tried changing to JSON in 'setPayload' :
%dw 2.0
import * from dw::core::Binaries
output application/json
---
fromBase64(payload)
However I get this exception :
""Unexpected character 'ÿ' at payload#[1:30] (line:column), expected
Expecting end of input but got `ÿ, while reading `payload` as Json.
1| {"Employee":{"EName":"abcd"}}
^" evaluating expression: "%dw 2.0 import * from dw::core::Binaries output application/json
--- fromBase64(payload)"."
Note - also tried to check datatype of payload using :
typeOf(payload)
This too gave the same exception as above ...
Please help
Update 1:
Based on comments from #aled few additional details :
I am making a GET request to my api endpoint which is reading data from an ERP system which returns data in XML
Then I use a custom component that is being used to transform the XML data to JSON
The o/p of this custom component is what is generating binary data which I am unable to parse
I agree regarding Base64 that was useless
I later even tried with Transform component and all of the below attempts failed with the same exception :
Attempt#1:
%dw 2.0
output application/json
---
if (isEmpty(payload.^raw)) {
"payload": "No payload!!"
}
else {
"payload": payload
}
Attempt#2:
%dw 2.0
output application/json
---
payload
The above XML payload is a short excerpt received from ERP
I need to parse the payload and identify if there is an error present in the payload .
NOTE : if I simply log the payload it all shows up fine in the logs.
is it not possible to convert binary data to json ? do you think there is something wrong in the data received either from ERP or in the XML to JSON Transformer ?
Update#2
So based on earlier comments from #aled ( thanks ! ) started focusing on payload o/p of the ERP component ( which is XML ) BEFORE it is fed to our custom transformer ( which is generating binary data )
Attaching screen print for clarity and will also paste non proprietary code :
Now when I inspect the payload I can see it as :
<?xml version = "1.0" encoding="utf-8" ?>
<Output>
<ErrorDetails>
<Status>1</Status>
<Details>Invalid user or organisation</Details>
</ErrorDetails>
<Employee>
<EName>abcd</EName>
</Employee>
</Output>
and in debug mode I can see that payload is String and NOT Binary
Now I can access individual elements of the payload .
Example:
<set-variable value="#[output application/java --- payload[0].ErrorDetails['Status']]"
doc:name="extract status" variableName="status"/>
So this gives me the value 1 which is expected and good .
Conclusions:
1 The component that interacts with ERP is generating Non Binary / stream data
2 This data as expected is parseable - all good here
3 AFTER this data is fed to our custom component - the o/p payload is now binary and am running into issues parsing it ...
Question:
I know this being a custom component i cannot share code details here and thereby cannot expect inputs BUT regarding #aled comments about hexdump do you think I should try and use it against the output of the custom component ?
Once again a big thanks for your patience and help

Using function fromBase64() is just wrong. The value is not a Base64 string. It doesn't makes sense to try it. Using toBase64() would be useless because you would not be able to do anything with it, other than sending it somewhere else.
Your payload seems to be a JSON, and the media type is application/json in your screenshot, but it contains an UNICODE character at the end at least. You need. What are you trying to do with that payload exactly?
If the payload is truly a binary there is nothing you can do about that in the application. If you are receiving it from somewhere else check with the source. You didn't provide details of the source. Also ensure it is not your application corrupting the payload somehow.

Related

Multipart/form-data broken after apiKit router Mule4

having issues with reading data from multipart/form-data after the request goes through ApiKit router in Mule4 app.
There is a RAML in place, used in ApiKit router to validate & route the requests.
#%RAML 1.0
title: ACC race data API
description: API for designing and updating race data
version: 1
protocols: [ HTTPS, HTTP ]
/acc/reverse/entrylist:
description: Used to manipulate entrylist for ACC reverse grid.
post:
description: Creates new entrylist based on supplied quali result, race one result and race one entrylist.
queryParameters:
positionsRotated:
description: Parameter defining how many positions to rotate.
required: false
type: number
example: 30
body:
multipart/form-data:
properties:
qualifyResult:
description: File containing qualifying result.
type: file
fileTypes: ['application/json']
required: true
raceOneResult:
description: File containing race 1 result.
type: file
fileTypes: ['application/json']
required: true
The validation works fine - if the data are sent in incorrect format, the exception is returned. For correctly formated data request is routed to the flow post:\acc\reverse\entrylist:multipart\form-data:acc-race-data-config.
Between the steps, payload format is changed (by Mule) from payload in readable form (see below) to java.io.ByteArrayInputStream#379ebdd5
Readable format of payload data (received to the app) before ApiKit router:
----------------------------180928595588258919887097
Content-Disposition: form-data; name="qualifyResult"; filename="json1.json"
Content-Type: application/json
{
"json1": "1"
}
----------------------------180928595588258919887097
Content-Disposition: form-data; name="raceOneResult"; filename="json2.json"
Content-Type: application/json
{
"json2": "2"
}
----------------------------180928595588258919887097--
The following dataweave script works fine if used before apiKit, but it doesn't work in the flow called by ApiKit:
%dw 2.0
output application/json
---
payload.parts[1].content
The example of output if above DW is used before ApiKit:
{
"json2": "2"
}
The example of output if the same DW is used after ApiKit:
org.mule.runtime.core.api.expression.ExpressionRuntimeException: "javax.mail.internet.ParseException - Missing start boundary, while reading `payload` as MultiPart.
Trace:
at main (Unknown)" evaluating expression: "%dw 2.0
output application/json
---
payload.parts[1].content".
Testing in Anypoint Studio 7.8.0, supposed to be used on Mule4-CE runtime once finished and ready to deploy.
Using Postman v8.5.1 for testing. sending form-data body with qualifyResult and raceOneResult parts containing JSON data, default headers, basic auth, query param positionsRotated=30.
Url called: https://localhost:443/api/acc/reverse/entrylist?positionsRotated=30
Tried to generate multipart/form-data body manually using a RAW type of payload, but the results were the same. Everything works fine if there is no ApiKit.. but I would like to use it to validate request validity.
Thanks to everyone replying for any useful hints!
I tested a similar scenario I had with the latest versions and I didn't had that problem. If you are using older versions of the HTTP connector and APIKit module try upgrading to the latest releases. Anypoint Studio has a feature to detect newer releases of connectors: https://docs.mulesoft.com/studio/7.9/update-modules

How to accept any multipart file (image/pdf/csv) from postman and create the same file into SFTP location in Mule 4?

When I tried this usecase for some MIME types the content is coming in payload.parts[0].content, for some MIME types the content is coming in payload.parts.file.content. How to create the file in the same MIME type as how we received from postman in SFTP?
Quite not sure in what ways you have used and tested the both expression. Please find the screenshot below. I have added different format of datas in single request. If you use mutiple format in single request and wanting to extract individually. Use payload.parts[0].content and payload.parts[1].content
Note: This expression ideally mean whatever you add (form-data) in postman body request can be extract using index.
Example: if you have added content-type: application/xml in the below picture 3rd row, should be using payload.parts[2].content
If you're using single request payload.parts.file.content can be used.
Once after the request reached HTTP/HTTPS ( source listener), use dataweave as below to convert binary in to the defined structure data
%dw 2.0
output multipart/form-data
---
payload
After extracting the contents using the expressions, you can write in to SFTP (Screenshot) here I have stored in Vars (based on your logic)

How to retrieve data using a db connector in ESB

I am trying to retrieve names of employees from the database by calling a stored procedure and logging it.
But Instead of the names , this is the output of the log:
INFO 2019-12-11 11:48:39,994 [[MuleRuntime].cpuLight.10: [testrun].testrunFlow.CPU_LITE #3bc66a50] [event: 0-12970c00-1bde-11ea-92a5-38dead78b440] org.mule.runtime.core.internal.processor.LoggerMessageProcessor:
org.mule.runtime.core.internal.message.DefaultMessageBuilder$MessageImplementation
{
payload=org.mule.extension.db.internal.result.statement.CloseableMap#8194c34
mediaType=application/java; charset=UTF-8
attributes=<not set>
attributesMediaType=*/*
}
what changes can I make so that I recieve the list of names from db.
your database connector by defaults returns a resultset. convert your resultset into a json by adding a transformer after DB call and set output type to json. if you want only one particular field then you need to have your transform done accordingly
as you can see in your logger that its mentioned the payload type
payload=org.mule.extension.db.internal.result.statement.CloseableMap#8194c34
that is the internal representation of the result set read back from the stored procedure. so just convert it into json or whatever format you want. Put a transform message component after DB call and do the following:
%dw 2.0
output application/json
---
payload
if you want any particular field or something then you can see how your payload is getting structured and do the dataweave accordingly

How to get payload mime type in Mule 4

I am trying to find out what is the payload type by printing it out using logger. I am aware that in Mule 3 you could use this #[payload.getClass().getSimpleName()] or #[message.dataType.type].
How to achieve this in Mule 4? I did my google search but could not find any solution.
You can use:
#[payload.^class]
Try logging:
#[payload.^]
Should print:
{class=java.util.ArrayList, encoding=UTF-8, mimeType=application/java, raw=[{}]}
But if your payload is not application/java mime/type but json or something instead it won't have class attribute.
Get payload media type: #[payload.^mediaType]
Reference: https://docs.mulesoft.com/dataweave/2.4/dataweave-cookbook-extract-data

Unknown Payload in transform message Anypoint Studio

I have a problem with transform message. I need to recieve messages in SOAP/WSDL sistem. In my case i have this structure for process the responses.
This is my flow and in trnasform message component i have this problem:
Y put credentials in transform message but if i see the web service component, say that payload is unknown.
The problem is the sem for two transform message. I don't know how can i do to put the credentials good for doing a reception message aplication with SOAP/WSDL sistem.
Thanks for help.
Please try to log the payload as [message.payloadAs(java.lang.String)]
If you can see the actual payload in XML or JSON format whatever you are POSTing, then extract the specific elements.
If your GET then in query params retrieve them using [message.inboundProperties.'httpqueryparams']
Extract the specfic elements into flowvars and set them to Webservice consumer add [flowVars.userid] and [flowVars.pwd]
That will echo back as response.
I hope this is going to help.
Bit of a guess here because you haven't shown exactly what your WSDL is expecting:
I think you may want a transform a bit like this
%dw 1.0
%output application/xml encoding="UTF-8"
---
{
gdspHeader #(xmlns: "http://ws.gdsp.vf.gdspcustomer.com/" ) :{
gsdpCredentials: {
password: "Mule",
userId: "Monty"
}
}
}
Which transforms to
<?xml version='1.0' encoding='UTF-8'?>
<gdspHeader xmlns="http://ws.gdsp.vf.gdspcustomer.com/">
<gsdpCredentials>
<password>Mule</password>
<userId>Monty</userId>
</gsdpCredentials>
</gdspHeader>