Multipart/form-data broken after apiKit router Mule4 - mule

having issues with reading data from multipart/form-data after the request goes through ApiKit router in Mule4 app.
There is a RAML in place, used in ApiKit router to validate & route the requests.
#%RAML 1.0
title: ACC race data API
description: API for designing and updating race data
version: 1
protocols: [ HTTPS, HTTP ]
/acc/reverse/entrylist:
description: Used to manipulate entrylist for ACC reverse grid.
post:
description: Creates new entrylist based on supplied quali result, race one result and race one entrylist.
queryParameters:
positionsRotated:
description: Parameter defining how many positions to rotate.
required: false
type: number
example: 30
body:
multipart/form-data:
properties:
qualifyResult:
description: File containing qualifying result.
type: file
fileTypes: ['application/json']
required: true
raceOneResult:
description: File containing race 1 result.
type: file
fileTypes: ['application/json']
required: true
The validation works fine - if the data are sent in incorrect format, the exception is returned. For correctly formated data request is routed to the flow post:\acc\reverse\entrylist:multipart\form-data:acc-race-data-config.
Between the steps, payload format is changed (by Mule) from payload in readable form (see below) to java.io.ByteArrayInputStream#379ebdd5
Readable format of payload data (received to the app) before ApiKit router:
----------------------------180928595588258919887097
Content-Disposition: form-data; name="qualifyResult"; filename="json1.json"
Content-Type: application/json
{
"json1": "1"
}
----------------------------180928595588258919887097
Content-Disposition: form-data; name="raceOneResult"; filename="json2.json"
Content-Type: application/json
{
"json2": "2"
}
----------------------------180928595588258919887097--
The following dataweave script works fine if used before apiKit, but it doesn't work in the flow called by ApiKit:
%dw 2.0
output application/json
---
payload.parts[1].content
The example of output if above DW is used before ApiKit:
{
"json2": "2"
}
The example of output if the same DW is used after ApiKit:
org.mule.runtime.core.api.expression.ExpressionRuntimeException: "javax.mail.internet.ParseException - Missing start boundary, while reading `payload` as MultiPart.
Trace:
at main (Unknown)" evaluating expression: "%dw 2.0
output application/json
---
payload.parts[1].content".
Testing in Anypoint Studio 7.8.0, supposed to be used on Mule4-CE runtime once finished and ready to deploy.
Using Postman v8.5.1 for testing. sending form-data body with qualifyResult and raceOneResult parts containing JSON data, default headers, basic auth, query param positionsRotated=30.
Url called: https://localhost:443/api/acc/reverse/entrylist?positionsRotated=30
Tried to generate multipart/form-data body manually using a RAW type of payload, but the results were the same. Everything works fine if there is no ApiKit.. but I would like to use it to validate request validity.
Thanks to everyone replying for any useful hints!

I tested a similar scenario I had with the latest versions and I didn't had that problem. If you are using older versions of the HTTP connector and APIKit module try upgrading to the latest releases. Anypoint Studio has a feature to detect newer releases of connectors: https://docs.mulesoft.com/studio/7.9/update-modules

Related

Accessing multiple files in a multipart request

Most of the documentation in Mule regarding multipart relates to making a request and not processing an incoming request.
I'm looking to receive multiple files in an incoming request as an array and then loop through them to upload to another service (Salesforce).
Due to the use case I'm sending two properties:
data which is metadata to create a case record
files which is an array of files to be associated with the case record
My Postman request is:
This is the payload I receive in a Mule 4 application (redacted the file data itself):
----------------------------778704367722595657997650
Content-Disposition: form-data; name="data"
Content-Type: application/json
[
{
"subject": "test
}
]
----------------------------778704367722595657997650
Content-Disposition: form-data; name="files"; filename="logo.jpeg"
Content-Type: image/jpeg
jpegdata
----------------------------778704367722595657997650
Content-Disposition: form-data; name="files"; filename="icon.png"
Content-Type: image/png
pngdata
----------------------------778704367722595657997650--
I attempt to get the files with Dataweave:
payload.parts.files
But this returns only the first file 'logo.jpeg'
I've attempted to send the files as files[] but it doesn't seem to send as an array.
It seems that I should loop through the boundary instead? If so, should my request set the boundary so it's the same each time?
Or is there a different way to access the files?
Secondarily, is multipart the correct method or should I utilize application/json instead?
The use case is to send files from a contact form in another application to Salesforce.
When DataWeave parses a multi part input it outputs an object in which each key is the name of the part. Your input has two entries with the same key name. As usual in DataWeave you should use the multi valued selector to get all the values with the same key: payload.parts.*files

RAML, specify API request Content-Length value

I'm trying to set the value of the Content-Length property to 0 in my RAML file.
I first stumbled upon this thread : How to specify API request header value in RAML
When I tried implementing it with this code :
/update:
post:
description: Store updated data in database
headers:
Content-Length:
enum: [0]
body:
...
I got the response 400 Bad Request :
{
"code": "REQUEST_VALIDATION_ERROR",
"message": "Required header Content-Length is missing."
}
My IDE told me a string was expected but feeding with ["0"] didn't solve the problem.
(I'm testing my API specification in Mulesoft Design Center.)
So is there something I did wrong or is there another way to set the Content-Length to 0 in the request's header ?
You should only try to define custom headers in RAML. A required header of the HTTP protocol like Content-Length must not be defined. It should be automatically generated to prevent errors.

Mule 4.4 convert Binary payload into JSON

So I have gone through a number of questions related to this topic and tried the various options but none of them seem to work .
Environment - Mule runtime 4.4 , Anypoint Studio 7.11.1
Our limitation is we are not using Enterprise edition only community edition
So cannot use Transform component . However can use dataweave in setPayload / set Variable
I am receiving payload as Binary : see below :
Data appears as JSON but is Binary
I tried changing to JSON in 'setPayload' :
%dw 2.0
import * from dw::core::Binaries
output application/json
---
fromBase64(payload)
However I get this exception :
""Unexpected character 'ÿ' at payload#[1:30] (line:column), expected
Expecting end of input but got `ÿ, while reading `payload` as Json.
1| {"Employee":{"EName":"abcd"}}
^" evaluating expression: "%dw 2.0 import * from dw::core::Binaries output application/json
--- fromBase64(payload)"."
Note - also tried to check datatype of payload using :
typeOf(payload)
This too gave the same exception as above ...
Please help
Update 1:
Based on comments from #aled few additional details :
I am making a GET request to my api endpoint which is reading data from an ERP system which returns data in XML
Then I use a custom component that is being used to transform the XML data to JSON
The o/p of this custom component is what is generating binary data which I am unable to parse
I agree regarding Base64 that was useless
I later even tried with Transform component and all of the below attempts failed with the same exception :
Attempt#1:
%dw 2.0
output application/json
---
if (isEmpty(payload.^raw)) {
"payload": "No payload!!"
}
else {
"payload": payload
}
Attempt#2:
%dw 2.0
output application/json
---
payload
The above XML payload is a short excerpt received from ERP
I need to parse the payload and identify if there is an error present in the payload .
NOTE : if I simply log the payload it all shows up fine in the logs.
is it not possible to convert binary data to json ? do you think there is something wrong in the data received either from ERP or in the XML to JSON Transformer ?
Update#2
So based on earlier comments from #aled ( thanks ! ) started focusing on payload o/p of the ERP component ( which is XML ) BEFORE it is fed to our custom transformer ( which is generating binary data )
Attaching screen print for clarity and will also paste non proprietary code :
Now when I inspect the payload I can see it as :
<?xml version = "1.0" encoding="utf-8" ?>
<Output>
<ErrorDetails>
<Status>1</Status>
<Details>Invalid user or organisation</Details>
</ErrorDetails>
<Employee>
<EName>abcd</EName>
</Employee>
</Output>
and in debug mode I can see that payload is String and NOT Binary
Now I can access individual elements of the payload .
Example:
<set-variable value="#[output application/java --- payload[0].ErrorDetails['Status']]"
doc:name="extract status" variableName="status"/>
So this gives me the value 1 which is expected and good .
Conclusions:
1 The component that interacts with ERP is generating Non Binary / stream data
2 This data as expected is parseable - all good here
3 AFTER this data is fed to our custom component - the o/p payload is now binary and am running into issues parsing it ...
Question:
I know this being a custom component i cannot share code details here and thereby cannot expect inputs BUT regarding #aled comments about hexdump do you think I should try and use it against the output of the custom component ?
Once again a big thanks for your patience and help
Using function fromBase64() is just wrong. The value is not a Base64 string. It doesn't makes sense to try it. Using toBase64() would be useless because you would not be able to do anything with it, other than sending it somewhere else.
Your payload seems to be a JSON, and the media type is application/json in your screenshot, but it contains an UNICODE character at the end at least. You need. What are you trying to do with that payload exactly?
If the payload is truly a binary there is nothing you can do about that in the application. If you are receiving it from somewhere else check with the source. You didn't provide details of the source. Also ensure it is not your application corrupting the payload somehow.

Mulesoft_ passing client id and Client_secret under HTTP header

I have a requirement where I need to add both client id and client_secret under HTTP headers(under attribute,headers)
I tried to use the below code and unfortunately, it is not working, so please check and guide us to get the expected code.
Transform message create an attribute and applied the below DataWeave code:
output application/java
---
{
headers: {
client_id: '68eee04d10774836b6e5c40189ed3efb',
client_secret: '149541B02E23487EBE517708496a2920'
}
And in the consume connector header part, I have added the below line code.
Attributes.headers
It is difficult to say without seeing exactly what the application is doing but I suspect you are mixing SOAP headers with HTTP headers. HTTP Headers go into the transport headers element.
Example assuming both kind of headers in different variables. Note that you can set each header individually or pass an object which contains them all:
<wsc:consume doc:name="Consume" config-ref="Web_Service_Consumer_Config" operation="GetContent">
<wsc:message>
<wsc:headers><![CDATA[#[vars.soapHeaders]]]></wsc:headers>
</wsc:message>
<wsc:transport-headers>
<wsc:transport-header key="client_id" value="#[vars.httpHeaders.client_id]" />
<wsc:transport-header key="client_secret" value="#[vars.httpHeaders.client_secret]" />
</wsc:transport-headers>
</wsc:consume>

File Upload with Additional Parameters as JSON

I am using the below code to upload image with some additional parameters.
[HttpPost]
public HttpResponseMessage Upload(Data data)
{
var count = HttpContext.Current.Request.Files.Count;
return null;
}
I tried checked this method using postman chrome extension. I passed these values
Under Headers
enctype: multipart/form-data
Content-Type: application/json
Under form-data added one image file
Under raw
{
"Id": "1",
"ModifiedBy" : "1"
}
But the problem i am getting 0 for count
The Content-Type should be multipart/form-data not application/json for your code to work. A typical content type looks like this:
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryvmxBBVAoH3KRsa9L
The actual post data then contains blocks of 'data' separated by the boundary. These blocks can contain a content type, but they don't have to. If you're doing file uploads the content type should not matter - the filename is what determines the type.
All that said, with WebAPI you shouldn't be using the ASP.NET HttpContext object, but rather the lower level Web API semantics. While what you're doing works as long as you run inside of the ASP.NET stack, if you self-host or maybe in the future run ontop of a different stack like OWin/Katana this code will no longer work.
To handle file uploads with Web API specific code check out this blog post from Filip W.
http://www.strathweb.com/2012/04/html5-drag-and-drop-asynchronous-multi-file-upload-with-asp-net-webapi/