AFHttpSessionManager - Multipart POST causing org.codehaus.jackson.JsonParseException - jackson

Am trying to use AFNetworking 2's AFHTTPSessionManager to post pdf content to my web service. I have subclassed AFHTTPSessionManager and set request and response serializers to the corresponding AFJsonXXXSerializer. However when I POST the form data as a multi part request, I get a JsonParseException. Am pretty sure this is a straight forward use case and I might be missing something. Pls help, Thanks in advance !
[accessMgr postDataToURL:#"objects-d2" usingParams:nil fileData:imageData andDelegate:self];
which calls the below method to POST
-(void) postDataToURL:(NSString *) urlString usingParams:(id)parameters fileData: (NSData *)fileData andDelegate:(id<RemoteAccessDelegate>) remoteDelegate
{
//RESTSessionManager extends AFHTTPSessionMaanger
RESTSessionManager *manager = [self getManager:remoteDelegate];
[manager POST:urlString parameters: parameters constructingBodyWithBlock:^(id<AFMultipartFormData> formData) {
[formData appendPartWithFileData:fileData name:#"content" fileName:#"test.pdf" mimeType:#"application/pdf"];
The request goes with the following headers and the form data
POST /d2fs/repositories/ls67sp2/objects-d2 HTTP/1.1
Host: localhost:8080
Content-Type: multipart/form-data; boundary=Boundary+422AF86226B68040
Connection: keep-alive
Transfer-Encoding: Chunked
Accept: application/json
User-Agent: IIG Mobile/1.1 (iPad Simulator; iOS 7.1; Scale/2.00)
Accept-Language: en;q=1, fr;q=0.9, de;q=0.8, zh-Hans;q=0.7, zh-Hant;q=0.6, ja;q=0.5
Authorization: Basic bHNhZG1pbjpsc2FkbWlu
Accept-Encoding: gzip, deflate
--Boundary+422AF86226B68040
Content-Disposition: form-data; name="content"; filename="test.pdf"
Content-Type: application/pdf
%PDF-1.3
%ƒÂÚÂÎßÛ†–ƒ∆
<more file content>
I have verified that the trailing boundary is also set. The response am getting from my web service is
{
"status": 400,
"code": "E_INPUT_ILLEGAL_ARGUMENTS",
"message": "There are illegal arguments provided.",
"details": "org.codehaus.jackson.JsonParseException: Unexpected character ('%' (code 37)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: org.apache.commons.fileupload.MultipartStream$ItemInputStream#22a5cdce; line: 1, column: 2];Unexpected character ('%' (code 37)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: org.apache.commons.fileupload.MultipartStream$ItemInputStream#22a5cdce; line: 1, column: 2]"
}

I was able to resolve this issue today. I had to append a json object which was metadata about the object before appending the file data and the upload worked as a charm ! Thanks.
[formData appendPartWithFormData:data name:#"object"];
[formData appendPartWithFileData:fileData name:#"content" fileName:#"test.pdf" mimeType:#"application/pdf"];

Related

Content-Type header property value quoting doesn't work with third-party server

Is it correct to quote the boundary property value of Content-Type header?
I have sent an http-request with two files to a third-party server and get the following response:
Boundary '--"38b14895-fd44-4acc-8287-9f0378691da2"' not found in message body
because RestSharp quotes the boundary value, but the server doesn't unquote it. I can neither change the third-party server nor customize RestSharp header quoting.
What is the problem? Does the http spec allow escaped strings in header property values? I've read the spec, but haven't found a place where this would be explicitly defined.
I create the RestRequest something like this:
private RestRequest CreateRequest( ... )
{
var request_url = $"url?param=value";
var request = new RestRequest( request_url, Method.Post );
request.AddFile( "file1", ..., "file1", "application/xml" );
request.AddFile( "file2", ..., "file2", "audio/x-wav" );
request.AddHeader( "Content-Type", "multipart/form-data" );
return request;
}
and get the following HTTP-request:
POST /url?param=value
Host: 192.168.1.1:80
Accept: application/json, text/json, text/x-json, text/javascript, application/xml, text/xml
User-Agent: RestSharp/108.0.1.0
Accept-Encoding: gzip, deflate, br
Content-Type: multipart/form-data; boundary="38b14895-fd44-4acc-8287-9f0378691da2"
Content-Length: 227841
--38b14895-fd44-4acc-8287-9f0378691da2
Content-Type: application/xml
Content-Disposition: form-data; name="file1"; filename="file1"
[data]
--38b14895-fd44-4acc-8287-9f0378691da2
Content-Type: audio/x-wav
Content-Disposition: form-data; name="file2"; filename="file2"
[data]
--38b14895-fd44-4acc-8287-9f0378691da2--
Okay so I have just been dealing with a very similar issue. I found this GitHub Issue which gives some good context into the actual issue that is occurring here but I have also found a fix.
Firstly, you shouldn't manually add the "Content-Type" header. RestSharp will do this for you since you are using the AddFile() method.
I am assuming you are using the latest version of RestSharp, if so you can set the request.OnBeforeRequest property of the RestRequest to handle this and strip out the double quotes around the boundary before the request is sent:
request.OnBeforeRequest = (http) =>
{
var boundary = http.Content.Headers.ContentType.Parameters.First(o => o.Name == "boundary");
boundary.Value = boundary.Value.Replace("\"", String.Empty);
return default;
};
Hope this helps!

How to send multipart/form-data for image binary with Python requests lib (with CRLF chars)?

I'm trying to upload a local image to the Medium /images endpoint (documented here).
Their example request looks like:
Host: api.medium.com
Authorization: Bearer 181d415f34379af07b2c11d144dfbe35d
Content-Type: multipart/form-data; boundary=FormBoundaryXYZ
Accept: application/json
Accept-Charset: utf-8
--FormBoundaryXYZ
Content-Disposition: form-data; name="image"; filename="filename.png"
Content-Type: image/png
IMAGE_DATA
--FormBoundaryXYZ--
The type of image data is never specified, but I'm assuming it's raw binary based on this very similar API call from Ancestry for uploading images.
Currently, I have:
headers['Content-Type'] = 'multipart/form-data; boundary=FormBoundaryXYZ'
imageData = '''--FormBoundaryXYZ
Content-Disposition: form-data; name="image"; filename="filename.jpg"
Content-Type: image/jpeg
0000000000000011111111111111111111111111
0000000000000011111111111111111111111111
0000000000000011111111111111111111111111
0000000000000000011111111111111111111111
0000000000000000000111111111111111111111
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000001000000000000000000
0000000000000000000100000000000000000000
0000000000000000010000000000000000000000
0000000000000000000000000000000000000000
0000000010000000000000000000000000000000
0000000000001100000000000000000000000000
0000000000000100000001000000000000000000
0000000000000000000001000000000000000010
0000000000000000001001111111110000000001
0000000000000000000000100110000000000000
0000000000000000000000000001110000000000
0000000000000000011110001111111101100000
0000000000000000111111111111111111111111
0000000001000001111111111111111111111111
0000000000000011111111111111111111111111
0000000100001011111111111111111111111111
0000001100000001111111111111111111111111
0000110000000000000000000000000000000000
0001000000000000000000000000000000000000
--FormBoundaryXYZ--'''
def post_image(imgData):
req = r.post(base_url+'/images', headers=headers, data=imgData)
res = req.json()
return res
print(json.dumps(post_image(imageData), indent=2))
When executed, I get the error response:
{
"errors": [
{
"message": "Expected CR Received 10",
"code": -1
}
]
}
The documentation mentions
All lines in the body must be terminated with \r\n.
and I know that the "10" in the response must be the "\n", so I'm missing the "\r". I've tried explicitly adding "\r\n" to the end of each line termination, but then I get an error like
{
"errors": [
{
"message": "Expected alphabetic character, received 10",
"code": -1
}
]
}
Or
{
"errors": [
{
"message": "stream ended unexpectedly",
"code": -1
}
]
}
depending on where I place the /r/n or remove it.
My question is: Is there a way to format this that I'm missing or overthinking? I'm hoping I'm just inexperienced in multipart/form-data encoding and I could be doing this programmatically instead.
Thanks! I'd be happy to add details if needed!
(As a side note: I got the image's binary data with the dcode.fr/binary-image tool.)

Creating multipart/mixed request

I have a very specific case I'm trying to test with Karate.
PUT https://test-api.com/endpoint
Content-Type: multipart/mixed; boundary=BOUNDARY
--BOUNDARY
Content-Type: application/vnd.api+json
{"type": "json-api-object"}
--BOUNDARY
Content-Disposition: attachment; name="fieldname"; filename="filename.jpg"
Content-Type: image/jpeg
Content-Encoding: base64
<binary data>
--BOUNDARY--
Examples show multipart/mixed requests, but they don't show how to set the content-type header on each part. I tried using And multipart header... but that didn't parse correctly.
https://github.com/intuit/karate/blob/master/karate-demo/src/test/java/demo/upload/upload.feature
If I can get this figured out with your help, I'll make a PR against the examples to hopefully help someone in the future.
Yeah this is the first time I'm seeing a need for a custom content-type for each part. This will need a change in the code, so yes an example will expedite a fix.
Meanwhile you can customize the content-type for normal requests.
I had success using mulipart files
* configure headers = {"Content-Type": 'multipart/mixed'}
* configure charset = null
* def mFiles = {}
* set mFiles.jsondata = { value: {'type': 'json-api-object'}, contentType: 'application/vnd.api+json' }
* set mFiles.fieldname = { read: 'classpath:path/to/somePhoto.jpg', filename: 'filename.jpg', contentType: 'image/jpeg' }
Given url "https://my-api.com/put"
And multipart files mFiles
When method PUT
According to this comment, multipart file is preferred for anything other than a string in a multipart field.

AFNetworking - occasionally get "unacceptable content type: application/json" on http response

I can't figure out why this message is being thrown by AFNetworking 2.0. My understanding is that application/json is the default serialization scheme, so if the server returns JSON with content-type:application/json, why would AFNetworking throw this error?
Failure with messages Error Domain=com.alamofire.error.serialization.response Code=-1016 "Request failed: unacceptable content-type: application/json" UserInfo=0x17e2ed60 {com.alamofire.serialization.response.error.response=<NSHTTPURLResponse: 0x17e84100> { URL: http://XXX/XXX/XXX } { status code: 200, headers {
Connection = "keep-alive";
"Content-Type" = "application/json;charset=UTF-8";
Date = "Fri, 14 Aug 2015 16:16:52 GMT";
Server = "Apache-Coyote/1.1";
"Transfer-Encoding" = Identity;
"X-Application-Context" = application;
} }, NSErrorFailingURLKey=http://XXX/XXX/XXX, NSLocalizedDescription=Request failed: unacceptable content-type: application/json, com.alamofire.serialization.response.error.data=<7b227374 61747573 223a3230 302c2263 6f646522 3a225636 594d227d>}
Commented out URL's with X's in the code block. This only happens sometimes, and it causes the request to fail. If I issue another request, it succeeds.
I'm using an AFHTTPSessionManager singleton to issue all requests.
Try this:
sessionManager.responseSerializer.acceptableContentTypes = [NSSet setWithObjects:#"application/json", #"text/json", #"text/javascript", #"text/html", nil];

Restkit loading gzipped json

I'm using RestKit to load a gzipped JSON with RKRequest:
RKRequest* request = [[RKClient sharedClient] requestWithResourcePath:urlString delegate:self];
[request send];
but I receive a status 406. When using AsiHttpRequest everything works, the response gets unzipped and I can work with the JSON. When I turn off gzip on server the RKRequest works.
What is wrong? I found no way to tell RKRequest, that the response will be zipped. Any ideas?
EDIT:
It is strange. Sometimes I get
Headers: {
Connection = "Keep-Alive";
"Content-Length" = 14;
"Content-Type" = "text/html; charset=UTF-8";
Date = "Fri, 16 Mar 2012 13:44:16 GMT";
"Keep-Alive" = "timeout=2, max=500";
Server = Apache;
"X-Powered-By" = "Servlet/2.5 JSP/2.1";
}
and sometimes I get application/gzip which is handled correct. My problem is why I get "Content-Type" = "text/html; charset=UTF-8"; sometimes.
And the same request opened in Safari results in a gzip-response always.
Can you post what's in your headers using an HTTP Proxy (like Charles)?
You may need to modify your "request headers" in the post call.
Make sure your firewall is able to accept POST calls. This might be an https issue.
EDIT:
You may need to configure your server to always return the response as a GZIP and DEFLATE based on the extension type. This is based on here (http://betterexplained.com/articles/how-to-optimize-your-site-with-gzip-compression/).
Example:
# compress json format in .htaccess (for apache servers):
AddOutputFilterByType DEFLATE application/json
You can find the 'mod_deflate' documentation here (http://httpd.apache.org/docs/2.0/mod/mod_deflate.html)
If you can post the outgoing headers, that would also be useful, as they should include:
Accept-Encoding: gzip, deflate
Similar issues
https://groups.google.com/forum/?fromgroups#!topic/restkit/Xo84PH1l5kM
Weird "406 not acceptable" error
https://serverfault.com/questions/183843/content-length-not-sent-when-gzip-compression-enabled-in-apache
EDIT:
Make sure you also do this:
[[RKClient sharedClient] setValue:#"gzip" forHTTPHeaderField:#"Accept-Encoding"];
or this
[[RKClient sharedClient] setValue:#"gzip, deflate" forHTTPHeaderField:#"Accept-Encoding"];
This should set the value of your header to accept "gzip" for encoding the response. I noticed these github issues:
https://github.com/RestKit/RestKit/pull/540
https://github.com/RestKit/RestKit/issues/511