How to send multipart/form-data for image binary with Python requests lib (with CRLF chars)? - file-upload

I'm trying to upload a local image to the Medium /images endpoint (documented here).
Their example request looks like:
Host: api.medium.com
Authorization: Bearer 181d415f34379af07b2c11d144dfbe35d
Content-Type: multipart/form-data; boundary=FormBoundaryXYZ
Accept: application/json
Accept-Charset: utf-8
--FormBoundaryXYZ
Content-Disposition: form-data; name="image"; filename="filename.png"
Content-Type: image/png
IMAGE_DATA
--FormBoundaryXYZ--
The type of image data is never specified, but I'm assuming it's raw binary based on this very similar API call from Ancestry for uploading images.
Currently, I have:
headers['Content-Type'] = 'multipart/form-data; boundary=FormBoundaryXYZ'
imageData = '''--FormBoundaryXYZ
Content-Disposition: form-data; name="image"; filename="filename.jpg"
Content-Type: image/jpeg
0000000000000011111111111111111111111111
0000000000000011111111111111111111111111
0000000000000011111111111111111111111111
0000000000000000011111111111111111111111
0000000000000000000111111111111111111111
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000000000000000000000000
0000000000000000000001000000000000000000
0000000000000000000100000000000000000000
0000000000000000010000000000000000000000
0000000000000000000000000000000000000000
0000000010000000000000000000000000000000
0000000000001100000000000000000000000000
0000000000000100000001000000000000000000
0000000000000000000001000000000000000010
0000000000000000001001111111110000000001
0000000000000000000000100110000000000000
0000000000000000000000000001110000000000
0000000000000000011110001111111101100000
0000000000000000111111111111111111111111
0000000001000001111111111111111111111111
0000000000000011111111111111111111111111
0000000100001011111111111111111111111111
0000001100000001111111111111111111111111
0000110000000000000000000000000000000000
0001000000000000000000000000000000000000
--FormBoundaryXYZ--'''
def post_image(imgData):
req = r.post(base_url+'/images', headers=headers, data=imgData)
res = req.json()
return res
print(json.dumps(post_image(imageData), indent=2))
When executed, I get the error response:
{
"errors": [
{
"message": "Expected CR Received 10",
"code": -1
}
]
}
The documentation mentions
All lines in the body must be terminated with \r\n.
and I know that the "10" in the response must be the "\n", so I'm missing the "\r". I've tried explicitly adding "\r\n" to the end of each line termination, but then I get an error like
{
"errors": [
{
"message": "Expected alphabetic character, received 10",
"code": -1
}
]
}
Or
{
"errors": [
{
"message": "stream ended unexpectedly",
"code": -1
}
]
}
depending on where I place the /r/n or remove it.
My question is: Is there a way to format this that I'm missing or overthinking? I'm hoping I'm just inexperienced in multipart/form-data encoding and I could be doing this programmatically instead.
Thanks! I'd be happy to add details if needed!
(As a side note: I got the image's binary data with the dcode.fr/binary-image tool.)

Related

SAP Business One Service Layer - Upload Attachment - Internal Error (-43)

I'm having an issue uploading an attachment into SAP B1 via the Service Layer. I am able to upload an attachment using the SAP client without issue. I'm using Postman to test sending the request before I embed this into a real application. I've gone over the documentation available in the SAP B1 Service Layer Guide (located here https://help.sap.com/doc/0d2533ad95ba4ad7a702e83570a21c32/9.3/en-US/Working_with_SAP_Business_One_Service_Layer.pdf) on page 106.
Here's what my HTTP request looks like... and it matches what the documentation expects.
POST /b1s/v1/Attachments2 HTTP/1.1
Host: 129.159.XXX.XXX:50000
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Cookie: B1SESSION=62bbdc3e-d634-11ec-8000-02001703771b; ROUTEID=.node3
Content-Length: 197
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="files"; filename="JSON-Tshirt-Master Product.txt"
Content-Type: text/plain
(data)
----WebKitFormBoundary7MA4YWxkTrZu0gW
Here's the error that I'm getting which doesn't make sense to me. I can't find anything on this error and I was wondering if anyone had seen something like this before?
{
"error": {
"code": -43,
"message": {
"lang": "en-us",
"value": "Internal error (-43) occurred"
}
}
}

Power BI - Get JWT Token from Basic authentication

I'm getting a token from a REST API with Postman with this request:
URL:
https://www.wathever.com/fmi/data/v1/databases/sessions
Headers:
Authorization: Basic (binary generated by Postman)==
Postman-Token: <calculated when request is sent>
Content-Length: 0
Host: <calculated when request is sent>
User-Agent: PostmanRuntime/7.26.2
Accept: */*
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Type: application/json
and I succesfully get a token:
{
"response": {
"token": "fff***********************3ecfc7f01a*****716"
},
"messages": [
{
"code": "0",
"message": "OK"
}
]
}
Now, I'm trying to get the token through Power BI with the following code (credential anonymous):
let server = "https://www.wathever.com/fmi/data/v1/databases/",
database = "thedatabase",
username = "username",
psw = "apassword",
url = server & database & "/sessions",
bytes = Text.ToBinary(username & ":" & psw),
encoded = Binary.ToText(bytes, BinaryEncoding.Base64),
body = "",
data = Json.Document(Web.Contents(url,
[
Headers = [
#"Authorization"=("base64-encoded " &encoded),
#"Content-Type"="application/json"
]
]))
in
data
But I get the following error message:
We couldn't authenticate with the credentials provided. Please try
again.
I also tried to connect with credential basic, without passing headers, but I get the following message:
DataSource.Error: Web.Contents failed to get contents from
'https://www.wathever.com/fmi/data/v1/databases/sessions' (405):
Method Not Allowed
Any idea of what I am missing?
Thanks!

imgur API v3 upload image keep getting back status 1203: "Imgur is over capacity. Please try again."

This is the request I am posting
POST /3/image HTTP/1.1
Host: api.imgur.com
Authorization: Client-ID <My client ID>
Cache-Control: no-cache
----WebKitFormBoundaryE19zNvXGzXaLvS5C
Content-Disposition: form-data; name="image"
http://png1.findicons.com/files/icons/1580/devine_icons_part_2/128/home.png
----WebKitFormBoundaryE19zNvXGzXaLvS5C
However, the response I get is:
{
"data": {
"error": "Imgur is over capacity. Please try again.",
"request": "/3/image",
"method": "POST"
},
"success": false,
"status": 1203
}
I just want to use anonymous upload but the documentation is saying limit of 10MB right? anything I have done wrongly? what should I do to fix this error?
Imgur is frequently over capacity, so perhaps you can try again later. I doubt it's a problem on your end.

OneDrive Expects Boundary for Content-Type: application/json

This is a recent development. We run integration tests against this API on every commit.
POST https://apis.live.net/v5.0/me/skydrive
Authorization: Bearer ACCESS_TOKEN
Content-Type: application/json
{
"name": "My example folder"
}
Response:
{
"error": {
"code": "request_header_invalid",
"message": "The header 'Content-Type' is missing the required parameter: 'boundary'."
}
}
Requiring a boundary for an application/json content-type is just wrong. But lets try it:
POST https://apis.live.net/v5.0/me/skydrive
Authorization: Bearer ACCESS_TOKEN
Content-Type: application/json; boundary=idontactuallybelonghere
{
"name": "My example folder"
}
Response:
{
"error": {
"code": "request_header_invalid",
"message": "The header 'Content-Type' is missing the required parameter: 'boundary'."
}
}
Are you trying to create a folder? That error sounds like it thinks you're trying to upload a file using a multipart/form-data POST.
Are you using the exact URL as in your example? I tried that exact request against my account and it worked. I was able to reproduce your error, though, by using a URL that ends in '/files' like I would for a file upload.

AFHttpSessionManager - Multipart POST causing org.codehaus.jackson.JsonParseException

Am trying to use AFNetworking 2's AFHTTPSessionManager to post pdf content to my web service. I have subclassed AFHTTPSessionManager and set request and response serializers to the corresponding AFJsonXXXSerializer. However when I POST the form data as a multi part request, I get a JsonParseException. Am pretty sure this is a straight forward use case and I might be missing something. Pls help, Thanks in advance !
[accessMgr postDataToURL:#"objects-d2" usingParams:nil fileData:imageData andDelegate:self];
which calls the below method to POST
-(void) postDataToURL:(NSString *) urlString usingParams:(id)parameters fileData: (NSData *)fileData andDelegate:(id<RemoteAccessDelegate>) remoteDelegate
{
//RESTSessionManager extends AFHTTPSessionMaanger
RESTSessionManager *manager = [self getManager:remoteDelegate];
[manager POST:urlString parameters: parameters constructingBodyWithBlock:^(id<AFMultipartFormData> formData) {
[formData appendPartWithFileData:fileData name:#"content" fileName:#"test.pdf" mimeType:#"application/pdf"];
The request goes with the following headers and the form data
POST /d2fs/repositories/ls67sp2/objects-d2 HTTP/1.1
Host: localhost:8080
Content-Type: multipart/form-data; boundary=Boundary+422AF86226B68040
Connection: keep-alive
Transfer-Encoding: Chunked
Accept: application/json
User-Agent: IIG Mobile/1.1 (iPad Simulator; iOS 7.1; Scale/2.00)
Accept-Language: en;q=1, fr;q=0.9, de;q=0.8, zh-Hans;q=0.7, zh-Hant;q=0.6, ja;q=0.5
Authorization: Basic bHNhZG1pbjpsc2FkbWlu
Accept-Encoding: gzip, deflate
--Boundary+422AF86226B68040
Content-Disposition: form-data; name="content"; filename="test.pdf"
Content-Type: application/pdf
%PDF-1.3
%ƒÂÚÂÎßÛ†–ƒ∆
<more file content>
I have verified that the trailing boundary is also set. The response am getting from my web service is
{
"status": 400,
"code": "E_INPUT_ILLEGAL_ARGUMENTS",
"message": "There are illegal arguments provided.",
"details": "org.codehaus.jackson.JsonParseException: Unexpected character ('%' (code 37)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: org.apache.commons.fileupload.MultipartStream$ItemInputStream#22a5cdce; line: 1, column: 2];Unexpected character ('%' (code 37)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: org.apache.commons.fileupload.MultipartStream$ItemInputStream#22a5cdce; line: 1, column: 2]"
}
I was able to resolve this issue today. I had to append a json object which was metadata about the object before appending the file data and the upload worked as a charm ! Thanks.
[formData appendPartWithFormData:data name:#"object"];
[formData appendPartWithFileData:fileData name:#"content" fileName:#"test.pdf" mimeType:#"application/pdf"];