cURL vs MIME: POSTing a file - api

Question
I have written a very simple API using Flask, and I would like to upload a file to it using a POST command. I can easily make it work using cURL, but not so much using a logic app.
I have been using the Mozilla MIME Guide trying to construct the HTTP call, but I am not sure what to use in the header and body.
What I know is:
I would like to be able to send any file type, so I think I have to use the following:
Content-Type: application/octet-stream
Content-Disposition: attachment; filename="filename.xxx"
I have my file encoded with Base64, so I need to write that somehow, and place it in the body
I would like to use chunking. Does this make any difference?
My API
from flask import Flask, request, redirect
app = Flask(__name__)
#app.route('/', methods=['POST'])
def print_hello():
if request.files:
request.files['file'].save("/home/ebbemonster/cool_file.txt")
return "Hello World"
return "Goodbye World"
if __name__=="__main__":
app.run(host='0.0.0.0')
cURL
curl -X POST 13.81.62.87:5000 -F file=#GH019654.MP4
Logic App

So I figured out how to convert a cURL POST to a HTTP header/body.
Fetching header/body details
# Logging post call
curl -X POST XX.XX.XX.XX:5000 -F file=#GH019654.MP4 --trace-ascii post.log
# Fetching header
head -n 30 post.log
=> Send header, 216 bytes (0xd8)
0000: POST / HTTP/1.1
0011: Host: XX.XX.XX.XX:5000
0029: User-Agent: curl/7.58.0
0042: Accept: */*
004f: Content-Length: 300745456
006a: Content-Type: multipart/form-data; boundary=--------------------
00aa: ----ec1aab65fb2d68fd
00c0: Expect: 100-continue
# Fetching body
sed -n '18,25p' post.log
0000: --------------------------ec1aab65fb2d68fd
002c: Content-Disposition: form-data; name="file"; filename="GH019654.
006c: MP4"
0072: Content-Type: application/octet-stream
009a:
009c: ....ftypmp41 ...mp41....mdatGPRO#...HD7.01.01.70.00LAJ9022436601
00dc: 517...................................1...US.8'.f..C328132710684
011c: 1.HERO7 Black........................E.....1...US.8'.f..........
# Fetching end of body
tail -n 30 google.log
02c2: --------------------------ec1aab65fb2d68fd--
== Info: HTTP 1.0, assume close after body
Logic App Header/Body

Related

Download pdf files with curl using SOAP request

I am trying to download a pdf file using CURL command by sending HTTP POST request.
When I send my CURL request it download a pdf file but the file is not readable.
This the request I send: curl -H "Content-Type: text/xml; charset=UTF-8" -H "http://xxxxxxx/webservice/content/getContent:" -d #content_request.txt -X POST http://xxxxxx/xxxx/ContentService?wsdl/ -o sortieContent.pdf
(I replace the real adress by xxxxx for privacy reasons)
The thing is that even I download a pdf file it is not readable like it was corrupted.
What I understand is that curl answers with a file (cat of the content below) but there is different informations which are not the same format so the file received get corrupted.
--uuid:b47a2d96-bf98-4de9-99ae-9308d18ae599
Content-Id: rootpart*b47a2d96-bf98-4de9-99ae-9308d18ae599#example.jaxws.sun.com
Content-Type: application/xop+xml;charset=utf-8;type="text/xml"
Content-Transfer-Encoding: binary
79740application/pdfxxxxxxxFUOBqIAILPaDCmTvBRDXPhWnQQliV0ygEYrgPFVvDXw=
--uuid:b47a2d96-bf98-4de9-99ae-9308d18ae599
Content-Id: 5c3a7832-7ce4-4405-9cf6-20cb304972ca#example.jaxws.sun.com
Content-Type: application/octet-stream
Content-Transfer-Encoding: binary
%PDF-1.5
I tried replacing Content-Type: text/xml by Content-Type: application/pdf or Content-Type: application/octet-stream but it doesn't even download the content.
Is it possible to download only the pdf not the other informations so my file will be readable? How can I do it?
Thank you

HTTP Post Request (CSV File to ebay) with VBA

The eBay CSV Manager enables to upload a CSV file to mark all paid and sent items as sent in eBays seller tool. My Excel Macro creates such a CSV file. Now, I want to upload this file via HTTP Post Request.
In eBays CSV Manager manual they say:
Files can only be uploaded via Token. This Token has to be send with the file every upload. It is virtually the key for the entry to the CSV Manager.
I already have the Token.
It goes on to say that
a HTTP Post request has to be sent to the eBay CSV Manager Server to upload the file via script. The HTTPS connection has to be started and the data has to be send together with the Token to the CSV Manager web adress https://bulksell.ebay.com/ws/eBayISA...ExchangeUpload
eBay has an example for the request in its CSV Manager manual:
POST /path/to/upload/script HTTP/1.0
Connection: Keep-Alive
User-Agent: My Client App v1.0
Host:
https://bulksell.ebay.com/ws/eBayISA...ExchangeUpload
Content-type: multipart/form-data;
boundary=THIS_STRING_SEPARATES
Content-Length: 256
--THIS_STRING_SEPARATES
Content-Disposition: form-data; name="token"
12345678987654321
--THIS_STRING_SEPARATES
Content-Disposition: form-data; name="file";
filename="listings.csv"
Content-Type: text/csv
... contents of listings.csv ...
--THIS_STRING_SEPARATES
The explanation of the example is:
State which method has to be applied to the ressource:
POST /path/to/upload/script HTTP/1.0
Connection type, user-agent and information to the host:
Connection: Keep-Alive
User-Agent: My Client App v1.0
Host:https://bulksell.ebay.com/ws/eBayISA...ExchangeUpload
Header with information to the content and the lenght of the file:
Content-type: multipart/form-data; boundary=THIS_STRING_SEPARATES
Content-Length: 256
Safety Token and content to be uploaded:
--THIS_STRING_SEPARATES
Content-Disposition: form-data; name="token"
12345678987654321
--THIS_STRING_SEPARATES
Content-Disposition: form-data; name="file"; filename="listings.csv"
Content-Type: text/csv
... contents of listings.csv ...
--THIS_STRING_SEPARATES
How to do the HTTP Request and how to integrate it to the VBA Code?
The concept of REST API is that you post a data file to the API end point. You need to use one of microsoft http add-on to conscruct the payload which contains the header and body and post it to the require API.
You would have multi part in your macro to achieve this function
A login function where you would store the token
A body constructor where you would read your excel data and transform it into a json format.
A header constructor
A post function where post the relevant data to the gateway api using the microsoft http document handler.
But if you are planning to send the CSV as data then its best to write a vbscript to quickly post the file to the API.

Postman shows "Could not get any response" even though response is OK

I have a WCF service which I make API requests to.
This API call returns a JSON response object and also is able to return it in GZIP compression as well when "gzip" value is used in "Accept-Encoding" header.
The problem is when I try to get the response in GZIP, Postman shows "Could not get any response" although I see response and response's content are OK (200 status code) in Fiddler and can easily decompress the response content in my C# client.
I took a look in Postman Console but all I see is "Error: incorrect header check".
I hardly tried to find any documentation regarding this header check but couldn't find any.
These are the request headers:
POST /correction/v1/document?lang=US HTTP/1.1
Content-Type: text/plain
Accept-Encoding: gzip
User-Agent: PostmanRuntime/7.6.0
Accept: */*
content-length: 630
Connection: close
These are the response headers:
HTTP/1.1 200 OK
Content-Length: 512
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
Server: Microsoft-HTTPAPI/2.0
Date: Sun, 24 Feb 2019 14:05:50 GMT
Connection: close
The only thing I suspect is wrong is this message from Fiddler:
I integrated this code into mine in order to use GZIP in WCF.
https://github.com/carlosfigueira/WCFSamples/tree/master/MessageEncoder/GZipEncoderAndAutoFormatSelection
Basically, it captures the response before returning to client and use GZipStream for compression.
I got the same issue, I added the following header to fix this issue.
Accept-Encoding : *
I was able to solve a similar issue by using the header Accept-Encoding: */* or if you want to be specific do Accept-Encoding: */* that way the HTTP client will be able to process the response based on the type of encoding received, in the case of a gzip, it will decode the response and show it as normal text.
For me, I removed 'Accept-Encoding' in the request header.
I got this issue when the REST service was returning a zip content (aka. WinZip format). I solved the error by compressing the data using 7zip to produce true gzip format.

Generate HTTP POST form with multipart-form-data without curl

So I'm trying to generate HTTP POST form in my embedded application. However I get server 400 error that indicates that something is wrong with my post. I do not have any curl-like libraries, or such, so I need to form the post header from scratch.
const static char *post_header = "POST /v1/avs/speechrecognizer/recognize HTTP/1.1\r\n\
Host: access-alexa-na.amazon.com\r\n\
Authorization: Bearer %s\r\n\
Content-Type: multipart/form-data; boundary=BOUNDARY1234\r\n\
Transfer-Encoding: chunked\r\n\
Content-Length: %d\r\n\
\r\n\r\n\
--BOUNDARY1234\r\n\
Content-Disposition: form-data; name=\"metadata\"\r\n\
Content-Type: application/json; charset=UTF-8\r\n\
\r\n\
{\"messageHeader\": {},\"messageBody\": {\"profile\": \"alexa-close-talk\",\"locale\": \"en-us\",\"format\": \"audio/L16; rate=16000; channels=1\"}}\r\n\
\r\n\r\n\
--BOUNDARY1234\r\n\
Content-Disposition: form-data; name=\"audio\"\r\n\
Content-Type: audio/L16; rate=16000; channels=1\r\n\n";
After the last "\n" I have the wav header and payload itself. I don't have null termination between the wav header and the last request header content. Altough I've tried it and it doesn't seem to make any difference.
My authentication token should be OK (I've verified it with curl). I've used these scripts (https://miguelmota.com/blog/alexa-voice-service-with-curl/) and Amazon documentation as a base. The blogpost has a script that generates multipart payload and it's identical (compared binary dumps) to mine. My only obvious questionmarks are the first part of the query:
"POST /v1/avs/speechrecognizer/recognize HTTP/1.1\r\n\
Host: access-alexa-na.amazon.com\r\n\
Authorization: Bearer %s\r\n\
Content-Type: multipart/form-data; boundary=BOUNDARY1234\r\n\
Transfer-Encoding: chunked\r\n\
Content-Length: %d\r\n\
\r\n\r\n\"
and the curl call with especially the --data-binary part. Should it effect the request body shomehow?
curl -X POST \
-H "Authorization: Bearer ${TOKEN}" \
-H "Content-Type: multipart/form-data; boundary=${BOUNDARY}" \
--data-binary #multipart_body.txt \
https://access-alexa-na.amazon.com/v1/avs/speechrecognizer/recognize \
> response.txt
Any ideas anyone? I'm gettin a bit frustrated with this.
EDIT 1: Just to clarify. The total size of the data is about 200kbytes with the audio data included. The header size with the token is about 1200bytes. I'm sending the stuff in 1k blobs and I get the error after 4k or so. So I don't manage to send the whole thing before the server responds with the error. Also some of the similar cases in Amazon side indicates that 400 in this case points to problem with the header. However they aren't manually forming the posts so I cannot see the whole thing anywhere.
EDIT2:
Also as this is chunked data, I wonder how it affects this?
I mean if I fe chunk the header into parts defined by the --BOUNDARY1234 and max of 512 bytes, how would that work? I mean fe:
--BOUNDARY1234\r\n\ Content-Disposition: form-data; name=\"metadata\"\r\n\ Content-Type: application/json;
charset=UTF-8\r\n\ \r\n\ {\"messageHeader\": {},\"messageBody\":
{\"profile\": \"alexa-close-talk\",\"locale\": \"en-us\",\"format\":
\"audio/L16; rate=16000; channels=1\"}}\r\n\ \r\n\r\n\
Should the there be chunk size right in the start of the transfer before the --BOUNDARY1234 or does the "Content-Disposition" or "Content-Type" affect this somehow? Or should I add the chunk size only to binary payload? Problem here is that the max send block with my HW is 1k. And the total header size is ~1,5k.

Is it possible to send GZip'd HTTP POST request by using PAW

Is it possible to send gzip'd HTTP POST requests in Paw?
I have accept-encoding, content-encoding set to gzip but it seems to be not enough to get it to work.
POST / HTTP/1.1
Content-Type: application/json
Content-Encoding: gzip
Accept: application/json
Accept-Encoding: gzip
X-Protocol-Version: 2
User-Agent: test-ua
Host: host123:8080
Connection: close
Content-Length: 120
[{"user-id":"123","p-id":"1"}]
I can confirm that. It is not possible at the moment to send gzip'ed body automatically.
If you use "Content-Encoding: gzip" you must gzip the body and drop the file into Paw. Paw is sending the gzip'ed body without any issue. I am using Paw version 2.2.5.
I think you need to gzip the body yourself and then paste it in. I could be wrong and if so would love this auto gzip feature!