I'm trying to upload a pdf file with form-data to a server. The upload works but the file gets corrupted for some reason (I can't open the uploaded version). Here's my code:
post_url = 'https://myposturl'
headers = {
'Content-Type':'multipart/form-data; charset=UTF-8; boundary=MyBoundary'
}
with open('./myfile.pdf', 'rb') as f:
body = f'--MyBoundary\r\nContent-Disposition: form-data; name="file"; filename="myfile.pdf"\r\nContent-Type: application/pdf\r\n\r\n{f.read()}\r\n--MyBoundary--\r\n'
res = s.post(post_url, headers = headers, data = body)
I thought it was coming from the \r\n, I tried a replace('\n', '\r\n') on the f.read() output but it didn't work.
Also, when using https://httpbin.org to check the POST request, I get \\\\r\\\\n for each new line in the pdf binary data. I'm wondering if this is normal, maybe that could help.
Thank you in advance for your suggestions.
With requests it can be done a bit easier.
import requests
post_url = 'https://myposturl'
files = {'file': open('./myfile.pdf', 'rb')}
r = requests.post(post_url, files=files)
More docs.
The reason you get corrupted file is probably because you're setting headers and body manually. requests usually sets theses things implicitly, so you should not break this concept and follow official guides.
Related
I need to upload a pdf file through backend API using JMeter. So for that, I passed a multipart API request. To upload the file I am using BeanShell Preprocessor.
FileInputStream in = new FileInputStream("C:\\Users\\XYZ\\Downloads\\PT_003.pdf");
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
for (int i; (i = in.read(buffer)) != -1; ) {
bos.write(buffer, 0, i);
}
in.close();
byte[] binarydata = bos.toByteArray();
bos.close();
vars.put("binarydata", new String(binarydata));
Multipart Request Body :
--AaC07x
Content-Type: application/json; charset=utf-8
{
"token":"a6b8J000000055JQPU",
"flow":"Development"
}
--AaC07x
content-disposition: form-data; name="File"; filename="PT_003.pdf"
Content-Type: application/pdf
Content-Transfer-Encoding: bytecode
${binarydata}
--AaC07x--
Header Manager:-
Content-Type multipart/related;boundary="AaC07x"
File uploaded but content in that pdf file is missing means when I tried to open the uploaded pdf file it's blank/corrupted.
So can you please anyone help me to fix that issue??
It looks like your byte array to string conversion is the problem. Moreover as per JMeter 5.5 you cannot simply send a byte array using HTTP Request sampler as given you put everything into "Body Data" tab it will be treated as a String
Since JMeter 3.1 you should avoid using Beanshell, the recommended scripting language is Groovy
If you have problems coming up with a proper request definition you can just record it using HTTP(S) Test Script Recorder, just make sure that your PT_003.pdf is in JMeter's "bin" folder during both recording and replay
I am trying to migrate one of api test to karate framework. However I am unable to write the corrent step defined in karate documentation. Maybe I am missing some basic syntax..but could anyone have any idea how we write following steps in karate feature
requestPostDoc.header("x-api-key","FG6dcYHN9N7PYKfWCUlGo5QGTwZhv2Re1MrDSOTV");//New chnages
requestPostDoc.contentType("multipart/form-data").multiPart("part2-file",file).formParam("part1-json",objDocumentWrite.toJSONString());
requestPostDoc.queryParam("loadProperties",true); //New changes
responseForNewCaseDocFile=requestPostDoc.post("https://vrh0oox3hl.execute-api.eu-central-1.amazonaws.com/default/");//New changes
filterableRequestSpecification = (FilterableRequestSpecification) requestPostDoc;
filterableRequestSpecification.removeQueryParam("loadProperties");
I have written following feature file in karate:
Given url 'https://vrh0oox3hl.execute-api.eu-central-1.amazonaws.com/default/'
And header x-api-key = 'FG6dcYHN9N7PYKfWCUlGo5QGTwZhv2Re1MrDSOTV'
And header Authorization = 'Bearer ' + jwt
And param loadProperties = true
And multipart file info = { read: 'classpath:testData/documentWrite.json', filename: 'documentWrite.json' }
And multipart file Uploading = { read: 'classpath:testData/TextFile.txt', filename: 'TextFile.txt' }
When method post
Then print response
Then status 200
When I execute this test i am getting 400 response code
status code was: 400, expected: 200, response time in milliseconds: 252, url: https://vrh0oox3hl.execute-api.eu-central-1.amazonaws.com/default/?loadProperties=true, response:
Based on the cURL command in the comments, this is my best guess. The rest is up to your research. Read the docs and tweak the Content-Type and other sub-headers if needed. You need to figure this out depending on what your server wants: https://github.com/karatelabs/karate#multipart-file
* multipart file part1-json = { read: 'documentWrite.json' }
* multipart file part2-file = { read: 'TextFile.txt' }
For anyone coming across this question in the future and if you are stuck, get a friend if needed and go through this exercise together: https://github.com/karatelabs/karate/issues/1645#issuecomment-862502881
This stuff can be hard and needs time. There are no short cuts.
I tried few methods, not able to get it working.
Client side(React), I am sending a zip file as follows using POST,
const data = new FormData();
data.append('file', file);
data.append('filename', file.name);
let params = {
headers: {
'Content-Type': 'multipart/form-data'
},
body: data
};
Server side:(API Gateway/Lambda/Nodejs)
I added 'multipart/form-data' to Binary Media Type on Gateway side.
When parsing through lambda event.body is not well formed.
It looks like this:
{"body": "e30=",
"isBase64Encoded": true }
Any ideas what might be happening? Any takes on how to parse?
Although Ariz's answer is correct, I strongly recommend you to look into AWS Pre-Signed Upload URLs. It allows your clients to upload the file first to an AWS S3 Bucket, from where your lambda function can later access the object.
Especially when you're working with large binary files, the former approach can lead to a lot of problems (-> memory issues, which is sparse in Lambda).
I have written a short blog post about this in the past.
you are getting base64 encoded data, following is one of the ways to decode.
However it's an empty object.
var base64 = 'e30='
var decodedData = Buffer.from(base64, 'base64').toString();
console.log(decodedData)
I created an API function to work with S3. I imported the template swagger. After deployment, I tested with a Node.js project by the npm module aws-api-gateway-client.
It works well with: get bucket lists, get bucket info, get one item, put a bucket, put a plain text object, however I am blocked with put a binary file.
firstly, I ensure ACL is allowed with all permissions on S3. secondly, binary support also added
image/gif
application/octet-stream
The code snippet is as below. The behaviors are:
1) after invokeAPI, the callback function is never hit, after sometime, the Node.js project did not respond. no any error message. The file size (such as an image) is very small.
2) with only two times, the uploading seemed to work, but the result file size is bigger (around 2M bigger) than the original file, so the file is corrupt.
Could you help me out? Thank you!
var filepathname = './items/';
var filename = 'image1.png';
fs.stat(filepathname+filename, function (err, stats) {
var fileSize = stats.size ;
fs.readFile(filepathname+filename,'binary',function(err,data){
var len = data.length;
console.log('file len' + len);
var pathTemplate = '/my-test-bucket/' +filename ;
var method = 'PUT';
var params = {
folder: '',
item:''
};
var additionalParams = {
headers: {
'Content-Type': 'application/octet-stream',
//'Content-Type': 'image/gif',
'Content-Length': len
}
};
var result1 = apigClient.invokeApi(params,pathTemplate,method,additionalParams,data)
.then(function(result){
//never hit :(
console.log(result);
}).catch( function(result){
//never hit :(
console.log(result);
});;
});
});
We encountered the same problem. API Gateway is meant for limited data (10MB as of now), limits shown here,
http://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html
Self Signed URL to S3:
Create an S3 self signed URL for POST from the lambda or the endpoint where you are trying to post.
How do I put object to amazon s3 using presigned url?
Now POST the image directly to S3.
Presigned POST:
Apart from posting the image if you want to post additional properties, you can post it in multi-form format as well.
http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#createPresignedPost-property
If you want to process the file after delivering to S3, you can create a trigger from S3 upon creation and process with your Lambda or anypoint that need to process.
Hope it helps.
Hello im trying to send a multipart post request and using jython with commons httpclient 4.3.1.
http_client = DefaultHttpClient()
http_post = HttpPost(url)
bin = FileBody(File(file_name), ContentType.APPLICATION_XML)
me = MultipartEntity()
me.addPart('datei', bin)
http_post.setEntity(me)
print "Executing Post Request:", http_post.getRequestLine()
http_response = http_client.execute(http_post)
result_entity = http_response.getEntity()
return EntityUtils.toString(result_entity)
In my opinion the return value should a str with the content, but it is:
"type 'org.apache.http.conn.BasicManagedEntity'"
Where is my mistake?
I do not know the exact answer to your question. And today I have been trying to make a multipart post work from my jython script for a skiuli project I am working on.
Here is how I was able to do it.
First get ez_setup.py from: http://peak.telecommunity.com/dist/ez_setup.py
Then run jython ez_setup.py from the cmd prompt
Second get poster from: http://atlee.ca/software/poster/dist/0.8.1/
Unzip the file
Then run jython setup.py install
Last you can follow this sample code or see more on the atlee site:
from poster.encode import multipart_encode
from poster.streaminghttp import register_openers
import urllib2
register_openers()
upload_url = "http://your.website.com/upload-multipart.php"
#this will upload a file and a json object
#in php the file is in $_FILES and the json is in $_POST
params = {'file': open("test.txt", "rb"), 'jsonData': '[{"foo":"bar", "foo1":"bar1"}]'}
datagen, headers = multipart_encode(params)
request = urllib2.Request(upload_url, datagen, headers)
result = urllib2.urlopen(request)
print result.read()
I hope this helps someone out there. This example will also work for post multipart in python.