Jmeter - image uploaded to s3 as binary/broken image - amazon-s3

I'm sending a request to server service called path-generator which gives me a generated url and I'm uploading images to this url which moves the images to s3 bucket.
I'm able to upload the file to the bucket, but it arrives as broken image (when i'm uploading the file with 'Accept: application/json, text/plain' header)
or as 'Content-Transfer-Encoding: binary' when not using the header
The requests:
With header:
Connection: keep-alive
Content-type: image/png
Accept: application/json, text/plain
:
Content-Length: 201571
Host: {some host}
User-Agent: Apache-HttpClient/4.5.6 (Java/11.0.1)
without header:
Connection: keep-alive
Content-type: application/json
Content-Length: 221702
Host: {some host}
User-Agent: Apache-HttpClient/4.5.6 (Java/11.0.1)
I'm using the exact same flow as the client so it must be something wrong I'm doing with Jmeter

When you tick Use multipart/form-data box JMeter doesn't use Content-Type header specified in the HTTP Header Manager, most probably this is the reason for your request failure.
Try recording the file upload request using HTTP(S) Test Script Recorder (make sure to copy the file to "bin" folder of your JMeter installation) to see if JMeter is capable of properly capture the upload request(s). If it is - you should be good to go. If not - you will have to amend JMeter configuration to 100% match request specification, check out Testing REST API File Uploads in JMeter article for example test plan.

S3 PUT requests only need file content and no extra fields.
Do not pass parameter name and MIME type, only pass filePath correctly. If required add header Content-Type: image/jpg or video/mp4 in case it's a video. Similarly for pdf, text, etc.
Additionally, when you download the broken file and open it in notepad++
along with the actual file which was used to upload in notepad++
you can see the difference: the broken file has some extra text in it. If you remove it, it will work as expected
Also do not try this is notepad, use notepad++ only.

Related

How to send a csv file upload request from Jmeter

I have a request in which the user uploads a csv file and then the file is validated based on a few criteria. I have captured the request from blazemeter (refer the screenshot of the request captured)
But its failing when I hit this using Jmeter. (I am trying to send the exact same request along with the file attached in the "File Upload" option).
I've checked in Developer's tool and the actually request sent is :
------WebKitFormBoundaryBk6Qf9RTb4vFvkKw
Content-Disposition: form-data; name="file"; filename="EmployeeCSV.csv"
Content-Type: application/vnd.ms-excel
------WebKitFormBoundaryBk6Qf9RTb4vFvkKw
Content-Disposition: form-data; name="fileExtension"
csv
------WebKitFormBoundaryBk6Qf9RTb4vFvkKw
Content-Disposition: form-data; name="organisationId"
ZUVMbzBudFM5T2Y2bVNtUHFWWmZCRmJhWFZFTkJGMEVuNk9WKzBsQ2dUWT0=
------WebKitFormBoundaryBk6Qf9RTb4vFvkKw
Content-Disposition: form-data; name="schemeIdSf"
d0dqVXd5b25wVFArWTFkc3l0dUV5R3V0STlqZ2owVE5FVCtxVGhEZUxESmJydVAzYjBLSkxqdWxLMmQva29WSg==
------WebKitFormBoundaryBk6Qf9RTb4vFvkKw
Content-Disposition: form-data; name="fileGUID"
1612429088749
------WebKitFormBoundaryBk6Qf9RTb4vFvkKw--
Can someone help me out on how can I send the above mentioned request in Jmeter?
The HTTP Request sampler setup doesn't seem correct to me, you need to define the file(s) you're uploading under "Files Upload" tab like:
I would recommend re-recording your file upload event using JMeter's HTTP(S) Test Script Recorder, JMeter should be smart enough to generate proper HTTP Request sampler and HTTP Header Manager configuration.
Just make sure to copy your EmployeeCSV.csv to "bin" folder of your JMeter installation during recording, only this way JMeter will be able to properly capture and build the request. More information: Recording File Uploads with JMeter

Multipart File Upload failing with new IntelliJ HTTP Client

I´m trying to get a file upload to my JAX-RS controller working with multipart. I´ve been using the integrated REST HTTP Client of IntelliJ, that is recommending me to switch to the new HTTP Client by using .http files.
However, once I´m doing that I´m unable to do any uploads and I´m running into this exception
exception
java.io.IOException: RESTEASY007550: Unable to get boundary for multipart
request.http
POST http://localhost:8080/content/upload
Accept: */*
Cache-Control: no-cache
Accept-Encoding: multipart/form-data
--null
< path/to/my/file
--null--
What is the correct syntax for performing multipart uploads?
Here's my .http file. Worked.
POST http://127.0.0.1:8000/v1/dataSource/1/table/
Content-Type: multipart/form-data; boundary=WebAppBoundary
--WebAppBoundary
Content-Disposition: form-data; name="csv_files"; filename="insight_modeling_base_node.csv"
< /home/core/Code/work/KG_test/data/insight_modeling_base_node.csv
--WebAppBoundary--
In your case, try adding boundary=null. Or just copy what I pasted here and make some adaptation.

Why am I getting headers embedded to files uploaded to S3 using JMeter?

Whenever uploading a file (json) to S3 using JMeter's HTTP Request sampler the uploaded file contains the HTTP headers at the top of the file. I am using a signed URL (with actual values):
https://something-s3bucket-something.s3.eu-west-1.amazonaws.com/afda5939-c232-d746-06f7-68790abde85b-91e962d6-4643-8091-fab8-9d0f78f35810.jsonTemp?X-Amz-Expires=18000&x-amz-security-token=somethinglongandcomplicated&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=somethingspecific&X-Amz-Date=20190627T070453Z&X-Amz-SignedHeaders=host;x-amz-security-token&X-Amz-Signature=something
HTTP Request Settings:
Resulting json:
--Y0B3WMeM2M6xYSUHpjPUIj72y3xnO_pswRA12Oh
Content-Disposition: form-data; name="filename"; filename="500payslips.json"
Content-Type: binary/octet-stream
Content-Transfer-Encoding: binary
{
... json content ...
}
}
--Y0B3WMeM2M6xYSUHpjPUIj72y3xnO_pswRA12Oh--
I know I could add the s3 java libraries to JMeter and then code the upload but I would prefer to just simply use the HTTP Request sampler (if possible).
More of a workaround than a fix(?). Instead of sending the file as a multipart/form-data I just read the contents of the file into a variable and sent the content as part of the body of the request.

Issue faced while uploading file using JMeter (Getting error - Import could not start - Object reference not set to an instance of an object)

I'm facing issue while uploading a file through JMeter. When file was uploaded manually, it was successful. The file is CSV file and it's contents are: Mr.,Perf,Driver4,LIC1,10003,12/31/2025,12/31/2025,CA,USA
Request Captured in fiddler:
POST https://WebSiteName/MainPage/Drivers/UploadDrivers HTTP/1.1
Accept: text/html, application/xhtml+xml, */*
Referer: https://WebSiteName/MainPage/Drivers/Main
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko
Content-Type: multipart/form-data; boundary=---------------------------7e21cd231003b8
Accept-Encoding: gzip, deflate
Host: WebSiteName
Content-Length: 691
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: intercom-lou-zjtmncjn=1; __utma=1.1400002961.1527162610.1533116464.1533118526.10; __utmz=1.1527162610.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); MainPageusername=akalambe; MainPagerememberchkbox=true; __utmv=1.|1=FleetId=67468d0b-4f3d-4693-9d06-ed226d27198c=1^2=UserId=f0467301-c934-4354-87a1-d875f56359de=1^3=RegionName=FW02=1; __utmt=1; __utmb=1.2.10.1533118526; AWSELB=6703C92B1A36D2911BDDEF67F947B6D8FA09E46F7AB993BF5EEE818483244FFE1C5B9ED7D743F6F7966ACAFFC21671252299970FE39D531FCECB082F45AABB8B505FB5E6AA; __utmc=1; DirectorAuth=CDBE3FDB1C993ABF0A408A386AFEDC5DFA8B423E6B55F11100AC49D6DD5FB5D66721E8F3111D77E689ABAA72CF5D8E88A94C07F3F58B310CDCFD350A604038BF703448EA9E3E21F4F815E8288C781E64C6E1180FDF81426C1F3F24CF513E5C4026072351C5BEC06F0C28EBA57502937D551A7CEFA67BBAE5FF2C59137A33498B341ED0AD90B83919ACB0A1630B9C80AFC9992D7004C1DF9D7AE042420BE76EA2AB96D43A5E7F8956BDE18ACC8BE8AF813AE476A376F58C3750E3A02B8BC148006B427790F7AD8B7E88276D5CEAEC16D8624E7D087DB8D06AE7727DF4F71A6EB01798E119D71D13E8B6556F21CDB6B58D30FD0078D57E1E4C334F5600C72DA0A636F14529506298657A45E5BD009A556EDB6837BBEA1B26F71EF8304A4401F47B12785D5F; MainPageUserPreferredLocale=en-US; DirectorSession=oceg01jzjoqjq5iybq4mg3im
-----------------------------7e21cd231003b8
Content-Disposition: form-data; name="file"; filename="EntityImportTemplate.csv"
Content-Type: text/plain
Title,FirstName,LastName,LicenceClass,LicenceNo,ExpirationDate,LastPhysical,LicenceState(Abbreviated),"LicenceCountry(Abbreviations include: USA,CAN,MEX,AUS,NZ)"
Mr.,Perf,Driver4,LIC1,10003,12/31/2025,12/31/2025,CA,USA
-----------------------------7e21cd231003b8
Content-Disposition: form-data; name="Filename"
EntityImportTemplate.csv
-----------------------------7e21cd231003b8
Content-Disposition: form-data; name="OwnerId"
67468d0b-4f3d-4693-9d06-ed226d27198c
-----------------------------7e21cd231003b8—
Response Captured in fiddler:
{"Success":true,"ImportMessages":[{"ErrorDescription":"Driver verified","Tag":null,"TimeStampUtc":"/Date(1533123584180)/","MessageType":0,"Text":"Driver verified"}]}
But response in JMeter for same file upload is:
{"Success":false,"ImportMessages":[{"ErrorDescription":"Import could not start - Object reference not set to an instance of an object.","Tag":null,"TimeStampUtc":"/Date(1533717886829)/","MessageType":2,"Text":"Import could not start"}]}
JMeter HTTP Request
HTTP Header Manager
HTTP Cookie Manager
Result
The way of building your upload request is quite flaky, I would rather recommend using Files Upload tab of the HTTP Request sampler and specify your CSV file there like:
Also don't forget to tick Use multipart/form-data for POST box
In general the best way to build a file upload request with JMeter is just recording it with HTTP(S) Test Script Recorder
Copy your EntityImportTemplate.csv file to "bin" folder of your JMeter installation
Start HTTP(S) Test Script Recorder
Configure your browser to use JMeter as the proxy
Issue your upload request
The relevant HTTP Request sampler should be created under the Recording Controller
More information: Recording File Uploads with JMeter
Just go to Advanced Tab and select "Java" as implementation.I hope it will work correctly.
It mostly depends on how web application under test responds to Jmeter scripts.
For me, once I was trying to upload docx through Jmeter and it was uploaded to file share location but thrown an error - "File format is not supported" even with correct uploading API configurations as recommended.
File Path: filename.docx, Parameter name: class name of UI element used to upload document e.g. button to browse file from disk MIME TYPE: vnd.openxmlformats-officedocument.wordprocessingml.document Select 'Use multipart/form-data' It worked successfully without any error and un-corrupted document content with below configuration some how:
File Path: filename.docx, Parameter name: Blank (Yes.. do not provide any value in here for Parameter column just leave it blank) MIME TYPE: vnd.openxmlformats-officedocument.wordprocessingml.document Deslect 'Use multipart/form-data'

Attach a file (picture) to a conversation

I would like to be able to attach a file to a conversation, using the REST API. Is it possible? There is a «attachments» to /conversations/{convId}/messages/{itemId} but is that usable? How? The description of that field is not available.
The file API of Circuit supports the upload of attachments. As soon as you received your access token you can POST a message with the byte data. The following example wold upload a file with name test.jpg
POST /rest/v2/fileapi HTTP/1.1
Host: local.circuit.com
Authorization: Bearer <access token>
Content-Length: 100
Content-Disposition: attachment; filename="test.jpg"
Cache-Control: no-cache
<your content in binary form here>
Usually I am using Postman for my tests since it is very easy to use and supports OAuth 2.0 token generation (https://www.getpostman.com/)
You will receive an result that looks like
{"fileId":"fb211fd6-df53-4b82-824d-986dac47b3e7","attachmentId":"ZmIyMT..."}
If you want to validate your upload you can check it via
GET /rest/v2/fileapi?fileid=fb211fd6-df53-4b82-824d-986dac47b3e7 HTTP/1.1
Host: local.circuit.com
Authorization: Bearer <access token>
Cache-Control: no-cache
Well that was the easy part, now that you have uploaded the file to the backend you must attach it to a conversation item. Today we do not support UPDATE, i.e. you need to create a new one.
POST /rest/v2/conversations/<conv ID>/messages HTTP/1.1
Host: local.circuit.com
Authorization: Bearer <access token>
Content-Type: application/x-www-form-urlencoded
Cache-Control: no-cache
content=New+Text+Message&attachments=ZmIyMT...
You have to pass the generated attachment ID. After the execution of this requests the file is attached to the conversation.
If you skip the second step the file will not be linked to any conversation, is only accessible by the user who initiates the upload and will be deleted within the next 24 - 48h automatically.
Hope this helps, let me know if you have additional questions.