GET Bucket op response + AWS S3 + Content-Length header - amazon-s3

Just wanted to know if the GET Bucket op response ever skips the Content-Length header. I tested this and i saw that there was no Content-Length header in the response for GET Bucket op.
How does an application reading the response understand where the body of the response ends if the response doesn't contain Content-Length header?
Request-Response Snippet:
GET /?max-keys=1000&prefix&delimiter=%2F HTTP/1.1
Date: Sat, 09 Apr 2016 18:27:23 GMT
x-amz-request-payer: requester
Authorization: AWS AKIAIP3KAUILC4GG7A2A:UG3bGvIjayrxrkxEX1mfrvETy/M=
Connection: Keep-Alive
User-Agent: Cyberduck/4.9.19632 (Mac OS X/10.10.5) (x86_64)
HTTP/1.1 200 OK
x-amz-id-2: yg76HSq5j0mi0oR6dXF8ZfGq722kHBWiMQmNvXPqiLxr1S4nGj5GVn1RVrPQrOUfNynxxaMSYEY=
x-amz-request-id: B4468E68E10B6AEF
Date: Sat, 09 Apr 2016 18:27:25 GMT
x-amz-bucket-region: us-east-1
Content-Type: application/xml
Server: AmazonS3
Connection: close
<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">......</ListBucketResult>
Thanks!

The Content-Length header is optional in response. And it may not reflect the real content-length even if it presents. Think about gzipped response. So to answer the question: When no Content-Length is received, the client keeps reading until the server closes the connection.
In Java, keep calling InputStream.read() until it returns -1.
Is the Content-Length header required for a HTTP/1.0 response?

Related

Jmeter not showing up proper response instead giving details of server and connection details

I am using Jmeter 5.4.1 version, my API is of oauth1.0 type. When I ran my api through postman , it gave my proper json response for example an proper id, but the same api when ran through jmeter gives 200 response code but giving details of server and connection in response body and not the reponse that is expected(a proper id).
Below is the response :
HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Wed, 12 May 2021 12:33:10 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Vary: Accept-Encoding
Set-Cookie: PHPSESSID=eqvp0l22u2jo30moqn194meugp; expires=Wed, 12-May-2021 13:33:10 GMT; Max-Age=3600; path=/; domain=dev.moorup.no; HttpOnly
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Pragma: no-cache
X-Frame-Options: SAMEORIGIN
Cache-Control: no-store
enter image description here
You're looking at Response Headers tab of the View Results Tree listener therefore you're seeing the HTTP Response Headers
Just switch to Response Body tab and you will be able to see "raw" HTML Response and several options of rendering it:
Also be aware that it is possible to convert your Postman scripts to JMeter, for OAuth you will still have to do some correlation, but for the main logic record and replay should work more or less fine

How to fetch HTTP response body with lua?

I need to parse retreived text from HTTP GET request. I am using Lua on NodeMCU and I am not very familiar with it.
I am using script to fetch response and to split it one row at a time using this script
local nStart, nEnd = string.find(c, "\n\n")
if (nEnde == nil) then
nStart, nEnd = string.find(c, "\r\n\r\n")
end
c = string.sub(c,nEnd+1)
print("length: "..string.len(c))
data = mysplit(c, "\n") -- fill the field with filenames
HTTP GET request looks like this
GET /lua/node.php?id=4022029&list HTTP/1.1
Host: mydomain.com
Accept: */*
User-Agent: Mozilla/4.0 (compatible; esp8266 Lua;)
When I print HTTP response, it looks like this
HTTP/1.1 200 OK
Date: Tue, 28 Nov 2017 01:05:12 GMT
Server: Apache
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=23p8rtds43pd1662ncm5cjhrl3; path=/
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8
When I type mydomain.com/lua/node.php?id=4022029&list, I get a list of files as a text one bellow other, but this script is not fetching anything. It looks like there is no body. What am I missing here?
Update,
My script is working when fetching data from HTTP which is not encoding data chunked, but I am not able to fetch it from chunked encoding.
Transfer-Encoding: chunked
The server you are connecting to is using chunked encoding, which means the header should be followed by one or more chunks that consist of chunk length and the content. It looks like either you haven't finished reading the content or the library you are using doesn't handle the chunked content for some reason.

How to test with api-designer (RAML editor)?

Just like postman does, does api-designer support that?
I try to POST my request to my service (127.0.0.1:8180) and if I click 'API is behind a firewall', it POST http://127.0.0.1:8180/v1/device,and the response is:
Status
0
Headers
If uncheck 'API is behind a firewall', it POST /proxy/http://127.0.0.1:8180/v1/device, and the response is
Status
404
Headers
connection:
keep-alive
content-length:
51
content-type:
text/html; charset=utf-8
date:
Fri, 09 Sep 2016 14:24:06 GMT
x-content-type-options:
nosniff
x-powered-by:
Express
Body
Cannot POST /proxy/http://127.0.0.1:8180/v1/device
1
Cannot POST /proxy/http://127.0.0.1:8180/v1/device
CozyAzure.
Unfortenly, api-designer has several problems with the proxy. I didn't spend much time for the issue and started to use postman collections for this. You can Importing RAML folders as Collections and make a request with postman.

Fiddler doesn't decompress gzip responses

I use Fiddler to debug my application. Whenever the response is compressed by server, instead of decompressed response, Fiddler shows unreadable binary data:
/* Response to my request (POST) */
HTTP/1.1 200 OK
Server: xyz.com
Date: Tue, 07 Jun 2011 22:22:21 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.3.3
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Encoding: gzip
14
����������������
0
How can I get the response decompressed?
I use fiddler version 2.3.4.4 and just noticed that in the Inspectors tab ("Raw" sub-tab), above the response section (in case of gzip-ed response), appears "Response is encoded and may need to be decoded before inspection. Click here to transform."
If you click on that, the response becomes readable.
The settings are pretty much the default, I just installed Fiddler and did not change anything.
If you don't want to have to click per response as in the accepted answer, using the menu, click Rules -> Remove All Encodings.
From the fiddler faq
Q: I like to navigate around a site then do a "search" for a text on all the logged request/responses. I was curious if Fiddler automatically decompressed gzipped responses during search?
A: Fiddler does not decompress during searches by default, since it would need to keep both the compressed and decompressed body in memory (for data integrity reasons).
In current versions of Fiddler, you can tick the "Decode Compressed Content" checkbox on the Find dialog.
Here is a link to the site
http://www.fiddler2.com/fiddler/help/faq.asp

Cliqset.com Error 403 while trying to update any data through API

I used both Fiddler and my test C# application to update my profile information in Cliqset.com through their API but I always get 403 error. It works well when I just want to get my profile info, not to update. I use all necessary HTTP headers, like CliqsetConsumerKey and Authorization.
Thank you.
When I try to get my status, I use this:
GET https://api.cliqset.com/user/cosmorocket/status HTTP/1.1
Authorization: Basic Y**********rZXQ6ZnVsY3J1bTA=
CliqsetConsumerKey: 11**********3c2ccd26c4ca946e3d8e
Host: api.cliqset.com
And it returns this:
HTTP/1.1 200 OK
Date: Fri, 10 Jul 2009 05:17:37 GMT
Server: Apache-Coyote/1.1
X-Powered-By: Servlet 2.4; JBoss-4.2.3.GA (build: SVNTag=JBoss_4_2_3_GA date=200807181417)/JBossWeb-2.0
Content-Type: application/xml
Content-Length: 153
Set-Cookie: csp=appInst2; path=/
Connection: close
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><StatusEntry><Updated>2009-06-09T18:09:29.315Z</Updated><Status>cliqsetting all day long</Status></StatusEntry>
But if I use the same credentials for updating with this request:
PUT https://api.cliqset.com/200905/user/cosmorocket/status HTTP/1.1
I get this:
HTTP/1.1 403 Forbidden
Date: Fri, 10 Jul 2009 05:25:04 GMT
Vary: Accept-Encoding
Content-Length: 9
Connection: close
Content-Type: text/html; charset=iso-8859-1
Error 403
I contacted tech support and solved the problem. Each time a new user uses my registered application it's necessary to make an authenticated GET
request to 'https://api.cliqset.com/200905/user/useridentifier'. Notice that you don't change 'useridentifier' to any other login name, just leave it as it is. This
will setup the appropriate permissions to update data.
I did this request and now it works well.