My website serves gzipped content. I verified with Firebug and YSlow. However, JMeter does not request the gzipped content. Therefore, it gets all uncompressed content. As a result, my test cases take much longer (6-10x longer) than they do in reality.
How can I make JMeter request gzipped content from a website?
FYI, I am using the latest stable build: JMeter 2.3.4 r785646.
Add an HTTP Header Manager to the Thread Group in your Test Plan.
Add the name-value pair:
Name: Accept-Encoding
Value: gzip,deflate,sdch
This will ensure that all JMeter requests use HTTP compression.
To verify:
Add this Listener to the Thread Group: View the Results Tree
Run your test plan
View the Sampler result tab for one of the webpages.
Do you see these name-value pairs?
Content-Encoding: gzip
Vary: Accept-Encoding
Transfer-Encoding: chunked
If yes, then you've successfully setup gzip requests in JMeter. Congrats.
Another way to verify is in the Summary Report stats:
You'll see that the Avg Bytes values are the uncompressed sizes. That's OK. For whatever reason, that's how JMeter works. Pay attention to the KB/sec column. That will show an improvement of 6-10x with gzip enabled.
Related
I am trying to set up JMeter 5.5 to upload a file into our system as if it were happening in the user interface, but I continue to get a 502 proxy error. How can I get the file to upload successfully?
I am using the parameter hivUpload. If I change the parameter from hivUpload to anything else, I get a 500 error instead of a 502, so I think hivUpload is the correct parameter.
I have the file in the JMeter bin folder, but it's not clear if I need to include the full file path or not. I have tried it both ways, and neither has been successful.
Setup
Results
Source file location
Request
POST https://cdc-ew.lutherhq1b.int/rest/v1/upload
POST data:
--s6kF9JKRBTVi1qsnV4rm1hbf6gd6HMeH62
Content-Disposition: form-data; name="hivUpload"; filename="AgencyInfo 2_0 - good file.xml"
Content-Type: application/xml
Content-Transfer-Encoding: binary
<actual file content, not shown here>
--s6kF9JKRBTVi1qsnV4rm1hbf6gd6HMeH62--
[no cookies]
Response
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request<p>Reason: <strong>Error reading from remote server</strong></p></p>
</body></html>
I am able to get other processes to happen successfully by sending in a JSON request as if I were doing data entry. Note the successful green entries in the results, so I believe all of the other items are set up correctly. However, we have not tried using JMeter to upload files before. I know it is possible to do this I have seen articles about it (like this one https://artoftesting.com/fileuploadinjmeter#:~:text=File%20upload%20in%20JMeter%20will,checkbox%20in%20HTTP%20Request%20sampler.), but it's not clear to me what I need to tweak in my setup.
We cannot comment on your configuration because we don't know how exactly the file needs to be supplied as there are multiple ways of uploading the file.
You could try just recording it using JMeter's HTTP(S) Test Script Recorder, just make sure that the file you're uploading will be in "bin" folder of JMeter installation, only this way JMeter will be able to intercept the request and generate proper HTTP Request sampler and HTTP Header Manager
More information: JMeter Performance Testing: Upload and Download Scenarios
I'm sending a request to server service called path-generator which gives me a generated url and I'm uploading images to this url which moves the images to s3 bucket.
I'm able to upload the file to the bucket, but it arrives as broken image (when i'm uploading the file with 'Accept: application/json, text/plain' header)
or as 'Content-Transfer-Encoding: binary' when not using the header
The requests:
With header:
Connection: keep-alive
Content-type: image/png
Accept: application/json, text/plain
:
Content-Length: 201571
Host: {some host}
User-Agent: Apache-HttpClient/4.5.6 (Java/11.0.1)
without header:
Connection: keep-alive
Content-type: application/json
Content-Length: 221702
Host: {some host}
User-Agent: Apache-HttpClient/4.5.6 (Java/11.0.1)
I'm using the exact same flow as the client so it must be something wrong I'm doing with Jmeter
When you tick Use multipart/form-data box JMeter doesn't use Content-Type header specified in the HTTP Header Manager, most probably this is the reason for your request failure.
Try recording the file upload request using HTTP(S) Test Script Recorder (make sure to copy the file to "bin" folder of your JMeter installation) to see if JMeter is capable of properly capture the upload request(s). If it is - you should be good to go. If not - you will have to amend JMeter configuration to 100% match request specification, check out Testing REST API File Uploads in JMeter article for example test plan.
S3 PUT requests only need file content and no extra fields.
Do not pass parameter name and MIME type, only pass filePath correctly. If required add header Content-Type: image/jpg or video/mp4 in case it's a video. Similarly for pdf, text, etc.
Additionally, when you download the broken file and open it in notepad++
along with the actual file which was used to upload in notepad++
you can see the difference: the broken file has some extra text in it. If you remove it, it will work as expected
Also do not try this is notepad, use notepad++ only.
The newest version of Safari (mobile & desktop) buffers videos 4x slower than other browsers because it sends many small sized range-bytes requests opposed to a few large ones. An example request and response is below (this request continues with a small size of 64kb until enough data is loaded for the video to play, in Chrome, Firefox and other browsers the range-bytes request is much larger and so the data is delivered much faster in one stream).
Is it possible to get around this issue by forcing my web server (apache) to ignore Safari's small range-byte request of 64kb, and instead send a larger amount of data (about 5MB)? The request is made directly to the video file.
Summary
URL: http://example.org/video.mp4?rand=942824
Status: 206 Partial Content
Source: Network
Request
GET /video.mp4 HTTP/1.1
Accept: */*
Connection: keep-alive
Range: bytes=0-65535
Accept-Encoding: identity
Response
HTTP/1.1 206 Partial Content
Content-Type: video/mp4
Content-Range: bytes 0-65535/467342440
Accept-Ranges: 0-467342440
Content-Length: 65536
Connection: keep-alive
Server: nginx/1.2.1
UPDATE: I managed to change the request range header using the below code, however even though the 5mb is downloaded quickly, safari continues sending these small 64kb range requests and ignores the 5mb that was downloaded so this is not a solution.
SetEnvIf Range bytes=0-65535 HAVE_MyRequestHeader
RequestHeader unset Range env=HAVE_MyRequestHeader
RequestHeader set Range bytes=0-5000000 env=HAVE_MyRequestHeader
No. You can not change it server side. The client makes a request the server fulfills the request. Sending data the client didn’t ask for will likely cause errors.
Please note: This is not a complain about a shoddy CMS.
Just toying with Apache Bench and got terrible results with our custom CMS, more exactly i got:
Requests per second: 0.37 [#/sec] (mean)
When i run another test with a plain php file i got:
Requests per second: 4786.07 [#/sec] (mean)
Another test with a previous version of the CMS:
Requests per second: 6068.66 [#/sec] (mean)
The website(s) are working fine, no problems detected, Google's Webmaster Tools reports our sites as faster than 80% of the pages which is fine, i think.
The test was:
ab -t 30 -c 10 http://example.com/
Maybe some kind of Apache problem? Bad .htaccess config, or similar?
Update:
Just ran a simple test with sockets and the results are similar. Page loads very, very slowly. If i ran my script with another website everything is fine.
Also, there's a small hint about a chunk length problem. (Bad Apache Headers, or line endings?)
The site is gzipped, and when verbose logging turned on, i see these lines in the response:
LOG: Response code = 200
LOG: header received:
HTTP/1.1 200 OK
Date: Tue, 04 Oct 2011 13:10:49 GMT
Server: Apache
Set-Cookie: PHPSESSID=ibnfoqir9fee2koirfl5mhm633; path=/
Expires: Sat, 26 Jul 1997 05:00:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Cache-Control: post-check=0, pre-check=0
Vary: Accept-Encoding
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
2ef6
Always at the same place, in the middle of the HTML-source, then <!DOCTYPE HTML> again.
Please, help.
Update #2:
Just checked my HTTP headers with Rex Swain's HTTP Viewer and got these results:
HTTP/1.1·200·OK(CR)(LF)
Date:·Wed,·05·Oct·2011·08:33:51·GMT(CR)(LF)
Server:·Apache(CR)(LF)
Set-Cookie:·PHPSESSID=n88g3qcvv9p6irm1fo0qfse8m2;·path=/(CR)(LF)
Expires:·Sat,·26·Jul·1997·05:00:00·GMT(CR)(LF)
Cache-Control:·no-store,·no-cache,·must-revalidate(CR)(LF)
Pragma:·no-cache(CR)(LF)
Cache-Control:·post-check=0,·pre-check=0(CR)(LF)
Vary:·Accept-Encoding(CR)(LF)
Connection:·close(CR)(LF)
Transfer-Encoding:·chunked(CR)(LF)
Content-Type:·text/html;·charset=UTF-8(CR)(LF)
(CR)(LF)
Do you notice anything unusual?
If it works well with ordinary web browsers (as you mentioned in the comments) the CMS handle the requests from Apache Benchmark differently.
A quick checklist:
AFAIK Apache Benchmark just send simple requests without any cookie handling, so try to set -C with a valid cookie (copy the values from a web browser).
Try to send exactly the same headers to the CMS as the web browser sends. Save a dump of a valid request with netcat, HttpFox or a packet sniffer and set the missing headers with -H.
Profile the CMS on the server while you're sending to it a request with Apache Benchmark. Maybe you found the bottleneck. Two poor man's error_log calls with a timestamp in the first and the last line of the index.php (or the tested script's entry point) could show how fast is the PHP script and help to calculate the overhead of the Apache HTTP Server and network.
If you run socket tests and browser tests from different machines it's could be a DNS issue (turn off HostnameLookups in Apache). Try to run them from the same machine.
Try ab -k ... or ab -H "Connection: close" ....
I guess the CMS does some costly initialization when it initializes the session and it's happens when it processes the first request. Since Apache Benchmark does not send the cookies back the CMS it creates a new session for every request and it's the cause of the slow answers.
A second guess is that the CMS handle the incoming http headers differently and the headers which was sent (or the lack of them) by Apache Benchmark trigger some costly/slow processing. It looks more appropriate since the report of the Google's Webmaster Tools.
Apache Benchmark sends HTTP 1.0 request, for example:
GET / HTTP/1.0
Host: localhost:9100
User-Agent: ApacheBench/2.3
Accept: */*
It looks to me that your server does not send any http header about Keep-Alive settings but it assumes that the client uses keep-alive when the client uses HTTP 1.0. It's not an RFC compliant behaviour:
From RFC 2616, 19.6.2 Compatibility with HTTP/1.0 Persistent Connections:
Some clients and servers might wish to be compatible with some
previous implementations of persistent connections in HTTP/1.0
clients and servers. Persistent connections in HTTP/1.0 are
explicitly negotiated as they are not the default behavior.
By default Apache Benchmark doesn't use keep-alive so it waits when the response arrives for the closing of the socket. The server closes it after 15 seconds idle. Downloading the main page with wget also takes 15 seconds. Wget also uses HTTP 1.0 in the request.
I think it's a bug in the PHP code of the CMS since ab works well on the same server with a plain php file. Anyway, you can workaround it with using keep-alive connections (-k):
ab -k -t 30 -c 10 http://example.com/
or with explicitly disabling persistent connections:
ab -H "Connection: close" -t 30 -c 10 http://example.com/
but it's still a server side issue and your original ab commands is right.
Please note that this bug probably affects only HTTP 1.0 clients (like Apache Benchmark, wget) and clients with regular browsers will not notice it.
How to enable glass fish compression? I enabled compression in http-lister properties
but no changed response
Login to admin console: localhost:4848
Go to the Network Config > Network Listener
Select the listener for which you want to enable gzip > HTTP Tab
Check if you have min compression size set. I don't remember from the top of my head if glassfish has a default min compression size. Pretty much if the resource does not exceed this value in size, it won't be compressed.
Check if you have correct compressableMimeType set. application/xml is not the same as text/xml, even if they're really both XMLs.
Responses to HTTP requests in version 1.0 are not compressed. You must send your requests in HTTP 1.1 to get gzipped responses from your glassfish server.
More over, you must add the header "Accept-Encoding: gzip" in your http requests.