How to enable glass fish compression? I enabled compression in http-lister properties
but no changed response
Login to admin console: localhost:4848
Go to the Network Config > Network Listener
Select the listener for which you want to enable gzip > HTTP Tab
Check if you have min compression size set. I don't remember from the top of my head if glassfish has a default min compression size. Pretty much if the resource does not exceed this value in size, it won't be compressed.
Check if you have correct compressableMimeType set. application/xml is not the same as text/xml, even if they're really both XMLs.
Responses to HTTP requests in version 1.0 are not compressed. You must send your requests in HTTP 1.1 to get gzipped responses from your glassfish server.
More over, you must add the header "Accept-Encoding: gzip" in your http requests.
Related
I am testing a REST API service that supports json and gzip data. I have tested the response from the json request payload. How to test if it handles gzip data properly? Please help?
Basically the client sends a request with accept-encoding as gzip in the header, service handles the compression and the client takes care of the decompression. But I need a way to confirm that service indeed handles compressed gzip data
gzip is basically a header + deflate + a checksum.
Gatling will retain the original Content-Encoding response header so you can check if the payload was gzipped, and then trust the gzip codec to do the checksum verification and throw an error if the payload was malformed.
I have hundreds of hardware devices at customers which need to send HTTP data through a telnet interface.
The destination is an Apache 2 Webserver with a PHP script waiting for the data.
This is already working however we found that the hardware involved is not able to handle hw-flow-control, this means that once data is filled (around 250 bytes) the buffer can overflow resulting in data corruption.
Fixing the HW-flow is not an option, the "modem" firmware is closed and can not be modified by the vendor anymore as it's quite old hardware.
Normally we'd use this:
POST / HTTP/1.1
Host: api.server
User-Agent: P8
Content-Type: application/x-www-form-urlencoded
Accept: */*
Content-Length: 767
VARIABLE=URLENCODED_DATA(total length 767 bytes)
This would work perfectly fine with flow-control, but in my case the 767 bytes are too much.
After around 200 bytes buffers would be overwritten and some bytes are lost.
The only current way to get it working now was using a delay when sending to the "modem" so it can empty it's buffers in time. However in the field this will not work due to instable internet connections with unpredictable timings.
I am not an expert in HTTP, I just hope it is possible to fragment a package.
I thought about using "Connection: keep-alive" or something similar.
My main question:
Is there a way to send POST data ($VARIABLE) to a Apache 2 server in smaller chunks in a way that makes the HTTP server combine them to one stream internally ?
Pseudo code:
POST / HTTP/1.1
Host: api.server
User-Agent: P8
Content-Type: application/x-www-form-urlencoded
Accept: */*
Content-Length: 400
Connection: keep-alive
VARIABLE=URLENCODED_DATA(200 bytes)
END\n\n
Server responds in TCP stream once received with "OK".
Next chunk is sent:
VARIABLE=URLENCODED_DATA(200 bytes)
Connection is closed.
As 400 bytes have been reached the process is ready, Apache forwards VARIABLE to PHP scripts POST input.
So like a HTTP flow-control within an open TCP connection.
Maybe there is a HTTP feature which is built for that purpose, or something that can be "ab"used to act in that way. keep-alive was just a guess.
If current HTTP protocols do not have such a feature the only way I can think about solving my issue is to implement flow-control on PHP side.
I hope for a better way than that though.
Update:
Meanwhile I found two interesting parameters:
Expect: 100-continue
Transfer-Encoding: chunked
What I would need is a mix of both.
A chunked transfer encoding which is expecting a 100-continue after each chunk !
This is a very interesting question, and it really has nothing to do with HTTP but with TCP.
The way to solve this is to use an intermediary proxy that takes care of spoon-feeding your devices. Ideally, this device will be able to set the window size on the TCP packet ACKs to whatever the size of the buffer the device is. That window size will close to zero when the device cannot handle any more. If you do this, you will be utilizing TCP's built-in flow control and solve the problem in a simple way.
Another thing you can do is keep this entirely in the application layer and have this intermediary proxy buffer all of the data from the response. For most normal HTTP responses this will be okay.
Varnish will hold compressed obj in Cache, but when client don't support gzip.
How does Varnish deal with it? Does it hold the other uncompressed obj in Cache too, or decompress the compressed obj?
Varnish 3.0 supports Gzip as mentioned in the "Compression" chapter of the official tutorial. All HTTP requests to the backend will include a request for gzipped content, so by default all objects are stored in memory gzipped.
If the backend does not support gzip, you can ask Varnish to compress the response before storing it by setting beresp.do_gzip in vcl_fetch.
If a request comes in from a client that does not support gzip, Varnish will gunzip the stored object before delivering it.
Varnish 2.x does not compress or de-compress: if the client supports Gzip, it will hold a gzipped version of the page in cache. If the client does not support it, another copy will be placed in the cache for the plain-content, without compression.
So yes: depending on the Accept-Encoding header (which should be normalized), multiple version of a page will be held in cache for each supported compression algoritme.
*Updated: editted for clarity, the above is for Varnish 2.0 or 2.1 only. Varnish 3.x supports gzip, as explained above.
Please note: This is not a complain about a shoddy CMS.
Just toying with Apache Bench and got terrible results with our custom CMS, more exactly i got:
Requests per second: 0.37 [#/sec] (mean)
When i run another test with a plain php file i got:
Requests per second: 4786.07 [#/sec] (mean)
Another test with a previous version of the CMS:
Requests per second: 6068.66 [#/sec] (mean)
The website(s) are working fine, no problems detected, Google's Webmaster Tools reports our sites as faster than 80% of the pages which is fine, i think.
The test was:
ab -t 30 -c 10 http://example.com/
Maybe some kind of Apache problem? Bad .htaccess config, or similar?
Update:
Just ran a simple test with sockets and the results are similar. Page loads very, very slowly. If i ran my script with another website everything is fine.
Also, there's a small hint about a chunk length problem. (Bad Apache Headers, or line endings?)
The site is gzipped, and when verbose logging turned on, i see these lines in the response:
LOG: Response code = 200
LOG: header received:
HTTP/1.1 200 OK
Date: Tue, 04 Oct 2011 13:10:49 GMT
Server: Apache
Set-Cookie: PHPSESSID=ibnfoqir9fee2koirfl5mhm633; path=/
Expires: Sat, 26 Jul 1997 05:00:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Cache-Control: post-check=0, pre-check=0
Vary: Accept-Encoding
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
2ef6
Always at the same place, in the middle of the HTML-source, then <!DOCTYPE HTML> again.
Please, help.
Update #2:
Just checked my HTTP headers with Rex Swain's HTTP Viewer and got these results:
HTTP/1.1·200·OK(CR)(LF)
Date:·Wed,·05·Oct·2011·08:33:51·GMT(CR)(LF)
Server:·Apache(CR)(LF)
Set-Cookie:·PHPSESSID=n88g3qcvv9p6irm1fo0qfse8m2;·path=/(CR)(LF)
Expires:·Sat,·26·Jul·1997·05:00:00·GMT(CR)(LF)
Cache-Control:·no-store,·no-cache,·must-revalidate(CR)(LF)
Pragma:·no-cache(CR)(LF)
Cache-Control:·post-check=0,·pre-check=0(CR)(LF)
Vary:·Accept-Encoding(CR)(LF)
Connection:·close(CR)(LF)
Transfer-Encoding:·chunked(CR)(LF)
Content-Type:·text/html;·charset=UTF-8(CR)(LF)
(CR)(LF)
Do you notice anything unusual?
If it works well with ordinary web browsers (as you mentioned in the comments) the CMS handle the requests from Apache Benchmark differently.
A quick checklist:
AFAIK Apache Benchmark just send simple requests without any cookie handling, so try to set -C with a valid cookie (copy the values from a web browser).
Try to send exactly the same headers to the CMS as the web browser sends. Save a dump of a valid request with netcat, HttpFox or a packet sniffer and set the missing headers with -H.
Profile the CMS on the server while you're sending to it a request with Apache Benchmark. Maybe you found the bottleneck. Two poor man's error_log calls with a timestamp in the first and the last line of the index.php (or the tested script's entry point) could show how fast is the PHP script and help to calculate the overhead of the Apache HTTP Server and network.
If you run socket tests and browser tests from different machines it's could be a DNS issue (turn off HostnameLookups in Apache). Try to run them from the same machine.
Try ab -k ... or ab -H "Connection: close" ....
I guess the CMS does some costly initialization when it initializes the session and it's happens when it processes the first request. Since Apache Benchmark does not send the cookies back the CMS it creates a new session for every request and it's the cause of the slow answers.
A second guess is that the CMS handle the incoming http headers differently and the headers which was sent (or the lack of them) by Apache Benchmark trigger some costly/slow processing. It looks more appropriate since the report of the Google's Webmaster Tools.
Apache Benchmark sends HTTP 1.0 request, for example:
GET / HTTP/1.0
Host: localhost:9100
User-Agent: ApacheBench/2.3
Accept: */*
It looks to me that your server does not send any http header about Keep-Alive settings but it assumes that the client uses keep-alive when the client uses HTTP 1.0. It's not an RFC compliant behaviour:
From RFC 2616, 19.6.2 Compatibility with HTTP/1.0 Persistent Connections:
Some clients and servers might wish to be compatible with some
previous implementations of persistent connections in HTTP/1.0
clients and servers. Persistent connections in HTTP/1.0 are
explicitly negotiated as they are not the default behavior.
By default Apache Benchmark doesn't use keep-alive so it waits when the response arrives for the closing of the socket. The server closes it after 15 seconds idle. Downloading the main page with wget also takes 15 seconds. Wget also uses HTTP 1.0 in the request.
I think it's a bug in the PHP code of the CMS since ab works well on the same server with a plain php file. Anyway, you can workaround it with using keep-alive connections (-k):
ab -k -t 30 -c 10 http://example.com/
or with explicitly disabling persistent connections:
ab -H "Connection: close" -t 30 -c 10 http://example.com/
but it's still a server side issue and your original ab commands is right.
Please note that this bug probably affects only HTTP 1.0 clients (like Apache Benchmark, wget) and clients with regular browsers will not notice it.
My website serves gzipped content. I verified with Firebug and YSlow. However, JMeter does not request the gzipped content. Therefore, it gets all uncompressed content. As a result, my test cases take much longer (6-10x longer) than they do in reality.
How can I make JMeter request gzipped content from a website?
FYI, I am using the latest stable build: JMeter 2.3.4 r785646.
Add an HTTP Header Manager to the Thread Group in your Test Plan.
Add the name-value pair:
Name: Accept-Encoding
Value: gzip,deflate,sdch
This will ensure that all JMeter requests use HTTP compression.
To verify:
Add this Listener to the Thread Group: View the Results Tree
Run your test plan
View the Sampler result tab for one of the webpages.
Do you see these name-value pairs?
Content-Encoding: gzip
Vary: Accept-Encoding
Transfer-Encoding: chunked
If yes, then you've successfully setup gzip requests in JMeter. Congrats.
Another way to verify is in the Summary Report stats:
You'll see that the Avg Bytes values are the uncompressed sizes. That's OK. For whatever reason, that's how JMeter works. Pay attention to the KB/sec column. That will show an improvement of 6-10x with gzip enabled.