I got few questions regarding to gzip compression in my self hosted WCF service.
I implemented a custom gzip compression (using the msdn sample) and everything works o.k. (requests/responses are all well compressed).
currently the client config includes all the additional configurations for using the compression (bindingElementExtensions, policyImporters pointing the gzip encoder binary etc.).
Actually i thought that the client doesn't need any external configuration for using the service and only need to have "gzip" mentioned at the "Accept-Encoding" and "Content-Encoding"...
how do i configure my client to work that way ? i want my client to handle an encoded message if he gets one only...
Another question - is that possible to set a minimal size of message which the gzip compression will encode starting from that size only? the motivation is for optimizing the server's load and prevent a unneccessary compression of small messages.
or is it using the gzip compression or not using it at all?
Thanks in advance!!!
Related
We have moved to using PLupload for file uploads and found that it can support "chunked" file uploads. The problem is that our server sees one large file upload as multiple smaller files coming in multiple POST requests.
Does anybody know if Apache Commons FileUpload supports chunked uploads?
FWIW looking at the PLupload webpage the "Chunking" they are talking about is not "HTTP Chunking". http://www.plupload.com/index.php
Their marketing term "Chunking" is their concept of sending a large payload up in small and separate HTTP requests. The server is required to have logic to group, stitch up and verify all the small parts. You are better off getting help on their forum on this. There is no reason why this logic can not be created by you on the server side and maybe they have example Java code implementing it.
Useful info and pointer to their upload.php example (maybe you convert to Java and on top of Apache Commons FileUpload) :
http://www.plupload.com/punbb/viewtopic.php?id=1484
What you are observing the small segments of a file arriving like they are separate files is exactly how the "PLupload Chunking" mechanism works. This technique is not defined in any standard, but it is also not an uncommon solution to the problems it addresses.
The "HTTP Chunking" is standard for defining how to transfer a single HTTP Request (and/or HTTP Response) between click/server using a HTTP transfer encoding. This is supported by all webservers and all browsers and has been around for a long time (since HTTP/1.1).
I'm looking for an alternative to Amazon S3 for hosting static sites, but that allows GZIP compression depending on the Accept-Encoding header.
In other words, I'm looking for something that will return a different version of a file depending on the client's Accept-Encoding header. I don't mind uploading compressed and uncompressed files myself (it can be automated easily).
Besides that, the service needs to be capable of hosting websites (allows an index page and 404 page to be set). I don't need CNAME capabilities since I'll be using a CDN. I'd also like the capability to set caching headers (Expires, Last-Modified).
FYI, my startup, Site44, does static hosting by syncing with Dropbox. We support all of this except the customized caching headers, but we'd be open to that feature request. (Today we set the Cache-Control header using a heuristic based on the last modified date, with a maximum value of 24 hours.)
Rackspace Cloudfiles supports everything mentioned.
I was just doing some testing with YSlow and it's telling me:
Grade F on Compress components with gzip: There are 10 plain text
components that should be sent compressed
I know that Apache 1.3 uses mod_gzip while Apache 2.x uses mod_deflate, and so the easiest solution to remedy this is to use mod_deflate on an Apache 2 server.
However, I've checked with two shared hosting companies and one local company and they've all told me that they don't support mod_deflate.
I know that some older browsers have trouble accepting gzipped / deflated content, and I'm not suggesting it be enabled by default, but are there any negatives for making mod_deflate available? Is it just extra load on the server's processors?
Also, are there any alternatives? I saw that if you are using a CMS like Wordpress you could potentially install a caching plugin which would serve out gzipped cached versions of the pages initially generated via PHP.
Compression takes CPU time. Maybe the hosting company decided they care more about CPU than network traffic. Maybe they offer it with a more expensive package. MAybe they simply didn't add it. Only your hosting company would know.
When using PHP you can check whether your PHP setup has zlib support enabled. If that is the case you can use ob_start("ob_gzhandler"); in code to enable an output buffer which will compress your data or set zlib.output_compression in your php configuration for instance by using php_flag zlib.outout_compression on in your .htaccessfile.
http://php.net/ob_gzhandler
http://php.net/zlib.output-compression
I'm evaluating the front end performance of a secure (SSL) web app here at work and I'm wondering if it's possible to compress text files (html/css/javascript) over SSL. I've done some googling around but haven't found anything specifically related to SSL. If it's possible, is it even worth the extra CPU cycles since responses are also being encrypted? Would compressing responses hurt performance?
Also, I'm wanting to make sure we're keeping the SSL connection alive so we're not making SSL handshakes over and over. I'm not seeing Connection: Keep-Alive in the response headers. I do see Keep-Alive: 115 in the request headers but that's only keeping the connection alive for 115 milliseconds (seems like the app server is closing the connection after a single request is processed?) Wouldn't you want the server to be setting that response header for as long as the session inactivity timeout is?
I understand browsers don't cache SSL content to disk so we're serving the same files over and over and over on subsequent visits even though nothing has changed. The main optimization recommendations are reducing the number of http requests, minification, moving scripts to bottom, image optimization, possible domain sharding (though need to weigh the cost of another SSL handshake), things of that nature.
Yes, compression can be used over SSL; it takes place before the data is encrypted so can help over slow links. It should be noted that this is a bad idea: this also opens a vulnerability.
After the initial handshake, SSL is less of an overhead than many people think* - even if the client reconnects, there's a mechanism to continue existing sessions without renegotiating keys, resulting in less CPU usage and fewer round-trips.
Load balancers can screw with the continuation mechanism, though: if requests alternate between servers then more full handshakes are required, which can have a noticeable impact (~few hundred ms per request). Configure your load balancer to forward all requests from the same IP to the same app server.
Which app server are you using? If it can't be configured to use keep-alive, compress files and so on then consider putting it behind a reverse proxy that can (and while you're at it, relax the cache headers sent with static content - HttpWatchSupport's linked article has some useful hints on that front).
(*SSL hardware vendors will say things like "up to 5 times more CPU" but some chaps from Google reported that when Gmail went to SSL by default, it only accounted for ~1% CPU load)
You should probably never use TLS compression. Some user agents (at least Chrome) will disable it anyways.
You can selectively use HTTP compression
You can always minify
Let's talk about caching too
I am going to assume you are using an HTTPS Everywhere style web site.
Scenario:
Static content like css or js:
Use HTTP compression
Use minification
Long cache period (like a year)
etag is only marginally useful (due to long cache)
Include some sort of version number in the URL in your HTML pointing to this asset so you can cache-bust
HTML content with ZERO sensitive info (like an About Us page):
Use HTTP compression
Use HTML minification
Use a short cache period
Use etag
HTML content with ANY sensitive info (like a CSRF token or bank account number):
NO HTTP compression
Use HTML minification
Cache-Control: no-store, must-revalidate
etag is pointless here (due to revalidation)
some logic to redirect the page after session timeout (taking into account multiple tabs). If someone presses the browser's Back button, the sensitive info is not displayed due to the cache header.
You can use HTTP compression with sensitive data IF:
You never return user input in the response (got a search box? don't use HTTP compression)
Or you do return user input in the response but randomly pad the response
Using compression with SSL opens you up to vulnerabilities like BREACH, CRIME, or other chosen plain-text attacks.
You should disable compression as SSL/TLS have no way to currently mitigate against these length oracle attacks.
To your first question: SSL is working on a different layer than compression. In a sense these two are features of a web server that can work together and not overlap. Yes, by enabling compression you'll use more CPU on your server but have less of outgoing traffic. So it's more of a tradeoff.
To your second question: Keep-Alive behavior is really dependent on HTTP version. You could move your static content to a non-ssl server (may include images, movies, audio, etc)
Is it possible to send pre-compressed files that are contained within an EARfile? More specifically, the jsp and js files within the WAR file. I am using Apache HTTP as the web server and although it is simple to turn on the deflate module and set it up to use a pre-compressed version of the files, I would like to apply this to files that are contained within an EAR file that is deployed to JBoss. The reason being that the content is quite static and compressing it on the fly each time is quite costly in terms of cpu time.
Quite frankly, I am not entirely familiar with how JBoss deploys these EAR files and 'serves' them. The gist of what I want to do is pre-compress the files contained inside the war so that when they are requested they are sent back to the client with gzip for Content-Encoding.
In theory, you could compress them before packging them in the EAR, and then serve them up with a custom controller which adds the http header to the response which tells the client they're compressed, but that seems like a lot of effort to go to.
When you say that on-the-fly compression is quite costly, have you actually measured it? Have you tried requesting a large number of uncompressed pages, measured the cpu usage, then tied it again with compressed pages? I think you may be over-estimating the impact. It uses quite low-intensity stream compression, designed to use little CPU resources.
You need to be very sure that you have a real performance problem before going to such lengths to mitigate it.
I don't frequent this site often and I seem to have left this thread hanging. Sorry about that. I did succeed in getting compression to my javascript and css files. What I did was I precompress them in the ant build process using the gzip. I then had to spoof the name to get rid of the gzip extension. So I had foo.js and compressed it into foo.js.gzip. I renamed this foo.js.gzip to foo.js and this is the file that gets packaged into the WAR file. So that handles the precompression part. To get this file served up properly, we just have to tell the browser that this file is compressed, via the content-encoding header of the http response. This was done via a output filter that is applied to files that matched the *.js extension (some Java/JBoss, WEB-INF/web.xml if it helps. I'm not too familiar with this so sorry guys).