My original problem was that I can't get mod_deflate working despite ensuring it is properly installed and enabled and trying all possible .htaccess combinations. What is even more strange is that the document itself is gzipped while JS and CSS files aren't.
So this led to me another strange observation: response headers contain a line Server: nginx, even though I am convinced I am using Apache. How am I convinced? If I run service apache2 stop, the website is down. So it seems rather strange to me and I am curious why it happens and maybe it will help me to solve my original problem.
OS I am using is Debian 7, Plesk is used for configuration of Apache, and the website is running Laravel (which is of little significance I guess, but still).
I hope someone can help me, thanks in advance.
I use the same configuration as you with one of my servers. Nginx is used to manage the SSD cache after Apache.
You can change Nginx configuration with Plesk:
Websites & Domains > domain > Web Server Settings (Additional nginx directives).
I use Gzip with:
gzip on;
gzip_comp_level 9;
gzip_disable “MSIE [1-6]\.(?!.*SV1)”;
gzip_types text/plain text/html text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript text/x-js;
I am using mod_defalte, as so:
<IfModule mod_deflate.c>
AddOutputFilterByType DEFLATE text/html
AddOutputFilterByType DEFLATE text/css
AddOutputFilterByType DEFLATE application/javascript
AddOutputFilterByType DEFLATE image/svg+xml
AddOutputFilterByType DEFLATE image/x-icon
</IfModule>
I am wondering, but cannot find the answer to: Does the above mean all content that matches those rules will be gzipped to any request? Or does mod_defalte on compress when the HTTP request states it can accept gzip?
Further, I am reading some posts where people are disabling certain browsers with bugs in their gzip implementation. But there is no explanation for this. Does anyone have a definitive set of rules for this. Or is it not needed?
mod_deflate is capable of compressing using gzip encoding.
Sometimes the module skips certain files because they are either too small or thought to have no significant gain.
The request header tells the server whether to compress or not.
Most of the bugs are related to proxy server on the client side where gzipped content is cached because a browser that accepts the encoding requested a resource first, but other browsers behind the same cache cannot. This is the reason to use the Vary header.
We have an odd, intermittent error that occurs with mod_proxy_ajp, i.e. using apache as a front end to a tomcat server.
The error
User clicks on a link browser prompts
user to "save as...." (e.g. in
Firefox "You have chosen top open
thread.jsp which is a
application/octet-stream"...What
should firefox do with this file)
User says "Huh?" and presses "Cancel"
User clicks again on the same link
Browser displays the page correctly
This error occurs intermittently, but unfortunately rarely on our test server and frequently on production.
In firefox's LiveHttpHeaders I see the following in the above usecase:
first page download (i.e. click on link) is "text/plain"
second download is "text/html"
I thought the problem may stem from ProxyPassReverse (i.e. muddling up whether to use http or ajp), but all these proxypassreverse settings resulted in the same error:
ProxyPassReverse /ajp://localhost:8080/
ProxyPassReverse /pe http://localhost/pe
ProxyPassReverse /pe http://forumstest.company.com/pe
Additionally, I've checked the apache error logs (set to debug) and see no warnings or errors...
** But it works with mod_proxy_http ?? **
It appears that switching to mod_proxy_http 'solves' the problem. Limited testing, I have not been able to recreate the problem in the test environment.
Because the problem is intermittent, I'm not 100% sure that mod_proxy_http "solves" the problem
Environment
Apache 2.2 Windows
Jboss 4.2.2 back end (tomcat 6)
One other data point
For better or worse, a servlet filter in tomcat gzips the html before sending it to apache. (which means extra work as apache must unzip before it performs ProxyPassReverse's "find and replace"). I don't know if "gzip" messes up.
Questions
anyone seen this before?
what tools help analyze the cause?
thanks
Addendum 1: Here is the LiveHttpHeaders output
Browser Incorrectly sees html as "text/plain"
http://forums.customer.com/pe/action/forums/displaythread?rootPostID=10842016&channelID=1&portalPageId=1002
GET http://forums.customer.com/pe/action/forums/displaythread?rootPostID=10842016&channelID=1&portalPageId=1002 HTTP/1.1
Host: forums.customer.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 ( .NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Proxy-Connection: keep-alive
Cookie: __utma=156962862.829309431.1260304144.1297956514.1297958674.234; __utmz=156962862.1296760237.232.50.utmcsr=forumstest.customer.com|utmccn=(referral)|utmcmd=referral|utmcct=/pe/action/forums/displaythread; s_vi=[CS]v1|258F5B88051D3FC3-40000105C056085F[CE]; inqVital=xd|0^sesMgr|{"sID":4,"lsts":1292598007}^incMgr|{"id":"755563420055418864","group":"CHAT","ltt":1292598006741,"sid":"755563549194447187","igds":"1290627502757","exempt":false}^inq|{"customerID":"755562378269271622"}^saleMgr|{"state":"UNSOLD","qDat":{},"sDat":{}}; inqState=sLnd|1^Lnd|{"c":4,"flt":1274728016,"lldt":17869990,"pgs":{"201198":{"c":1,"flt":1274728016,"lldt":0},"0":{"c":3,"flt":1274845009,"lldt":17752997}},"pq":["0","0","0","201198"],"fsld":1274728016697}; adv_search_results_page=10; ep_beta=1; visitorID=57307059; JSESSIONID=6jXLNdHRDjR9Th3B5gvTVkw1dZLn1zvhvKLR2r4GTLjylHJgjY3Q!683274050; __utmc=156962862; JSESSIONID=6jXLNdHRDjR9Th3B5gvTVkw1dZLn1zvhvKLR2r4GTLjylHJgjY3Q!683274050; TLTHID=5CCA50304DE99E28DB79A7B3267D4231; TLTSID=9DFCDE8045B374AAB752CC98A30E8311; AreCookiesEnabled=1; s_cc=true; SC_LINKS=%5B%5BB%5D%5D; s_sq=%5B%5BB%5D%5D; __utmb=156962862.64.10.1297958674; memberexists=T; ev1=greywolf%20hdtv%20whmx
Cache-Control: max-age=0
HTTP/1.0 200 OK
Date: Thu, 17 Feb 2011 17:38:42 GMT
Content-Type: text/plain
X-Cache: MISS from samus.company.com
X-Cache-Lookup: MISS from samus.company.com:3128
Via: 1.0 samus.company.com:3128 (squid/2.6.STABLE20)
Proxy-Connection: close
----------------------------------------------------------
Browser Correctly sees html as "text/html"
http://forums.customer.com/pe/action/forums/displaythread?rootPostID=10842016&channelID=1&portalPageId=1002
GET http://forums.customer.com/pe/action/forums/displaythread?rootPostID=10842016&channelID=1&portalPageId=1002 HTTP/1.1
Host: forums.customer.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13 ( .NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Proxy-Connection: keep-alive
Cookie: __utma=156962862.829309431.1260304144.1297956514.1297958674.234; __utmz=156962862.1296760237.232.50.utmcsr=forumstest.customer.com|utmccn=(referral)|utmcmd=referral|utmcct=/pe/action/forums/displaythread; s_vi=[CS]v1|258F5B88051D3FC3-40000105C056085F[CE]; inqVital=xd|0^sesMgr|{"sID":4,"lsts":1292598007}^incMgr|{"id":"755563420055418864","group":"CHAT","ltt":1292598006741,"sid":"755563549194447187","igds":"1290627502757","exempt":false}^inq|{"customerID":"755562378269271622"}^saleMgr|{"state":"UNSOLD","qDat":{},"sDat":{}}; inqState=sLnd|1^Lnd|{"c":4,"flt":1274728016,"lldt":17869990,"pgs":{"201198":{"c":1,"flt":1274728016,"lldt":0},"0":{"c":3,"flt":1274845009,"lldt":17752997}},"pq":["0","0","0","201198"],"fsld":1274728016697}; adv_search_results_page=10; ep_beta=1; visitorID=57307059; JSESSIONID=6jXLNdHRDjR9Th3B5gvTVkw1dZLn1zvhvKLR2r4GTLjylHJgjY3Q!683274050; __utmc=156962862; JSESSIONID=6jXLNdHRDjR9Th3B5gvTVkw1dZLn1zvhvKLR2r4GTLjylHJgjY3Q!683274050; TLTHID=5CCA50304DE99E28DB79A7B3267D4231; TLTSID=9DFCDE8045B374AAB752CC98A30E8311; AreCookiesEnabled=1; s_cc=true; SC_LINKS=%5B%5BB%5D%5D; s_sq=%5B%5BB%5D%5D; __utmb=156962862.64.10.1297958674; memberexists=T; ev1=greywolf%20hdtv%20whmx
Cache-Control: max-age=0
HTTP/1.0 200 OK
Date: Thu, 17 Feb 2011 17:38:44 GMT
X-Powered-By: Servlet 2.4; JBoss-4.2.1.GA (build: SVNTag=JBoss_4_2_1_GA date=200707131605)/Tomcat-5.5
Content-Encoding: gzip
Content-Type: text/html;charset=UTF-8
Content-Length: 24739
X-Cache: MISS from samus.company.com
X-Cache-Lookup: MISS from samus.company.com:3128
Via: 1.0 samus.company.com:3128 (squid/2.6.STABLE20)
Proxy-Connection: keep-alive
----------------------------------------------------------
Addendum 2: Additional Information
The browser did receive the "gzipped" file. I had earlier clicked "save as..." when a few of these errors occurred. Gunzip successfully processed the files and converted them to html.
Answer is here: How to preserve the Content-Type header of a Tomcat HTTP response sent through an AJP connector to Apache using mod_proxy
set DefaultType to None in apache configuration.
# DefaultType: the default MIME type the server will use for a document
# if it cannot otherwise determine one, such as from filename extensions.
# If your server contains mostly text or HTML documents, "text/plain" is
# a good value. If most of your content is binary, such as applications
# or images, you may want to use "application/octet-stream" instead to
# keep browsers from trying to display binary files as though they are
# text.
#
DefaultType None
The first page download is of mime type "application/octet-stream". May be the zipped stream is being sent back to the browser? (You can confirm by saving the file and looking inside it.)
It would be really helpful to post the Live HTTP Header request / response traces for problematic and normal cases. If the content is same in both cases - the response HTML - may be forcing the mime type to be text/html (using ForceType directive) for that particular context can fix the issue.
If it turns out to be the case that gzipped content is being sent to the browser in the problematic case - then digging deeper would be necessary. Is this browser specific - only happens with Firefox or happens with all browsers?
Ok, based on the new information provided - looks like Squid is caching the problematic response and not sending the right headers back to the client. So the browsers are doing the right thing here - without the Content-Encoding and right Content-Type they cannot do much else.
Missing response headers for the problematic request :
X-Powered-By: Servlet 2.4; JBoss-4.2.1.GA (build: SVNTag=JBoss_4_2_1_GA date=200707131605)/Tomcat-5.5
Content-Encoding: gzip
Content-Type: text/html;charset=UTF-8
X-Powered-By : It sounds like the problematic requests don't actually hit your JBoss server. Can you verify this by checking access logs? Is Squid caching the response and then sending it back as text/plain?
Reconfiguring Squid to not cache that particular URL should help - see http://www.lirmm.fr/doc/Doc/FAQ/FAQ-7.html section 7.8 for example (which is specific to Squid 2 but later versions should have similar capabilities.)
Seems like a content-negotiation problem. Apache is guessing the content type using the "magic" byte and setting the content type incorrectly. That explains why it happens intermittently. Try disabling mod_negotiation and see what happens. See http://httpd.apache.org/docs/2.0/content-negotiation.html for more info.
I saw the following in your settings
ProxyPassReverse /ajp://localhost:8080/
But port 8080 is not ajp port. The default ajp port is 8009. Could this be your problem?
There is most likely something wrong with your web application, not Apache. If your web app sends back the correct Content-Type, Apache will gladly forward it to the client. No content negotiation will be done in that case. If you do not return any Content-Type, Apache will almost surely substitute text/plain, which is not what you want.
Test your web app without Apache in the middle, make sure that it sends back the correct Content-Type.
It uses to be when apache serves secure content in a non secure channel.
trying to enable gzip compression for javascripts (jquery libs) I have on my site.
I do have enabled deflate in Apache's httpd.config file, and I have added next lines in .htaccess:
RewriteEngine On
AddOutputFilterByType DEFLATE text/html text/plain text/javascript
But, when I check with Google's page speed web performance tool, it gives me information that js is not compressed.
Can you tell me what I do wrong and how can I enable gzip compression for my web app?
Thank you in advance!
Javascript isn't text/javascript, it is application/javascript.
EDIT I have found that the problem is actually php minify. This was sending the deflated content instead of Apache. I'll find more on this.
According to High Performance Web Sites, if I enable mod_deflate in Apache 2.x, by adding the following line, it should send gzipped/delfated content: -
AddOutputFilterByType DEFLATE text/html text/css application/x-javascript
The book also says that gzip is more effective than deflate.
I have enabled in httpd.conf by adding the same line. But Apache sends Content-Encoding: deflate.
I tested with CURL using: -
curl -i -H "Accept-Encoding: gzip" "http://192.168.1.33/s.js" >> e:\curl_log.txt
It returns 'gzipped' content. But when I send the command: -
curl -i -H "Accept-Encoding: gzip, deflate" "http://192.168.1.33/s.js" >> e:\curl_log.txt
It returns 'deflated' content.
So, if the browser supports both deflated and gzipped, Apache send deflated. How to tell Apache to prefer gzip over deflate?
FYI: -
I could not find anything in:
http://httpd.apache.org/docs/2.0/mod/mod_deflate.html.
There is no occurrence of no-gzip in
the Apache conf.
Server: Apache/2.2.9 (Win32) PHP/5.2.6
FF sends request header as: "Accept-Encoding: gzip, deflate"
As I see the cause was already found. To further on help getting out of possible confusions:
mod_deflate despite its name is currently only supporting gzip.
gzip is more "effective" because of the following
deflate - despite its name the zlib compression (RFC 1950) should be used (in combination with the deflate compression (RFC 1951)) as described in the RFC 2616. The implementation in the real world however seems to vary between the zlib compression and the (raw) deflate compression[3][4]. Due to this confusion, gzip has positioned itself as the more reliable default method (March 2011).
gzip and zlib are file/stream formats that by default wrap around deflate and amongst other things add a checksum which make them more secure and a little slower. The increase in size on the other hand should not be of any concern.
Also see HTTP_compression - Wikipedia
deflate would be more "efficient" (Frequently Asked Questions about zlib - What's the difference between the "gzip" and "deflate" HTTP 1.1 encodings)
See http://httpd.apache.org/docs/2.0/mod/mod_deflate.html for all the gory details -- are you sure you don't have occurrences of no-gzip elsewhere in the configuration? What happens as you vary your "browser", e.g. by using wget with various values for -U?
I suspect whatever you're using to test is not sending ...
Accept-Encoding: gzip
... in the request header.