CSS is not always gzipped why? - gzip

Question

In my Firefox or Chrome if I check the HTTP header the result are always with Content-Encoding: gzip. But I have customers reporting that they see "transfer-encoding: chunked" instead and the request are not gzipped.
http://www.example.com/public/css/style.min.css
If I or the customer do a gzip compression online check it's confirmed gzip is active.
https://checkgzipcompression.com = gzip!
But if I use a checker like this one. http://onlinecurl.com/
I also get the transfer-encoding: chunked
Request:
GET /style/css.css HTTP/1.1
Host: www.example.com
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
User-Agent: ...
Accept: /
Referer: http://www.example.com/
Accept-Encoding: gzip, deflate
Accept-Language: ...
Cookie: ...
Response:
HTTP/1.1 200 OK
Age: 532948
cache-control: public, max-age=604800
Content-Type: text/css
Date: Wed, 28 Jun 2017 12:35:07 GMT
ETag: "5349e8d595dfd21:0"
Last-Modified: Wed, 07 Jun 2017 13:56:17 GMT
Server: Microsoft-IIS/7.5
Vary: X-UA,Accept-Encoding, User-Agent
X-Cache: HIT
X-Cache-Hits: 6327
X-CacheReason: Static-js-css.
X-Powered-By: ASP.NET
X-Served-By: ip-xxx-xxx-xxx-xx.name.xxx
x-stale: true
X-UA-Device: pc
X-Varnish: 993020034 905795837
X-Varnish-beresp-grace: 43200.000
X-Varnish-beresp-status: 200
X-Varnish-beresp-ttl: 604800.000
transfer-encoding: chunked
Connection: keep-alive
Why are some requests not gzipped, when it should, this is my Varnish config (the part relevant for gzip):
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|flv|swf)$") {
# No point in compressing these
remove req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
# unkown algorithm
remove req.http.Accept-Encoding;
}
}
# Enabling GZIP
if (beresp.http.Content-Type ~ "(text/css|application/x-javascript|application/javascript)") {
set beresp.do_gzip = true;
}
if (beresp.http.Content-Encoding ~ "gzip" ) {
if (beresp.http.Content-Length == "0") {
unset beresp.http.Content-Encoding;
}
}
set beresp.http.Vary = regsub(beresp.http.Vary, "(?i)^(.*?)X-Forwarded-URI,?(.*)$", "\1\2");
set beresp.http.Vary = regsub(beresp.http.Vary, "(?i)^(.*?)User-Agent,?(.*)$", "\1\2");
set beresp.http.Vary = regsub(beresp.http.Vary, "^(.*?),?$", "X-UA,\1");
set beresp.http.Vary = regsub(beresp.http.Vary, "^(.*?),?$", "\1");
Any ideas, thank you.

Answer 1

Responses will only be gzipped if the request indicates that it can accept a gzipped response. This is indicated by the Accept-Encoding header in the request. So perhaps your online curl is not sending that header. It may be the same for your clients who are seeing this. You really have customers who are reporting that they are not getting responses gzipped?
Update
Ah, I see what you're doing now. Are you using a recent version of Varnish? There's no need to do all this yourself now. Varnish handles it all natively. All you need to do is set do_gzip to on for the content types where you want it, and Varnish takes care of the rest, including the Accept-Encoding header. See the documentation here.
So just remove all of your gzip/encoding related code except the part directly under # Enabling GZIP:
# Enabling GZIP
if (beresp.http.Content-Type ~ "(text/css|application/x-javascript|application/javascript)") {
set beresp.do_gzip = true;
}
And that will probably get everything working. It works fine for me that way. The best amount of VCL is as little as possible, Varnish is very good at handling things itself. Don't forget to restart Varnish or otherwise clear the cache for this site after making the change.
In case it's useful, I use the following VCL for this:
if (
beresp.status == 200
&& beresp.http.content-type ~ "\b((text/(html|plain|css|javascript|xml|xsl))|(application/(javascript|xml|xhtml\+xml)))\b"
) {
set beresp.do_gzip = true;
}
Which checks for more content types that can benefit from compression, including HTML. I don't bother with application/x-javascript as it's ancient and not used.
On another note, are you sure you need to be modifying the Vary header in the way that you are doing there?