Varnish not ignoring subdomain despite vcl rules - apache

I am running a basic lamp server with apache on port 80, and varnish on port 81. I am attempting to exclude a subdomain of the primary site entirely, however I have had no luck in doing so thus far, and I'm not sure why.
As you can see below, I have a rule in place to A) skip logged in users on the subdomain, and B) skip the subdomain entirely. Neither of these seem to work however. Is there something wrong with my vcl configuration?
backend default {
.host = "my.server.ip.address";
.port = "80";
}
sub vcl_recv {
call identify_device;
# Allow the back-end to serve up stale content if it is responding slowly.
set req.grace = 2m;
# Always cache the following file types for all users.
if ( req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js|html|htm)(\?[a-z0-9]+)?$" ) {
unset req.http.cookie;
}
# Don't serve cached pages to logged in users
if ( req.http.cookie ~ "wordpress_logged_in" || req.url ~ "vaultpress=true" ) {
return( pass );
}
#Lets skip the logged in users on subdomain too!
if ( req.http.cookie ~ "dmr_user" ) {
return (pass);
}
#skip subdomain.domain.com
if (req.http.host ~ "subdomain.domain.com") {
return (pass);
}
#Following for WooCommerce and comments
if (req.url ~ "^/(cart|my-account|checkout|addons|wp-comments-post)") {
return (pass);
}
#Lets skip the logged in users on entries too!
if ( req.http.cookie ~ "dmr_user" ) {
return (pass);
}
if ( req.url ~ "\?add-to-cart=" ) {
return (pass);
}
# Drop any cookies sent to WordPress.
if ( ! ( req.url ~ "wp-(login|admin)" ) ) {
unset req.http.cookie;
}
}
sub vcl_fetch {
if (beresp.ttl < 180s) {
set beresp.ttl = 180s;
}
if (!(req.url ~ "wp-(login|admin)")) {
unset beresp.http.set-cookie;
}
}
sub vcl_hash {
hash_data(req.http.X-Device);
}
sub vcl_deliver {
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT";
} else {
set resp.http.X-Cache = "MISS";
}
}

You are only skipping the processing of the subdomain halfway into your handling, ie instructions are executed in order. Moving the skip domain check directly at the top of sub vcl_recv should ensure no other rules gets executed against requests to that subdomain.

Well guys, it turns out that what I needed was to use pipe instead of pass.
#skip subdomain.domain.com
if (req.http.host ~ "subdomain.domain.com") {
return (pass);
}
is now
#skip subdomain.domain.com
if (req.http.host ~ "subdomain.domain.com") {
return (pipe);
}
I also went ahead and moved it up to the top of the config. Altogether it works like a charm now. Thanks to everyone for their help!

Related

Redirect HTTPS on multidomain Varnish

i have got two domain based on same framework (magento2)
domain1.it
domain2.com
I would like to redirect them to their respective SSL version.
https://domain1.it
https://domain2.com
Domain 1 is correctly configured to redirect to HTTPS and my varnish Config file is:
sub vcl_recv {
if ( (req.http.host ~ "^(?i)www.domain1.it" || req.http.host ~ "^(?i)domain1.it") && req.http.X-Forwarded-Proto !~ "(?i)https") {
return (synth(750, ""));
}
sub vcl_synth {
if (resp.status == 750) {
set resp.status = 301;
set resp.http.Location = "https://domain1.it" + req.url;
return(deliver);
}
the problem is the synth always redirect to the same domain.
I should add an if condition where i could call a subroutines that redirect to https for domain2
For the love of everything that is good, please stop using otherworldly status codes, 301 and 302 are perfectly fine, clearer and save you a line.
I would advise against using x-forwarded-proto and use an SSL/TLS terminator that supports the PROXY protocol, but since this is what you have, here you go:
sub vcl_recv {
if (req.http.X-Forwarded-Proto !~ "https") {
set req.http.location = "https://" + req.http.host + req.url;
return(synth(301));
}
}
sub vcl_synth {
if (resp.status == 301 || resp.status == 302) {
set resp.http.location = req.http.location;
return (deliver);
}
}
relevant link: https://info.varnish-software.com/blog/rewriting-urls-with-varnish-redirection
Bitnami Engineer here. I just reviewed the Varnish documentation and found this:
sub vcl_recv {
if (client.ip != "127.0.0.1" && std.port(server.ip) == 80 && req.http.host ~ "^(?i)example.com") {
set req.http.x-redir = "https://" + req.http.host + req.url;
return(synth(850, "Moved permanently"));
}
}
sub vcl_synth {
if (resp.status == 850) {
set resp.http.Location = req.http.x-redir;
set resp.status = 302;
return (deliver);
}
}
This is useful when you want to redirect the clients to an SSL-version of your site. More info here:
https://varnish-cache.org/trac/wiki/VCLExampleRedirectInVCL

varnish or squid reverse proxy behind corp proxy

How to configure varnish(or if someone can hint me to squid im fine) to cache requests from backend, but connect to backend through http_proxy
So I try to:
backend default {
.host = "10.1.1.1";
.port = "8080";
}
backend corp_proxy {
.host = "proxy";
.port = "8080";
}
sub vcl_recv {
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
set req.backend_hint = corp_proxy;
set req.url ="http://" + req.http.host + req.url;
}
Varnish (or other web caching proxies) caches a request, based on its cache-related headers (like Cache-Control).
Unfortunately, many web applications don't set these headers correctly. So we should use a more aggressive approach to cache some well-known items, e.g., pictures, .js or .css files.
Moreover, this line set req.url ="http://" + req.http.host + req.url; is not required, because Varnish sends the request, as is, to your designated backend.
Here is my recommended config:
backend corp_proxy {
.host = "proxy";
.port = "8080";
}
sub vcl_recv {
// Determine backend
if ( req.http.host ~ ".example.com" ) {
set req.backend_hint = corp_proxy;
// Determine cacheable items
if( req.url ~ "\.(css|js|jpg|jpeg|png|gif|ico) {
unset req.http.Cookie;
unset req.http.Cache-Control;
}
}
}
sub vcl_backend_response {
if( bereq.http.host ~ ".example.com" ) {
if (bereq.url ~ "\.(css|jsjpg|jpeg|png|gif|ico)") {
set beresp.ttl = 20m; // I opt for 20 minutes of caching by your mileage may vary
}
}

How to exclude virtual host from varnish 3?

I want to exclude a single virtual host from varnish 3 config
Cache: [www].domain.tld
Dont Cache: host.domain.tld
### default.vcl
backend foo { .host = "domain.tld"; .port = "8880"; }
backend bar { .host = "host.domain.tld"; .port = "8880";}
# ...
if (req.url == "host.domain.tld") { set req.backend = bar; }
if (req.url == "host.domain.tld") { return(pass); }
if (req.http.Cookie && req.url == "host.domain.tld") { return(pass); }
# ...
### httpd.conf
Listen 8880
<VirtualHost vhost.domain.tld:8880>
DocumentRoot /var/www/foo/
ServerName vhost.doman.tld
</VirtualHost>
The request never reaches the virtual host. I guess the problem is that I request on port 80 and the host listens on port 8880.
What I can do to fix this?
req.url doesn't contain the domain. You want something like this instead:
sub vcl_recv {
if (req.http.host == "host.domain.tld") {
set req.backend = bar;
return (pass);
}
}
This will set the backend for any request going to the domain "host.domain.tld" to "bar" and pass directly to the backend (bypassing the cache).

Nginx $server_name in if statement

It seems like this is not working:
server_name blabla.bla;
location ~* (wp-comments-posts|wp-login)\.php$ {
if ($http_referer !~ ^(http://$servername) ) {
return 405;
}
}
While
server_name blabla.bla;
location ~* (wp-comments-posts|wp-login)\.php$ {
if ($http_referer !~ ^(http://blabla.bla) ) {
return 405;
}
}
works just fine. Is this expected and if so why? Or am I doing something wrong here?
Regular expressions are compiled while reading configuration, thus they cannot contain variables.
Also please note:
http://wiki.nginx.org/IfIsEvil
http://nginx.org/en/docs/http/ngx_http_referer_module.html
If you have the referer module you might like this one, this will ONLY allow the current server names to be valid referrers. All others will return as 405 error.
location ~* (wp-comments-post)\.php$ {
valid_referers server_names;
if ( $invalid_referer ) {
return 405;
}
### Do your stuff here
}

gzip compression using varnish cache

Im trying to provide gzip compression using varnish cache. But when I set content-encoding as gzip using my below mentioned configuration for varnish (default.vcl). Browser failed to download those content for which i set content-encoding as gzipped.
Varnish configuration file:
backend default {
.host = "127.0.0.1";
.port = "9000";
}
backend socketIO {
.host = "127.0.0.1";
.port = "8083";
}
acl purge {
"127.0.0.1";
"192.168.15.0"/24;
}
sub vcl_fetch {
/* If the request is for pictures, javascript, css, etc */
if (req.url ~ "^/public/" || req.url ~ "\.js"){
unset req.http.cookie;
set beresp.http.Content-Encoding= "gzip";
set beresp.ttl = 86400s;
set beresp.http.Cache-Control = "public, max-age=3600";
/*set the expires time to response header*/
set beresp.http.expires=beresp.ttl;
/* marker for vcl_deliver to reset Age: */
set beresp.http.magicmarker = "1";
}
if (!beresp.cacheable) {
return (pass);
}
return (deliver);
}
sub vcl_deliver {
if (resp.http.magicmarker) {
/* Remove the magic marker */
unset resp.http.magicmarker;
/* By definition we have a fresh object */
set resp.http.age = "0";
}
if(obj.hits > 0) {
set resp.http.X-Varnish-Cache = "HIT";
}else {
set resp.http.X-Varnish-Cache = "MISS";
}
return (deliver);
}
sub vcl_recv {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For ", " client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
if (req.request != "GET" &&
req.request != "HEAD" &&
req.request != "PUT" &&
req.request != "POST" &&
req.request != "TRACE" &&
req.request != "OPTIONS" &&
req.request != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}
# Pass requests that are not GET or HEAD
if (req.request != "GET" && req.request != "HEAD") {
return(pass);
}
#pipe websocket connections directly to Node.js
if (req.http.Upgrade ~ "(?i)websocket") {
set req.backend = socketIO;
return (pipe);
}
# Properly handle different encoding types
if (req.http.Accept-Encoding) {
if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|js|css)$") {
# No point in compressing these
remove req.http.Accept-Encoding;
} elsif (req.http.Accept-Encoding ~ "gzip") {
set req.http.Accept-Encoding = "gzip";
} elsif (req.http.Accept-Encoding ~ "deflate") {
set req.http.Accept-Encoding = "deflate";
} else {
# unkown algorithm
remove req.http.Accept-Encoding;
}
}
# allow PURGE from localhost and 192.168.15...
if (req.request == "PURGE") {
if (!client.ip ~ purge) {
error 405 "Not allowed.";
}
return (lookup);
}
return (lookup);
}
sub vcl_hit {
if (req.request == "PURGE") {
purge_url(req.url);
error 200 "Purged.";
}
}
sub vcl_miss {
if (req.request == "PURGE") {
purge_url(req.url);
error 200 "Purged.";
}
}
sub vcl_pipe {
if (req.http.upgrade) {
set bereq.http.upgrade = req.http.upgrade;
}
}
Response Header:
Cache-Control:public, max-age=3600
Connection:keep-alive
Content-Encoding:gzip
Content-Length:11520
Content-Type:application/javascript
Date:Fri, 06 Apr 2012 04:53:41 GMT
ETag:"1330493670000--987570445"
Last-Modified:Wed, 29 Feb 2012 05:34:30 GMT
Server:Play! Framework;1.2.x-localbuild;dev
Via:1.1 varnish
X-Varnish:118464579 118464571
X-Varnish-Cache:HIT
age:0
expires:86400.000
Any suggestion on how to fix it and how to provide gzip compression using varnish.
Varnish Cache 3.0 does most of the handling of Accept-Encoding automatically and you shouldn't mess with it.
Basically, if you want Varnish to compress an object just set beresp.do_gzip in vcl_fetch and it will compress it before storing it in cache. Uncompression happens automatically when needed.
The content-length seems to be wrong, try un-setting it. Also, why are you using the beresp instead of obj in vcl_fetch?
...
obj.http.Content-Encoding="gzip";
remove obj.http.Content-Length;
...
Check and see if you are with apache, and have mod_deflate enabled,
try disabling it.