We are running NGINX in front of our backend server.
We are attempting to enable the proxy_cache_background_update feature to allow NGINX to async updates to the cache and serve STALE content while it does this.
However, we are noticing that it still delivers STALE content slowly as if it's not serving from the cache. The time it takes after an item expires is very slow and clearly not served from cache - you can tell it's going to the backend server, getting an update and delivering it to the client.
Here is our configuration from NGINX:
proxy_cache_revalidate on;
proxy_ignore_headers Expires;
proxy_cache_background_update on;
Our backend server is delivering the following headers:
HTTP/1.1 200 OK
Date: Thu, 28 Feb 2019 21:07:09 GMT
Server: Apache
Cache-Control: max-age=1800, stale-while-revalidate=604800
Content-Type: text/html; charset=UTF-8
When attempting an expired page fetch we do notice the following header:
X-Cache: STALE
However, when providing this response it is very slow as if it's contacted the backend server and done it in realtime.
NGINX version:
$ nginx -v
nginx version: nginx/1.15.9
Any suggestions, tips and config changes are greatly appreciated.
When I send a http request to my couchdb server like it is shown in the docs here CouchDB Proxy Authentication, it doesn't give the response shown in the docs, just empty data. What am I doing wrong?
Also, am I able to start a session with this Proxy Auth? If I try a POST /_session, I get 500 error code.
GET /_session HTTP/1.1
Host: 127.0.0.2:5984
User-Agent: curl/7.51.0
Accept: application/json
Content-Type: application/json; charset=utf-8
X-Auth-CouchDB-UserName: john
X-Auth-CouchDB-Roles: blogger
< HTTP/1.1 200 OK
< Cache-Control: must-revalidate
< Content-Length: 132
< Content-Type: application/json
< Date: Sun, 06 Nov 2016 01:10:58 GMT
< Server: CouchDB/2.0.0 (Erlang OTP/17)
<
{"ok":true,"userCtx":{"name":null,"roles":[]},"info":{"authentication_db":"_users","authentication_handlers":["cookie","default"]}}
I found in the CouchDB issue tracker that the Proxy Authentication is broken in version 2.0.0. Either that or the docs aren't updated to indicate that it only works with clusters or something. I changed back to version 1.6.1 and everything works fine. I must say that the documentation for how Proxy Authentication works is very poor.
How it works is you need your third party authentication server to have the "[couch_httpd_auth] secret" and when a client authenticates, you need to generate a HMAC-SHA1 token by combining the username and secret. Then, on any http requests you make from the client to the CouchDB server, if you include all the headers:
X-Auth-CouchDB-Roles
X-Auth-CouchDB-UserName
X-Auth-CouchDB-Token
that request will be authenticated as a user client.
Also, it is not mentioned in the docs, but POST on the /_session API using these headers does nothing.
It's not the Proxy Authentication itself which is broken in CouchDB 2.0, it's just that in the current release there's no way to configure the authentication handlers like there was in the old 1.6 days.
There are some patches mentioned in the issue tracker which add proxy authentication to the list of authentication handlers. Furthermore there was a pull request which was accepted and merged which brings back configurability to CouchDB 2.0.
However in order to take advantage of those I'm afraid you either have to wait until the next release, or build CouchDB 2.0 yourself from the sources.
Proxy authentication is fixed as of CouchDB 2.1.1. The latest (>2.1.1) documentation shows how to configure proxy authentication again, along with the important proxy_use_secret option.
My iOS app makes Rest calls to my WCF web service.
The responding speed is very slow, over 3 min.
However, when I set up Fiddler as a Proxy to monitor the iOS traffic. The call was finished in 1 sec.
What does make Fiddler magically accelerate the Rest call from iOS?
p.s. Fiddler is setup on a windows PC where uses the same network with iOS App.
The rest call example (from Fiddler)
Request
GET https://xxxx.xxxx.com/Deals HTTP/1.1
Host: xxx.xxxx.com
Proxy-Connection: keep-alive
Accept-Encoding: gzip
Content-Type: application/json
Cookie: ASPXAUTH=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Connection: keep-alive
User-Agent: Natural xxxx x.x.x (iPad; iPhone OS 7.0.2; en_US)
Response
HTTP/1.1 200 OK
Cache-Control: private
Content-Length: 891437
Content-Type: application/json; charset=utf-8
Server: Microsoft-IIS/7.5
LastFetchDateTimeUTC: 2014-02-14T16:52:43.5465273Z
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Fri, 14 Feb 2014 16:52:45 GMT
Response body is a large json (2MB)
p.s.
Except for Fiddler, we also tried to install wireshark and use it to capture traffic on the mac while running the app from on the simulator.
We see a lot DUP ACK, I guess that's causing tcp re-transmission
p.s.
We pinged from iOS too, there is no delay to the WCF web service.
Help!
UPDATE:
We found out a problem, looks like the respond time decreases with the length of the body. Does it mean anything?
The WireShark logs should provide you plenty of information about what happens in each case. When Fiddler "magically" makes things faster, it's typically due to:
Better connection reuse (e.g. Fiddler may reuse connections better than client)
Better buffer sizes (e.g. not using tiny buffers for read/write)
Non-broken proxy determination behavior
I wrote a bit about these in this blog post.
We solved this problem by proving that server is a shitty one. We deployed the same service on another VM and it works. Must be the a broken network card
We have an Tomcat server where we're trying to log the HTTP version which the response is sent with. We've seen a few times that it seems to be HTTP/0.9, which kills the content (not supported I guess?). We would like to get some stats on this by using the access log in apache. However, since the header line for this isn't prefixed by anything, we cannot use the %{xxx}o logging.
Is there a way to get this?
An example:
Response is:
HTTP/1.1 503 This application is not currently available
Server: Apache-Coyote/1.1
Content-Type: text/html;charset=utf-8
Content-Length: 1090
Date: Wed, 12 May 2010 12:53:16 GMT
Connection: close
And we'd like the catch HTTP/1.1 (alternatively, HTTP/1.1 503 This application is not currently available.
Is this possible? We do not have access to the application being served, so we need to do this either as a Java filter, or in the tomcat access log - Preferably in the access log.
Enabling the <Valve className="org.apache.catalina.valves.RequestDumperValve"/> in server.xml writes out the request and response headers for each request.
Example:
19-May-2010 12:26:18 org.apache.catalina.valves.RequestDumperValve invoke
INFO: protocol=HTTP/1.1
I need to either find a file in which the version is encoded or a way of polling it across the web so it reveals its version. The server is running at a host who will not provide me command line access, although I can browse the install location via FTP.
I have tried HEAD and do not get a version number reported.
If I try a missing page to get a 404 it is intercepted, and a stock page is returned which has no server information on it. I guess that points to the server being hardened.
Still no closer...
I put a PHP file up as suggested, but I can't browse to it and can't quite figure out the URL path that would load it. In any case I am getting plenty of access denied messages and the same stock 404 page. I am taking some comfort from knowing that the server is quite robustly protected.
The method
Connect to port 80 on the host and send it
HEAD / HTTP/1.0
This needs to be followed by carriage-return + line-feed twice
You'll get back something like this
HTTP/1.1 200 OK
Date: Fri, 03 Oct 2008 12:39:43 GMT
Server: Apache/2.2.9 (Ubuntu) DAV/2 SVN/1.5.0 PHP/5.2.6-1ubuntu4 with Suhosin-Patch mod_perl/2.0.4 Perl/v5.10.0
Last-Modified: Thu, 02 Aug 2007 20:50:09 GMT
ETag: "438118-197-436bd96872240"
Accept-Ranges: bytes
Content-Length: 407
Connection: close
Content-Type: text/html; charset=UTF-8
You can then extract the apache version from the Server: header
Typical tools you can use
You could use the HEAD utility which comes with a full install of Perl's LWP library, e.g.
HEAD http://your.webserver.com/
Or, use the curl utility, e.g.
curl --head http://your.webserver.com/
You could also use a browser extension which lets you view server headers, such as Live HTTP Headers or Firebug for Firefox, or Fiddler for IE
Stuck with Windows?
Finally. if you're on Windows, and have nothing else at your disposal, open a command prompt (Start Menu->Run, type "cmd" and press return), and then type this
telnet your.webserver.com 80
Then type (carefully, your characters won't be echoed back)
HEAD / HTTP/1.0
Press return twice and you'll see the server headers.
Other methods
As mentioned by cfeduke and Veynom, the server may be set to return limited information in the Server: header. Try and upload a PHP script to your host with this in it
<?php phpinfo() ?>
Request the page with a web browser and you should see the Apache version reported there.
You could also try and use PHPShell to have a poke around, try a command like
/usr/sbin/apache2 -V
httpd -v will give you the version of Apache running on your server (if you have SSH/shell access).
The output should be something like this:
Server version: Apache/2.2.3
Server built: Oct 20 2011 17:00:12
As has been suggested you can also do apachectl -v which will give you the same output, but will be supported by more flavours of Linux.
Warning, some Apache servers do not always send their version number when using HEAD, like in this case:
HTTP/1.1 200 OK
Date: Fri, 03 Oct 2008 13:09:45 GMT
Server: Apache
X-Powered-By: PHP/5.2.6RC4-pl0-gentoo
Set-Cookie: PHPSESSID=a97a60f86539b5502ad1109f6759585c; path=/
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Connection: close
Content-Type: text/html
Connection to host lost.
If PHP is installed then indeed, just use the php info command:
<?php phpinfo(); ?>
Rarely, a hardened HTTP server is configured to give no server information or misleading server information. In those scenarios if the server has PHP enabled you can add:
<?php phpinfo(); ?>
in a file and browse to it and look for the
_SERVER["SERVER_SOFTWARE"]
entry. This is susceptible to the same hardening lack of information/misleading though I would imagine that it's not altered often, because this method first requires access to the machine to create the PHP file.
The level of version information given out by an Apache server can be configured by the ServerTokens setting in its configuration.
I believe there is also a setting that controls whether the version appears in server error pages, although I can't remember what it is off the top of my head. If you don't have direct access to the server, and the server administrator is competent and doesn't want you to know the version they're running... I think you may be SOL.
Telnet to the host at port 80.
Type:
get / http1.1
::enter::
::enter::
It is kind of an HTTP request, but it's not valid so the 500 error it gives you will probably give you the information you want. The blank lines at the end are important otherwise it will just seem to hang.
If they have error pages enabled, you can go to a non-existent page and look at the bottom of the 404 page.
Your best option is through PHP:
All version requests from the client side cannot be trusted since your Apache could be configured with ServerTokens Prod and ServerSignature Off. See: http://www.petefreitag.com/item/419.cfm
In the default installation, call a page that doesn't exist and you get an error with the version at the end:
Object not found!
The requested URL was not found on this server. If you entered the URL manually please
check your spelling and try again.
If you think this is a server error, please contact the webmaster.
Error 404
localhost
10/03/08 14:41:45
Apache/2.2.8 (Win32) DAV/2 mod_ssl/2.2.8 OpenSSL/0.9.8g mod_autoindex_color PHP/5.2.5
Simply use something like the following - the string should be there already:
<?php
if(isset($_SERVER['SERVER_SOFTWARE'])){
echo $_SERVER['SERVER_SOFTWARE'];
}
?>
Use this PHP script:
$version = apache_get_version();
echo "$version\n";
Se apache_get_version.