We try to identify the ssl_protocol is using when it communicated to Inbound Connectivity and so we tried to append this attribute \"%{SSL_PROTOCOL }e\" under LogFormat in httpd.conf file.
We formatted as in below manner.
LogFormat "%h %l %u %t %T\"%r\" %>s %b \"%{SSL_PROTOCOL}e\" " common
But still we are not able to capture the SSL_PROTOCOL details. Is that any pre-requisite needs to be done or any other alternative is to be done?
IBM HTTP Server uses %{SSL_PROTOCOL_VERSION}e and %{HTTPS_CIPHER}e for the handshake details.
Related
The server with my website has started working with load balancer.
Previous log format
LogFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"" combined
this format now contains the IP Address of the load balancer
I found the following solution to this problem on the php side
$_SERVER["HTTP_CLIENT_IPS"]
Is there a solution to this problem in log formats?
You need to setup the correct flag in the custom log format: http://httpd.apache.org/docs/current/mod/mod_log_config.html
Also consider using X-Forwarded-For header: https://www.rfc-editor.org/rfc/rfc7239
I am using Apache's mod_logio https://httpd.apache.org/docs/2.4/mod/mod_logio.html to debug some slow requests on Ubuntu.
%^FB shows "Delay in microseconds between when the request arrived and
the first byte of the response headers are written."
I have turned it on in apache2.conf and include %^FB in the log format that I use.
LogIOTrackTTFB ON
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %^FB" myformat
The time is included, but only with GET requests. It is not included for POST requests nor in requests that are requested as: script src="requesturl"
Is there a way to get this info for POST and script src requests?
So it turns out that it does work with all requests. I had just forgotten to use the new logging format both for secure and non-secure requests. That was the difference.
For the life of me, I couldn't get mod_remoteip to get client IPs in my Apache access logs. I'm working on a Virtualmin setup with Varnish 4 installed in front of Apache 2.4.7. How do you get it working?
I finally got the client IPs in the log and I found the last step here:
Here are the steps to getting it to work:
Get Varnish to pass a header to Apache with the client IP. You do this by including this bit of code (found in this answer) at the very beginning of your vcl_recv:
if (req.restarts == 0) {
if (req.http.X-Forwarded-For) {
set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
Now enable mod_remoteip in Apache.
Edit your Apache config to tell mod_remoteip what header contains the client IP (from Apache docs). I'm using X-Forwarded-For, but I suppose this could be anything as long as it matches what you've configured Varnish to pass along:
RemoteIPHeader X-Forwarded-For
If you restarted Apache and Varnish right now, I bet Apache will now reference the client IP instead of 127.0.0.1 everywhere. Except in the access logs which is where I've been checking. To get the access logs to show the client IP, we need to modify the log format Apache it's using. In my case, this was the "combined" format. This was my breakthrough and I found it here which links to this excellent information for our purposes.
This is what my combined log format looked like:
LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
And I just substituted %a in where %h was and this is what it looks like:
LogFormat "%a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
And finally, here's what a block of my Apache config file looks like (mod_remoteip loading prior to it):
# Note that the use of %{X-Forwarded-For}i instead of %h is not recommended.
# Use mod_remoteip instead.
RemoteIPHeader X-Forwarded-For
LogFormat "%a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
I am attempting to perform some data analysis on a set of Apache access logs that were passed on to me, but I noticed these logs do not seem to be in a conventional format (based on a few other Apache log examples I found online). Following is one row extracted from one of my log files (after some anonymization):
2013-08-25 10:06:11 EDT - "GET http://www.siteaddress.com/section/aaa/z/directory HTTP/1.1" 404 1677 1.2.181.171 "-" "Mozilla/4.0 (compatible; MSIE 4.01; Windows 98)" - 0 155311 -
Is there any way I can find out what format is being used to record these logs? That is, how would I go about getting some sort of a header for this file? Ps.: I have access to the server that is capturing these logs and could use it to find that information out.
Edit 1: I was told to check the content of /etc/apache2/httpd.conf, which I found to be empty.
Edit 2: The following relevant piece was found within apache2.conf but I'm not quite sure these match what I'm seeing in the logs.
# The following directives define some format nicknames for use with
# a CustomLog directive (see below).
# If you are behind a reverse proxy, you might want to change %h into %{X-Forwarded-For}i
LogFormat "%v:%p %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
Edit 3: Found this within /etc/apache2/sites-available/hub and it seems to match my format (Thanks a lot!)
LogFormat "%{%Y-%m-%d %H:%M:%S %Z}t %u \"%r\" %>s %B %a \"%{Referer}i\" \"%{User-Agent}i\" - %T %D -"
In debian based distros, the config is split into multiple files. Main configuration is /etc/apache2/apache2.conf, while all vHosts (that are used by default) are configured in /etc/apache2/sites-available (they might not be active though, when activating virtual hosts using a2ensite a symlink in /etc/apache2/sites-enabled/ is created, from where the files get included)
If you have never messed with the config, the log format should either be set in
/etc/apache2/sites-available/default inside the VirtualHost container ot in apache2.conf
You can find the format specification in one of the Apache configuration files; depending on your server configuration, that may be the primary config file (/etc/apache2/apache2.conf, probably; that's the default on Debian and its derivates, while /etc/httpd/httpd.conf is common among the Rat Head family), or in the configuration file for the virtual host whose logs you're looking at (/etc/apache2/sites-enabled/* for Debian and company, God only knows where for Rat Head).
The configuration directive you're after will be either LogFormat, which aliases a format string to a short name, or CustomLog, which uses either a format string, or a short name defined earlier in a LogFormat directive, to specify an actual logging format.
The format string syntax is detailed in the Apache documentation.
To save you some time, from eyeballing the sample line and having had considerable experience of Apache logs, most of the format appears to be:
<datetime> - "<request method> <url> <HTTP version>" <response status> <response length> <client address> "<request Referer: header value?>" "<request User-Agent: header value>" - ?? <response duration in microseconds?> - <newline>
I don't know offhand what the parts involving question marks are, but the rest seem pretty obvious.
I have Apache running as a proxy and I want to log the cookie string as part of the requests that come through the proxy, but I there are certain cookies I want to exclude since they contain sensitive info.
I know that Apache mod_log_config allows logging the whole cookie string with:
%{Cookie}i
And I can log one particular cookie with:
%{Foobar}C
But I want to log everything except certain cookies if they exist. One possible tactic I tried was to use SetEnvIf combined with %{FOO}e, which prints an environment variable, so something like:
SetEnvIf = Cookie "(.*?)(Foobar=([^\s;]*;? ?))?(.*)" FILTERED_COOKIES=$1$4
and
%{FILTERED_COOKIES}e
This works well if Foobar exists in the cookie string, but if Foobar doesn't exist, if just logs a "-", so I don't get all the other cookies.
It's possible that my regex might be poor, but it seems to work in the matching case, but Apache doesn't seem to handle it right in the non-matching case.
Short of writing a custom Apache mod, is there a way to exclude cookies from being logged?
One way to do this would be to have multiple log formats. In your main config file:
SetEnvIf Cookie "(.*?)(Foobar=([^\s;]*;? ?))?(.*)" FILTERED_COOKIES=$1$4
LogFormat "%h %l %u %t \"%r\" %>s %b %{Cookie}i" common
LogFormat "%h %l %u %t \"%r\" %>s %b %{FILTERED_COOKIES}e" filtered
Then in your VirtualHost, add something like this:
CustomLog logs/access_log filtered env=FILTERED_COOKIES
CustomLog logs/access_log common env=!FILTERED_COOKIES