Strange "pattern not match" error on fluentd - apache

Question

Can someone tell if its normal that fluentd raise this error in td-agent.log file?
2015-07-31 13:15:19 +0000 [warn]: pattern not match: "- - - [31/Jul/2015:13:15:19 +0000] GET http://172.31.108.218/ HTTP/1.1 200 0 \"-\" \"ELB-HealthChecker/1.0\""
While this is a well formated apache2 log:
- - - [31/Jul/2015:13:15:19 +0000] GET http://172.31.108.218/ HTTP/1.1 200 0 \"-\" \"ELB-HealthChecker/1.0\"
And here is the source configuration:
<source>
type tail
format apache2
path /var/log/varnish/varnishncsa.log
pos_file /var/log/td-agent/tmp/access.log.pos
tag "apache2.varnish-access"
</source>
I can't figure out what's wrong there above.

Answer 1

Instead of finding some way to filter out logs from ELB-HealthChecker, you can set your own format for the Apache access log that is a little more flexible in terms of the first couple fields. I ran into this same error when getting /server-status checks from collectd (using it to monitor for SignalFx).
Setting the source like so:
<source>
type tail
format /^(?<host>[^ ]*(?:\s+[^ ]+)*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<code>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?$/
time_format %d/%b/%Y:%H:%M:%S %z
path /var/log/apache2/access.log
pos_file /var/log/td-agent/apache2.pos
tag apache2.log
</source>
Allows both log lines like:
172.18.0.2:80 127.0.0.1 - - [08/Aug/2017:19:58:38 +0000] "GET /server-status?auto HTTP/1.1" 200 508 "-" "collectd/5.7.2.sfx0"
As well as:
192.168.0.1 - - [28/Feb/2013:12:00:00 +0900] "GET / HTTP/1.1" 200 777 "-" "Opera/12.0"
You can test format regex matching using Fluentular.
See related: Fluentd apache log format with multiple host ip

Answer 2

The problem is that these ELB-HealthChecker line log has an empty referer ip field. And then the log doesn't match apache2 log format for fluentd.
So the way to fix that is to filter logs with ELB-HealthChecker user-agent.