We're running mod_perl on Apache 2 and get seemingly random header related errors that we just can't figure out. Due to the nature of the site we get hit by a ton of bots, so I'm thinking these are caused by bad or malformed requests from bots, but I'd like to figure it out for sure one way or another so I know where to go from here. Here's an example of the 2 most common errors we see in the logs:
[Thu Nov 13 21:40:48 2014] [warn] /whatever did not send an HTTP header
[Thu Nov 13 21:40:48 2014] [error] [client x] malformed header from script. Bad header=\x86z\x03d\x86z\x03d\x86z\x03d\x86z\x03d\x86z\x03d\x86z\x03d\x86z\x03d\x86z: index.cgi
[Fri Nov 14 00:04:17 2014] [warn] /whatever did not send an HTTP header
[Fri Nov 14 00:04:17 2014] [error] [client x] Premature end of script headers: index.cgi
We get 100s of 1,000s of requests to these same URLs daily, and they work fine 99.999% of the time. I don't believe it's our scripts - we always output correct headers. No real users have ever complained about any errors on our site, etc. so I'm hoping this is just caused by some bad requests from bots.
And if so, what if anything can we do to make these stop? It's a real pain because these errors trip our monitoring systems and my techie gets about 20-30 fake error alerts every day.
Turns out it was a problem with Safari browsers and mod_deflate compression.
The simple solution:
BrowserMatch Safari gzip-only-text/html
Related
I'm using apache-2.4.53 and having problems with caching. Perodically I see errors related to "premature end of cache headers" and don't know how to troubleshoot it. This is on fedora34. The site is using Cloudflare.
[Wed Jul 06 04:23:49.577237 2022] [cache_disk:error] [pid 3202400:tid 3202451] (70014)End of file found: [client 162.158.190.138:47866] AH00717: Premature end of cache headers.
[Wed Jul 06 04:23:49.577247 2022] [cache_disk:debug] [pid 3202400:tid 3202451] mod_cache_disk.c(883): [client 162.158.190.138:47866] AH02987: Error reading response headers from /var/cache/httpd/W_#/Ro6/7ihAG5M_Eyw0t7jA.header.vary/#Iu/98u/9#ot3lTARaKl3p8g.header for https://example.com:443/index.php?
The 162.158.190.138 is a cloudflare address.
There seems to be a ongoing related apache bug related to this issue since 2016, but I don't know that it's the same thing. I don't know how to reproduce it. Where do I start to look?
I can correlate the lines from the error_log with the access_log based on time, but I can't be sure they're directly related. There were three requests during that same second, all of which were bots. One was a 200, one was a 301 and one was a 404 for a file that was never there.
The file the error_log references is there on the filesystem:
find . -name \*7ihAG5M_Eyw0t7jA\*
./W_#/Ro6/7ihAG5M_Eyw0t7jA.header.vary
./W_#/Ro6/7ihAG5M_Eyw0t7jA.header
Here is the bug report from 2016.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59744
Here are the cache options from the virtual host config:
CacheQuickHandler off
CacheLock on
CacheLockPath /tmp/mod_cache-lock
CacheLockMaxAge 5
CacheIgnoreHeaders Set-Cookie
CacheRoot "/var/cache/httpd"
# Enable the X-Cache-Detail header
CacheDetailHeader on
CacheEnable disk "/"
CacheHeader on
CacheDefaultExpire 800
CacheMaxExpire 64000
CacheIgnoreNoLastMod On
CacheDirLevels 2
CacheDirLength 3
I also notice the cache directory (/var/cache/httpd) grows boundlessly. At one time htcacheclean was running from systemd, but that doesn't look to be the case any longer.
Should I be investigating the HTTP cache control headers? Is that related or helpful?
Do you have any recommendations for optimal disk cache sizes?
I have only allowed GET, POST methods in my apache server. It shows lot of times error like below which is of no use to me. How can I block these errors to come in apache error log
[Mon Aug 22 18:43:27.232168 2016] [allowmethods:error] [pid 19314:tid 139797637039872] [demowebsite.com] [client 224.0.0.0:80] AH01623: client method denied by server configuration: 'PURGE' to /var/www/demowebsite/
I also want to know what is causing it. I am using apache 2.4 + php 5.5 + mod_pagespeed + varnish.
Please help me.
Since you seem to be using Apache 2.4.X
Just by setting:
LogLevel allowmethods:crit
you will be rising the level necessary to log to error log to critical level in that module so they won't show up for errors.
My Adsense-supported website's error.log file has lots entries like this which I'm fairly sure are advert related:
[Fri Apr 12 07:19:57 2013] [error] [client IP] File does not exist: /var/www/[mywebsite]/htdocs/eyeblaster, referer: http://apac-bidder.mathtag.com/notify/iframe? [snip lots of junk]
What is it and is it harmful?
Should I create an empty eyeblaster.html file to shut it up?
I also have another two that are like it - are they related?:
[Fri Apr 12 07:08:52 2013] [error] [client IP] File does not exist: /var/www/[mywebsite]/htdocs/7196176924447058959
[Fri Apr 12 07:13:58 2013] [error] [client IP] File does not exist: /var/www/[mywebsite]/htdocs/_sans
Take a look at StackOverflow question above - Determine which advertisement made a request to /eyeblaster/addineyev2.html. I think all will be revealed :).
Eyeblaster is an online advertising company now called Sizmek (formerly Mediamind).
As John mentioned, this file is used as an iframe buster, just like DoubleClick's iframe buster, which should be placed by a publisher on [www.example.com]/doubleclick/DARTIframe.html
Another solution is to disallow eyeblaster.
Just add these to your robots.txt:
Disallow: /eyeblaster
Disallow: /addineyeV2.html
Or I prefer to redirect eyeblaster to index.html inside .htaccess file
RewriteCond %{REQUEST_URI} (eyeblaster|addineyeV2) [NC]
RewriteRule ^(.*)$ /index.html? [R=301,L]
eyeblaster is just another piece of malware which most machines end up getting on at one point from what I can tell it hasn't caused any series issues but it is best to remove it from your machine whenever your anti-malware software pics it up
I'm getting the following errors in my erorr.log file on every request
[Fri Jan 29 14:44:17 2010] [debug] mod_deflate.c(619): [client 10.128.99.99] Zlib: Compressed 6025 to 1847 : URL
about 2 gigs worth (high load server)
any idea what this error is referring to?
Make sure you only have LogLevel specified once, or that you're changing it for the correct virtual host. And you'll need to kick apache of course.
doh! just found it... someone had set a specific error log for this particular virtual host and the loglevel was set to debug.
I am receiving some mysterious errors in my apache error log.
For example:
File does not exist: /home/httpd/vhosts/studentlife.co.il/httpdocs/help_center/div.hot-jobs
File does not exist: /home/httpd/vhosts/studentlife.co.il/httpdocs/solutions/function.filesize
File does not exist: /home/httpd/vhosts/studentlife.co.il/httpdocs/work_abroad/div.publish
Now I don't understand - how did my divs and functions suddenly turn into files?? Of course they do not exist, they are divs and functions written in the code. So I am not sure why I am getting these file does not exist errors.
Also, how can I debug this, since in no place am I trying to access files with these names?
Any insight will be great.
I would say don't worry about it.
I can make a request for /the/answer/to/life/the/universe/and/everything.42 and it would show up in your error log.
Probably some spider is hunting for vulnerabilities in your site and it's randomly trying URLs. If you see the same pages a lot then start checking to see whether you have any broken links, otherwise forget about it.
You could use FireBug to see the return codes (you are looking for 404) of all HTTP requests made when you open your website. Maybe you have some typo in your HTML.
I had this same issue in this scenario:
wamp installation
system was running perfectly til dec 01/2010.
restarted computer on dec 02/2010.
apache wouldn't start up and was logging [Thu Dec 02 09:15:45 2010] [error] [client 127.0.0.1] File does not exist: C:/wamp/www/announce
then, only then, I reminded myself I had just installed SKYPE on dec 01/2010 and that, on windows, we've got this conflict between the phone program and apache fighting for the port 80, I guess.
closed Skype, restarted services and voi là: my system is back. =)
One of my clients had a similar issue with Apache on Windows.
On any request he was getting:
[error] [client 192.168.1.66] File does not exist: D:/apache
I realized that the DocumentRoot was missing form the httpd.conf.
I agree with Gareth, a spider could be hunting for vulnerabilities in your site and it's randomly trying URLs. However, there may be another reason, probably more likely.
After debugging my own scripts to see what was going on, I noticed the following. The error of the form:
[Mon May 06 21:47:29 2013] [error] [client ip] File does not exist: /path/, referer: http://domain/example.html
is visible in the error log and even though it will sound weird, it does not have to do with the static HTML file or script that generates the page. In fact, the source of this problem may be the HTML of the page referencing a broken link to an image/css/js file, so that when the browser receives the HTML and fetches these resources Apache can't find them and prints the "File Does Not Exist" error.
It is nice of Apache as it warns us about broken links.