I'm getting the following errors in my erorr.log file on every request
[Fri Jan 29 14:44:17 2010] [debug] mod_deflate.c(619): [client 10.128.99.99] Zlib: Compressed 6025 to 1847 : URL
about 2 gigs worth (high load server)
any idea what this error is referring to?
Make sure you only have LogLevel specified once, or that you're changing it for the correct virtual host. And you'll need to kick apache of course.
doh! just found it... someone had set a specific error log for this particular virtual host and the loglevel was set to debug.
Related
I'm using apache-2.4.53 and having problems with caching. Perodically I see errors related to "premature end of cache headers" and don't know how to troubleshoot it. This is on fedora34. The site is using Cloudflare.
[Wed Jul 06 04:23:49.577237 2022] [cache_disk:error] [pid 3202400:tid 3202451] (70014)End of file found: [client 162.158.190.138:47866] AH00717: Premature end of cache headers.
[Wed Jul 06 04:23:49.577247 2022] [cache_disk:debug] [pid 3202400:tid 3202451] mod_cache_disk.c(883): [client 162.158.190.138:47866] AH02987: Error reading response headers from /var/cache/httpd/W_#/Ro6/7ihAG5M_Eyw0t7jA.header.vary/#Iu/98u/9#ot3lTARaKl3p8g.header for https://example.com:443/index.php?
The 162.158.190.138 is a cloudflare address.
There seems to be a ongoing related apache bug related to this issue since 2016, but I don't know that it's the same thing. I don't know how to reproduce it. Where do I start to look?
I can correlate the lines from the error_log with the access_log based on time, but I can't be sure they're directly related. There were three requests during that same second, all of which were bots. One was a 200, one was a 301 and one was a 404 for a file that was never there.
The file the error_log references is there on the filesystem:
find . -name \*7ihAG5M_Eyw0t7jA\*
./W_#/Ro6/7ihAG5M_Eyw0t7jA.header.vary
./W_#/Ro6/7ihAG5M_Eyw0t7jA.header
Here is the bug report from 2016.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59744
Here are the cache options from the virtual host config:
CacheQuickHandler off
CacheLock on
CacheLockPath /tmp/mod_cache-lock
CacheLockMaxAge 5
CacheIgnoreHeaders Set-Cookie
CacheRoot "/var/cache/httpd"
# Enable the X-Cache-Detail header
CacheDetailHeader on
CacheEnable disk "/"
CacheHeader on
CacheDefaultExpire 800
CacheMaxExpire 64000
CacheIgnoreNoLastMod On
CacheDirLevels 2
CacheDirLength 3
I also notice the cache directory (/var/cache/httpd) grows boundlessly. At one time htcacheclean was running from systemd, but that doesn't look to be the case any longer.
Should I be investigating the HTTP cache control headers? Is that related or helpful?
Do you have any recommendations for optimal disk cache sizes?
I have only allowed GET, POST methods in my apache server. It shows lot of times error like below which is of no use to me. How can I block these errors to come in apache error log
[Mon Aug 22 18:43:27.232168 2016] [allowmethods:error] [pid 19314:tid 139797637039872] [demowebsite.com] [client 224.0.0.0:80] AH01623: client method denied by server configuration: 'PURGE' to /var/www/demowebsite/
I also want to know what is causing it. I am using apache 2.4 + php 5.5 + mod_pagespeed + varnish.
Please help me.
Since you seem to be using Apache 2.4.X
Just by setting:
LogLevel allowmethods:crit
you will be rising the level necessary to log to error log to critical level in that module so they won't show up for errors.
I'm trying to configure the following environment: a VPS running apache and mod_proxy to proxy another server running at home (the backend). I'm able to download files but when I try to upload files the POST request fails with this error:
Proxy Error
The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request POST /upload/upload.php.
Reason: Error reading from remote server
What I don't understand is why it works for files as low as 500 bytes. And it does quickly! However, when I try to upload a tiny 4kb file, it takes forever until the error is reached. As expected, the upload works flawlessly when the backend is accessed directly, without the VPS. I tried many configurations for both sides, also tried to increase the timeout but I don't think this is the way to go. The backend has mod_access installed and it doesn't log anything when the file upload fails.
The apache logs the following:
[Thu Nov 07 22:26:03.044309 2013] [proxy_http:error] [pid 9173] (70007)The timeout specified has expired: [client 177.148.252.99:54097] AH01102: error reading status line from remote server myhome.com, referer: http://frontend.com/upload/
[Thu Nov 07 22:26:03.044423 2013] [proxy:error] [pid 9173] [client 177.148.252.99:54097] AH00898: Error reading from remote server returned by /upload/upload.php, referer: http://frontend.com/upload/
The VPS is running Apache 2.4.6 and the server running at home is a Lighttpd 1.4.32 with SSL.
The virtual host redirecting to the backend is configured as follows:
<VirtualHost *:80>
ServerAdmin webmaster#frontend.com
ServerName frontend.com
ProxyPass / http://backend.com/
ProxyPassReverse / http://backend.com/
</VirtualHost>
Front-end:
http://frontend.com/upload/
Back-end:
http://backend.com/upload/
Do you have any ideas?
The error you're seeing is due to a timeout of the proxy connection to the back-end system. You need to set the ProxyTimeout value to something larger than the default. I would recommend that you start with a value of 60 seconds and see how that works.
ProxyTimeout 60
In addition, I agree with Varghese that you want to set the environment variable in order to configure the connection to send the data in chunks. Unfortunately, there is some confusion over whether the correct setting should be, so you can try either of these:
SetEnv proxy-sendchunked 1
or
SetEnv proxy-sendchunks 1
Good luck. It's a frustrating problem.
Environment variables availables in mod_proxy:
https://httpd.apache.org/docs/2.4/mod/mod_proxy_http.html
I have faced the similar kind of issue and got resolved with the statement, I put after the ProxyPassReverse statement.
The command you have to use is : SetEnv proxy-sendchunks 1
We are having problems with our web server (which is configured ssl -> apache -> jetty) randomly rejecting multipart upload POST requests with a 400 Bad Request error code. The apache error log (on info level) shows the following two errors:
[info] [client x1.y1.z1.w1] (70007)The timeout specified has expired: SSL input filter read failed.
[error] proxy: pass request body failed to x.y.z.w:8087 from x1.y1.z1.w1
[info] [client x1.y1.z1.w1] Connection closed to child 74 with standard shutdown
or
[info] [client x2.y2.z2.w2] (70014)End of file found: SSL input filter read failed.
[error] proxy: pass request body failed to x.y.z.w:8087 from x2.y2.z2.w2
[info] [client x2.y2.z2.w2] Connection closed to child 209 with standard shutdown
both cases result from the client side in a 400 Bad Request. Sometimes our jetty server doesn't even see the request meaning that it gets rejected on apaches side, sometimes it starts processing it only to be rejected (this manifests itself as a MultipartException in our UploadFilter)
We have mod_proxy setup to use a fallback load balancing scheme but the logs show that a fallback has not yet been triggered causing me to believe this is not the cause of the problem.
I tried setting SetEnv proxy-sendcl 1 but that didn't change anything.
The upload requests are arount 1mb. Only these multipart file POST requests fail, we have multiple GET requests comming in at the same time and they always work as expected.
If anyone can share any advice or suggestions I would be very grateful! Thank you
If you are using some ajp-enabled backend server, like Tomcat, you may try using mod_proxy_ajp instead of mod_proxy_http. I had a similar problem on a heavy upload app and I fixed it by changing
ProxyPass /myapp http://localhost:8080/myapp
ProxyPassReverse /myapp http://localhost:8080/myapp
by
ProxyPass /myapp ajp://localhost:8009/myapp
ProxyPassReverse /myapp ajp://localhost:8009/myapp
It's also required to enable the ajp connector on tomcat side, of course.
Hope it helps!
Please check this one: https://issues.apache.org/bugzilla/show_bug.cgi?id=44592
The problem could be caused by the HTTP keepalive (KeepAliveTimeout directive) that is killing an established connection with a slow client (tcp latency, slow request body creation, etc).
Try to raise the KeepAliveTimeout in your apache conf, but don't keep it too high to avoid killin' your server (DOS).
You may be seeing this due to timeouts resulting from Apache trying to buffer the entire POST body before passing it through.
Enabling proxy-sendcl may exacerbate this, since this can force Apache to spool a large POST to disk just to calculate the Content-Length when "the original body was sent with chunked encoding (and is large)".
To avoid this, set the environment variable proxy-sendchunked.
After fixing the main problem with the upload, I was still getting some weird logs like:
[reqtimeout:info] [pid 18164:tid 140462752990976] [client 201.76.162.37:41473] AH01382: Request header read timeout
I was able to reduce drastically the frequency it coccurs by increasing the limits of mod_reqtimeout, changing the values of RequestReadTimeout parameter. You can change the default values or redeclare this parameter on your VirtualHost.
I am receiving some mysterious errors in my apache error log.
For example:
File does not exist: /home/httpd/vhosts/studentlife.co.il/httpdocs/help_center/div.hot-jobs
File does not exist: /home/httpd/vhosts/studentlife.co.il/httpdocs/solutions/function.filesize
File does not exist: /home/httpd/vhosts/studentlife.co.il/httpdocs/work_abroad/div.publish
Now I don't understand - how did my divs and functions suddenly turn into files?? Of course they do not exist, they are divs and functions written in the code. So I am not sure why I am getting these file does not exist errors.
Also, how can I debug this, since in no place am I trying to access files with these names?
Any insight will be great.
I would say don't worry about it.
I can make a request for /the/answer/to/life/the/universe/and/everything.42 and it would show up in your error log.
Probably some spider is hunting for vulnerabilities in your site and it's randomly trying URLs. If you see the same pages a lot then start checking to see whether you have any broken links, otherwise forget about it.
You could use FireBug to see the return codes (you are looking for 404) of all HTTP requests made when you open your website. Maybe you have some typo in your HTML.
I had this same issue in this scenario:
wamp installation
system was running perfectly til dec 01/2010.
restarted computer on dec 02/2010.
apache wouldn't start up and was logging [Thu Dec 02 09:15:45 2010] [error] [client 127.0.0.1] File does not exist: C:/wamp/www/announce
then, only then, I reminded myself I had just installed SKYPE on dec 01/2010 and that, on windows, we've got this conflict between the phone program and apache fighting for the port 80, I guess.
closed Skype, restarted services and voi là: my system is back. =)
One of my clients had a similar issue with Apache on Windows.
On any request he was getting:
[error] [client 192.168.1.66] File does not exist: D:/apache
I realized that the DocumentRoot was missing form the httpd.conf.
I agree with Gareth, a spider could be hunting for vulnerabilities in your site and it's randomly trying URLs. However, there may be another reason, probably more likely.
After debugging my own scripts to see what was going on, I noticed the following. The error of the form:
[Mon May 06 21:47:29 2013] [error] [client ip] File does not exist: /path/, referer: http://domain/example.html
is visible in the error log and even though it will sound weird, it does not have to do with the static HTML file or script that generates the page. In fact, the source of this problem may be the HTML of the page referencing a broken link to an image/css/js file, so that when the browser receives the HTML and fetches these resources Apache can't find them and prints the "File Does Not Exist" error.
It is nice of Apache as it warns us about broken links.