access_log is huge, not being archived. how to reset it? - apache

I have discovered that my access_log is occupying most of my HDD. It's over 200 GB in size. How can I reset it ?
I am using Apache 2.2.3 on a CentOS server with Plesk.
Thank you guys !

knx'answer is good, but I would suggest to rename the log, and create a new one, so that you can restart apache without waiting for the access log to be compressed, which can take a while if it's big.
needs access to ssh
First, rename the current log file:
mv /var/log/apache/access.log /var/log/apache/access.log.1
Second, create a new log file and give the same permissions, owner/group and selinux context as the original one:
touch /var/log/apache/access.log
chown --reference=/var/log/apache/access.log.1 /var/log/apache/access.log
chmod --reference=/var/log/apache/access.log.1 /var/log/apache/access.log
restorecon --reference=/var/log/apache/access.log.1 /var/log/apache/access.log
(probably need to be root to do that)
Next, restart apache
Then Gzip the old file (text files compression ratios are really good). If we assume the file is named /var/log/apache/access.log then do this:
gzip -c /var/log/apache/access.log.1 > /var/log/apache/access.log.1.gz
these 4 points are what logrotate do automatically.

If you have access by SSH to the server, then you can:
1) Gzip the old file (text files compression ratios are really good). If we assume the file is named /var/log/apache/access.log then do this:
gzip -c /var/log/apache/access.log > /var/log/apache/access.log.gz
2) Clear the current file
echo > /var/log/apache/access.log
3) Restart apache
Also as Dez has suggested consider using logrotate for production grade apache log archiving.

Use the logrotate daemon in order to have a clean maintenance of your logs, specially, the apache related logs.
A brief info about logrotate: http://www.scriptinstallation.in/logrotate.html

If on Ubuntu do:
sudo su
cd /var/log/apache2
rm access.log
rm error.log
touch access.log
When creating that access log it magically starts the error log too.

I know this post is ages old, but I just had same problem and no answer covers it correctly.
The point is the apache creates the file as access_log, according to its configuration. However, logrotate only looks for *.log, hence the name does not match the search pattern.
Solutions: Either you add *_log to logrotate configuration, or change the apache configuration to make it create the log file named access.log. Changing apache configuration requires apache reload.

A simple solution is to disable access_log, commenting only one line on the configuration file.
Source: https://www.mydigitallife.info/how-to-disable-and-turn-off-apache-httpd-access-and-error-log/
For Plesk users:
https://stackoverflow.com/a/41000240/1792240

Rename the file, create a new access_log, then restart Apache.

Rename the file to different filename and create new file with the name access_log and restart apache (otherwise apache keeps the lock on the file and don't "see" the file change)

Related

How to run a bat file in apache server using Rotatelogs configuration ?

I am trying to rotate the log file in apache server. I tried a lot using rotatelog configuration in httpd.conf in apache configuration. But I can't.
while i configuring the rotatelog in httpd.conf, i found that Rotatelog having the option to run a program that can be acheived by using parameter -p. So i wrote a bat file to rotate the logs. And configured with apache server like below command
ErrorLog "|bin/rotatelogs.exe -l -p logs/apachelog.bat errorlogs 1M"
Whether My way of configuration is correct or not. Once error log file reached that particular file size , i want to execute the bat file. So i tried like this.
Thanks in advance.

How do I find my httpd.conf file?

*Edit: Anyone who's using MAMP PRO on a Mac running Mac OS X Lion may find it helpful to know that I finally tracked down my httpd.conf file in my personal library at Application Support > appsolute > MAMP PRO > httpd.conf. The reason I couldn't find it with a normal search is that it's a hidden directory. To access it, open Finder, Choose Go > Go To Folder, then type in ~/Library After tracking it down, you may then discover that it's almost impossible to edit your httpd.conf file. ); *
This is a weird one. I'm using MAMP on a Mac and want to modify my httpd.conf file so it defaults my URL's to lower case. So I navigated to Applications > MAMP > conf > apache > httpd.conf and added this line of code:
RewriteMap tolower int:tolower
I then modified one of my .htaccess files accordingly, but nothing happened. (Yes, I restarted my servers.)
I then reopened my httpd.conf file and added this code, which I understand is supposed to display "verbose" messages in my log file:
LogLevel trace8
I then opened my log file # Application > MAMP > logs > apache_error.log. There's code associated with the web pages I most recently visited, but they're just generic messages - not "verbose" messages.
Someone who understands Apache far better than me told me this indicates that the httpd.conf file I modified isn't being used by my server or sites. In other words, there must be another httpd.conf file out there somewhere.
Sure enough, I typed httpd.conf into Apple's finder and found half a dozens files with that name. I don't know where they came from; I recently tried to install Drupal and Aqquia, so that might be where some of them originated.
Anyway, I added the code (including LogLevel trace8) to EVERY httpd.conf file, restarted my servers, and I STILL don't see any changes on my websites, and I still just see generic code in my log file.
I'm completely snowed. It looks like the httpd.conf file in my MAMP folder is out of the loop, yet the Apache log file in the same folder is being used...or can errors be written to more than one Apache log file at the same time?
If I rename the httpd.conf file in my MAMP folder and restart my servers, my sites display fine, further suggesting that this httpd.conf is outside the flow. Yet if I rename a file that sits beside it named mime.types, my web pages don't display.
Anyway, does anyone have any idea how I can figure out what's going on?
If not, then can you advise me on Plan B? If I can't find my httpd.conf file, then I'll have to abandon ship and just do it online. I've found the path to my online httpd.conf file. I don't know how to open online files, but I assume I can copy it to my computer with an ftp program, open it and modify it, then publish it online.
So here are my questions: Am I correct in understanding that I can safely add the code RewriteMap tolower int:tolower to my online httpd.conf file without affecting any of my websites UNTIL I also modify a .htaccess file(s)? And if I modify my online config file, do i have to restart the server, just as I do on my computer?
Thanks.
First, use the ps command to find the full path to the running Apache process:
$ ps ax | grep httpd
28 ?? Ss 3:01.99 /usr/sbin/httpd -D FOREGROUND
On this system, Apache is /usr/sbin/httpd.
Second, run that binary with the -V option to show its configuration. This should include the path to the config file.
$ /usr/sbin/httpd -V
...
Server compiled with....
..
-D SERVER_CONFIG_FILE="/private/etc/apache2/httpd.conf"
On this system, the running copy of Apache reads its config from /private/etc/apache2/httpd.conf.

Where does PHP store the error log? (PHP 5, Apache, FastCGI, and cPanel)

I am on shared hosting and have cPanel, Apache, and PHP is run by FastCGI. Where does PHP store the error log?
Is there another way I can find the error log on a shared hosting environment instead of having to go through entire site structure to look for error_log files?
I have access to the php.ini file (I am using PHP version 5.2.16).
PHP stores error logs in /var/log/apache2 if PHP is an apache2 module.
Shared hosts are often storing log files in your root directory /log subfolder.
But...if you have access to a php.ini file you can do this:
error_log = /var/log/php-scripts.log
According to rinogo's comment: If you're using cPanel, the master log file you're probably looking for is stored (by default) at
/usr/local/apache/logs/error_log
If all else fails you can check the location of the log file using
<?php phpinfo(); ?>
Try phpinfo() and check for "error_log"
Linux
php --info | grep error
The terminal will output the error log location.
Windows
php --info | findstr /r /c:"error_log"
The command prompt will output the error log location.
To set the log location
Open your php.ini and add the following line:
error_log = /log/myCustomLog.log
Thanks chelmertz and Boom for these (comments on the question).
On a LAMP environment the PHP errors are be default directed to this below file.
/var/log/httpd/error_log
All access logs come under:
/var/log/httpd/access_log
How to find your PHP error log on Linux:
sudo updatedb
[sudo] password for eric:
sudo locate error_log
/var/log/httpd/error_log
Another equivalent way:
sudo find / -name "error_log" 2>/dev/null
/var/log/httpd/error_log
It can also be /var/log/apache2/error.log if you are in Google Compute Engine.
And you can view the tail like this:
tail -f /var/log/apache2/error.log
It appears that, by default, PHP does not log errors anywhere. The error_log key in php.ini is commented out in all the installs I've seen.
Generally I:
look for php.ini files. locate php.ini.
Search these files for the error_reporting value;
Which should be set to whatever amalgamation of PHP log levels are enough for you.,
For example: E_ALL & ~E_DEPRECATED & ~E_STRICT
Check the error_log value to make sure it points to an actual place and is not commented out.
The default value doesn't give a full path, only a file name, I don't know where this path resolves to normally. Probably /var/log/.
You should use absolute path when setting error_log variable in your php.ini file, otherwise, error logs will be stored according to your relative path.
error_log = /var/log/php.errors
Other solution would be writing simple script which would list all error logs files from directory tree.
php --info | grep error
This is helpful.
NGINX usually stores it in /var/log/nginx/error.log or access.log (on Ubuntu in any case).
The best way is to look in your httpd.conf file and see what the default is. It could also be overridden by your specific virtual host. I start by looking at /etc/httpd/conf/httpd.conf or /etc/apache2/httpd.conf and search for error_log. It could be listed as either /var/log/httpd/error_log or /var/log/apache2/error_log, but it might also be listed as simply logs/error_log.
In this case it is a relative path, which means it will be under /etc/httpd/logs/error_log. If you still can't find it, check the bottom of your httpd.conf file and see where your virtual hosts are included. It might be in /etc/httpd/conf.d/ <- as "other" or "extra". Your virtual host could override it then with ErrorLog "/path/to/error_log".
If you have build Apache and PHP from source, then the error logs by default is generated at your ${Apache install dir}/logs/error_log, i.e., generally /usr/local/apache2/logs/error_log.
Else, if you have installed it from the repository, you will find it at /var/log/apache2/error_log.
You can set the path in your php.ini also and verify it by invoking phpinfo().
Wherever you want it to, if you set it in your function call:
error_log($errorMessageforLog . "\n", 4, 'somePath/SomeFileName.som');
If you use the php5-fpm log default, it should be under:
/var/log/php5-fpm.log
Search the httpd.conf file for ErrorLog by running cat <file location> | grep ErrorLog on the command line. For example:
cat /etc/apache2/httpd.conf | grep ErrorLog
Output:
# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
ErrorLog "/private/var/log/apache2/error_log"
Find the line that starts with ErrorLog and there's your answer.
Note: For virtual hosts, you can edit the virtual hosts file httpd-vhosts.conf to specify a different log file location.
cPanel Error logs are located in:
/usr/local/cpanel/logs/
/usr/local/apache/logs/
By default Apache logs are located inside:
/var/log/apache
or
/var/log/apache2
If anyone is using a custom log location then you can check it by running this command:
cat /etc/apache2/**conf**/httpd.conf | grep ErrorLog
If you are getting an error that the apache2 directory does not exist then you can run this command to find correct location by:
whereis apache
or
whereis apache2
When configuring your error log file in php.ini, you can use an absolute path or a relative path. A relative path will be resolved based on the location of the generating script, and you'll get a log file in each directory you have scripts in. If you want all your error messages to go to the same file, use an absolute path to the file.
See more in Error Handling Functions.
something like this :
sudo locate error.log | xargs -IX grep -iH "errorlog" X
or
sudo locate error_log | xargs -IX grep -iH "errorlog" X
or
sudo find / -iname "error?log" 2>/dev/null | xargs -IX grep -iH "errorlog" X
I can guarantee you, I am not the only person who has been driven to madness at least once in a frustrating search for a log file. It seems like it should be the easiest thing to find in the whole system.
A definitive guide on where the PHP error log is stored would be a complicated bit of work. The official PHP manual does not even try to address the whole topic, because there are dependencies on systems outside PHP, such as the operating system (Linux vs. Windows, which distribution of Linux), including settings within Windows and Linux that affect the name and location of the PHP error log.
Until someone takes the time to write a complete, cross-system guide, the best you are going to get is general directions where you can inquire. Every PHP developer has had to endure agony in this pursuit, with one exception. If you work in one place and the information is provided when you first need it, then you have the information need forever, that is, until you find yourself in a new working environment. There are such fortunate people.
If the information is not given to you on a silver platter, so to speak, you have some hunting to do. The hunt is not the longest you will face in your career, but it is not the simplest either.
As is evident from the many answers already posted, a smart place to begin is the output of phpinfo(). To view it, create a PHP file containing this:
<?php
phpinfo();
Either browse to that file or run it from the command line. If you do both, you likely will find the error_log is in different places, depending on command line vs. web server use of PHP. That is because the PHP interpreter that runs on a web server is not the same PHP interpreter that runs from the command line, even when the command line is on the same machine as the web server. The answers already posted in here mostly are making an unstated assumption that PHP is running as part of a web server.
The default for error_log is no value
Whatever the value is, it comes from the php.ini files used to configure PHP. There can be many php.ini files. Finding your way among them is confusing at first, but you do not need to deal with this to find your PHP log.
If the output from phpinfo() shows a full path to a file, that is where the log is. You are lucky.
The trick is there usually is not a full path indicated in phpinfo(). When there is not a full path, the location depends on:
Whether error_log is no value. If it is, the log file location will depend on the operating system and the mode PHP is running. If PHP is running as an Apache module, on Linux the log often is in /var/log/apache2/error.log. Another likely spot is in a logs directory in your account home directory, ~/logs/error.log.
If there is a file name without a path, the location depends on whether the file name has the value syslog. If it syslog, then the PHP error log is injected into the syslog for the server, which varies by Linux distribution. A common location is /var/log/syslog, but it can be anywhere. Even the name of the syslog varies by distribution.
If the name without a path is not syslog, a frequent home for the file is is the document root of the website (a.k.a., website home directory, not to be confused with the home directory for your account).
This cheat sheet has been helpful in some situations, but I regret to have to admit it is not nearly universal. You have my condolences.
For CentOS 8 it is var/log/httpd/error_log.
You are on a shared environment and cannot find error log. Always check if cPanel has option Errors in your cPanel dashboard. If you are not being able to find the error log, then you can find it there.
In the cPanel search bar, search Error, it will show Error Pages which are basically lists of different HTTP error pages and other Error is where the error logs are displayed.
Other places to look in a shared environment:
/home/yourusername/logs
/home/yourusername/public_html/error_log
You can go in File Manager and check the logs folder.
Check the log file in a public_html folder.
Check "php phpinfo()" file for where the log is stored.
Mine for some reason stored it in:
/var/log/php-errors.log
I'm using Ubuntu Server 16.04 (Xenial Xerus) and PHP 7.1.28.
WordPress
WordPress will direct error_log() messages to /wp-content/debug.log when WP_DEBUG_LOG is set to true.
See WordPress documentation for WP_DEBUG_LOG
For PHP-FPM, just search the configuration file for error_log:
cat /etc/php-fpm.d/www.conf | grep error_log
php_admin_value[error_log] = /var/log/php-fpm/www-error.log
In a shared cPanel environment you cannot find the error log, if your hosting provider doesn’t provide any option in the cPanel dashboard. You can search "error" and see if your provider have any.
Otherwise normally you will find a file called "error_log" in your public_html file, which have all the PHP error recorded.
You can see the php errors for an account under:
/home/username/logs/domain_tld.php.error.log
You can set php error log globally and locally.
Globally through WHM:
WHM -->> MultiPHP INI Editor --> "Editor Mode"
Select the version of PHP you would like to set the error log
Edit the parameter error_log.
error_log = "/some/path"
Change the filename to the desired error log file
Save changes
Locally though cPanel:
CPanel -->> MultiPHP INI Editor --> "Editor Mode"
Select the version of PHP you would like to set the error log
Edit the parameter error_log.
error_log = "/some/path"
Change the filename to the desired error log file
Home directory will set the option for all domains, addons, and subdomains configured on the account
A specific domain will limit the change to that domain, or, in the case of an addon domain, the addon and the matching subdomain
Save changes
If PHP-fpm is enabled:
WHM --> MultiPHP Manager
Find the domain you'd like to change the error log location for
Click "Edit PHP-FPM"
Change the log location in "The error log file (error_log)" relative to the folder "logs" within the user's home directory
Save changes

How to define apache's logrotation's file name using an external shell script

I have apache running on different servers, I would like to rsync log files back to a centralised server on a daily basis. I can use log rotate to create log file for a day and put it in a directory that gets rsync'd to the central server. However is there any way to set the log filename with a designation that could be read from an external file?
The documentation says I can do the following,
CustomLog "|/usr/sbin/rotatelogs -f /var/log/httpd/log 86400" common
I have an external config file which has a designation field which I can parse from command line to get its value, is there any way I can add it when file name gets defined. I'm curious to know if this designation can be passed to apache as an environment variable and use that in apache's config. Is this possible?
Many Thanks again guys!
I think you can do this by configuring Apache and logrotate. Logrotate is configured by scripts in the /etc/logrotate.d/ directory. For example, my default apache logrotate config file is:
/etc/logrotate.d/httpd
/var/log/httpd/*log {
missingok
notifempty
sharedscripts
postrotate
/bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true
endscript
}
Check the logrotate man pages for the postrotate command. During the postrotate phase you could rename the log file and place it somewhere to be picked up by rsync.

Does Apache need to be stopped to edit "/etc/apache2/sites-available/default"?

I am attempting to edit the "default" file located at ..
"/etc/apache2/sites-available/default"
on my Ubuntu machine running Apache 2.2.8.
I want to do this in order to enable the use of .htaccess files. I have downloaded the "default" file and edited it and now I am trying to upload it back to the server via SFTP. I keep getting permission denied errors.
Could it be because Apache is running and making use of the file? I am an admin on the machine so I would expect to be able to overwrite the file. Thanks for any assistance.
No it does not need to be stopped.
Try accessing the file through ssh, and make sure you access it with root privileges:
sudo nano /etc/apache2/sites-available/default
You would still need to force-reload Apache after changing the config files, as tux21b suggested in a comment below:
sudo /etc/init.d/apache2 force-reload
For Apache/2.4.7 , the file that you want to edit is:
/etc/apache2/apache2.conf