I am trying to configure Apache to handle a larger number of requests on Ubuntu 14.04. I have done so by adding the following to /etc/apache2/httpd.conf
MaxKeepAliveRequests 500
ServerLimit 20
StartServers 2
MaxRequestWorkers 1000
ThreadsPerChild 50
This doesn't seem to be having the desired affect as I am still receiving the error message "server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers settings"
I would like to confirm that I have set the values correctly and that they are not being overridden anywhere else by checking what values Apache actually has for those settings. Is this possible with Apache?
As it turns out, I missed that httpd.conf is not included/sourced in apache2.conf by default :-/
Related
In our production Apache servers i am observing error AH00286: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
My Environment:
I am using Apache Server version: Apache/2.4.18 (Ubuntu).
MPM worker config as bellow.
<IfModule mpm_worker_module>
ServerLimit 25
StartServers 2
ThreadLimit 64
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
ListenBacklog 511
</IfModule>
During this time there are concurrent request observed to the server however the HTTP response code for all requests are 200 for almost all request excluding very few 302.
I am running Bitbucket server behind reverse proxy.
Question1) how to find my ListenBacklog that is 511 is getting filled OR how to identify its not enough?
Question2) How do i reproduce this error ( i have used AB tool in the past but seeking advice to on this)
Question3) When i get this error, will new user requests & existing user request which are in the queue will be rejected?
Question4) I have enabled monitoring of thread usage based on the scoreboard usage (/usr/sbin/apache2ctl fullstatus) numbers, it prints busy worker thread as 150 and get reset to 0 in next Min when error is observed. is Apache reset all the worker thread when it gets an error as mentioned above?
Thanks in advance for attention on this matter.
I am trying to configure mod_evasive for Apache 2.4.6 on CentOS, release 7.5.1804. I got clean install of CentOS, with clean install of Apache without serving any pages or anything (just example index.html saying hello world), and I installed mod_evasive using this tutorial: https://www.digitalocean.com/community/tutorials/how-to-protect-against-dos-and-ddos-with-mod_evasive-for-apache-on-centos-7
Everything works fine, unless i have to run the testing script which should send requests to server and get 403 error because of mod_evasive. Instead, I am getting 400 Bad Request.
I switched firewalld for IPtables, and I have port 80 open, in fact, the example page works ok from browser. Also, SELinux mode is set to permissive.
Any suggestions?
You will most likely have to change the perl script (usually saved in /usr/share/doc/libapache2-mod-evasive/examples/test.pl) to make this work, e.g.
Original line:
print $SOCKET "GET /?$_ HTTP/1.0\n\n";
Re-worked line:
print $SOCKET "GET /?$_ HTTP/1.0\r\nHost: 127.0.0.1\r\n\r\n";
From https://centosfaq.org/centos/apache-mod_evasive-problem-with-testpl/
The issue was not with mod_evasive or its configuration. In my case I had to tweak the configuration of mpm_prefork_module like below to get mod_evasive configuration to work:
StartServers 10
MinSpareServers 10
MaxSpareServers 10
MaxRequestWorkers 80
MaxConnectionsPerChild 0
Basically fix the number of servers to constant by setting StartServers = MinSpareServers = MaxSpareServers = {your_magic_number} and set MaxConnectionsPerChild=0, so that no new server processes are spawned and no re-cycling of connections happen, allowing Child to hold infinitely many concurrent connections.
Hope this saves your day!
After a few days, I found that there was an error in testing script provided with mod_evasive...
I corrected it and found out that installation was ok.
I have a webserver that should handle about 800 simultaneous connections.
For this I have configured Apache 2.4 in worker mode (on Centos 6) with this values:
ServerLimit 40
StartServers 25
MaxClients 1000
MinSpareThreads 25
MaxSpareThreads 75
MaxRequestWorkers 1000
ThreadsPerChild 25
MaxConnectionsPerChild 0
However, when I check the server-status I can not see more than 400 workers simultaneously.
In error_log file I can see this: "[mpm_worker:error] [pid 2559:tid 140190319810368] AH00286: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting", but increasing this value has no effect. (with stop/start apache after each change in configuration file).
I checked the ulimits and for apache user the value of nproc is 2048.
I mention that Apache was compiled by me and I didn't set explicitly any limit at compilation time.
What could be the cause of this limitation?
So I figured out what was causing the issue. In httpd.conf the settings for worker module were declared before loading the module, so they were ignored by apache. Moving them after "LoadModule" section fixed the problem.
I an using Apache and Tomcat with mod-jk in my project. My web application is in Tomcat instances. I am using Apache jmeter for testing http load on tomcat. But I'm getting the error Scoreboard is full,not at MaxRequestWorkers even with 1000 threads.
[mpm_event:error] [pid 24313:tid 3075319488] AH00485: scoreboard is full, not at MaxRequestWorkers
[mpm_event:error] [pid 24313:tid 3075319488] AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
I've configured http-mpm for approximately 5000 threads.
httd-mpm.conf file is:
<IfModule mpm_event_module>
ServerLimit 200
StartServers 3
MinSpareThreads 75
MaxSpareThreads 250
MaxClients 5000
ThreadsPerChild 25
MaxRequestWorkers 5000
MaxConnectionsPerChild 5000
</IfModule>
Can anybody please guide me what can be the problem in this configuration?
It seems that Apache tries to gracefully finish some of the processes and hangs while doing so. It might be a problem of Apache (https://bz.apache.org/bugzilla/show_bug.cgi?id=53555) or a problem of the application itself.
You can easily check it by looking at the server-status page of apache. In order to do that you need to enable the status.conf apache module. Then go to your-server.org/server-status. You will see all the processes and threads currently being used or gracefully finishing ("G"-state). If you have to many G's that don't go away, you found your problem. It would look sth like this:
Too many threads gracefully finishing...
The solution is pretty simple. Just set MaxConnectionsPerChild to 0 or comment it out (mpm_event.conf). What this does is just let the process continue it's work without restarting it. That's a good thing in our case, since restarting the process let to the problem of having to many G's filling up the scoreboard and eventually crashing Apache.
Hope that helped.
I had this same problem. I tried different Apache versions and MPMs.
I seem to get this alot with MPM Worker. Also error does not reoccur using Apache 2.2.2
Are you using cPanel? IF so try /upcp --force and increase StartServers to a higher amount like 50 as that's all I did to get this error away.
This serverfault Q&A suggests that the event mpm may have a bug. The recommended settings for at least partial workaround:
StartServers 3
MinSpareServers 5
MaxSpareServers 10
ServerLimit 250
MaxRequestWorkers 250
MaxConnectionsPerChild 1000
KeepAlive Off
Try EnableMMAP Off
in 00_default_settings.conf
we have a very strong server (32-cores cpu, 96GB ram) and have apache running in prefork mode. our apache2.conf file includes such settings :
<IfModule mpm_prefork_module>
StartServers 2
MinSpareServers 5
MaxSpareServers 20
ServerLimit 3000
MaxClients 3000
MaxRequestsPerChild 1000
</IfModule>
the problem is, when our website is under heavy load (when apache process count reaches 1000 to be precise) (or when setting StartServers beyond 1000), apache2 freezes and needs to be restarted. Yet there is still plenty of ram, cpu is underused and apache process count is far beyond maxclients.
My question is, what should i do to allow apache to reach the maxclients configured in the conf file ?
please consider we have already played with /etc/security/limits.conf to set max opened files and nprocs to 5000 (ulimit -a showed these values were well taken into account).
No errors are shown in /var/log/apache2/error.log
Your Apache server may have a compiled in hard limit. To change it you need to recompile your webserver. The default is 200000 which should be high enough - but packages from your linux distribution may differ.
I would rather recommend to get of static file serving from your webserver. Put an nginx or lighttp server in front of your apache. Let it serve static content (images, css, javascript, etc.) and forward dynamic request to your apache.