I'm using mod_proxy_fcgi with apache 2.4 on a debian Jessie with my C++ application which does ServerSentEvents with libfcgipp.
My problem is, that apache still buffers my response data. I confirmed that it isn't buffered by the libfcgipp library by using wireshark: After starting the fcgi application via spawn-fcgi, the data gets send to the apache web server as soon as possible. But in my browser (which I use for testing, later there will be a C++ client) it only shows up after I "killed"/closed the sending request in the server application.
So I assume I need to disable buffering for either apache or mod_proxy_fcgi (or both). But I cannot find the appropriate documentation on how to do this.
As the result of a subsequent discussion on the httpd-dev mailing list, support for flushpackets and flushwait was added to mod_proxy_fcgi in r1802040 and backported for Apache 2.4.31 in r1825765. If you are using Apache 2.4.31 or later, you can disable buffering using <Proxy flushpackets=on> as described in the BigPipe documentation:
<FilesMatch "\.php$">
# Note: The only part that varies is /path/to/app.sock
SetHandler "proxy:unix:/path/to/app.sock|fcgi://localhost/"
</FilesMatch>
# Define a matching worker.
# The part that is matched to the SetHandler is the part that
# follows the pipe. If you need to distinguish, "localhost; can
# be anything unique.
<Proxy "fcgi://localhost/" enablereuse=on flushpackets=on max=10>
</Proxy>
Note: flushpackets and flushwait are currently only included in the Apache mod_proxy_fcgi documentation for trunk because r1808129 has not been backported to the 2.4.x branch.
A few notes, since I just spent the past few hours experimenting to find the answer to this question:
It's not possible to entirely disable output buffering when using mod_proxy/mod_proxy_fcgi, however, you can still have responses streamed in chunks.
It seems, based on my experimentation, that chunks have to be at least 4096 bytes before the output will be flushed to the browser.
You can disable output buffering with the mod_fastcgi or mod_fcgi module, but those mods aren't as popular/widely used with Apache 2.4.
If you have mod_deflate enabled and don't set SetEnv no-gzip 1 for the virtualhost/directory/etc. that's streaming data, then gzip will not allow the buffer to flush until the request is complete.
I was testing things out to see how to best use Drupal 8's new BigPipe functionality for streaming requests to the client, and I posted some more notes in this GitHub issue.
Related
I have CentOS 7, with Apache 2.4.46. I have done pretty much every thing available on internet. I am getting the below error
The mpm module (prefork.c) is not supported by mod_http2. The mpm
determines how things are processed in your server. HTTP/2 has more
demands in this regard and the currently selected mpm will just not
do. This is an advisory warning. Your server will continue to work,
but the HTTP/2 protocol will be inactive.
My apache is running from /usr/local/apache2/.
I have seen, I may need to use php-fpm module to run this.
I have made changes on my live server and It is kinda stuck now. Though work impact is very less but I really want to learn what wrong am I doing
You would need to:
Disable mod_mpm_prefork
Disable mod_php (disable the php module, what ever it's named, example: libphp7)
Enable mod_mpm_event
Enable mod_http2
Enable mod_proxy
Enable mod_proxy_fcgi
Enable mod_setenvif
Install PHP-FPM using yum install php-fpm, or you might need to specify the version yum install php7-fpm or php8-fpm, and then start the PHP-FPM service after installation of that, using systemctl start php-fpm or php7-fpm/php8-fpm.
Add the following to Apache configuration:
<FilesMatch "*\.php">
SetHandler "proxy:fcgi://localhost:9000"
</FilesMatch>
If it is running as an UNIX socket, you would need to change it to:
<FilesMatch "*\.php">
SetHandler "proxy:unix:/path/to/php-fpm.sock|fcgi://localhost:9000"
</FilesMatch>
Restart Apache
I recently replaced Google Analytics by the self-hosted analytics tool Piwik.
This means that every time someone connects my website http://www.mywebsite.com, a Javascript tracking code is executed on the client, that calls my Piwik server http://www.mywebsite.com/piwik/piwik.php
Result:
on my server's Apache access.log, there is a line about http://www.mywebsite.com, that's normal
in my Piwik database, an information is stored about this visit, this is normal
on my server's Apache access.log, there is a line about the fact my Piwik server received a tracking request (executed by client with JS)
The logging part 3. is clearly too much!
From now, since Piwik in installed, my access.log is double sized!
How to remove the fact that Apache logs in access.log the connection to http://www.mywebsite.com/piwik/piwik.php ? i.e. client JS tracking code <--> Piwik server ?
The solution is to disable logging of certain requests (for example in
/etc/apache2/sites-available/000-default.conf with Debian 8):
<VirtualHost *:80>
ServerName www.mywebsite.com
DocumentRoot /home/www/mywebsite
...
SetEnvIf Request_URI "^/piwik(.*)$" dontlog
CustomLog ${APACHE_LOG_DIR}/other_vhosts_access.log vhost_combined env=!dontlog
</VirtualHost>
The Apache manual contains a section on conditional logging
https://httpd.apache.org/docs/2.4/logs.html
What you need to do is set an environment variable when a condition is met (path is piwik/piwik.php)? Then you can use that environment variable in the apache log file configuration.
Disabling your tracking logs in Apache log file is not the best idea. If your Piwik will crash for some reason or your tracking will not work for some period of time (eg. over the weekend) you will loose your data.
Apache logs can save you here, you can then replay your traffic using LogAnalytics: http://piwik.org/log-analytics/#logfile
It is better to have reasonable log file storing policy then removing data from your log.
Please, Can you help me for best Apache Configuration
I own the servers for files download, Download files by direct links
ex: domain.com/files.rar
Without programming or php function
The problem: Sometimes I having a high load or stop servers
For this can you help me for best Apache Configuration
Such as:
Server Limit
Max Clients
Max Requests Per Child
Keep-Alive
Keep-Alive Timeout
Max Keep-Alive Requests
Etc.
My servers with 4GB RAM and HDD drives, and 100Mb-ps and 1GBMb-ps
Thanks.
Separate Static and Dynamic Content
Use separate servers for static and dynamic content. Apache processes serving dynamic content will carry overhead and swell to the size of the content being served, never decreasing in size. Each process will incur the size of any loaded PHP or Perl libraries. A 6MB-30MB process size [or 10% of server's memory] is not unusual, and becomes a waist of resources for serving static content.
For a more efficient use of system memory, either use mod_proxy to pass specific requests onto another Apache Server, or use a lightweight server to handle static requests:
Nginx
lighttpd
Or use a front-end caching proxy such as Squid-Cache or Varnish-Cache
The Server handling the static content goes up front.
Note that configuration settings will be quite different between a dynamic content Server and a static content Server.
mod_deflate
Reduce bandwidth by 75% and improve response time by using mod_deflate.
LoadModule deflate_module modules/mod_deflate.so
<Location />
AddOutputFilterByType DEFLATE text/html text/plain text/css text/xml application/x-javascript
</Location>
Loaded Modules
Reduce memory footprint by loading only the required modules.
Some also advise to statically compile in the needed modules, over building DSOs (Dynamic Shared Objects). Very bad advice. You will need to manually rebuild Apache every time a new version or security advisory for a module is put out, creating more work, more build related headaches, and more downtime.
mod_expires
Include mod_expires for the ability to set expiration dates for specific content; utilizing the 'If-Modified-Since' header cache control sent by the user's browser/proxy. Will save bandwidth and drastically speed up your site for [repeat] visitors.
Note that this can also be implemented with mod_headers.
KeepAlive
Enable HTTP persistent connections to improve latency times and reduce server load significantly [25% of original load is not uncommon].
prefork MPM:
KeepAlive On
KeepAliveTimeout 2
MaxKeepAliveRequests 100
worker and winnt MPMs:
KeepAlive On
KeepAliveTimeout 15
MaxKeepAliveRequests 100
With the prefork MPM, it is recommended to set 'KeepAlive' to 'Off'. Otherwise, a client will tie up an entire process for that span of time. Though in my experience, it is more useful to simply set the 'KeepAliveTimeout' value to something very low [2 seconds seems to be the ideal value]. This is not a problem with the worker MPM [thread-based], or under Windows [which only has the thread-based winnt MPM].
With the worker and winnt MPMs, the default 15 second timeout is setup to keep the connection open for the next page request; to better handle a client going from link to link. Check logs to see how long a client remains on each page before moving on to another link. Set value appropriately [do not set higher than 60 seconds].
SymLinks
Make sure 'Options +FollowSymLinks -SymLinksIfOwnerMatch' is set for all directories. Otherwise, Apache will issue an extra system call per filename component to substantiate that the filename is NOT a symlink; and more system calls to match an owner.
<Directory />
Options FollowSymLinks
</Directory>
AllowOverride
Set a default 'AllowOverride None' for your filesystem. Otherwise, for a given URL to path translation, Apache will attempt to detect an .htaccess file under every directory level of the given path.
<Directory />
AllowOverride None
</Directory>
ExtendedStatus
If mod_status is included, make sure that directive 'ExtendedStatus' is set to 'Off'. Otherwise, Apache will issue several extra time-related system calls on every request made.
ExtendedStatus Off
Timeout
Lower the amount of time the server will wait before failing a request.
Timeout 45
If you are having load-problems with your apache setup, you could also consider migrating to another system. From my personal experience I would suggest you to try nginx to serve static files.
I'm working with Apache 2.4.2 and I need to change the LimitRequestFieldSize. supposedly (according to some Google researchs) that can be done in the httpd.conf file but I cant find that LimitRequestFieldSize neither the httpd.conf or any file within the Apache. Have any idea how I can do it?
In the end I solved simply adding LimitRequestFieldSize 500000 to the file httpd-default.conf
What you just did is open the door to a DoS attack.
Take a look at the LimitRequestFieldSize directive in the Apache documentation:
Quoting from that source:
This directive specifies the number of bytes that will be allowed in
an HTTP request header.
The LimitRequestFieldSize directive allows the server administrator to
set the limit on the allowed size of an HTTP request header field. A
server needs this value to be large enough to hold any one header
field from a normal client request. The size of a normal request
header field will vary greatly among different client implementations,
often depending upon the extent to which a user has configured their
browser to support detailed content negotiation. SPNEGO authentication
headers can be up to 12392 bytes.
This directive gives the server administrator greater control over
abnormal client request behavior, which may be useful for avoiding
some forms of denial-of-service attacks.
The documentation also specifies that the context of that directive is server config (which means server-wide) and virtual host (you can apply this directive on a per-vhost basis).
In addition, you do not mention what your OS is. In case it's Linux (which I'm more familiar with):
The configuration file, httpd.conf, is found in /etc/httpd/conf/httpd.conf (RHEL, CentOS, Fedora, Scientific Linux).
In Debian, and derivatives like Ubuntu (I don't think that is the case here, but I am mentioning it anyway just for the record), the configuration file is apache2.conf and can be found in /etc/apache2/apache2.conf.
Hope it helps.
And last but not least, you may want to check out the Unix and Linux Q&A here in StackExchange for questions like this (assuming Linux or other *Nix OS). You may have better luck at getting an answer.
This issue can be solved by updating the directive LimitRequestFieldSize either in the apache httpd.conf or in the virtual hosts.
How to add the prop in the virtual host
<VirtualHost 10.10.50.50:80>
ServerName www.mysite.com
LimitRequestFieldSize 16384
RewriteEngine On
...
...
</VirtualHost>
How to add in the httpd.conf which is inside , apache2/conf/httpd.conf
LimitRequestFieldSize 16384
But even after doing this i am still getting bad request error.
Summary:
Apache 2.4's mod_proxy does not seem to be passing the Authorization headers to PHP-FPM. Is there any way to fix this?
Long version:
I am running a server with Apache 2.4 and PHP-FPM. I am using APC for both opcode caching and user caching. As recommended by the Internet, I am using Apache 2.4's mod_proxy_fcgi to proxy the requests to FPM, like this:
ProxyPassMatch ^/(.*\.php)$ fcgi://127.0.0.1:9000/foo/bar/$1
The setup works fine, except one thing: APC's bundled apc.php, used to monitor the status of APC does not allow me to log in (required for looking at user cache entries). When I click "User cache entries" to see the user cache, it asks me to log in, clicking on the login button displays the usual HTTP login form, but entering the correct login and password yields no success. This function is working perfectly when running with mod_php instead of mod_proxy + php-fpm.
After some googling I found that other people had the same issue and figured out that it was because Apache was not passing the Authorization HTTP headers to the external FastCgi process. Unfortunately I only found a fix for mod_fastcgi, which looked like this:
FastCgiExternalServer /usr/lib/cgi-bin/php5-fcgi -host 127.0.0.1:9000 -pass-header Authorization
Is there an equivalent setting or some workaround which would also work with mod_proxy_fcgi?
Various Apache modules will strip the Authorization header, usually for "security reasons". They all have different obscure settings you can tweak to overrule this behaviour, but you'll need to determine exactly which module is to blame.
You can work around this issue by passing the header directly to PHP via the env:
SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1
See also Zend Server Windows - Authorization header is not passed to PHP script
In some scenarios, even this won't work directly and you must also change your PHP code to access $_SERVER['REDIRECT_HTTP_AUTHORIZATION'] rather than $_SERVER['HTTP_AUTHORIZATION']. See When setting environment variables in Apache RewriteRule directives, what causes the variable name to be prefixed with "REDIRECT_"?
This took me a long time to crack, since it's not documented under mod_proxy or mod_proxy_fcgi.
Add the following directive to your apache conf or .htaccess:
CGIPassAuth on
See here for details.
Recently I haven'd problem with this arch.
In my environement, the proxy to php-fpm was configured as follow:
<IfModule proxy_module>
ProxyPassMatch ^/(.*\.php)$ fcgi://127.0.0.1:9000/usr/local/apache2/htdocs/$1
ProxyTimeout 1800
</IfModule>
I fixed the issue set up the SetEnvIf directive as follow:
<IfModule proxy_module>
SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1
ProxyPassMatch ^/(.*\.php)$ fcgi://127.0.0.1:9000/usr/local/apache2/htdocs/$1
ProxyTimeout 1800
</IfModule>
I didn't find any similar settings with mod_proxy_fcgi BUT it just works for me by default. It asks for user authorization (.htaccess as usual) and the php gets it, and works like with mod_php or fastcgi and pass-header. I don't know if I was helpful...
EDIT:
it only works on teszt.com/ when using the DirectoryIndex... If i pass the php file name (even if the index.php!) it just doesn't work, don't pass the auth to the php. This is a blocker for me, but I don't want to downgrade to apache 2.2 (and mod_fastgi) so I migrate to nginx (on this machine too).