I am running an Apache 2.2.3 proxy server to hide my backend machines from users. I added a file upload service to my webservices; however, files larger than 128 kb are returning http Status Code of 413. I know this means Request entity too large, and I have scoured the internet looking for a solution.
I have changed my php.ini file to have max_execution_time = 3000, max_input_time = 6000, memory_limit = 128M, post_max_size = 20M, upload_max_filesize = 20M, default_socket_timeout = 6000. This didn't help, as I suspected it wouldn't. I am doing a Rest call from Java for the webservice it is not PHP.
I have changed the maxHttpHeaderSize in server.xml to 20000000 on the proxy connector to try to allow for more information to flow through. Again this did nothing and my limit is still at 128 kb.
I have also added the LimitRequestBody 20000000 Directive to the Location block for the webservice files will be uploaded from. This again didn't work.
Currently all 3 are in place without any improvement. I am still only able to send max 128 kb files through the proxy.
When I try to send a file directly to the backend machine without using the proxy it works perfectly fine without taking into account the size.
Any suggestions on how to fix this will be very much appreciated.
Thank you.
I have figured out what the problem was, and where the 128k limit occurs.
In mod_ssl it uses the default ssl negotiation size as 128k, when doing an upload we automatically renegotiate for security purposes.
I had to add and modify the SSLRenegBufferSize directive in the Locations and Directories that needed a larger than 128k buffer on renegotiation. This has worked like a charm for me.
Hope it helps anyone else that experiences this limit, or had this question.
Related
I need to be able to upload files ranging in size in 500MBs in CKan. I have installed CKAN using packager in Ubuntu 16x version. It works nice with me being able to set up organizations and creating new datasets. However, I am not able to upload files more than 100mb in size. I get error
413 Request Entity Too Large error' nginx/1.4.6 (Ubuntu)
Based on various forums and suggestions, I have changed
client_max_body_size to 1g in file /etc/nginx/nginx.conf. I have tried various ways such as setting this parameter to 1000M/1g/1G values one at a time and nothing seems to work. My all uploads beyond 100MB keep failing.
I also learnt that changing production.ini or development.ini(ckan.max_resource_size) file would help and I tried that too but it doesn't work. Please suggest what could be done. nginx is a proxy server and apache is web server that comes with default cKan packager.
In the end of /etc/nginx/nginx.conf, you have this include directive :
include /etc/nginx/sites-enabled/*;
that will include /etc/nginx/sites-enabled/ckan. This file contains the directive :
client_max_body_size 100M;
Change it, don't forget to change the ckan.max_resource_size /etc/ckan/default/production.ini, restart nginx and apache and it will work normally.
Please, Can you help me for best Apache Configuration
I own the servers for files download, Download files by direct links
ex: domain.com/files.rar
Without programming or php function
The problem: Sometimes I having a high load or stop servers
For this can you help me for best Apache Configuration
Such as:
Server Limit
Max Clients
Max Requests Per Child
Keep-Alive
Keep-Alive Timeout
Max Keep-Alive Requests
Etc.
My servers with 4GB RAM and HDD drives, and 100Mb-ps and 1GBMb-ps
Thanks.
Separate Static and Dynamic Content
Use separate servers for static and dynamic content. Apache processes serving dynamic content will carry overhead and swell to the size of the content being served, never decreasing in size. Each process will incur the size of any loaded PHP or Perl libraries. A 6MB-30MB process size [or 10% of server's memory] is not unusual, and becomes a waist of resources for serving static content.
For a more efficient use of system memory, either use mod_proxy to pass specific requests onto another Apache Server, or use a lightweight server to handle static requests:
Nginx
lighttpd
Or use a front-end caching proxy such as Squid-Cache or Varnish-Cache
The Server handling the static content goes up front.
Note that configuration settings will be quite different between a dynamic content Server and a static content Server.
mod_deflate
Reduce bandwidth by 75% and improve response time by using mod_deflate.
LoadModule deflate_module modules/mod_deflate.so
<Location />
AddOutputFilterByType DEFLATE text/html text/plain text/css text/xml application/x-javascript
</Location>
Loaded Modules
Reduce memory footprint by loading only the required modules.
Some also advise to statically compile in the needed modules, over building DSOs (Dynamic Shared Objects). Very bad advice. You will need to manually rebuild Apache every time a new version or security advisory for a module is put out, creating more work, more build related headaches, and more downtime.
mod_expires
Include mod_expires for the ability to set expiration dates for specific content; utilizing the 'If-Modified-Since' header cache control sent by the user's browser/proxy. Will save bandwidth and drastically speed up your site for [repeat] visitors.
Note that this can also be implemented with mod_headers.
KeepAlive
Enable HTTP persistent connections to improve latency times and reduce server load significantly [25% of original load is not uncommon].
prefork MPM:
KeepAlive On
KeepAliveTimeout 2
MaxKeepAliveRequests 100
worker and winnt MPMs:
KeepAlive On
KeepAliveTimeout 15
MaxKeepAliveRequests 100
With the prefork MPM, it is recommended to set 'KeepAlive' to 'Off'. Otherwise, a client will tie up an entire process for that span of time. Though in my experience, it is more useful to simply set the 'KeepAliveTimeout' value to something very low [2 seconds seems to be the ideal value]. This is not a problem with the worker MPM [thread-based], or under Windows [which only has the thread-based winnt MPM].
With the worker and winnt MPMs, the default 15 second timeout is setup to keep the connection open for the next page request; to better handle a client going from link to link. Check logs to see how long a client remains on each page before moving on to another link. Set value appropriately [do not set higher than 60 seconds].
SymLinks
Make sure 'Options +FollowSymLinks -SymLinksIfOwnerMatch' is set for all directories. Otherwise, Apache will issue an extra system call per filename component to substantiate that the filename is NOT a symlink; and more system calls to match an owner.
<Directory />
Options FollowSymLinks
</Directory>
AllowOverride
Set a default 'AllowOverride None' for your filesystem. Otherwise, for a given URL to path translation, Apache will attempt to detect an .htaccess file under every directory level of the given path.
<Directory />
AllowOverride None
</Directory>
ExtendedStatus
If mod_status is included, make sure that directive 'ExtendedStatus' is set to 'Off'. Otherwise, Apache will issue several extra time-related system calls on every request made.
ExtendedStatus Off
Timeout
Lower the amount of time the server will wait before failing a request.
Timeout 45
If you are having load-problems with your apache setup, you could also consider migrating to another system. From my personal experience I would suggest you to try nginx to serve static files.
I am using a slightly modified version of phUploader script that works fine for small files. But when I try to upload files larger than 1MB it fails by giving this undescriptive error:
General upload failure.
Nginx error log does not show any evidence. I have set
upload_max_filesize = 200M
post_max_size = 200M
memory_limit = 128M
in the php.ini
and
client_max_body_size 200M;
in nginx.conf.
The odd thing is that the script used to work well on the same server (before an upgrade on Debian 6 server) and already works well on another Debian 6 server with the same nginx and php configs. So I'm really out of ideas and appreciate your hints.
You're getting an undescriptive error because the script you're using isn't properly displaying the actual error (line 252).
Modify the script to echo or log the value of $_FILES['file']['error'] instead of "General upload failure." Then, you should see a more appropriate error code. The PHP Manual has a section that explains what the error codes mean.
I can download 1+ Gb files with no problem from my server. Today uploaded 2.3 GB file and discovered that I can't download it, downloading gets interrupted after 1GB and it happens for every user. I tried Firefox 15, latest Chrome, and IE9.
Server:
Apache 2.2.22
fcgi
eAccelerator
CPnginx
How can I resolve this?
I think I just faced the same problem, hopefully this might help others.
My server is set up to proxy requests to Apache through Nginx using the ngx_http_proxy_module. The proxy_max_temp_file_size directive of this module defaults to 1024m and is what was causing my problem.
Try adding the line proxy_max_temp_file_size 5120m; within a http, server or location directive in your nginx.conf file and reload nginx's config file with /etc/init.d/nginx reload.
Looks like the default max download size apache sets is around 2 Gb and you can override it by tweaking LimitRequestBody in your httpd.conf. Not sure why it would stop at 1 Gb though, that would make me think the problem is something else.. Hope this helps.
This is a rather complicated scenario, so I would highly appreciate any pointer to the correct direction.
So I have setup apache on server A to proxy https traffic το server B, that is a plone site behind varnish and apache.
I connect to A and can browse the site on https, everything is fine. However, problems start when I upload files, via plone's POST forms. I can upload small files (~1 MB), but when I try to upload a 50MB file, I wait all the time till the file is uploaded, and when the indication is 100%, I get a Bad gateway (The proxy server received an invalid response from an upstream server.)
It seems to me that something timeouts between the communication of A and B and instead of being redirected to the correct url, I get a Bad gateway, not to mention that the file is not uploaded.
On the apache log I see
[error] proxy: pass request body failed
As suggested on other threads, I've experimented with the following values with no luck
force-proxy-request-1.0
proxy-nokeepalive
KeepAlive
KeepAliveTimeout
proxy-initial-not-pooled
Timeout
ProxyTimeout
Sooooo..any suggestions? Thanks a million in advance!
Did you check the varnish configuration? varnish has some timeouts of its own, I am familiar with send_timeout which usually breaks downloads if they fail to finish within a few seconds (Varnish really isn't any good for large downloads, because you end doing stupid things like configuring send_timeout=7200 to make it work).
Also, set first_byte_timeout to a larger number for that backend, because a large file upload might delay plone's response just enough to cause this.
Setting the Timeout and KeepAliveTimeout in the apache virtual host file worked for me.
Example:
Timeout 3600
KeepAliveTimeout 50