I have just read about new technique DoS that called HashDoS.
Details about it, https://cryptanalysis.eu/blog/2011/12/28/effective-dos-attacks-against-web-application-plattforms-hashdos/
This DoS technique POST a large number of parameter and trigger worse case of hashtable algorithm. Web server will take more time to do the job.
They said:
So you can keep about 10.000 Core i7 CPU cores busy processing PHP
requests using a gigabit internet connection. Alternatively for
ASP.NET, 30,000 Core2 CPU cores, or for Java Tomcat 100,000 Core i7
CPU cores, or for CRuby 1.8 1,000,000 Core i7 CPU cores, can be kept
busy with a single gigabit connection.
So, I want to limit number of parameter in POST content for my company website.
I know modsecurity can do that, but I'm not familiar with modsecurity.
Thanks in advance.
Modsecurity can limit the total length of, but not the number of, parameters. There is a module that has recently been updated with this feature (it's in beta): ModIfier: http://yoyo.org/~steve/mod_ifier.html
I took some time to understand modsecurity.
Then I found OWASP Modsecurity Core Rule Set, in file modsecurity_crs_23_request_limits.conf there was a rule that limited number of arguments in a request.
On my company webserver, using modsecurity core rule too, but there's not have this file. I don't know why :(
You can see it here (from line 30):
http://mod-security.svn.sourceforge.net/viewvc/mod-security/crs/tags/2.2.3/base_rules/modsecurity_crs_23_request_limits.conf?revision=1882&view=markup
You can find a great article that describes mod_security options on defending this here : http://blog.spiderlabs.com/2012/01/modsecurity-mitigations-for-aspnet-hashtable-dos-vulnerability-cve-2011-3414.html
Related
I am using Aerospike 3.4 and Python Clinet 1.0.41
I am able to achieve only around 1400 writes per second. This is by synchronous writes, single thread. Can anyone suggest how to improve the write speed on single thread. I didn't find Asynchronous write feature in Python client.
I have seen benchmark results on the web claiming around 8L writes per second on SSD.
My Configuration:
No of nodes:2,
CPUs: 16 per node,
Replication: 2,
Data Persistence: SSD
Thanks,
Dhanasekaran
Updated 2015-07-29:
(1) The Python Aerospike client is fully synchronous at the moment. There appeared to be no firm plans for async support in the discussion at https://discuss.aerospike.com/t/gevent-compatibility-or-async-api/1001
but Ronen has since confirmed below (see comments) that async support is planned for all clients in the future.
(2) Regarding 1.4k TPS, I experienced very similar results when hosting Aerospike in a VirtualBox VM and connecting from the physical host. This may be due to VirtualBox's networking issues. When the client (Java benchmark) was run on the same VM as the host database, my speed went up to about 8k TPS.
A good news here is 'C' client 4.0 has been released with asynchronous support. http://www.aerospike.com/download/client/c/notes.html.
Since python client wraps around C client, there are very good chances that python client will have this feature sooner.
Source code: https://github.com/sean-tan-columbia/aerospike-async-client
I've implemented an Aerospike asynchronous client as an open-source project, the source code is as above. It has been tested on Aerospike 3.3 with Aerospike Python Client 1.0.38 and Python 2.7.
I just recently started it so it's not yet mature, welcome to improve it!
We're experiencing a strange problem with our current Varnish configuration.
4x Web Servers (IIS 6.5 on Windows 2003 Server, each installed on a Intel(R) Xeon(R) CPU E5450 # 3.00GHz Quad Core, 4GB RAM)
3x Varnish Servers (varnish-3.0.3 revision 9e6a70f on Ubuntu 12.04.2 LTS - 64 bit/precise, Kernel Linux 3.2.0-29-generic, each installed on a Intel(R) Xeon(R) CPU E5450 # 3.00GHz Quad Core, 4GB RAM)
The 3 Varnish Servers have a pretty much standard, vanilla cfg: the only thing we changed was the vcl_recv and vcl_fetch in order to handle the session cookies. They are currently configured to use in-memory cache, but we already tried switching to HDD cache using an high-performance Raid Drive with the same exact results.
We had put this in place almost two years ago without problems on our old web farm, and everything worked like a blast. Now, using the machines described above and after a clean reinstall, our customers are experiencing a lot of connection problems (pending request on clients, 404 errors, missing files, etc.) when our websites are under heavy traffic. From the console log we can clearly see that these issues start happening when each Varnish reaches roughly 700 request per seconds: it just seems like they can't handle anything more. We can easily reproduce the critical scenario at any tme by shutting down one or two Varnish Servers, and see how the others react: they always start to skip beats everytime the req per seconds count reaches 700. Considering what we've experienced in the past, and looking to the Varnish specs, this doesn't seem to be normal at all.
We're trying to improve our Varnish servers performances and/or understand where the problem actually is: in order to do that, we could really use some kind of "benchmark" from other companies who are using it in a similar fashion in order to help us understand how far we are from the expected performances (I assume we are).
EDIT (added CFG files):
This is our default.vcl file.
This is the output of varnishadm >param.show output console cmd.
I'll also try to post a small part of our varnishlog file.
Thanks in advance,
To answer the question in the headline: A single Varnish server with the specifications you describe should easily serve 20k+ requests/sec with no other tuning than increasing the number of threads.
You don't give enough information (vcl, varnishlog) to answer your remaining questions.
My guess would be that you somehow end up serialising the backend requests. Check out your hit_for_pass objects and make sure they have a valid TTL set. (120s is fine)
I don't actually have any problem, just a bit curious of things.
I make a python web framework based on bottle (http://bottlepy.org/). Today I try to do a bit comparison to compare bottle WSGI server and apache server performance. I work on lubuntu 12.04, using apache 2, python 2.7, bottle development version (0.12) and get this surprising result:
As stated in the bottle documentation, the included WSGI Server is only intended for development purpose. The question is, why the development server is faster than the deployment one (apache)?
As far as I know, development server is usually slower, since it provide some "debugging" features.
Also, I never has any response in less than 100 ms when developing PHP application. But look, it is just 13 ms in bottle.
Can anybody please explain this? This is just doesn't make sense for me. A deployment server should be faster than the development one.
Development servers are not necessarily faster than production grade servers, so such an answer is a bit misleading.
The real reason in this case is likely going to be due to lazy loading of your web application on the first request that hits a process. Especially if you don't configure Apache correctly, you could hit this lazy loading quite a bit if your site doesn't get much traffic.
I would suggest you go watch my PyCon talk which deals with some of these issues.
http://lanyrd.com/2013/pycon/scdyzk/
Especially make sure you aren't using prefork MPM. Use mod_wsgi daemon mode in preference.
A deployment server should be faster than the development one.
True. And it generally is faster... in a "typical" web server environment. To test this, try spinning up 20 concurrent clients and have them make continuous requests to each version of your server. You see, you've only tested 1 request at a time--certainly not a typical web environment. I suspect you'll see different results (we're thinking of both latency AND throughput here) with tens or hundreds of concurrent requests per second.
To put it another way: At 10, 20, 100 requests per second, you might still see ~200ms latency from Apache, but you'd see much worse latency from Bottle's server.
Incidentally, the Bottle docs do refer to concurrency:
The built-in default server is based on wsgiref WSGIServer. This
non-threading HTTP server is perfectly fine for development and early
production, but may become a performance bottleneck when server load
increases.
It's also worth noting that Apache is doing a lot more than the Bottle reference server is (checking .htaccess files, dispatching to child process/thread, robust logging, etc.) and all those features necessarily add to request latency.
Finally, I'd ask whether you tuned the Apache installation. It's possible that you could configure it to be faster than it is now, e.g. by tuning the MPM, simplifying logging, disabling .htaccess checks.
Hope this helps. And if you do run a concurrent benchmark, please do share the results with us.
I'm thinking of getting my own dedicated server with the following stats:
Processor: Celeron 440 2.0 GHz
Memory: 1 GB
Primary Hard Drive : 160 GB SATA II
This will be running Windows. I have some experience with my local IIS and playing around with servers, but I have never set one up (at least a Windows one) and I've never dealt with DNS/backup/security issues.
My question has two parts:
Will this server be able to run Windows 2008, SQL Server, and possible Exchange on it without trouble. I'm worried about the processor and RAM.
Are there any guides/tutorials that talk about how to admin a windows server from start to finish. (I'm looking for something like the FAQs slicehost has for *nix based servers).
You WILL run into a problems with RAM. Refer to MS documentation and minimum requirements (SQL Server and Exchange). Also please mind that new releases of Exchange run only on 64bit systems.
Personally I would recommend installing CORE version of W2K8 if you plan to go with your described configuration.
It depends from user load. If you have about 1k unique users / month this means that probably, you will have 30 users per day - roughly 1 per hour. I think you will use more CPU working on this computer personally. So it really depends from user load.
If I were you, I would add more RAM to have something about 4 GB. RAM is the cheapest upgrade available.
You state "I've never dealt with DNS/backup/security issues."
I would suggest to you that these are the most important issues. You need to stay on top of security, applying security patches, insuring firewalls are properly configured etc.
Having been called after the fact for websites that have been hacked, I can tell you it is not pretty. Learn all you can before you stand up this server on the internet.
I have a dedicated server where i'm only the user in it.
Processor : AMD Sempron 3100+
Memory : 1GB DDR I
I'm using PHP for website. Its mostly used for downloading stuff and uploading and so.
I currently using apache, it eats too much processor.
So i came across few better then apache. I need to know which one of this good for downloading/uploading, nginx, lighttpd or litespeed?
Thanks
Its hard to beat apache in my opinion, perhaps look at enabling disable mod_deflate etc might speed things up for you.
Take a look at the benchmarks for lighttpd vs apache
I have used PHP in machines as “low end” as an AMD Geode LX800 (500 MHz, 256 MiB of RAM), using a stock Debian install and the Apache 2, PHP5 and PostgreSQL packages provided by Debian. In general, most things work well, but you want to take care of lenghty operations (e.g. avoid resizing big images with the GD extension) and always be aware of the implied cost of operations which usually seems “easy”. My particular application was serving about 25 simultaneous clients without performance problems, and in my tests it maintained a decent time-per-request up to a hundred of simultaneous clients.
You may find that installing APC will help a lot. Without it, or another byte-code cache, Apache will have to re-compile the PHP files on every invocation. While it doesn't take much effort, it does add up surprsingly quickly. You'll be surprised how useful 64MB for APC (which out of 1024Mb is not too much) will help your system, depending on how much code you are actually running (you may only need half or a quarter of that given to APC).
If it's a busy site, then optimising it with Yslow will also help, as will taking the static content (like image) away from having Apache server them. It's here that Nginx can make a small, fast improvement to page times, and memory use. I've used just that technique of a separate image server myself, and to excellent effect.
You might want to try Nginx reverse-proxying requests to a php-cgi instance. Doesn't get any more spartan than that.
But I agree with Paul, Apache is hard to beat as far as maintainability / configurability goes.
My guess is that your performance problems are related to the PHP code and not Apache. So look if you can optimize your PHP code instead.
Zeus is a high-performance web server aimed at the *Ahem* 'Static Content' industry. It will serve biblical volumes of files with minimal resources. I believe it uses asynchronous I/O, and is very quick on modest hardware.
I would recommend Apache but only 2.2.x
Here's a small benchmark that was done. and as you can see, serving php, Apache 2.2.2 is better than lighty
Definitely, I suggest lighttpd. I'm using it on different heavy load servers and it helped a lot!