I see some strange results while using apache benchmark with nginx. Please see details below -
terminal:~ directory$ ab -c 100 -n 10000 http://localhost/banner.jpg
This is ApacheBench, Version 2.3
Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests
Server Software: nginx/1.2.0
Server Hostname: localhost
Server Port: 80
Document Path: /banner.jpg
Document Length: 16697 bytes
Concurrency Level: 100
Time taken for tests: 1.224 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Total transferred: 169226562 bytes
HTML transferred: 167094858 bytes
Requests per second: 8170.40 [#/sec] (mean)
Time per request: 12.239 [ms] (mean)
Time per request: 0.122 [ms] (mean, across all concurrent requests)
Transfer rate: 135024.21 [Kbytes/sec] received
But when I increase the number of requests from 10K to 15K, then the time for test increases from 1.124 seconds to 6.760 seconds.
terminal:~ directory$ ab -c 100 -n 15000 http://localhost/banner.jpg
This is ApacheBench, Version 2.3
Benchmarking localhost (be patient)
Completed 1500 requests
Completed 3000 requests
Completed 4500 requests
Completed 6000 requests
Completed 7500 requests
Completed 9000 requests
Completed 10500 requests
Completed 12000 requests
Completed 13500 requests
Completed 15000 requests
Finished 15000 requests
Server Software: nginx/1.2.0
Server Hostname: localhost
Server Port: 80
Document Path: /banner.jpg
Document Length: 16697 bytes
Concurrency Level: 100
Time taken for tests: 6.760 seconds
Complete requests: 15000
Failed requests: 0
Write errors: 0
Total transferred: 254201718 bytes
HTML transferred: 250999689 bytes
Requests per second: 2218.92 [#/sec] (mean)
Time per request: 45.067 [ms] (mean)
Time per request: 0.451 [ms] (mean, across all concurrent requests)
Transfer rate: 36722.28 [Kbytes/sec] received
Did anyone notice this issue? Is there any tool to analyze nginx or apache benchmark tool and see what is taking some much of time?
Related
I try to apply limit_conn to my server for test purposes but it doesn't work.
There is my nginx.conf
events {}
http {
include mime.types;
limit_conn_zone $binary_remote_addr zone=addr:10m;
server {
listen 80;
location /downloads/ {
limit_conn addr 1;
return 200 "Hello";
}
}
}
Then I run Apache Benchmark
ab -n 10 -c 10 http://MY_IP_ADDRESS_OF_SERVER/downloads/
I get this output data:
This is ApacheBench, Version 2.3 <$Revision: 1757674 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking MY_IP_ADDRESS_OF_SERVER (be patient).....done
Server Software: nginx/1.16.1
Server Hostname: MY_IP_ADDRESS_OF_SERVER
Server Port: 80
Document Path: /downloads/
Document Length: 16 bytes
Concurrency Level: 10
Time taken for tests: 0.004 seconds
Complete requests: 10
Failed requests: 0
Total transferred: 1760 bytes
HTML transferred: 160 bytes
Requests per second: 2693.97 [#/sec] (mean)
Time per request: 3.712 [ms] (mean)
Time per request: 0.371 [ms] (mean, across all concurrent requests)
Transfer rate: 463.03 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 1 2 0.3 2 2
Processing: 0 1 0.6 1 2
Waiting: 0 1 0.5 1 2
Total: 2 3 0.3 3 3
Percentage of the requests served within a certain time (ms)
50% 3
66% 3
75% 3
80% 3
90% 3
95% 3
98% 3
99% 3
100% 3 (longest request)
But I expect to get in output something like this
Complete requests: 10
Failed requests: 1
Non-2xx responses: 9
It seems that limit_conn doesn't work in my server. Why doesn't it work and how can i solve this problem?
I have an Angular website with static assets of around 1.5 mb and gzipped it is around 400 kb, I have nginx as my webserver & reverse proxy to the API server, when I test nginx with Apache benchmark tool, I find huge drop in performance if I test the https site compared to the http (https is 10 times slower) & the cpu utilization & memory is not high at all (cpu 30% memory is only 1 mb!!)
I have been searching for hours & tried all possible enhancements but none worked, as far as I have read https shall not be that much slower on modern web servers (http around 1500 req/sec & https is 46 req/sec for nginx), this is mostly from the Nginx https very high connect time but I have no clue how to solve this.
Can someone advise how to improve this?
(Also to my surprise, Apache performs much better in both cases but doesn't respond if I set concurrent connections to more than 200) & this is not nginx vs apache I am just stating my situation.
Important note:
I am not comparing the 2 web servers that is not the point of this site, but generally they have comparable performance so if https in nginx is 10 times slower than Apache I feel that something is wrong in my Nginx configuration & I want to fix it.
All test are on my windows machine i7 & 16 gb ram.
Nginx http only:
C:\Apache24\bin>ab -n 5000 -c 200 http://localhost:8100/abc/index.html?param=abc
This is ApacheBench, Version 2.3 <$Revision: 1826891 $>
Server Software: nginx/1.15.4
Server Hostname: localhost
Server Port: 8100
Document Path: /abc/index.html?param=abc
Document Length: 1099 bytes
Concurrency Level: 200
Time taken for tests: 3.246 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 6665000 bytes
HTML transferred: 5495000 bytes
Requests per second: 1540.32 [#/sec] (mean)
Time per request: 129.843 [ms] (mean)
Time per request: 0.649 [ms] (mean, across all concurrent requests)
Transfer rate: 2005.12 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.3 0 16
Processing: 31 87 12.8 94 124
Waiting: 0 87 13.7 94 124
Total: 31 87 12.8 94 124
Percentage of the requests served within a certain time (ms)
50% 94
66% 94
75% 94
80% 94
90% 99
95% 109
98% 109
99% 113
100% 124 (longest request)
Nginx https (with http2 enabled)
C:\Apache24\bin>abs -n 5000 -c 200 https://localhost:8200/abc/index.html?param=abc
This is ApacheBench, Version 2.3 <$Revision: 1826891 $>
Server Software: nginx/1.15.4
Server Hostname: localhost
Server Port: 8200
SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
TLS Server Name: localhost
Document Path: /abc/index.html?param=abc
Document Length: 1099 bytes
Concurrency Level: 200
Time taken for tests: 108.985 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 6780000 bytes
HTML transferred: 5495000 bytes
Requests per second: 45.88 [#/sec] (mean)
Time per request: 4359.386 [ms] (mean)
Time per request: 21.797 [ms] (mean, across all concurrent requests)
Transfer rate: 60.75 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 16 4201 506.8 4251 4755
Processing: 0 32 12.6 31 88
Waiting: 0 32 12.6 31 88
Total: 62 4232 506.9 4283 4800
Percentage of the requests served within a certain time (ms)
50% 4283
66% 4342
75% 4413
80% 4439
90% 4484
95% 4547
98% 4694
99% 4727
100% 4800 (longest request)
Compared to Apache http (here CPU is around 90 to 100% utilized)
C:\Apache24\bin>ab -n 5000 -c 200 http://localhost:6200/abc/index.html?param=abc
Server Software: Apache/2.4.33
Server Hostname: localhost
Server Port: 6200
Document Path: /abc/index.html?param=abc
Document Length: 1099 bytes
Concurrency Level: 200
Time taken for tests: 1.781 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 6810000 bytes
HTML transferred: 5495000 bytes
Requests per second: 2806.99 [#/sec] (mean)
Time per request: 71.251 [ms] (mean)
Time per request: 0.356 [ms] (mean, across all concurrent requests)
Transfer rate: 3733.51 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 1.6 0 16
Processing: 16 69 16.0 63 125
Waiting: 0 57 16.0 63 125
Total: 16 69 16.0 63 125
Percentage of the requests served within a certain time (ms)
50% 63
66% 78
75% 78
80% 78
90% 94
95% 94
98% 94
99% 109
100% 125 (longest request)
And Apache https is as follows (http 1.1) & note that http 1.1 in nginx didn't improve its performance:
C:\Apache24\bin>abs -n 5000 -c 200 https://localhost:7200/abc/index.html?param=abc
This is ApacheBench, Version 2.3 <$Revision: 1826891 $>
Server Software: Apache/2.4.33
Server Hostname: localhost
Server Port: 7200
SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
TLS Server Name: localhost
Document Path: /abc/index.html?param=abc
Document Length: 1099 bytes
Concurrency Level: 200
Time taken for tests: 8.747 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 6810000 bytes
HTML transferred: 5495000 bytes
Requests per second: 571.60 [#/sec] (mean)
Time per request: 349.894 [ms] (mean)
Time per request: 1.749 [ms] (mean, across all concurrent requests)
Transfer rate: 760.27 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 198 42.7 188 391
Processing: 62 145 39.1 140 385
Waiting: 0 76 28.3 78 250
Total: 62 343 63.0 331 615
Percentage of the requests served within a certain time (ms)
50% 331
66% 369
75% 380
80% 389
90% 422
95% 465
98% 500
99% 536
100% 615 (longest request)
My nginx configuration:
worker_processes auto;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 8100;
server_name localhost;
location / {
root html;
index index.html index.htm;
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
server {
listen 8200 ssl http2;
server_name localhost;
ssl_certificate C:/nginx-1.13.12/conf/server.crt;
ssl_certificate_key C:/nginx-1.13.12/conf/server.key;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 10m;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
gzip on;
gzip_comp_level 1;
gzip_vary on;
gzip_types
text/css
text/javascript
text/xml
text/plain
text/x-component
application/javascript
application/json
application/xml
application/rss+xml
font/truetype
font/opentype
application/vnd.ms-fontobject
image/svg+xml;
gzip_static on;
location /ipo_reits/ {
root html;
index index.html index.htm;
## here we redirect to the homepage in case of nginx 404
try_files $uri $uri/ /ipo_reits/index.html;
# error_page 404 =301 /;
}
location /api/ {
proxy_pass https://localhost:7001/;
}
}
}
I hope that this will help someone else, It seems that is related to nginx on windows issue, I wrongly assumed that the performance of nginx on windows & linux is similar but clearly it is not.
I have tried the benchmark again with nginx on Linux on the same machine & got excellent performance as shown below
ab -n 5000 -c 200 https://localhost:8200/abc/index?param=abc
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Finished 5000 requests
Server Software: nginx/1.10.3
Server Hostname: localhost
Server Port: 8200
SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
Document Path: /abc/index?param=abc
Document Length: 1099 bytes
Concurrency Level: 200
Time taken for tests: 4.179 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 6825000 bytes
HTML transferred: 5495000 bytes
Requests per second: 1196.37 [#/sec] (mean)
Time per request: 167.173 [ms] (mean)
Time per request: 0.836 [ms] (mean, across all concurrent requests)
Transfer rate: 1594.77 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 15 141 185.3 106 1322
Processing: 1 22 13.1 20 82
Waiting: 1 14 9.5 13 81
Total: 24 163 185.7 128 1351
Percentage of the requests served within a certain time (ms)
50% 128
66% 142
75% 148
80% 155
90% 208
95% 260
98% 1100
99% 1164
100% 1351 (longest request)
Also for sustained higher load & concurrency, performance was still the same:
ab -n 25000 -c 1000 https://localhost:8200/abc/index?param=abc
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Benchmarking localhost (be patient)
Completed 2500 requests
....
Completed 25000 requests
Finished 25000 requests
Server Software: nginx/1.10.3
Server Hostname: localhost
Server Port: 8200
SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
Document Path: /abc/index?param=abc
Document Length: 1099 bytes
Concurrency Level: 1000
Time taken for tests: 20.149 seconds
Complete requests: 25000
Failed requests: 0
Total transferred: 34125000 bytes
HTML transferred: 27475000 bytes
Requests per second: 1240.76 [#/sec] (mean)
Time per request: 805.960 [ms] (mean)
Time per request: 0.806 [ms] (mean, across all concurrent requests)
Transfer rate: 1653.94 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 7 687 711.8 492 7694
Processing: 2 89 50.1 81 516
Waiting: 0 57 48.9 41 509
Total: 15 776 723.4 600 7756
Percentage of the requests served within a certain time (ms)
50% 600
66% 812
75% 1095
80% 1186
90% 1397
95% 1631
98% 3183
99% 3442
100% 7756 (longest request)
Avoid Old Cipher Suites
HTTP/2 has a huge blacklist of old and insecure ciphers, so we must avoid them. Cipher suites are a bunch of cryptographic algorithms, which describe how the transferring data should be encrypted.
We will use a really popular cipher set, whose security was approved by Internet giants like CloudFlare. It does not allow the usage of MD5 encryption (which was known as insecure since 1996, but despite this fact, its use is widespread even to this day).
Open the following configuration file:
sudo nano /etc/nginx/nginx.conf
Add this line after ssl_prefer_server_ciphers on;.
/etc/nginx/nginx.conf
ssl_ciphers EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
Save the file, and exit the text editor.
Once again, check the configuration for syntax errors:
sudo nginx -t
I have been working with apache bench for a while now and until now it worked just fine. However, today I started getting several Non-2xx responses:. In order to investigate further, I tried to run a test with a simple website, so I run:
ab -n 100 -c 10 http://www.yahoo.com/
And this is what I got:
This is ApacheBench, Version 2.3 <$Revision: 1796539 $> Copyright 1996
Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to
The Apache Software Foundation, http://www.apache.org/
Benchmarking www.yahoo.com (be patient).....done
Server Software: ATS
Server Hostname: www.yahoo.com
Server Port: 80
Document Path: /
Document Length: 8 bytes
Concurrency Level: 10
Time taken for tests: 4.898 seconds
Complete requests: 100
Failed requests: 0
Non-2xx responses: 100
Total transferred: 36875 bytes
HTML transferred: 800 bytes
Requests per second: 20.42 [#/sec] (mean)
Time per request: 489.817 [ms] (mean)
Time per request: 48.982 [ms] (mean, across all concurrent requests)
Transfer rate: 7.35 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 39 48 6.3 47 66
Processing: 50 416 89.3 415 521
Waiting: 49 254 121.0 261 512
Total: 93 464 92.1 460 575
Percentage of the requests served within a certain time (ms)
50% 460
66% 476
75% 511
80% 541
90% 569
95% 572
98% 574
99% 575
100% 575 (longest request)
As the output shows, even with an external url, I get 100% Non-2xx responses. Does anyone know how I could fix this?
Thank you!
It might be because when accessing Yahoo.com they are redirecting you, and therefore you'll get 30X responses and not 20X directly.
In the book Tomcat The Definitive Guide written by Jason Brittain with Ian F.Darwin, when using the ab tool to benchmark, the writters says,
you should benchmark by running a minimum of 100,000 HTTP requests.
Also , you may configure the test client to spawn as many client threads as you would like,
but you will not get helpful results if you set it higher than the maxThreads you set for you Connector in your Tomcat's conf/server.xml file.
By default, it is set to 150.
Then the writter recommends 149.
In my case, with 149 client threads, the running result is:
[user#apachetomcat ~]$ ab -k -n 100000 -c 149 http://10.138.0.2:8080/test.html
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 10.138.0.2 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests
Server Software:
Server Hostname: 10.138.0.2
Server Port: 8080
Document Path: /test.html
Document Length: 13 bytes
Concurrency Level: 149
Time taken for tests: 45.527 seconds
Complete requests: 100000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 99106
Total transferred: 23195530 bytes
HTML transferred: 1300000 bytes
Requests per second: 2196.48 [#/sec] (mean)
Time per request: 67.836 [ms] (mean)
Time per request: 0.455 [ms] (mean, across all concurrent requests)
Transfer rate: 497.54 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 6.8 0 70
Processing: 66 67 5.6 67 870
Waiting: 66 67 5.6 67 870
Total: 66 68 8.8 67 870
Percentage of the requests served within a certain time (ms)
50% 67
66% 67
75% 67
80% 67
90% 67
95% 68
98% 69
99% 133
100% 870 (longest request)
After increasing to 1000 client threads, the result is:
[user#apachetomcat ~]$ ab -k -n 100000 -c 1000 http://10.138.0.2:8080/test.html
This is ApacheBench, Version 2.3 <$Revision: 1430300 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 10.138.0.2 (be patient)
Completed 10000 requests
Completed 20000 requests
Completed 30000 requests
Completed 40000 requests
Completed 50000 requests
Completed 60000 requests
Completed 70000 requests
Completed 80000 requests
Completed 90000 requests
Completed 100000 requests
Finished 100000 requests
Server Software:
Server Hostname: 10.138.0.2
Server Port: 8080
Document Path: /test.html
Document Length: 13 bytes
Concurrency Level: 1000
Time taken for tests: 7.205 seconds
Complete requests: 100000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 99468
Total transferred: 23197340 bytes
HTML transferred: 1300000 bytes
Requests per second: 13879.80 [#/sec] (mean)
Time per request: 72.047 [ms] (mean)
Time per request: 0.072 [ms] (mean, across all concurrent requests)
Transfer rate: 3144.28 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 8.1 0 68
Processing: 66 69 22.3 67 1141
Waiting: 66 69 22.3 67 1141
Total: 66 70 27.5 67 1141
Percentage of the requests served within a certain time (ms)
50% 67
66% 67
75% 68
80% 68
90% 69
95% 71
98% 87
99% 139
100% 1141 (longest request)
The Requests per second increases from 2196.48/sec to 13879.80/sec, so I think this change is meaningful.
Why does the writer think it's not helpful when we set it higher than the maxThreads?
What does the incrementation of requests per second mean in my case?
I'm confused with the requests per second. It's very important to understand the writer's benchmarks in the following chapters of the book.
Can someone please walk me through the process of how I can load test my website using apache bench tool (ab)?
I want to know the following:
How many people per minute can the site handle?
Please walk me through the commands I should run to figure this out.
I tried every tutorial, and they are confusing.
The Apache benchmark tool is very basic, and while it will give you a solid idea of some performance, it is a bad idea to only depend on it if you plan to have your site exposed to serious stress in production.
Having said that, here's the most common and simplest parameters:
-c: ("Concurrency"). Indicates how many clients (people/users) will be hitting the site at the same time. While ab runs, there will be -c clients hitting the site. This is what actually decides the amount of stress your site will suffer during the benchmark.
-n: Indicates how many requests are going to be made. This just decides the length of the benchmark. A high -n value with a -c value that your server can support is a good idea to ensure that things don't break under sustained stress: it's not the same to support stress for 5 seconds than for 5 hours.
-k: This does the "KeepAlive" funcionality browsers do by nature. You don't need to pass a value for -k as it it "boolean" (meaning: it indicates that you desire for your test to use the Keep Alive header from HTTP and sustain the connection). Since browsers do this and you're likely to want to simulate the stress and flow that your site will have from browsers, it is recommended you do a benchmark with this.
The final argument is simply the host. By default it will hit http:// protocol if you don't specify it.
ab -k -c 350 -n 20000 example.com/
By issuing the command above, you will be hitting http://example.com/ with 350 simultaneous connections until 20 thousand requests are met. It will be done using the keep alive header.
After the process finishes the 20 thousand requests, you will receive feedback on stats. This will tell you how well the site performed under the stress you put it when using the parameters above.
For finding out how many people the site can handle at the same time, just see if the response times (means, min and max response times, failed requests, etc) are numbers your site can accept (different sites might desire different speeds). You can run the tool with different -c values until you hit the spot where you say "If I increase it, it starts to get failed requests and it breaks".
Depending on your website, you will expect an average number of requests per minute. This varies so much, you won't be able to simulate this with ab. However, think about it this way: If your average user will be hitting 5 requests per minute and the average response time that you find valid is 2 seconds, that means that 10 seconds out of a minute 1 user will be on requests, meaning only 1/6 of the time it will be hitting the site. This also means that if you have 6 users hitting the site with ab simultaneously, you are likely to have 36 users in simulation, even though your concurrency level (-c) is only 6.
This depends on the behavior you expect from your users using the site, but you can get it from "I expect my user to hit X requests per minute and I consider an average response time valid if it is 2 seconds". Then just modify your -c level until you are hitting 2 seconds of average response time (but make sure the max response time and stddev is still valid) and see how big you can make -c.
Please walk me through the commands I should run to figure this out.
The simplest test you can do is to perform 1000 requests, 10 at a time (which approximately simulates 10 concurrent users getting 100 pages each - over the length of the test).
ab -n 1000 -c 10 -k -H "Accept-Encoding: gzip, deflate" http://www.example.com/
-n 1000 is the number of requests to make.
-c 10 tells AB to do 10 requests at a time, instead of 1 request at a time, to better simulate concurrent visitors (vs. sequential visitors).
-k sends the KeepAlive header, which asks the web server to not shut down the connection after each request is done, but to instead keep reusing it.
I'm also sending the extra header Accept-Encoding: gzip, deflate because mod_deflate is almost always used to compress the text/html output 25%-75% - the effects of which should not be dismissed due to it's impact on the overall performance of the web server (i.e., can transfer 2x the data in the same amount of time, etc).
Results:
Benchmarking www.example.com (be patient)
Completed 100 requests
...
Finished 1000 requests
Server Software: Apache/2.4.10
Server Hostname: www.example.com
Server Port: 80
Document Path: /
Document Length: 428 bytes
Concurrency Level: 10
Time taken for tests: 1.420 seconds
Complete requests: 1000
Failed requests: 0
Keep-Alive requests: 995
Total transferred: 723778 bytes
HTML transferred: 428000 bytes
Requests per second: 704.23 [#/sec] (mean)
Time per request: 14.200 [ms] (mean)
Time per request: 1.420 [ms] (mean, across all concurrent requests)
Transfer rate: 497.76 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 1
Processing: 5 14 7.5 12 77
Waiting: 5 14 7.5 12 77
Total: 5 14 7.5 12 77
Percentage of the requests served within a certain time (ms)
50% 12
66% 14
75% 15
80% 16
90% 24
95% 29
98% 36
99% 41
100% 77 (longest request)
For the simplest interpretation, ignore everything BUT this line:
Requests per second: 704.23 [#/sec] (mean)
Multiply that by 60, and you have your requests per minute.
To get real world results, you'll want to test Wordpress instead of some static HTML or index.php file because you need to know how everything performs together: including complex PHP code, and multiple MySQL queries...
For example here is the results of testing a fresh install of Wordpress on the same system and WAMP environment (I'm using WampDeveloper, but there are also Xampp, WampServer, and others)...
Requests per second: 18.68 [#/sec] (mean)
That's 37x slower now!
After the load test, there are a number of things you can do to improve the overall performance (Requests Per Second), and also make the web server more stable under greater load (e.g., increasing the -n and the -c tends to crash Apache), that you can read about here:
Load Testing Apache with AB (Apache Bench)
Steps to set up Apache Bench(AB) on windows (IMO - Recommended).
Step 1 - Install Xampp.
Step 2 - Open CMD.
Step 3 - Go to the apache bench destination (cd C:\xampp\apache\bin) from CMD
Step 4 - Paste the command (ab -n 100 -c 10 -k -H "Accept-Encoding: gzip, deflate" http://localhost:yourport/)
Step 5 - Wait for it. Your done
Load testing your API by using just ab is not enough. However, I think it's a great tool to give you a basic idea how your site is performant.
If you want to use the ab command in to test multiple API endpoints, with different data, all at the same time in background, you need to use "nohup" command. It runs any command even when you close the terminal.
I wrote a simple script that automates the whole process, feel free to use it: http://blog.ikvasnica.com/entry/load-test-multiple-api-endpoints-concurrently-use-this-simple-shell-script
I was also curious if I can measure the speed of my script with apache abs or a construct / destruct php measure script or a php extension.
the last two have failed for me: they are approximate.
after which I thought to try "ab" and "abs".
the command "ab -k -c 350 -n 20000 example.com/" is beautiful because it's all easier!
but did anyone think to "localhost" on any apache server for example www.apachefriends.org?
you should create a folder such as "bench" in root where you have 2 files: test "bench.php" and reference "void.php".
and then: benchmark it!
bench.php
<?php
for($i=1;$i<50000;$i++){
print ('qwertyuiopasdfghjklzxcvbnm1234567890');
}
?>
void.php
<?php
?>
on your Desktop you should use a .bat file(in Windows) like this:
bench.bat
"c:\xampp\apache\bin\abs.exe" -n 10000 http://localhost/bench/void.php
"c:\xampp\apache\bin\abs.exe" -n 10000 http://localhost/bench/bench.php
pause
Now if you pay attention closely ...
the void script isn't produce zero results !!!
SO THE CONCLUSION IS: from the second result the first result should be decreased!!!
here i got :
c:\xampp\htdocs\bench>"c:\xampp\apache\bin\abs.exe" -n 10000 http://localhost/bench/void.php
This is ApacheBench, Version 2.3 <$Revision: 1826891 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests
Server Software: Apache/2.4.33
Server Hostname: localhost
Server Port: 80
Document Path: /bench/void.php
Document Length: 0 bytes
Concurrency Level: 1
Time taken for tests: 11.219 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 2150000 bytes
HTML transferred: 0 bytes
Requests per second: 891.34 [#/sec] (mean)
Time per request: 1.122 [ms] (mean)
Time per request: 1.122 [ms] (mean, across all concurrent requests)
Transfer rate: 187.15 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.3 0 1
Processing: 0 1 0.9 1 17
Waiting: 0 1 0.9 1 17
Total: 0 1 0.9 1 17
Percentage of the requests served within a certain time (ms)
50% 1
66% 1
75% 1
80% 1
90% 1
95% 2
98% 2
99% 3
100% 17 (longest request)
c:\xampp\htdocs\bench>"c:\xampp\apache\bin\abs.exe" -n 10000 http://localhost/bench/bench.php
This is ApacheBench, Version 2.3 <$Revision: 1826891 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests
Server Software: Apache/2.4.33
Server Hostname: localhost
Server Port: 80
Document Path: /bench/bench.php
Document Length: 1799964 bytes
Concurrency Level: 1
Time taken for tests: 177.006 seconds
Complete requests: 10000
Failed requests: 0
Total transferred: 18001600000 bytes
HTML transferred: 17999640000 bytes
Requests per second: 56.50 [#/sec] (mean)
Time per request: 17.701 [ms] (mean)
Time per request: 17.701 [ms] (mean, across all concurrent requests)
Transfer rate: 99317.00 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.3 0 1
Processing: 12 17 3.2 17 90
Waiting: 0 1 1.1 1 26
Total: 13 18 3.2 18 90
Percentage of the requests served within a certain time (ms)
50% 18
66% 19
75% 19
80% 20
90% 21
95% 22
98% 23
99% 26
100% 90 (longest request)
c:\xampp\htdocs\bench>pause
Press any key to continue . . .
90-17= 73 the result i expect !
Steps to perform:
Commands For load testing
Step 1 - Install Xampp.
Step 2 - Open CMD.
Step 3 - Go to the Apache bench destination (cd C:\xampp\apache\bin) from CMD.
Step 4 - Paste the command
ab -n 100 -c 10 -k -H "Accept-Encoding: gzip, deflate" http://localhost:port/
Step 5 - Wait for it. You're done
Main Command Only “Get”
ab -k -c 25 -n 2000 http://192.168.1.113:3001/api/v1/filters/3
One of the best solution for post request load testing. Click here