I want to know that how can I test my website (web-based program) performance with the factors of speed and response time when using MS-SQL Server and ASP.net
Actually I want to know when my users increased to 1,000,000 and more, how the speed and performance changed?
Thank you
There are a number of tools to run load tests against web sites; I like JMeter (http://jmeter.apache.org/) - open source, free, easy to use - but there are lots of others - google "web performance testing" and take your pick.
All those tools allow you to specify a number of concurrent users, wait times between page requests, and then specify one or more user journeys through the site. They will give you a report showing response times as the number of users changes.
You can install the load testing applications on any machine; most have the concept of "controller", and "load agent". The controller orchestrates the load test, while the load agents execute the tests. Generating the equivalent load of 1 million visitors is likely to require significant horse power - you may need to use one of the cloud providers of load testing solutions. Again, Google is your friend here.
Related
I was used a load of 100 using ultimate thread group for execution in NON GUI Mode .
The Execution takes place around 5 mins. only . After that my test environment got shut down. I am not able to drill down the issues. What could be the reason for server downs. my environment supports for 500 users.
How do you know your environment supports 500 users?
100 threads don't necessarily map to 100 real users, you need to consider a lot of stuff while designing your test, in particular:
Real users don't hammer the server non-stop, they need some time to "think" between operations. So make sure you add Timers between requests and configure them to represent reasonable think times.
Real users use real browsers, real browsers download embedded resources (images, scripts, styles, fonts, etc) but they do it only once, on subsequent requests the resources are being returned from cache and no actual request is being made. Make sure to add HTTP Cache Manager to your Test Plan
You need to add the load gradually, this way you will be able to state what was amount of threads (virtual users) where response time start exceeding acceptable values or errors start occurring. Generate a HTML Reporting Dashboard, look into metrics and correlate them with the increasing load.
Make sure that your application under test has enough headroom to operate in terms of CPU, RAM, Disk space, etc. You can monitor these counters using JMeter PerfMon Plugin.
Check your application logs, most probably they will have some clue to the root cause of the failure. If you're familiar with the programming language your application is written in - using a profiler tool during the load test can tell you the full story regarding what's going on, what are the most resources consuming functions and objects, etc.
I am using JMeter to test my own web application with the HTTP request. The final result seems okay. But I have one question are there any details of testing standard? Because I am writing a report which needs some data as a reference.
For example, something like the connected time and loading speed should lower than XXXXms or sample time should between XX and XX
I didn't find there are any references about this. So is there anyone knows about this which I can be used as reference data
I doubt you will be able to find "references". Normally when people invest into performance testing they have either non-functional requirements to check or they better spend money on performance testing to see if/when/where their system breaks instead of loosing it for every minute of system unexpected downtime.
So if you're developing an internal application for your company users will "have to" wait until it does its job as they don't have any alternative. On the other hand they will loose their valuable time so you will be like "serial programmer John"
If you're running a e-commerce website and it is not responsive enough - users just go to your competitors and never return.
If you still want some reference numbers:
According to a report by Akamai, 49% of respondents expected web pages to load in under 2 seconds, while 30% expect a 1-second response and 18% expected a site to load immediately. 50% of frustrated users will visit another website to accomplish their activity, while finally, 22% will leave and won't return to a website where problems have occurred
Similarly, a Dynatrace survey last year found that 75 percent of all smartphone and tablet users said they would abandon a retailer's mobile site or app if it was buggy, slow or prone to crashes.
See Why Performance Testing Matters - Looking Back at Past Black Friday Failures article for more information.
Feng,
There is no standard acceptance criteria for application performance. Most of the time Product owner takes the decision of acceptable response time, but we as a performance tester should always recommend to keep the response time within 2 seconds.
If you are running the performance testing first time of your application then its good to set the benchmark & baseline of your application based on that you can run your future tests and suggest the recommendation to the development team.
In performance testing, you can set benchmarks for following KPIs
Response time
Throughput
Also, its recommended to share detailed performance report to the stackholders so that they can easily take their decision. JMeter now provides Dashboard Report that has all the critical KPIs and performance related information.
I'm new with website performance testing field and will be using JMeter. After playing with it, I am still having troubles with identifying what to optimize in a website load time?
I'm currently still learning about the load testing - who should I give the performance report to? Developers/Programmers? or Network department? Example of an error I usually get is 502 error or timeouts.
Thanks in advance.
JMeter cannot identify anything, all it does is executing HTTP requests and measuring response times. Ideally it should be you, who takes JMeter raw results, performs analysis and creating the final report highlighting current problems and bottlenecks (and ideally what needs to be done to fix them)
Consider the following checklist:
You load test needs to be realistic, a test which doesn't represent real-life application usage does not make sense. So make sure your JMeter test carefully represents real users in terms of cookies, headers, cache, downloading images, styles and scripts, virtual user groups distribution, etc.
Increase and decrease the load gradually, this way you will be able to correlate such metrics as transactions per second and response time with increasing/decreasing number of users so make sure you apply reasonable ramp-up and ramp-down settings.
Monitor the application under test health. The reason of error may be as simple as lack of hardware resources (CPU, RAM, Disk, etc.). It can be done using i.e. PerfMon JMeter Plugin.
Do the same for JMeter instance(s). JMeter measures response time from "just before sending the request" until "last response byte arrives" so if JMeter is not able to send requests fast enough - you will have high response time without other visible reason.
Website load time is a combination of many factors including the browser rendering time, script execution time, resource download time etc. You can't use JMeter to validate the front end time. You can achieve it using chrome developer tools and other similar tools available for each browser. Refer https://developers.google.com/web/fundamentals/performance/
JMeter is primarily used for measuring the protocol level performance to ensure that you server can process the heavy workloads when it is subjected to real time stress conditions from several customers. It won't compute the java script execution time or HTML parsing time. Your JMeter script should be written in such a way that it emulates the logic of your java script executions and other presentation logic to form the request inputs and the subsequent requests.
Your question is way too open ended and you might have to start with a mentor who can help you with the whole process and train you.
Also, the mindset for functional testing and performance testing are totally different. Lot of key players in the performance area have suggested to measure the load time as part of the functional testing efforts while the majority of the server side performance is validated by the performance team.
I need to perform a load test using loadrunner to simulate load generated from external network (My home network) on servers placed in some organization in the same region.
The application which will be tested is a web site (Not Heavy one) which users can be logged into and get personal information.
I am very concerned that my home network bandwidth wouldn't be enough to generate the following load :
I need to simulate 250 Web concurrent users which will perform about 30,000 transactions in an hour.
My home network specs and statistics:
Download - 75M - 7.5 Megabyte/sec
Upload - 3.5 M - 350Kbyte / sec
From your experience is this would be enough to generate the desired load? If not what can be done to simulate load from external network?
One Load Generator is never enough from a process perspective. Consider at least three, two for primary load and one for a control set. So, right off of the bat you are likely to have issues.
Mentioned previously. Go to the cloud: Amazon, CloudAzure, GoDaddy, Rackspace, 1&1, etc... all have virtual machines that you can use for performance testing hosts running load generator software. More locations is better as this minimizes the influence of one host network over another if you are looking for representative experiences. Odds are your site will be on one backbone and some of your load generators may have to peer over from another backbone. This is not bad as this provides a more realistic view of your end user experiences from different locations.
Check your end user agreement from your home. Unless you have a business class agreement from your home such traffic may appear to be a DDOS event, setting off alarms at your service provider. Don't be surprised if you find yourself suddenly cut off from the internet without warning. I have seen this happen before with people attempting to generate load from their homes against a site.
As you can see in the comments, the amount of load you can generate is affected not only by the network bandwidth but also by the script itself and the LG machine specifications. What I mean is that there is no definitive answer to your question without taking all the parameters into account.
What you should do is create an account on one of the popular cloud providers (Amazon, Azure, HP) and create a machine with the exact specifications you need based on the parameters as you know them. Most of these services allow you to increase the machine size and the bandwidth if needed for some extra pay.
Good luck!
I want to analyze the performance (hence its weak points) of a sharepoint site doing stress test activity. What is needed to be done is call some methods exposed via web service that do the following things inside the sharepoint site:
-create a new group
-add a content to the group
-add an attachment to the content
-delete the content
-delete the previously created group
What is required is a simulation of a situation where there are 4500 users trying to do these operations concurrently (at the same time or more realistically within a short timespan, for example within 5 seconds).
We want to register the execution time of each operation (web method, for example of the "create new group"), too. I thought I could simulate these operations via a console applications using threads and stopwatchs. Is there anyone who has encountered a similar problem and can give me any existing solutions or hints to do it "the right way"? For
example how can I obtain that all threads start at the same instant? Thanks in advance.
I am a user of Visual Studio Load Testing since 2 years, and I find it very powerfull and easy to use. You can run integration tests, navigation in a web site, simulate database load, ... in fact, everything. Because it is a MS application, it is also fully compatible with all MS products like Sharepoint : it's easier to call a WCF service from a unit test than another technology (how to test nettcpbinding ?). You can also use the Visual Studio Profiler for instrumenting your code (and see what line of code is expensive or event ADO.net interactions). You can also easily extend the load testing by many extensibility points.
One important thing is that VS laod testing is "intrusive". It will note only collect response time, request lengths, ... but also all performance counters, database queries, ... All this metrics are saved in a dedicated database like SQLExpress for reporting. There is an AddOn for Excel.
Juste one important note (available for all load testing solutions !) :
You can run load tests from a developer machine or even a single dedicated machine, but you usually can't generate enough traffic to really see how the application responds (you machine can not simulate 500 concurrent users because of limited CPU/Memory/Network) . In order to simulate a lot of users, you'll set up what is known as a Load Test Rig.
A test rig is made up of a Test Controller machine and one or more Test Agent machines as shown in Figure 1. The controller manages and coordinates the agent machines and the agents generate load against the application. The test controller is also responsible for collecting performance monitor data from the servers under test and optionally from the test rig machines.
Here are some links :
MSDN
Dave's introduction
Not saying Visual Studio Load Testing is not a great tool. There are tools, like Tsung, Eventlet (and many others) that can support well over thousands of concurrent users.
Good luck.