Good day,
I need to be able to produce IIS usage reports for our SharePoint 2007 Custom application. The application runs on 2 IIS 6 Service farm for load balance/fail over purposes.
Here is the list of requirements that my management poses:
1.Daily Visitors (per farm).
2.Daily Hits (per farm).
3.Daily activity (hits, page views, visitors, avg. session duration).
4.Activity by hour of Day (for the whole farm).
5.Activity by day of week (for the whole farm combined).
6.Activity by month.
7.Page access statistics / Most popular pages.
8.Top authenticated users.
9.Browser use statistics.
10.Client OS Use statistics.
So I need to combine report results from the 2 IIS Boxes on the load balanced rotation.
So far I have tried these tools.:
1.Web Log Expert - produces desired report types, can combine IIS logs from multiple locations. But the tool has some major bugs, such as:
a. some important information is being missed from the report: in authenticated user report the test user I log into the application is missing from the report, the user is not specified in the ignore filter and that user is found in the ISS logs.
b. Bug with time and dates. Even though there is an option to adjust the time from GMT to whatever, that changes is not being obeyed by the software. It can be fixed however by converting the W3C standard log files into ncsa format with convlog utility. However, in this case, the Browser and OS usage data is gone and lost from the report.
2. Samurize - I am a bit perplexed with configuring it to report on the W3C log files. There is a lack of good tutorials or other information on that software as well.
Please recommend the tools that worked out for you and ideally answer at least a number of specified requirements.
Thanks.
Nintex has a program - no idea if it's any good (but their workflow does rock).
Related
I am using JMeter to test my own web application with the HTTP request. The final result seems okay. But I have one question are there any details of testing standard? Because I am writing a report which needs some data as a reference.
For example, something like the connected time and loading speed should lower than XXXXms or sample time should between XX and XX
I didn't find there are any references about this. So is there anyone knows about this which I can be used as reference data
I doubt you will be able to find "references". Normally when people invest into performance testing they have either non-functional requirements to check or they better spend money on performance testing to see if/when/where their system breaks instead of loosing it for every minute of system unexpected downtime.
So if you're developing an internal application for your company users will "have to" wait until it does its job as they don't have any alternative. On the other hand they will loose their valuable time so you will be like "serial programmer John"
If you're running a e-commerce website and it is not responsive enough - users just go to your competitors and never return.
If you still want some reference numbers:
According to a report by Akamai, 49% of respondents expected web pages to load in under 2 seconds, while 30% expect a 1-second response and 18% expected a site to load immediately. 50% of frustrated users will visit another website to accomplish their activity, while finally, 22% will leave and won't return to a website where problems have occurred
Similarly, a Dynatrace survey last year found that 75 percent of all smartphone and tablet users said they would abandon a retailer's mobile site or app if it was buggy, slow or prone to crashes.
See Why Performance Testing Matters - Looking Back at Past Black Friday Failures article for more information.
Feng,
There is no standard acceptance criteria for application performance. Most of the time Product owner takes the decision of acceptable response time, but we as a performance tester should always recommend to keep the response time within 2 seconds.
If you are running the performance testing first time of your application then its good to set the benchmark & baseline of your application based on that you can run your future tests and suggest the recommendation to the development team.
In performance testing, you can set benchmarks for following KPIs
Response time
Throughput
Also, its recommended to share detailed performance report to the stackholders so that they can easily take their decision. JMeter now provides Dashboard Report that has all the critical KPIs and performance related information.
I am hosting a small web-based application with Apache Web Server on EC2. On my monthly fee I usually see ~40GB usage of data transfer out, which cause about $5 or so a month.
Although this is no big money, I am curious on how these data transfer out were generated. I am sure at Midnight there won't be anyone actually visiting the web-based application. And yet there are some data transfer out at ~50M per hour (as I can see from the details report from amazon).
Is there any way to figure out what process actually generates those data-transfer out activity (even at Midnight when no one uses the web-application)?
thanks!
J.
How you looked at Boundary, may be they can help. They can monitor data going in and out of your EC2 instance (networking) You can see details like what ports the packets are coming from and where they are going to.
You have to install and agent on your machine and sign up for a trial.
Recently a Rails 3 app we built and host had some issues with the Google Analytics tracker installed. This resulted in vastly diminished statistics being tracked during the last month. We have our production logs from the app and I'm wondering if anyone knows of any way to parse these to produce visitor statistics (similar to what web analytics packages would provide). We need to deliver a stats report this week and would like to have some account for the missing visitors. Any suggestions or help would be greatly appreciated!
Probably the better place to look would be your web server logs. 5 or 10 years ago all the popular analytics software gobbled up web server logs, and there are a few free ones our there. Google "web log analytics" and see if there's anything suitable.
The problem is, web logs contain all traffic, and for many websites, this can be from all sorts of sources you don't care about, like GoogleBot and others that crawl your site to add to search indexes ... and many more. Look for software that will try to filter these out, and will also know to ignore assets (JS, CSS, images, etc.). Analytics doesn't have to worry about this kind of stuff since it's based on cookies and javascript running in a real visitor's browser.
No matter how good these programs are, there are two things you'll need to take into account.
Numbers will not align with GA, and you'll go crazy if you try to make them add up -- the differences can be astonishingly large, as much as 20% or more.
It may be more work than it's worth to get the software configured -- even if you do, the level of detail pales in comparison to GA.
If you're handy with grep, the Rails log might help you get some quick-and-dirty counts (although they also record all traffic, unless users need to log in, in which case logs may be a little less noisy).
A different approach might be to look in your database -- is there anything you can track that acts as a proxy for a visit or any other goal you have been tracking? How useful this is depends entirely on your app and what you store in the database.
Some combination of the above may be the best way to get at something, but I hate to be the bearer of bad news -- it's very likely that what you're able to glean from logs creates more confusion than it's worth. Been there, tried that :-(
Tom
I want to analyze the performance (hence its weak points) of a sharepoint site doing stress test activity. What is needed to be done is call some methods exposed via web service that do the following things inside the sharepoint site:
-create a new group
-add a content to the group
-add an attachment to the content
-delete the content
-delete the previously created group
What is required is a simulation of a situation where there are 4500 users trying to do these operations concurrently (at the same time or more realistically within a short timespan, for example within 5 seconds).
We want to register the execution time of each operation (web method, for example of the "create new group"), too. I thought I could simulate these operations via a console applications using threads and stopwatchs. Is there anyone who has encountered a similar problem and can give me any existing solutions or hints to do it "the right way"? For
example how can I obtain that all threads start at the same instant? Thanks in advance.
I am a user of Visual Studio Load Testing since 2 years, and I find it very powerfull and easy to use. You can run integration tests, navigation in a web site, simulate database load, ... in fact, everything. Because it is a MS application, it is also fully compatible with all MS products like Sharepoint : it's easier to call a WCF service from a unit test than another technology (how to test nettcpbinding ?). You can also use the Visual Studio Profiler for instrumenting your code (and see what line of code is expensive or event ADO.net interactions). You can also easily extend the load testing by many extensibility points.
One important thing is that VS laod testing is "intrusive". It will note only collect response time, request lengths, ... but also all performance counters, database queries, ... All this metrics are saved in a dedicated database like SQLExpress for reporting. There is an AddOn for Excel.
Juste one important note (available for all load testing solutions !) :
You can run load tests from a developer machine or even a single dedicated machine, but you usually can't generate enough traffic to really see how the application responds (you machine can not simulate 500 concurrent users because of limited CPU/Memory/Network) . In order to simulate a lot of users, you'll set up what is known as a Load Test Rig.
A test rig is made up of a Test Controller machine and one or more Test Agent machines as shown in Figure 1. The controller manages and coordinates the agent machines and the agents generate load against the application. The test controller is also responsible for collecting performance monitor data from the servers under test and optionally from the test rig machines.
Here are some links :
MSDN
Dave's introduction
Not saying Visual Studio Load Testing is not a great tool. There are tools, like Tsung, Eventlet (and many others) that can support well over thousands of concurrent users.
Good luck.
I have SQL 2008 R2 supporting a SharePoint 2010 environment. Some of the files will be stupidly large (i.e. 200mb). While Remote Blob Storage will be added to our platform in the future, I want to understand what reasonable performance might look like in getting a 200mb file to the user.
The network between the SharePoint WFE is only one part. Simply reading the blob from disk and passing it through the SharePoint layer MUST take some time, but I have no idea how to calculate this (or what additional information people need to help out)
Any suggestions?
That's a very complex question and requires knowledge of the environment in which you are working. The network as you rightly say is one aspect but there are many others. Traffic congestion, QoS, SQL Server versions, setup, hardware, etc Then there are issues with how the Web Front Ends are handing off the data and the HTTP pipe to the user, the browser in use, etc, etc.
Have a look at installing the Developer Dashboard for SharePoint 2010 and you'll be able to see all of the steps in fecthing and delivering files and how long each one will take you. You'll be quite surprised at how detailed the path is.
SharePoint 2010 Developer Dashboard
Regardless of the large size, you should consider activating the BlobCache feature if your large content is currently stored in a document library.
That will put it on your WFEs after first access, deliver it with proper expiration headers and completely cut the load from the SQL Server. Imagine 20 concurrent users accessing a 200mb file. If not in a blobcache, your farm will have a hard time swallowing that useless load.
The first access will be longer than your test scenario when you request it with as single user but any further access will be a fast as IIS 7 is able to deliver it and the network capacities up to your clients.
Hope it helped.