As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I'm looking for a more developer friendly alternative to HP (Mercury) Quality Center.
I'm interested to find out what other tools developers & testers are using out there to manage their functional and regression test cases.
Feel free to take a look at our software TestRail. TestRail is a fresh and modern approach to test management (e.g., all real-time statistics and activity charts are built-in and are always visible; there's not need to run complicated reports etc.).
(source: gurock.com)
HP Quality Center is not affordable for 99% of the companies.
Testlink (http://teamst.org) is free but has a bunch of limitations and they do not provide any commercial support.
XStudio http://www.xqual.com is the only free valuable alternative and there is an optional (very cheap - ~90€/user) commercial support.
You should also look at QMetry test management tool. It's a pretty sleek tool and integrates well with several defect trackers and test automation tools. The web address http://www.qmetry.com
For a much cheaper commercial alternative SpiraTest from Inflectra seems to be becoming quite popular these days. For open source check out TestLink.
There are probably more than 20 different software applications that offer testcase repositories, testcase management and test management. These three phrases mean different things. Prices vary from downright cheap to enterprise class.
of the applications that do test management, all the tools probably do 80% of the same thing. standard rule of 80/20 applies. There are a couple applications that are better because they offer valuable functionality the others don't. (everyone will have their own features but they are not universally valuable)
buyers need to ask themselves these 10 questions:
1) if they want to be locked into a suite of products that does everything, albeit not as well, vs best of breed technologies,
2)if they want to use a product as designed by the vendor or customize the software to make it their own platform.
3) if they want out of the box flexibility to integrate with any other infrastructure or pay professional services for customizations,
4) if they want to submit a bug from the TM tool into a JIRA/Bugzilla or want a UI that allows one to do all defect activities within the TM tool which saves time to do more testing
5) if they want to rely on users to manually refresh screens to know about changes or use technology to have the changes automatically pushed to them
6) if they want customisable dashboards per project or per customer or if one size fits all
7) if they want to use past project history to predict finish dates for current projects and plan future projects vs use one's gut feel.
8) if they want to have their outsourced company manage them or if the company wants to manage their providers
9) if they are willing to force their organization to change software app year over year in the event a selection of an entry level tool is made or if one's better selecting an application they will use for years to come.
10) if they want to make a business decision to manage the testing team with accountability and visibility or let them testers choose a tool they like for their day to day use.
Ask your vendors if they have can do the above. I hope this helps
Another very good solution for a QA Management suite ist aqua. It includes like Quality Center the test, requirements and defect management but also a project management module. It is really user (and developer) friendly.
You also have the possibility to use different test tools directly from aqua to have the flexibility to select the optimal tool based on your automation needs independent from the test management.
We use it already in multiple projects and I got very good feedback from all project members (developer, tester, project manager, ...)
You can use QPack ALM of Orcanos which is the closest replacement for quality center. The price is reasonable and they also provide a free version for small teams, including test management, requirements management and defect tracking
Jack
Have a look at StoryTestIQ (STIQ), a mashup of Selenium and FitNesse. It's way more developer friendly than QC and can be used too to create "executable specifications" (for BDD style development).
Try Vienna 2 from http://www.nmqa.com/ as far as I know this is the only serious free test management tool, partenered and certified by microsoft.
It will aid you during all the test cycle.
MSVS2010 Test Proffesional/Ultimate
When comparing tools against Quality Center you should start by assessing how they (the other tools) stack up to QC's functionality. "HP Quality Center has several modules—requirements management, release and cycle management, test plan, test lab, defects management, and dashboard reporting—that are seamlessly integrated to allow for the smooth flow of information across testing stages." more of the write found here: http://h20195.www2.hp.com/V2/GetPDF.aspx/4AA0-9587ENW.pdf
So it would be best if when providing the responses above, that the contributor make the connections between the suggested replacement tool to QC, for example state that "your suggestion" has a test plan, test lab and defects management section but not the others, etc.
I have been a long time user of QC, and I would be very interested in some hard facts/findings of how another tool can perform just like or better than QC currently does. I would have to say that if anyone has in depth knowledge of one of the tools stated above AND they have also used QC for more than 1 yr then they should give their "detailed" Pros & Cons on the subject. This way more of us that are used to QC and find ourselves moving to a company that does not have/want QC can feel confident they can provide the same level of functionality of QC in some other tool.
Maybe it should be the actual Tool creator who should take the time to tell us all how they stand up next to QC's functionality.
Just my professional input.
We've used Selenium before and it's been quite good for our front end testing. We've also used the ArtOfTest WebAii tools before and it had a lot of good support for .net developers.
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed last year.
Improve this question
I am Test Automation engineer and recently got opportunity to explore RPA tool blueprism. After exploring I found it similar to UI automation tools supporting various technologies. Can anyone tell me what value RPA adds compare to traditional tools. I was interested to see how it can use 'intelligence' but couldn't find any feature.
Can expert in this forum help me understand what RPA can do which traditional tool can not do ?
I see similar questions but they do not give any answers I am looking for.
Thanks,
Nilesh
The technological challenges of RPA and automation tools are quite similar. RPA and testing products differ in their user experience and reporting. While testing tools often offer features to assess risk or create testing data, RPA tools have bigger focus on bot creation and user data storage.
The main difference between the two very similar techniques Test (Process) Automation and Robotic Process Automation is the Goal. Almost all the points contained in the previous posts are, in my modest opinion, consequences of the goal of both techniques:
With a Test (Process) Automation tool you want to test an application or system under test. I.e.: Want to find bugs or to prove that the quality of the application has reached a certain level. The Test Process Automation will in general run in a test environment. If something goes wrong with your test automation code or tool breaking completely down the test environment, it is not that bad: You can reset the environment and have not hurt anyone.
With a RPA tool you want to implement a real life business process. The robot works in a productive environment. If something goes wrong you may really hurt someone, i.e. damage productive data or environment. The robot does the work of a user, not just simulates it. Therefore, the robot must be "save". It must also be possible to understand what the robot exactly did with the job it got.
I hope, this help to clarify.
PS: I include the word "Process" in the context of testing, because initializing or resetting a test environment, providing secondary data, booting a system under test, running a test, collecting results, comparing actual with expected results, creating reports for test management or DevOps is usually a process you automate using some kind of "Test Process Automation" not just Test Automation.
on a less official and serious note, RPA is a marketing term for a Test Automation Robot pumped up with some kind of a Workflow Editor and some remoting Technologies
We were using standard Test Automation Robots(UFT, Selenium etc) to do some RPA with the backlash that the automated workflow was rather coded than visualized and we had to have some effort invested into the infrastructure to support scaling. (launching them en-masse and automatically)
What does it solve?
- As mentioned above, visualising worfklows and scaling - although here it has limitations
What are the weak points?
The Test Automation Robot wrapped inside the RPA can be very limited - in many cases they are less mature than state of the art TA Robots.
The promise of record & replay and drag & drop your workflow. As always - we are not yet there
It solves a problem in a way it shouldn't be solved; The GUI is for the user the APIs are for the software (or call them robots in this case). These problems should be solved by writing integrations between systems or extending existing APIs (safer, cheaper, much more reliable etc)
RPA platforms provide you a singular place where various different type of applications can be automated.
These platforms fundamentally will try to consolidate and formalize the automation effort in an enterprise. and here the word "enterprise" is key.
for small businesses where they want to automate some task/s the intern can be asked to quickly build up something. no one cares what technology or tools were used. maybe he likes python, and someone else likes VBA. so a single task may be automated using several different technologies. no one cares as long as it works. the intern leaves and the next intern figures something new...
RPA platforms on the other hand are a larger "formal" effort that will try to automate tasks that otherwise require a lot of FTE (full time employee) count to accomplish. typical RPA use cases are repetitive tasks that humans are doing all day without using much brain. think of extracting each line item from a PO (purchase order) and putting it in an excel spreadsheet and then posting it on some internal application. now imagine a single guy doing this maybe for 100s of POs a day.
You cannot imagine how uneven the IT landscape in most of the enterprises is. old applications that were either built in house a long time ago or versions that arent being updated by the vendor any more. the bigger problem is when these applications do not have any integration points, so these RPA platforms provide the lease invasive (changes to old applications or upgrading even)
i can go on all day about RPA, do let me know if you have any follow up qns. i work for one of these RPA platforms, maybe i will be able to help.
There are many flavours of RPA.
Blueprism is not an ideal example of what modern RPA should look like, consider checking out Automation Anywhere or UiPath (both offer Community Edition you could download and try for free).
While technological differences may not be that vast (and indeed RPA vendors are now looking at test automation as a market for their products), biggest differences are in the ways the platforms are engineered, to name a few:
Security-oriented approach, RPA platform is designed to make sure it could handle important data responsibly.
Design for ease of use for non-technical people. Selenium is great but you need to know how to program to use it. UiPath requires easy drag-and-drop for the same things.
Working with unstructured data inputs, like OCR'ing documents and acting on them
ML integration, for decision making or extra capabilities. E.g. NLP stuff, sentiment analysis, helping OCR recognize new document formats etc.5. Integration with third-party like chat bots or BPM
Analytical and monitoring capabilities, to make sure that you know how long your bots take to do their work and to help them if they fail
Easy of use should not be discarded:
With RPA it's a half an hour job to receive a request by mail, take data from SAP, build pivot in Excel and upload to a website in JSON format. Could you do that in other tools? Sure! Is that as easy? Usually no.
So you could do poor man RPA with Selenium or AutoIT or bash or PowerShell, it will just be not as easy and will provide less capabilities while requiring more effort every step of the way. And if you do it properly you'll end up replicating one of the RPA platforms anyway.
Also in RPA there is usually but not always central coordination mechanism (ala Selenium Grid) to orchestrate several robots (up to 10k in UiPath case) to make sure they act in sync, have some sort of work queue, shift their workload, deploy processes to them etc. This makes all the difference for enterprise usage scenarios.
RPA and UI Automation tools have some technical features that intersect. For example;
UI Component utilization: These tools may utilize a UI screen image-based approach, OS platform frameworks (i.e. Microsoft Accessibility Frameworks), or technology-centric platforms extension (i.e Chrome or Firefox extension)
End-2-End application driving: These tools have the capabilities to drive applications to complete their duties. For example, log in to an application and get some data and shift to other legacy applications and enter data.
Screen scraping: These tools have screen scraping features to retrieve some data on screens there other techniques are not applicable.
3rd party application integration: Also these tools can integrate web services or databases to get data and use these in their application usage scenarios.
...
As you see lots of features these RPA and UI Automation tools share. But, the main concept is here not technology but application methodology. In this point of view, RPA Tools
Designed to drive real-life business flow in the production environment.
May have some cognitive power to complete human-exhibit tasks (i.e. document analysis, high OCR capability, pattern recognition)
Can work attendant and unattended
Doesn’t require any programming language knowledge. That non-technic staff can easily use and learn.
Contrast to below: for implementing complex flows, gaining scalability, achieve seamless integration to Third-Party application, and native integration of external technology into your business flows (i.e. third party microblog sentence classification A.I. library that you developed your own) Some RPA tools (Voodoo RPA) have their own Embedded Development Environment (EDE) for programmers.
Invented for doing the high-value repeatable task in 7/24 in reliable and secure manners
Enhanced workflow management, impersonation, and logging capabilities
In sum, RPA tools developed to easily implement high-volume repetitive tasks in a business environment but UI automation developed to test the application's UI and verify business rules suited for the baseline paradigm.
The main difference is a type of task we can automate using traditional
automation and RPA:
Traditional automation is mainly used to automate test cases of applications/products.
RPA is mainly used to automate business processes.
If we talk in terms of coding knowledge then traditional automation required more coding knowledge in comparison to RPA.
Traditional automation can either support desktop app automation or web app automation at a time without integration of 3rd party
tools. whereas RPA can support both web and desktop app automation.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am going to look for a job as a software tester (a SDET maybe), especially for website test. I have some vague impression of this area and got a couple of specific questions as below:
Among so many documents, such as functional spec, design spec, which should I pay more attention to? How to view them in a tester's view?
Any good suggestions about writing test spec?
Any attention should be paid to website test?
These are just some questions I got now, I'll update with more shortly.
I'd like to hear your voice very much. Many thanks.
Credentials: I'm an SDET with 5 years of experience, 2 of those years testing web applications.
1- I'd say testerab has a pretty good answer. There is no single document that you can invariably rely upon across companies or even teams within a single company. Pay attention to whichever document has information.
I'd augment that answer with this advice: Don't be surprised if the documentation is insufficient. Strike up strong relationships with people who help define the product (the dev, the business owner, the program manager, etc.). You will nearly always be relying on them for some of your specifications, since it is difficult to cover everything on paper (and, as you gain expertise as a tester, you will learn to see things that others don't notice). Try to write down any "verbal specs" as you hear them, and ideally get any requests for specification clarification in writing or email. Gathering them all in a public document is wise, and may help to uncover if two people have very different ideas about what the spec "ought" to be.
2- Testerab has a good answer to this question, also, here: How Do You Keep Automated Tests in Synch With Test Plans
"1) Who reads it? 2) Who should probably read it, but currently you suspect they don't bother? (Do you know why they don't bother?) 3) What information do they need to get from it? Does it give them that info? 4) How do you currently present that information? Does that work for your readers/non-readers? 5) What sort of feedback do you need to get from the readers of your test plan? 6) Do you have any regulatory requirements that you need to satisfy with your test planning? "
Test plans, like product specs, will vary greatly depending on the needs of your group. If you are in an Agile group you may spend very little time on your test plan, doing little more than outlining the areas you need to cover - or you might not even have a test plan at all, but just a conversation with the team about what will be sufficient testing for everyone to feel confident about making decisions about the product. Other companies will have very specific guidelines you will need to follow.
Cem Kaner's classic book "Testing Computer Software" is slightly outdated, but still a good place to start and discusses test planning. I'd recommend you buy a copy quite strongly, unless someone can recommend something as authoritative that is more current. Last I heard, this was still the software testing book.
3- I'm having a little trouble understanding this question, but will do my best. Do you mean, what specifically will you need to know to test websites? First, what do you mean by websites? Do you mean web applications? If so, you will probably need to understand server / client architecture, web services, databases and basic SQL, at least rudimentary security testing, integration testing, functional testing, and will benefit from an understanding or specialization in performance testing, load testing, more security testing, and familiarity with web GUI testing with Selenium or Watir.
Some helpful things for us to know to help you get started:
How much experience do you have, both as a developer and as a tester? If you are just getting started in your career, what is your educational background?
How much experience do you have working with web applications, and in what roles (dev, test, PM, etc.)?
And, you might want to try asking some of these questions over at http://www.softwaretestingclub.com - this is a site for software testers to build community. You will get a lot of good advice and support there, so long as you are active in the community, and many of the most influential software test writers hang out there. If you do stop by there, feel free to look me up!
Hope this helps!
Edit: Added some info to answer q. #2 and to mention Cem Kaner's book.
I'm a developer with 2 years .NET experience and 1.5 years previous testing experience and an ISTQB/ISEB Foundation qualification.
To answer your questions:
1: A test manager will (typically) have a test plan and awareness of the specification documents to be tested against. Using what a developer is using is a good start. If the development methodology is agile this will probably be "user story".
A good way to look at the documents is to go through and look at where individual elements of functionality are specified and create steps to exercise them (see some of the functional techniques below).
2: What do you mean by "test spec"?
You will need to prioritise the areas of the application that need testing and understand the coverage needed. A "Test case spec". (or test script) will fit into higher level documents (like Test Plans, and Test Strategies) can be efficiently and effectively written using some Black box (Functional) techniques including:
Equivalence Partitioning,
Boundary Value Analysis,
Decision Tables,
State Transition analysis,
Use Case analysis (which could be based on a user story)
to come up with scripts that contain test cases. These techniques can be looked up online.
White box (Structural) testing involves an awareness of the code and includes:
Statement Coverage,
Decision coverage
If you're are looking at a website, this may involve JavaScript; QUnit is a testing framework for automating JavaScript testing and would be useful to research. NUnit is a commonly used test framework for .NET applications (including web applications) - NUnit was ported from its Java equivanlent JUnit and has been expanded (most probably owing to the popularity of .NET).
3: I don't understand what you mean by this? A web application will need to be tested in many different ways, and contains server and client functionality that will be tested using different techniques and the testing needs will need to be analysed. It will depend on the project.
As mentioned in other answers there are also other types of testing:
Unit - modular testing of functions at the lowest possible levels
Integration - testing functionality between different functional areas
Regression - testing to ensure that previously working functionality hasn't been broken by changes
System testing (Functional) - ensuring that the code/system under test is working as specified
System testing (Non-functional) - ensuring that aspects of the system that may not be specified are appropriate e.g. performance, load, stress, interoperability, maintainability, reliability, portability, usability
Acceptance (something called User Acceptance Testing or UAT) - ensuring that the system under test is fit for use
As mentioned in other answers, you will be retesting existing defects and inclusion of these to your test scripts is a good idea.
Hopefully this answer has given you a lot of food for thought and a good base for research. Testing qualifications or a role as a Junior Tester in an established team to build your understanding and experience could prove to be very useful.
"Among so many documents, such as functional spec, design spec, which should I pay more attention to? How to view them in a tester's view?"
Being able to extract useful information from many different sources of documentation is a critical skill for a tester, so you're right to identify that as an area you need to look at. The documents you need to look at will vary from project to project, and from company to company, so there isn't one good answer about what document you need to look at - but having good specification analysis skills will mean you'll be able to cope with whatever you're given.
For that, I'd strongly recommend this BBST course on specification based testing - it will show you how to analyse specifications, applying the Satisfice Heuristic Test Strategy model. That should also help you with your second question about writing a test spec.
http://www.testingeducation.org/BBST/BBSTSpecificationTesting.html
I'd recommend the BBST courses in general - the course materials are all available freely online, at the website above.
If you're really serious about testing, you should also consider taking the online course from the Association of Software Testing. The Foundations course is free to members, and you'll get the opportunity to practice your skills online, gain really valuable feedback on how you present yourself and your ideas, and you'll also meet a lot of outstanding testers, both as fellow pupils and as instructors. It's hard work - but if you're willing to put the effort in you will really get a tremendous amount out of it. Being able to discuss the basics with other people will really help you to get a deeper understanding.
my 50c
If you don't have test specs, or any kind of specs, you can transform your bug reports into test plan.
For each bug report that occurs, create one test item. That way - you'll have list of tests that you can follow when doing regression testing.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
About three years ago I switched from using commercial testing tools to using an open source testing framework (WatiN), and a UI automation framework I developed myself.
Since doing this I think life is much better using these libraries and Visual Studio, than expensive dedicated testing tools with either their own languages, or VBA.
Are open source automated testing tools and frameworks better than commercial products
Or do expensive commercial automated testing tools such as WinRunner, QuickTest Pro, Testpartner etc ... still have a future?
It's hard to make that sort of determination for a generic category like "testing tools". Usually, it's best to evaluate both the commercial solution and the open source solution on a case-by-case basis. From the wording of your question, it sounds like you've found the open source solution a better fit for your needs.
However, there are some points you can use in your decision making process:
Commercial Tool Benefits:
Support - usually, the company is paying people to support the product. In addition, many commercial companies offer support contracts for various levels of support. If you need support in a crunch, commercial support is the way to go.
Open Source Tool Benefits
Price - pretty obvious...it's hard to compete against free
Openness - open source projects tend to adapt open standards more readily than commercial products (a lot of commercial products also adapt open standards, but open source software tends to do so more frequently).
Self-support - If your company allows it, you can fix the bugs you find in the tools yourself. No need to wait for a third party to get around to fixing them.
It's also worth mentioning that a lot of the commercial testing tools are built on their open source counterparts. If that is the case, then you might be better off going with the open source versions, unless you need the added support.
I think you'll find that commercial products and Open Source products tend to have similar feature sets. In other cases, they may solve the problem with completely different approaches. Again, you'll probably want to make the evaluation for every case.
PyWinAuto is a great open source tool to automate thick client or windows based standalone apps.
I got the experience, that tools for developers are the first and best class of software adopted in open-source. So you can see a big amount of great testing-tools in OS. So I think in most common environments you have test-frameworks in OS that work very well. But I do not code in every environment, so maybe you want to say which language/technology you use, and we can mention some good tools (OS or commercial).
All commercial tools have some +s's and -s's.
Not all of them can be considered as an exact fit in an automation environment which an AUT requires.
I have worked with all the big names it all depends on the automation Engineer how well He/She understands the concept and leverages the tool to the fullest.
Last 5 years I came to the conclusion it is high time we divorce these Commercial tools and go after Open Source Why?
I can customize the Open source to my needs.
I know where and what went wrong and at the same time know how to fix it.
I can customize things to the extent required write from planning to the execution to Reporting is in our control as automation engineers so creativity plays a role to implement the needs of the organization and the individuals who are affected by the automation testing.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
As a long time Microsoft developer, I find MSDN to be an invaluable resource.
However, when tinkering at home I am not able to play with the best latest technologies and the different offerings coming from Microsoft as I cannot justify paying such a hefty price for what is essentially a pastime.
The Express editions are great, but fall flat when trying to use the more advanced feature I am used to from the versions I use at work. I cannot get the latest betas and play with the new offerings, not legally, anyway.
Apart from getting an MVP, how would one go about getting an MSDN subscription for an acceptable price for a non-professional environment?
I am aware of the Empower program, but I thought it was geared towards getting commercial software to market. If this is not the case, it appears like the way for me to go. Thanks!
MSDN subscriptions are per user rather than per device so as long as you're the only person using them I think you should be free to use them at home. I'm not aware of any differentiation being applied to the workplace, unless of course your workplace itself lays down such a rule.
From http://msdn.microsoft.com/en-gb/subscriptions/aa948867.aspx:
MSDN Subscriptions are licensed to
individuals who may install the
provided software without restriction.
Software provided through MSDN
Subscriptions is licensed for design,
development, test and demonstration of
your applications.
See also http://msdn.microsoft.com/en-gb/subscriptions/aa948864.aspx.
There is an Empower program that Microsoft has available. It gives you several Premium subscriptions for cheap, with the catch that you have to be an ISV working towards an actual product.
This (Not available anymore - broken link) gives you all the software you'll need for development, and even a few "real world" licenses for certain apps (like Office)
After a couple of years, you have to pay full price though. The logic being that you should have a product on the market, and can afford it.
+1 Luke's comment about using work MSDN license at home. I think that's the best answer for the OP.
Also consider
DreamSpark (for students): http://www.dreamspark.com
BizSpark (for startups building "next gen web apps"): http://www.bizspark.com
Empower (for ISVs wanting to partner with Microsoft): http://www.empowerforisv.com
(Note there is some overlap between BizSpark and Empower ... many ISVs will find them both useful)
And finally ... don't overlook trial versions and VHD's. Most Microsoft software is available for trial (30-360 days). Many are available via the "VHD Test Drive"
VHD Test Drive: http://microsoft.com/vhd
Check out the Microsoft Action Pack Development and Design subscription. It is designed to replace the Empower program and gives you access to some MS products at a great price point.
https://partner.microsoft.com/global/40132997
In agreement with comments already made - get an Empower subscription, it's geared up towards people like yourself. As I recall, you have 2 years to bring a product/solution to market (where market is very loosely defined) that uses some element of MS technology (again, where this is quite loosely defined). In return for quite a modest outlay, you get MSDN, a bunch of OS licenses and access to development tools and end-user application programs (XP, Vista, Office being obvious examples).
For instance, I develop in Delphi but write code to run on SQL Express 2005 and full-blown SQL Server 2005+, and this entitles me to purchase an Empower agreement. I get all the goodies, plus things like Visual Studio, SQL Server, Office and OS licenses. If you don't bring a solution to market in the time allocated, you can pay to extend your agreement or... well, I must admit I'm not sure. It's hard to see what bad thing can befall you if you try to produce something but ultimately fail - it's the American dream, right? You have to stop using the software at the end of the period, etc. :-)
If you want to develop for desktop Windows you really need some level of MSDN access, or a good broadband connection and some patience while you access the online materials. Empower is a fairly pain-free method of getting your hands on all the best tools for very little outlay indeed - you end up with a large pile of DVDs and CDs, and a few updates during the year. I'd say it was an essential purchase - particularly if this is viewed as a career investment, or some element of training or progression. It's not a lot of money at all (I speak as an ISV - everything I have to pay out truly comes from my pocket!).
You may want to talk to your boss about your opportunities to join MSDN for free. I work at a company using all Microsoft Software, and I get a free subscription, which comes with access to almost all of microsoft's software.
If you have an MSDN subscription at work, odds are good that your subscription license has a provision for you to be able to install things at home as well.
I know with our subscriptions here I'm allowed to install copies of operating systems and development tools at home since I obviously can't use the copies at work and at home at the same time.
Edit: I'm assuming that since you said you were a longtime MSDN developer that you are currently employed doing development on Microsoft platforms.
Even with just one licence you can get MSDN Under a Volume Licence. This is cheaper and (depending on exactly which VL program) can allow the cost to be spread across the VL period (once fully paid the licences become permanent).
Also means you get the VL builds and keys for Office/Windows rather than just the retail.
Many MVP's have gift subscriptions that they can give away, so it pays off to be visible in the community.
Speak at your local user group, start (or participate) in an open source project, start a blog... just generally get your name out there.
Eventually you'll get one (or an MVP :)).
What I've found is that if you pay attention there are plenty of opportunities to snag a free copy of Office or Visual Studio at local Microsoft events.
Good luck!
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
We have hundreds of websites which were developed in asp, .net and java and we are paying lot of money for an external agency to do a penetration testing for our sites to check for security loopholes.
Are there any (good) software (paid or free) to do this?
or.. are there any technical articles which can help me develop this tool?
There are a couple different directions you can go with automated testing tools for web applications.
First, there are the commercial web scanners, of which HP WebInspect and Rational AppScan are the two most popular. These are "all-in-one", "fire-and-forget" tools that you download and install on an internal Windows desktop and then give a URL to spider your site, scan for well-known vulnerabilities (ie, the things that have hit Bugtraq), and probe for cross-site scripting and SQL injection vulnerabilities.
Second, there are the source-code scanning tools, of which Coverity and Fortify are probably the two best known. These are tools you install on a developer's desktop to process your Java or C# source code and look for well-known patterns of insecure code, like poor input validation.
Finally, there are the penetration test tools. By far the most popular web app penetration testing tool among security professionals is Burp Suite, which you can find at http://www.portswigger.net/proxy. Others include Spike Proxy and OWASP WebScarab. Again, you'll install this on an internal Windows desktop. It will run as an HTTP proxy, and you'll point your browser at it. You'll use your applications as a normal user would, while it records your actions. You can then go back to each individual page or HTTP action and probe it for security problems.
In a complex environment, and especially if you're considering anything DIY, I strongly recommend the penetration testing tools. Here's why:
Commercial web scanners provide a lot of "breadth", along with excellent reporting. However:
They tend to miss things, because every application is different.
They're expensive (WebInspect starts in the 10's of thousands).
You're paying for stuff you don't need (like databases of known bad CGIs from the '90s).
They're hard to customize.
They can produce noisy results.
Source code scanners are more thorough than web scanners. However:
They're even more expensive than the web scanners.
They require source code to operate.
To be effective, they often require you to annotate your source code (for instance, to pick out input pathways).
They have a tendency to produce false positives.
Both commercial scanners and source code scanners have a bad habit of becoming shelfware. Worse, even if they work, their cost is comparable to getting 1 or 2 entire applications audited by a consultancy; if you trust your consultants, you're guaranteed to get better results from them than from the tools.
Penetration testing tools have downsides too:
They're much harder to use than fire-and-forget commercial scanners.
They assume some expertise in web application vulnerabilities --- you have to know what you're looking for.
They produce little or no formal reporting.
On the other hand:
They're much, much cheaper --- the best of the lot, Burp Suite, costs only 99EU, and has a free version.
They're easy to customize and add to a testing workflow.
They're much better at helping you "get to know" your applications from the inside.
Here's something you'd do with a pen-test tool for a basic web application:
Log into the application through the proxy
Create a "hit list" of the major functional areas of the application, and exercise each once.
Use the "spider" tool in your pen-test application to find all the pages and actions and handlers in the application.
For each dynamic page and each HTML form the spider uncovers, use the "fuzzer" tool (Burp calls it an "intruder") to exercise every parameter with invalid inputs. Most fuzzers come with basic test strings that include:
SQL metacharacters
HTML/Javascript escapes and metacharacters
Internationalized variants of these to evade input filters
Well-known default form field names and values
Well-known directory names, file names, and handler verbs
Spend several hours filtering the resulting errors (a typical fuzz run for one form might generate 1000 of them) looking for suspicious responses.
This is a labor-intensive, "bare-metal" approach. But when your company owns the actual applications, the bare-metal approach pays off, because you can use it to build regression test suites that will run like clockwork at each dev cycle for each app. This is a win for a bunch of reasons:
Your security testing will take a predictable amount of time and resources per application, which allows you to budget and triage.
Your team will get maximally accurate and thorough results, since your testing is going to be tuned to your applications.
It's going to cost less than commercial scanners and less than consultants.
Of course, if you go this route, you're basically turning yourself into a security consultant for your company. I don't think that's a bad thing; if you don't want that expertise, WebInspect or Fortify isn't going to help you much anyways.
I know you asked specifically about pentesting tools, but since those have been amply answered (I usually go with a mix of AppScan and trained pentester), I think it's important to point out that pentesting is not the only way to "check for security loopholes", and is often not the most effective.
Source code review tools can provide you with much better visibility into your codebase, and find many flaws that pentesting won't.
These include Fortify and OunceLabs (expensive and for many languages), VisualStudio.NET CodeAnalysis (for .NET and C++, free with VSTS, decent but not great), OWASP's LAPSE for Java (free, decent not great), CheckMarx (not cheap, fanTASTic tool for .NET and Java, but high overhead), and many more.
An important point you must note - (most of) the automated tools do not find all the vulnerabilities, not even close. You can expect the automated tools to find approximately 35-40% of the secbugs that would be found by a professional pentester; the same goes for automated vs. manual source code review.
And of course a proper SDLC (Security Development Lifecycle), including Threat Modeling, Design Review, etc, will help even more...
McAfee Secure is not a solution. The service they provide is a joke.
See below:
http://blogs.zdnet.com/security/?p=1092&tag=rbxccnbzd1
http://blogs.zdnet.com/security/?p=1068&tag=rbxccnbzd1
http://blogs.zdnet.com/security/?p=1114&tag=rbxccnbzd1
I've heard good things about SpiDynamics WebInspect as far as paid solutions go, as well as Nikto (for a free solution) and other open source tools. Nessus is an excellent tool for infrastructure in case you need to check that layer as well. You can pick up a live cd with several tools on it called Nubuntu (Auditor, Helix, or any other security based distribution works too) and then Google up some tutorials for the specific tool. Always, always make sure to scan from the local network though. You run the risk of having yourself blocked by the data center if you scan a box from the WAN without authorization. Lesson learned the hard way. ;)
I know you asked specifically about pentesting tools, but since those have been amply answered (I usually go with a mix of AppScan and trained pentester), I think it's important to point out that pentesting is not the only way to "check for security loopholes", and is often not the most effective.
Source code review tools can provide you with much better visibility into your codebase, and find many flaws that pentesting won't.
These include Fortify and OunceLabs (expensive and for many languages), VisualStudio.NET CodeAnalysis (for .NET and C++, free with VSTS, decent but not great), OWASP's LAPSE for Java (free, decent not great), CheckMarx (not cheap, fanTASTic tool for .NET and Java, but high overhead), and many more.
An important point you must note - (most of) the automated tools do not find all the vulnerabilities, not even close. You can expect the automated tools to find approximately 35-40% of the secbugs that would be found by a professional pentester; the same goes for automated vs. manual source code review.
And of course a proper SDLC (Security Development Lifecycle), including Threat Modeling, Design Review, etc, will help even more...
Skipfish, w3af, arachni, ratproxy, ZAP, WebScarab : all free and very good IMO
http://www.nessus.org/nessus/ -- Nessus will help suggests ways to make your servers better. It can't really test custom apps by itself, though I think the plugins are relatively easy to create on your own.
Take a look at Rational App Scan (used to be called Watchfire). Its not free, but has a nice UI, is dead powerful, generates reports (bespoke and against standard compliance frameworks such as Basel2) and I believe you can script it into your CI build.
How about nikto ?
For this type of testing you really want to be looking at some type of fuzz tester. SPIKE Proxy is one of a couple of fuzz testers for web apps. It is open source and written in Python. I believe there are a couple of videos from BlackHat or DefCON on using SPIKE out there somewhere, but I'm having difficulty locating them.
There are a couple of high end professional software packages that will do the web app testing and much more. One of the more popular tools would be CoreImpact
If you do plan on going through with the Pen Testing on your own I highly recommend you read through much of the OWASP Project's documentation. Specifically the OWASP Application Security Verification and Testing/Development guides. The mindset you need to thoroughly test your application is a little different than your normal development mindset (not that it SHOULD be different, but it usually is).
what about rat proxy?
A semi-automated, largely passive web
application security audit tool,
optimized for an accurate and
sensitive detection, and automatic
annotation, of potential problems and
security-relevant design patterns
based on the observation of existing,
user-initiated traffic in complex web
2.0 environments.
Detects and prioritizes broad classes
of security problems, such as dynamic
cross-site trust model considerations,
script inclusion issues, content
serving problems, insufficient XSRF
and XSS defenses, and much more
Ratproxy is currently believed to support Linux, FreeBSD, MacOS X, and Windows (Cygwin) environments.
formerly hackersafe McAfee Secure.