As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
We have hundreds of websites which were developed in asp, .net and java and we are paying lot of money for an external agency to do a penetration testing for our sites to check for security loopholes.
Are there any (good) software (paid or free) to do this?
or.. are there any technical articles which can help me develop this tool?
There are a couple different directions you can go with automated testing tools for web applications.
First, there are the commercial web scanners, of which HP WebInspect and Rational AppScan are the two most popular. These are "all-in-one", "fire-and-forget" tools that you download and install on an internal Windows desktop and then give a URL to spider your site, scan for well-known vulnerabilities (ie, the things that have hit Bugtraq), and probe for cross-site scripting and SQL injection vulnerabilities.
Second, there are the source-code scanning tools, of which Coverity and Fortify are probably the two best known. These are tools you install on a developer's desktop to process your Java or C# source code and look for well-known patterns of insecure code, like poor input validation.
Finally, there are the penetration test tools. By far the most popular web app penetration testing tool among security professionals is Burp Suite, which you can find at http://www.portswigger.net/proxy. Others include Spike Proxy and OWASP WebScarab. Again, you'll install this on an internal Windows desktop. It will run as an HTTP proxy, and you'll point your browser at it. You'll use your applications as a normal user would, while it records your actions. You can then go back to each individual page or HTTP action and probe it for security problems.
In a complex environment, and especially if you're considering anything DIY, I strongly recommend the penetration testing tools. Here's why:
Commercial web scanners provide a lot of "breadth", along with excellent reporting. However:
They tend to miss things, because every application is different.
They're expensive (WebInspect starts in the 10's of thousands).
You're paying for stuff you don't need (like databases of known bad CGIs from the '90s).
They're hard to customize.
They can produce noisy results.
Source code scanners are more thorough than web scanners. However:
They're even more expensive than the web scanners.
They require source code to operate.
To be effective, they often require you to annotate your source code (for instance, to pick out input pathways).
They have a tendency to produce false positives.
Both commercial scanners and source code scanners have a bad habit of becoming shelfware. Worse, even if they work, their cost is comparable to getting 1 or 2 entire applications audited by a consultancy; if you trust your consultants, you're guaranteed to get better results from them than from the tools.
Penetration testing tools have downsides too:
They're much harder to use than fire-and-forget commercial scanners.
They assume some expertise in web application vulnerabilities --- you have to know what you're looking for.
They produce little or no formal reporting.
On the other hand:
They're much, much cheaper --- the best of the lot, Burp Suite, costs only 99EU, and has a free version.
They're easy to customize and add to a testing workflow.
They're much better at helping you "get to know" your applications from the inside.
Here's something you'd do with a pen-test tool for a basic web application:
Log into the application through the proxy
Create a "hit list" of the major functional areas of the application, and exercise each once.
Use the "spider" tool in your pen-test application to find all the pages and actions and handlers in the application.
For each dynamic page and each HTML form the spider uncovers, use the "fuzzer" tool (Burp calls it an "intruder") to exercise every parameter with invalid inputs. Most fuzzers come with basic test strings that include:
SQL metacharacters
HTML/Javascript escapes and metacharacters
Internationalized variants of these to evade input filters
Well-known default form field names and values
Well-known directory names, file names, and handler verbs
Spend several hours filtering the resulting errors (a typical fuzz run for one form might generate 1000 of them) looking for suspicious responses.
This is a labor-intensive, "bare-metal" approach. But when your company owns the actual applications, the bare-metal approach pays off, because you can use it to build regression test suites that will run like clockwork at each dev cycle for each app. This is a win for a bunch of reasons:
Your security testing will take a predictable amount of time and resources per application, which allows you to budget and triage.
Your team will get maximally accurate and thorough results, since your testing is going to be tuned to your applications.
It's going to cost less than commercial scanners and less than consultants.
Of course, if you go this route, you're basically turning yourself into a security consultant for your company. I don't think that's a bad thing; if you don't want that expertise, WebInspect or Fortify isn't going to help you much anyways.
I know you asked specifically about pentesting tools, but since those have been amply answered (I usually go with a mix of AppScan and trained pentester), I think it's important to point out that pentesting is not the only way to "check for security loopholes", and is often not the most effective.
Source code review tools can provide you with much better visibility into your codebase, and find many flaws that pentesting won't.
These include Fortify and OunceLabs (expensive and for many languages), VisualStudio.NET CodeAnalysis (for .NET and C++, free with VSTS, decent but not great), OWASP's LAPSE for Java (free, decent not great), CheckMarx (not cheap, fanTASTic tool for .NET and Java, but high overhead), and many more.
An important point you must note - (most of) the automated tools do not find all the vulnerabilities, not even close. You can expect the automated tools to find approximately 35-40% of the secbugs that would be found by a professional pentester; the same goes for automated vs. manual source code review.
And of course a proper SDLC (Security Development Lifecycle), including Threat Modeling, Design Review, etc, will help even more...
McAfee Secure is not a solution. The service they provide is a joke.
See below:
http://blogs.zdnet.com/security/?p=1092&tag=rbxccnbzd1
http://blogs.zdnet.com/security/?p=1068&tag=rbxccnbzd1
http://blogs.zdnet.com/security/?p=1114&tag=rbxccnbzd1
I've heard good things about SpiDynamics WebInspect as far as paid solutions go, as well as Nikto (for a free solution) and other open source tools. Nessus is an excellent tool for infrastructure in case you need to check that layer as well. You can pick up a live cd with several tools on it called Nubuntu (Auditor, Helix, or any other security based distribution works too) and then Google up some tutorials for the specific tool. Always, always make sure to scan from the local network though. You run the risk of having yourself blocked by the data center if you scan a box from the WAN without authorization. Lesson learned the hard way. ;)
I know you asked specifically about pentesting tools, but since those have been amply answered (I usually go with a mix of AppScan and trained pentester), I think it's important to point out that pentesting is not the only way to "check for security loopholes", and is often not the most effective.
Source code review tools can provide you with much better visibility into your codebase, and find many flaws that pentesting won't.
These include Fortify and OunceLabs (expensive and for many languages), VisualStudio.NET CodeAnalysis (for .NET and C++, free with VSTS, decent but not great), OWASP's LAPSE for Java (free, decent not great), CheckMarx (not cheap, fanTASTic tool for .NET and Java, but high overhead), and many more.
An important point you must note - (most of) the automated tools do not find all the vulnerabilities, not even close. You can expect the automated tools to find approximately 35-40% of the secbugs that would be found by a professional pentester; the same goes for automated vs. manual source code review.
And of course a proper SDLC (Security Development Lifecycle), including Threat Modeling, Design Review, etc, will help even more...
Skipfish, w3af, arachni, ratproxy, ZAP, WebScarab : all free and very good IMO
http://www.nessus.org/nessus/ -- Nessus will help suggests ways to make your servers better. It can't really test custom apps by itself, though I think the plugins are relatively easy to create on your own.
Take a look at Rational App Scan (used to be called Watchfire). Its not free, but has a nice UI, is dead powerful, generates reports (bespoke and against standard compliance frameworks such as Basel2) and I believe you can script it into your CI build.
How about nikto ?
For this type of testing you really want to be looking at some type of fuzz tester. SPIKE Proxy is one of a couple of fuzz testers for web apps. It is open source and written in Python. I believe there are a couple of videos from BlackHat or DefCON on using SPIKE out there somewhere, but I'm having difficulty locating them.
There are a couple of high end professional software packages that will do the web app testing and much more. One of the more popular tools would be CoreImpact
If you do plan on going through with the Pen Testing on your own I highly recommend you read through much of the OWASP Project's documentation. Specifically the OWASP Application Security Verification and Testing/Development guides. The mindset you need to thoroughly test your application is a little different than your normal development mindset (not that it SHOULD be different, but it usually is).
what about rat proxy?
A semi-automated, largely passive web
application security audit tool,
optimized for an accurate and
sensitive detection, and automatic
annotation, of potential problems and
security-relevant design patterns
based on the observation of existing,
user-initiated traffic in complex web
2.0 environments.
Detects and prioritizes broad classes
of security problems, such as dynamic
cross-site trust model considerations,
script inclusion issues, content
serving problems, insufficient XSRF
and XSS defenses, and much more
Ratproxy is currently believed to support Linux, FreeBSD, MacOS X, and Windows (Cygwin) environments.
formerly hackersafe McAfee Secure.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I am from network and OS operations and not from development background. I have some experience in writing Python and PHP code, and have studied software development in college.
As a hobby project (for now), I am planning on building a small website, which will have a component to store PII and sensitive information. I have to give security the first preference, and 2nd to performance (mainly of encryption/decryption).
My target is to have everything encrypted wherever possible, and also to have code which gives as little room as possible (by default) for exploitation. The site will be hosted on linux system.
The whole idea of the project is to learn a language in depth (as much as possible), and I feel I will be much more focused if I pick on some idea that I like. And that idea involves handling PII and other sensitive information. And, if the end product turns out good, then will open it up, hence wanting to make a good choice when choosing the language to write the code in.
I have done some reading, and saw people mentioning that for the backend c/c++ would be good, as it gives good performance and flexibility, but security is not easy. The next best choice would be Ada 2012, as that gives more security than C/C++, and also does not compromise on performance. Java can also provide security, but can be slightly slower. And then Python/Ruby.
I am thinking that Ada 2012 may be a good choice, but I don't want to get into a position wherein I learn it to some extent and then realize that I would have been better off with Python or Java or some other language.
I want to know from the experts answers to these 3 specific questions:
Which language will be ideal to develop this site, so that :
the best available encryption/decryption libraries can be used?
the features of the language can be leveraged to write inherently secure code?
Also, the more performance can be gained, the better?
Please advise. And also, if someone has done website (specially those handling PII) development using Ada, please share your experience.
I know each and every language has advantages and disadvantages, and the intent behind my query is to learn from the experience of those who have spent many years as website developers, and have used multiples languages and frameworks to develop websites handling sensitive data. If the mods think the question can start a good vs bad language war, I apologize as that is not the intent, and I will close the question.
The features of the language can be leveraged to write inherently secure code? Ada's type system supports writing code that validates data before usage. It's a feature of the language that helps with IT-security. But of course there is much more to IT-security than that. Configuring the firewalls, for example using systemd to specify how many processes of an executable is allowed to run simultaneously by the OS, how much memory each process is allowed to allocate, which directories the different applications have access to and permissions, and so on. I am sure there is lots I don't mention nor cover in this short response.
The best available encryption/decryption libraries can be used? The best library to my knowledge for cryptography is the Ada-Crypto-Library: https://github.com/cforler/Ada-Crypto-Library.
But what is asked for is making a safe web application. For encrypting the Secure Socket Layer (https) the Ada-Crypto-Library is not used in any http server implementation that I am aware of. If one wants to develop a web application in Ada there are three options that I see: AWS (Ada Web Server) from AdaCore and that is included in the Community Edition of the GNAT compiler (www.adacore.com), the http server implementation in Dmitry Kazakov's simple components (http://www.dmitry-kazakov.de/ada/components.htm) or GNOGA (www.gnoga.com) that is implemented on top of Dmitry Kazakov's Simple Componenets. Oh wait, Matreshka may also be used but I haven't used it yet so I cannot comment (http://forge.ada-ru.org/matreshka).
According to the documentation of AWS it can be compiled to use either OpenSSL, LibreSSL or GNUTLS (http://docs.adacore.com/live/wave/aws/html/aws_ug/building_aws.html#requirements).
With Simple Components and GNOGA the Secure Socket Layer implementation is provided by GNUTLS.
Another option for providing SSL to a web application is to use the Apache web server as a proxy that handles the encryption (I have never done such a setup, only heard of the existence of this possibility).
Also, the more performance can be gained, the better? I like performance and how to get the best performance is a vast subject. On the whole I think Ada is good programming language choice for those who like performance. Of the top of my head, to maximise performance using Ada one should:
1) When using the standard containers and using the GNAT compiler one may use "pragma Suppress (Tampering_Checks);" to increase the performance of ones application. Not everyone agrees with this view to have one debug build with the tampering checks turned on and then one release build with the checks off since one trades safety for performance, but it has a noticable impact on performance. An alternative to the standard containers one may use the Ada-Trait based containers (https://github.com/AdaCore/ada-traits-containers). They may be the World's most well designed containers for the Ada programming language.
2) Avoid usage of Unbounded_String in the standard library. One may use instead the XString unbounded string implementation in the GNATColl library and may give a 10x performance boost. Also consider allocating ordinary Strings inside memory pools (or subpools) if possible (I've done that in the Xml_Parser application in the repository: https://github.com/joakim-strandberg/wayland_ada_binding)
EDIT: I deliberately avoid arguing whether or not Ada, Java or Python is better and instead focus on, if you would do it in Ada, what would you need to do and consider.
short answer - No,such a system is never possible. PII is less sensitive than a nuclear program.
Long answer --
1. the best available encryption/decryption libraries can be used?
-As your question mentions encryption comes with decryption, the SHA-1 is broken now check alternatives (https://www.forbes.com/sites/forbestechcouncil/2017/04/13/sha-1-encryption-has-been-broken-now-what/#35e33f317ee7) and if you want to dig deep it is not about libraries it is about the algorithm used for the job.Any encryption can be broken sooner or later.
2. The features of the language can be leveraged to write inherently secure code?
There is nothing as secure language or features of language to save you there are few frameworks based on some security princiapls;just follow a set of practices to make code secure.
You follow them you would be safe if you don't there could be trouble and there are around 5000 free tools (unofficial number)that can be run on a website to break it.Are you willing to test your system against so many number of tools ?
3.Also, the more performance can be gained, the better?
-The stronger the encryption and security the more performance you lose always a trade off so choose your treadmill.
Security is a very vague and broad term and everyone gets hacked even the likes of yahoo and Symantec.(https://gizmodo.com/researchers-made-a-clever-tool-to-detect-hacks-companie-1821293404)
still not convinced here is the state of the art -https://en.wikipedia.org/wiki/Stuxnet but even this is 20 years old and just 500-kilobyte of threat.
My 2 cents - As we deal in 0 and 1 please define clear goals in terms of security and performance the make a poc(proof of concept) and run some benchmarks test.
Is there any free downloadable test management tool for Windows 7.
I do not want to use any web-based tool where-in I have to sign-up.
All of the test management tools and bug trackers I've used (SpiraTest, Quality Center, Jira) have been web-based. My personal opinion on test management tools, for a lone / single tester, is that they are often too complex and too restrictive to be useful. Sure you can link requirements to tests but you have to input everything into a requirements section, then link it to each test set. Seems much easier to just create a matrix or two in Excel and track your coverage.
All test management tools also seem to think its possible to record your tests in a simple step by step manner with expected and actual results. I personally find this type of system to be too restrictive. A good test design may not yield itself to simple step by step instructions.
Having a good bug tracker that you and your development team can use is an excellent idea. There are lots of free / cheap options however they are all also web-based. Bugzilla is free and takes a bit of time to setup, however you can probably find a free VM somewhere which will get you up and running quickly (Google around). Trac is a free wiki of sort with some bug reporting abilities. SpiraTest is cheap - like $50 for a single user and has test managment and bug tracking capabilities, although if you have a few developers you'll want a larger license that may cost upwards of $200/300.
I personally like Jira and Atlassian's software. I'm playing around with Jira, Confluence and Bonfire for my testing. I can create wikis, etc. for my test ideas in Confluence, report bugs in Jira and use Bonfire for exploratory testing. If you want to host the software yourself (install it on VMs on your machine) Atlassian has this really cool deal called Starter for just $10/each item.
My experience has taught me that having a web-based bug tracking tool is a must have but Test Management tools tend to be a waste. However I encourage you to look around and explore. Most, if not all, of these tools have trial programs. You'll have to sign up / register but that's the cost of doing business. Wikipedia has this list on Test management Tools so have at it.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am going to look for a job as a software tester (a SDET maybe), especially for website test. I have some vague impression of this area and got a couple of specific questions as below:
Among so many documents, such as functional spec, design spec, which should I pay more attention to? How to view them in a tester's view?
Any good suggestions about writing test spec?
Any attention should be paid to website test?
These are just some questions I got now, I'll update with more shortly.
I'd like to hear your voice very much. Many thanks.
Credentials: I'm an SDET with 5 years of experience, 2 of those years testing web applications.
1- I'd say testerab has a pretty good answer. There is no single document that you can invariably rely upon across companies or even teams within a single company. Pay attention to whichever document has information.
I'd augment that answer with this advice: Don't be surprised if the documentation is insufficient. Strike up strong relationships with people who help define the product (the dev, the business owner, the program manager, etc.). You will nearly always be relying on them for some of your specifications, since it is difficult to cover everything on paper (and, as you gain expertise as a tester, you will learn to see things that others don't notice). Try to write down any "verbal specs" as you hear them, and ideally get any requests for specification clarification in writing or email. Gathering them all in a public document is wise, and may help to uncover if two people have very different ideas about what the spec "ought" to be.
2- Testerab has a good answer to this question, also, here: How Do You Keep Automated Tests in Synch With Test Plans
"1) Who reads it? 2) Who should probably read it, but currently you suspect they don't bother? (Do you know why they don't bother?) 3) What information do they need to get from it? Does it give them that info? 4) How do you currently present that information? Does that work for your readers/non-readers? 5) What sort of feedback do you need to get from the readers of your test plan? 6) Do you have any regulatory requirements that you need to satisfy with your test planning? "
Test plans, like product specs, will vary greatly depending on the needs of your group. If you are in an Agile group you may spend very little time on your test plan, doing little more than outlining the areas you need to cover - or you might not even have a test plan at all, but just a conversation with the team about what will be sufficient testing for everyone to feel confident about making decisions about the product. Other companies will have very specific guidelines you will need to follow.
Cem Kaner's classic book "Testing Computer Software" is slightly outdated, but still a good place to start and discusses test planning. I'd recommend you buy a copy quite strongly, unless someone can recommend something as authoritative that is more current. Last I heard, this was still the software testing book.
3- I'm having a little trouble understanding this question, but will do my best. Do you mean, what specifically will you need to know to test websites? First, what do you mean by websites? Do you mean web applications? If so, you will probably need to understand server / client architecture, web services, databases and basic SQL, at least rudimentary security testing, integration testing, functional testing, and will benefit from an understanding or specialization in performance testing, load testing, more security testing, and familiarity with web GUI testing with Selenium or Watir.
Some helpful things for us to know to help you get started:
How much experience do you have, both as a developer and as a tester? If you are just getting started in your career, what is your educational background?
How much experience do you have working with web applications, and in what roles (dev, test, PM, etc.)?
And, you might want to try asking some of these questions over at http://www.softwaretestingclub.com - this is a site for software testers to build community. You will get a lot of good advice and support there, so long as you are active in the community, and many of the most influential software test writers hang out there. If you do stop by there, feel free to look me up!
Hope this helps!
Edit: Added some info to answer q. #2 and to mention Cem Kaner's book.
I'm a developer with 2 years .NET experience and 1.5 years previous testing experience and an ISTQB/ISEB Foundation qualification.
To answer your questions:
1: A test manager will (typically) have a test plan and awareness of the specification documents to be tested against. Using what a developer is using is a good start. If the development methodology is agile this will probably be "user story".
A good way to look at the documents is to go through and look at where individual elements of functionality are specified and create steps to exercise them (see some of the functional techniques below).
2: What do you mean by "test spec"?
You will need to prioritise the areas of the application that need testing and understand the coverage needed. A "Test case spec". (or test script) will fit into higher level documents (like Test Plans, and Test Strategies) can be efficiently and effectively written using some Black box (Functional) techniques including:
Equivalence Partitioning,
Boundary Value Analysis,
Decision Tables,
State Transition analysis,
Use Case analysis (which could be based on a user story)
to come up with scripts that contain test cases. These techniques can be looked up online.
White box (Structural) testing involves an awareness of the code and includes:
Statement Coverage,
Decision coverage
If you're are looking at a website, this may involve JavaScript; QUnit is a testing framework for automating JavaScript testing and would be useful to research. NUnit is a commonly used test framework for .NET applications (including web applications) - NUnit was ported from its Java equivanlent JUnit and has been expanded (most probably owing to the popularity of .NET).
3: I don't understand what you mean by this? A web application will need to be tested in many different ways, and contains server and client functionality that will be tested using different techniques and the testing needs will need to be analysed. It will depend on the project.
As mentioned in other answers there are also other types of testing:
Unit - modular testing of functions at the lowest possible levels
Integration - testing functionality between different functional areas
Regression - testing to ensure that previously working functionality hasn't been broken by changes
System testing (Functional) - ensuring that the code/system under test is working as specified
System testing (Non-functional) - ensuring that aspects of the system that may not be specified are appropriate e.g. performance, load, stress, interoperability, maintainability, reliability, portability, usability
Acceptance (something called User Acceptance Testing or UAT) - ensuring that the system under test is fit for use
As mentioned in other answers, you will be retesting existing defects and inclusion of these to your test scripts is a good idea.
Hopefully this answer has given you a lot of food for thought and a good base for research. Testing qualifications or a role as a Junior Tester in an established team to build your understanding and experience could prove to be very useful.
"Among so many documents, such as functional spec, design spec, which should I pay more attention to? How to view them in a tester's view?"
Being able to extract useful information from many different sources of documentation is a critical skill for a tester, so you're right to identify that as an area you need to look at. The documents you need to look at will vary from project to project, and from company to company, so there isn't one good answer about what document you need to look at - but having good specification analysis skills will mean you'll be able to cope with whatever you're given.
For that, I'd strongly recommend this BBST course on specification based testing - it will show you how to analyse specifications, applying the Satisfice Heuristic Test Strategy model. That should also help you with your second question about writing a test spec.
http://www.testingeducation.org/BBST/BBSTSpecificationTesting.html
I'd recommend the BBST courses in general - the course materials are all available freely online, at the website above.
If you're really serious about testing, you should also consider taking the online course from the Association of Software Testing. The Foundations course is free to members, and you'll get the opportunity to practice your skills online, gain really valuable feedback on how you present yourself and your ideas, and you'll also meet a lot of outstanding testers, both as fellow pupils and as instructors. It's hard work - but if you're willing to put the effort in you will really get a tremendous amount out of it. Being able to discuss the basics with other people will really help you to get a deeper understanding.
my 50c
If you don't have test specs, or any kind of specs, you can transform your bug reports into test plan.
For each bug report that occurs, create one test item. That way - you'll have list of tests that you can follow when doing regression testing.
I am nearly finished a web application. I need to test it and find the security issues before it release. Is there any methods / guideline to do this kind of testing? Or is there any tools to help me check my application is ready to go online? Thank you.
I would say:
check that there are no warnings or errors even in strict mode (error report).
In case you store any sensitive data (as passwords, credit cards, etc.) be sure they are encrypted with non-standard algorithms. Use SSL and try to be somehow paranoid with it.
Set your database with specific accesses by action and hosts, and do not use root account.
Perform exhaustive testing (use unit test when possible). Involve as many people you can.
Test it under the main browsers (Firefox, Chrome, Opera, Safari, IE) and if have time in others.
Validate all your HTML/CSS against standards (W3C). (recommendable)
Depends on the platform you are using, there are profilers which can help you identify bottlenecks in your code. (can be done in later stages).
Tune settings for your web server / script language.
Be sure it is search-engine friendly.
Pray once is online :)
This is not a complete list as it depends in:
which language/platform/web server you are using.
what kind of application you developed (social, financial, management, etc.)
who will use that application (the entirely world, an specific company, your family or just you).
are you going to sell it? then you must have at least most of the previous points.
is your application using very sensitive information (as credit cards)? if so, you should pay for some professional (company?) to check your code, settings and methods.
This is just my opinion, take it as it is. I would also like to hear what other people suggests.
Good Luck
As well as what's already been suggested, depending on what type of application it is, you can use a vulnerability scanner to scan your application for any vulnerabilities that could lead to hackers gaining entry.
There are quite a few good scanners out there, but note when using them that the results may or may not be 100%. It's hard to say.
For a list of scanners, commercial and free, see: http://projects.webappsec.org/Web-Application-Security-Scanner-List
For more information on scanners: http://en.wikipedia.org/wiki/Web_Application_Security_Scanner
Good luck.
Here you can find a practical checklist to use before launching a website
http://launchlist.net/
And here is a list of all the stuff you forgot to test
http://www.thebraidytester.com/downloads/YouAreNotDoneYet.pdf
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
About three years ago I switched from using commercial testing tools to using an open source testing framework (WatiN), and a UI automation framework I developed myself.
Since doing this I think life is much better using these libraries and Visual Studio, than expensive dedicated testing tools with either their own languages, or VBA.
Are open source automated testing tools and frameworks better than commercial products
Or do expensive commercial automated testing tools such as WinRunner, QuickTest Pro, Testpartner etc ... still have a future?
It's hard to make that sort of determination for a generic category like "testing tools". Usually, it's best to evaluate both the commercial solution and the open source solution on a case-by-case basis. From the wording of your question, it sounds like you've found the open source solution a better fit for your needs.
However, there are some points you can use in your decision making process:
Commercial Tool Benefits:
Support - usually, the company is paying people to support the product. In addition, many commercial companies offer support contracts for various levels of support. If you need support in a crunch, commercial support is the way to go.
Open Source Tool Benefits
Price - pretty obvious...it's hard to compete against free
Openness - open source projects tend to adapt open standards more readily than commercial products (a lot of commercial products also adapt open standards, but open source software tends to do so more frequently).
Self-support - If your company allows it, you can fix the bugs you find in the tools yourself. No need to wait for a third party to get around to fixing them.
It's also worth mentioning that a lot of the commercial testing tools are built on their open source counterparts. If that is the case, then you might be better off going with the open source versions, unless you need the added support.
I think you'll find that commercial products and Open Source products tend to have similar feature sets. In other cases, they may solve the problem with completely different approaches. Again, you'll probably want to make the evaluation for every case.
PyWinAuto is a great open source tool to automate thick client or windows based standalone apps.
I got the experience, that tools for developers are the first and best class of software adopted in open-source. So you can see a big amount of great testing-tools in OS. So I think in most common environments you have test-frameworks in OS that work very well. But I do not code in every environment, so maybe you want to say which language/technology you use, and we can mention some good tools (OS or commercial).
All commercial tools have some +s's and -s's.
Not all of them can be considered as an exact fit in an automation environment which an AUT requires.
I have worked with all the big names it all depends on the automation Engineer how well He/She understands the concept and leverages the tool to the fullest.
Last 5 years I came to the conclusion it is high time we divorce these Commercial tools and go after Open Source Why?
I can customize the Open source to my needs.
I know where and what went wrong and at the same time know how to fix it.
I can customize things to the extent required write from planning to the execution to Reporting is in our control as automation engineers so creativity plays a role to implement the needs of the organization and the individuals who are affected by the automation testing.