I've searched for a while, but I can't find anything related on Google or here.
Me and some friends were debating starting a company, so I figure it might be good to do a quick pilot project to see how well we can work together. We have a designer who can do HTML, CSS and Flash, enjoys doing art, but doesn't like to do HTML and CSS... And 2 programmers that are willing to do anything.
My question is, from an experienced site builder's perspective, what steps do we do - in chronological order - to properly handle a website? Does the designer design the look and feel of the site, then the programmers fill in the gaps with functionality? Or do the programmers create a "mock-up" of the site with most of the functionality, then the designer spices it up? Or is it more of a back-and-forth process?
I just want to know how a professional normally handles it.
Update:
A recap taking some of the notes from each post.
Step 1: Define requirements. What will your site/application do?
Step 2: Use cases. Who will use the application, and what will they do with it? This doesn't have to be done with a bunch of crazy UML diagrams, just use whatever visual aids you think work best for you. Find a CMS vendor, or a search vendor, or both. While planning, maybe do some competitor analysis, and see how those in similar fields have done theirs.
Step 3: Visual proof-of-concept. This is done by your designer, NOT your programmers... Programmers are notoriously bad at UI. Use an image program like Photoshop, not an HTML editor. Leave it fluid and simple at first. Select the three-color theme for the site (two primaries and an accent.) Get a sense of how you want to lay things out, keeping in mind the chosen CMS and/or search functionality. Focus hard on usability, add pizzaz later. Turn the created concept into JPEG mock-ups, or create a staging site to allow the client to view the work. A staging site will allow for future releases to be tested prior to moving it to production.
Step 4: Once the site is conceptualized by your designers, have your HTML/CSS developer turn it into markup. He/she should shoot for XHTML compliance and test on as many major browsers as you can. Also a good time to set up versioning/bug tracking/management systems, to keep track of changes, bugs, and feedback.
Step 5: Have your programmers start turning your requirements into software. This can and should be done in parallel with Step 4- there's no reason they can't be coding up the major pieces and writing tests while the UI is designed and developed.
Step 6: Marry up the final UI design with the code. Test, Test, Test!!
Step 7: Display end result to client, and get client sign-off.
Step 8: Deploy the site to production.
Rinse, Repeat...
Step 1: Define requirements. What will your site/application do?
Step 2: Use cases. Who will use the application, and what will they do with it? This doesn't have to be done with a bunch of crazy UML diagrams, just use whatever visual aids you think work best for you.
Step 3: Visual proof-of-concept. This is done by your designer, NOT your programmers. Use an image program like Photoshop, not an HTML editor. Leave it fluid and simple at first. Select the three-color theme for the site (two primaries and an accent.) Get a sense of how you want to lay things out. Focus hard on usability, add pizzaz later.
Step 4: Once the site is conceptualized by your designers, have your HTML/CSS developer turn it into markup. He/she should shoot for XHTML compliance and test on as many major browsers as you can.
Step 5: Have your programmers start turning your requirements into software. This can and should be done in parallel with Step 4- there's no reason they can't be coding up the major pieces and writing tests while the UI is designed and developed.
Step 6: Marry up the final UI design with the code. Test, Test, Test!!
Rinse, Repeat...
There is no one universal way. Every shop does it differently. Hence, a warning: gross generalizations follow.
Web development typically consists of much shorter release cycles, because it's so simple to push out a release, compared to client-side software. Thus the more "agile" methods are more frequently used than the "waterfall" models encountered in developing client software.
Figure out what, exactly, you're building.
Take care of all the legal stuff (e.g. what business entity you'll be forming, how will each team member be compensated for their work, will there be health benefits, etc).
Mockups. I suggest having the designers do the mockups since programmers are notoriously bad at UI design.
Set up some sort of bug tracking / case management system so that you have a centralized place for all your feature requests and bug reports.
Start coding.
Once you have a simple version of your app, get some people to test it out to make sure you're on the right path.
???
Profit!
As a first step, I'd recommend doing a bit of up-front design using an approach such as paper prototyping, to lock down what it is you want your website to do, and roughly how you want it to look.
Next up, read up on the Agile approach to software development and see if you like the sound of what it suggests. It tends to work best with smaller, well-motivated teams.
Figure out the minimum amount of functionality you can create that you can deliver as a product so that you can get user feedback as soon as possibly. Then expect to iteratively add functionality to the product over time.
The Web Style Guide provides a pretty detailed overview of the process.
You should mix and match the lists provided here for your needs.
I just want to make sure you know one thing...
Customers are "stoopid" when it comes to web design.
You will have to claw, scrape, drag, gnash, rip, and extricate every requirement from their naive little souls. If you fail to do so? Guess who gets the blame?
The road you now look down is a hard one filled with competition, stress, and risk. It requires endurance, faith, patience, and the ability to eat ramen 5 of 7 days a week.
To add (or repeat) Dave Swersky's list.
Gather requirements from clients
Do some competitor analysis. Gather
screen shots of competitor sites.
Build a sitemap /wireframe - What is
the structure/content of the site?
Get designers to create JPG mockups.
They may use the screen shots for
"inspiration"
Get feedback from
clients based on JPEG's
Create HTML
mockups from JPEG's
Get feedback
from clients. Go back to step 4 if
necessary
Implement HTML using
technology of choice
Unit test the site
UAT and obtain sign off.
Deploy to live
client feedback is critical, they should be involved in every step to ensure a successful implementation.
Hope this helps
In addition to the steps outlined in other answers, I'd add this (to be added somewhere near the end of the "cycle"):
x. Once you have a more or less end to end solution, set up a staging site.
y. Get client sign off on staging site.
z. Deploy to production site.
Celebrate! But not too hard, there's almost always going to be a few iterations of changes, because users rarely know exactly what they really want the first time around.
So, when (not if), the client asks for changes, you can work on the changes and promote them to the staging site first! This is important because a) it gives clients a chance to preview changes before the whole world sees them b) if the integrity of the data on the production site is important, you can hopefully weed out any issues on the staging site before they impact production data.
Just to give something on the other side of the coin. Where I work, we have for the past couple of years, worked on a redesign of the company's website. Here are some highlights of the process:
Identify vendors for various functions that will be needed. In this case that meant finding a Content Management System vendor as well as a Search vendor.
Get a new design for the site that can be applied to what was selected in the first step.
Using system integrators and in-house developers, start to build some of the functionality for the site and take the flexible, customizable software in 1 and make it useful for the organization. Note that this is where a couple of years have been spent getting this working and some business decisions ironed out.
Release a preview site to verify functionality and fix bugs, add enhancements as needed.
Note that in your case you may not have the same budget but there are various CMS frameworks out there to select as well as how much integration do you want to have for the site? Does it have to talk to a half-dozen different systems? In the case I mentioned above there are CRM integrations, ESB integrations, search integrations, and translation integrations to give a few examples of where things had to be wired up correctly.
In response to the comment, be sure you and the client know what is meant by "simple" as if there is any e-commerce functionality, forums, or personalization these are examples where it can be important to know what is needed now and have an idea of what is needed down the road as there can likely be a ton of things that customers may want but you have to figure out some of the nitty-gritty details at points in the future. For example, some people may think that Google is simple, and from an end-user perspective it is though how many computers does Google have running how many different applications doing how much processing 24/7? Quite a bit, I'd imagine. Simple is good, but sometimes making something look simple can be incredibly hard to do.
Related
The company I work for is starting a new web application, and I have requested that front end developers make this application Automation Friendly.
The previous application was using the react framework, very few elements had unique ID's (or any unique identifier at all). This time around, I have asked the developers to include a custom data attribute, specifically for automation.
I am looking for anyone who may have experience in this kind of situation.
What have you asked your developers to do to assist in automation?
Are there any standards, or guidelines for naming elements in an application to suit Selenium automation?
are custom data attributes the best way to go? are there other options?
Any advice/guidance would be greatly appreciated!
Web Applications can be difficult to test if they aren’t made with testing in mind. This is especially true for Single Page Applications (SPAs). SPAs support heavy interaction without incurring additional page loads (e.g. Facebook, Gmail). Instead of page loads, these SPAs use AJAX requests to relay data back and forth from the server.
As per #ChrisChua from ThousandEyes these are some of the best practices to keep in mind as you develop your web application to make testing easier:
Add classes that are meaningful.
Classes should indicate the element’s functionality and state.
Use functional names in IDs and classes for action elements.
Dynamically generated classes and IDs are not helpful for testing.
Add targetable DOM feedback to indicate application state.
Never hard-code content in test code!
Conclusion
It's true some of these are not easy changes, as the developer may have to think harder about using test-friendly designs rather than "something that just works". However, it will definitely help with maintainability of the testing, which would reduce costs in the long run.
tl; dr
A couple of references:
Nicolas Gallagher’s article on HTML Semantics and Front-end Architecture
Yandex’s Block Element Modifier
W3C’s recommendation on good class names
Pamela Fox’s article on Frontend Architectures
Test Driven Development makes it easier to automate. In my opinion, testers should be developers, or ex-developers who are good at finding and preventing mistakes. Write the tests in the same project that the solution is being developed in. Developers can also put in ids, even though they don’t want to sometimes, and if the “testers” are good they can even submit pull requests (in GitHub for example) for code improvements that will allow them to test better. Think of testers as a piece of your development team, where everyone is allowed to contribute code. It helps with accountability and improves autonomy. Everyone is there to help each other and if all the code is in the same project, and all code is reviewed and approved before merging to master, everyone is a potential developer if everyone is IT. The old days of manual testing are dying. Separating Dev from Testing is a brick wall. Tear it down.
I have the following dilema: My clients (mom-n-pop pawnshops) have been using my mgmt. system, developed with ISQL, for over 20 years. Throughout these two decades, I have customized the app to each clients desire, or when changes in Laws/Regulations have required it. Most clients are single-user sites. Some have multiple stores, but have never wanted a distributed db, don't trust the reliability or security of the internet or any other type of networking. So, they all use Standard Engines. I've been able to work around some SE limitations and done some clever tricks with ISQL and SE, but sooner or later, new laws may require images of pawnshop customers, merchandise, electronic transmision, etc. and then it will be time to upgrade to IDS, re-write the app in 4GL or change to another RDBMS. The logical and easiest route would be IDS/4GL, however, when I mentioned Linux or Unix-like platforms to my clients, they reacted negatively and demanded a Windows platform, so the easiest solution could be 4Js, Querix, etc.?.. or Access, Visual FoxPro or ???.. anyone have suggestions?
This whole issue probably comes down to a couple of issues that you'll have to deal with.
The first thing is what application programming and development language Are you willing to learn and work with?
The other thing is what kind of Internet capabilities to you want?
So for example while looking at a report do you want to be able to click on a button and have the report converted to a PDF document, and then launch the e-mail client with that PDF attached?
What about after they enter all the information data into the system, perhaps each store would like their own miniature web site in which people in town could go there to check what they've have place of having to phone up the store and ask if they have a $3 used lighter (the labor of phone and checking for these cheap items is MORE than the cost of selling the item – so web really great for this type of scenario).
The other issue is what kind of interface do you want? I assume you currently have some type of green screen or text based interface? Or perhaps over the years you did convert over to a GUI (graphical user interface).
If still green screen (text based) you now you have to sit down and give a considerable amount of effort and time into the layout and how you of screens will work with a graphical based system. I can remember when going from green screens to color, all of a sudden now the choices and effort of having to choose correct colors and layouts for that screen actually increased the workload by quite a bit. And then I went from color test screens to that of a graphical interface, then again all of a sudden now we're presented with a large number of new controls, colors, and in addition to that we have large choices in terms of different fonts and sizes.
And then now with the web, not only do you deal at different kinds a button styles (round, oval, shading, shadows, glow effects), but in addition to all those hover effects and shading effects etc, you now have to get down to some pretty serious issues in terms of what kind of colors (theme) your software will adopt for the whole web site.
This really comes down to how much learning and time you are willing to invest into new tools and how much software you can and will produce for given amount of time and effort.
I quite partial to RAD tools when you get down into the smaller business marketplace. Most of the smaller businesses can not afford rates for a .net developer (it not so much the rate, as the time to build an application). So, using ms-access is a good choice in the smaller business market place. Access is still a good 3 to 5 times many of the other tools in the marketplace. So quote by .net developer to develop something might be 12,000 bucks, and the same thing in Access might be $3000. I mean that small business can not afford to pay you to write unit testing code. This type of extra cost is just not going to happen on the smaller scale projects.
The other big issue you have to deal is what kind of report writing system are you going to build into the system? This is another reason why I like for the smaller business applications is access is because the report writer is really fantastic. Access reports have a whole bunch of abilities to bake connections in from forms and queries and pass filters and parameters into those reports. And, often the forms and queries that you spend time building already can talk to reports with parameters and pass values in a way that again really reduces the workload (development costs).
I think the number one issue that you'll have to address here however is what you're going to do for your web based strategy? You absolutely have to have one. Even if you build the front end part in access, you might still want to use a free edition of SQL server for the back end part. There are several reasons for this, but one reason is then it makes it easy to connect multiple stores up over the Internet.
Another advantage of putting your data in some type of server based system, is now you can set up some type of web server for all the stores to use, and build a tiny little customize system that allows each store to have their products and listings online (but, they use YOUR web server, or one that you paying $15 per month to host all of those customers). This web part could be an optional component that maybe perhaps all customers don't necessarily want. It would work off of the data they have to enter into the system anyway.
One great advantage of adopting these web based systems is not only does it allow these stores to serve their customers far better, but it also opens up the doors for you to convert your software into a monthly fee based system, or at least some part of it such as the optional web hosting part you offer.
When I converted so my longer time applications from green screen mainframe type software into windows desktop based applications it opened up large markets for me. With remote desktop, downloading software, issuing updates from a web site, then these new software systems make all of these nuts and bolts part of delivering software very easy now and especially so for supporting customers in different cities that you've never met face to face.
So, if you talking still primarily single user and one location, Access will reduce your development costs by a lot. It really depends on how complex and rich of an application you are talking about. If the size and scope of the project is beyond one developer, then you talking more about developer scaling (source code control, object development methodology, unit testing, cost and time of setting up a server based database system like SQL server etc). So they're certainly tipping point here when you go beyond that tipping point of cost time in complex city, then I actually don't recommend access. So this all comes down to the right horse for the right course.
Perhaps that the end of the day, it really comes down to what application development system are you willing to invest the time to learn?
Look at Aubit4GL - that is, I believe, available (or can be compiled on) Windows.
Yes, IDS is verging on overkill for a single-user system, but if SE doesn't provide all the features you need, or anticipate needing in the near future, it is a perfectly sensible choice. However, with a modicum of care, it can be set up to be (essentially) completely invisible to the user. And for a non-stressful application like this, the configuration is not complicated. You, as the supplier, would need to be fairly savvy about it. But there are features like silent install such that you could have your own installer run the IDS installer to get the software onto the customer's machine without extra ado. The total size of the system would go up - IDS is a lot bigger on disk than SE is (but you get a lot more functionality). There are also mechanisms to strip out the bigger chunks of code that you won't be using - in all probability. For example, you'd probably use ON-Tape for the backups; you would therefore omit ON-Bar and ISM from what you ship to customers.
IDS is used in embedded systems where there are no users and no managers working with the system. The hardware sits in the cupboard (closet) and works, communicating over the network.
It's good to see folks still getting value out of "old school" Informix Tools. I was never adept at Perform, but the ACE report writer always suited me. We skipped Perform and went straight for FourGen, and I lament that I've never been as productive as I was with FourGen. It had it own kind of elegance from its code generators to it funky, but actually quit powerful, stand alone menu system.
I appreciate the modern UI dynamics, but, damn, is it hard to write applications today. Not just tools, but simply industry requirements et al (such as you may be experiencing in your domain). And the Web is just flat out murder.
I guess part of it is that since most "green screen" apps look the same, it's hard to make one that looks bad! With GUIs and the Web etc., you can't simply get away with a good field order and the labels lining up.
But, alas, such as it is, that is what we have.
I have not used it in, what now, 15 years, but you may also want to look at Alpha 5. It was a pretty powerful, but not overly complicated, database development package, and (apparently) still going strong.
I wouldn't be too afraid of IDS. It runs pretty simply. Out of the box with zero or little tweaking, the DB works and is efficient, and it used to be pretty trivial to install. It was no SE, in that SE's access was tied to the application (using a library) vs an independent server that is IDS. But, operationally, it's really straightforward -- especially for an app like what you're talking about. I appreciate that it might be overkill, but even today, the resource requirements won't necessarily be insane. There's a lot of functionality, of course, and flexibility that you won't use. But frankly, beyond "flat file" DBase style databases, pretty much ALL of the server based SQL databases are very powerful and capable and potentially complicated. But they don't have to be. They can still be used "simply" and easily (well, save for Oracle -- Oracle can't do anything "simply").
As far as exploring other solutions, don't be too afraid of the "OOP" stuff, as most applications, while they leverage OOP libraries, aren't really OOP themselves (they can be, they just typically aren't, they simply don't need to be). The biggest issue with many of the OOPs systems, is they're simply to finely structured. Dealing with events at far too low of a level. While many programs need to access to that fine level of control, most applications, particularly the ones much like yours, do not. So, the extra flexibility simply gets in the way or creates more boiler plate.
That said, you shouldn't be frightened away from them per se, citing lacking of expertise. They can be picked up reasonably quickly. But I would certainly exhaust the more specialized tools (like Alpha 5, or Access, etc.) first to see if they don't offer what you want.
In terms of Visual FoxPro, was and remains a peerless tool (despite flak from people who know little about it). It has a fast, native database engine, built-in SQL and powerful report designer and so on. But you also have to consider that Microsoft support will be dropped for it in 2014, there will never be a 64-bit version, and so on. And the file locking method it uses will be increasingly flaky on future versions of Windows IMO.
When doing hallway usability tests do most of you make your apps fully or near fully functional? Or do you just make sure the links or flow chain correctly? Or do you just draw on paper and go with that?
I'm would like to test early on a prototype and am trying to find a good balance. But at the same time am worried that some non functional parts might actually not give representative results.
Thanks.
Usability tests, hallway or otherwise, only need the functionality that you need to test. In most usability tests, you should go in with specific design questions to answer and develop your prototype to the point where it can answer those questions. For example, if you need to test if users understand your indication of the sort order for a table, all you need is a paper picture of the table showing the sort indication (with the table contents blurred) and ask them how the table is sorted. If you need to test the IA, all you need is a bunch of web pages, empty except for a title, that are linked through the navigation menus.
You only need the pages relevant for the tasks you give your users. If you’re just testing the IA, then you only need the pages on the normative path. If you are also testing error recovery, then you need the pages off the normative path along with the full navigation controls. If you are also testing error detection, then you need content on the pages as well.
You can also simulate functionality when that’s easier to do. For example, in testing if users can figure out how to get a desired sort order, when the user clicks on a non-functioning control for sorting the table, you can say, “Okay, doing that will get you this,” and you take the mouse and select a bookmark that shows the table in the new sort order.
In hallway testing, if users breach the fidelity envelope, you can simply say, “I haven’t made that part yet. Let’s go back to A, and continue from there.” Of course, you should note that the user made a wrong turn in the task you intended for them. I haven’t had any problems with users complaining about non-functional features when I tell them up front it’s an incomplete prototype and we’re only testing the UI for features x, y, and z at the moment.
For low fidelity prototypes, I often call them “mockups” or “drawings” to users rather than “prototypes” to indicate the low functionality. You can put obvious placeholders in for missing content (e.g., “Blah, blah, blah…”, “TODO: Picture of product about here.”). If a user comments on something outside the fidelity envelope (e.g., “This symbol should be red to stand out more”), simply note it, and say that topic is under development (e.g., “Thanks. We haven’t started work on the colors yet. We’re just trying to figure out how to organize the site right now.”).
Usability testing with limited-fidelity prototypes is really necessary for iterative design to be feasible for most projects. Otherwise, you waste too much work developing things that have to be redone.
A couple things to remember:
Test early and often.
The goal of usability testing is to find problems with the UI, not Q/A your code.
Therefore, if users can see the parts of your UI you are interested in testing and interact with them in a realistic way (e.g., click on buttons and links), you should be able to collect useful data. If some links are dead-ends, that's okay, as long as there's some way for users to recover and continue on. Basically, with prototypes, the "correct" path should work, but it's okay if incorrect paths don't (as long as there's a reasonably quick way to get back on the correct path). Even static storyboards (non-functioning drawings of a UI) can provide you with some information if you ask the right questions, e.g., "What would you do on this screen if you wanted to view your shopping cart?").
I would suggest a couple rounds of usability testing. First on paper, perhaps later on screen, generally throughout the application lifecycle (take an Agile approach to it).
There is a good argument to be made for paper prototypes. When users see a screen, even limited functionality, they may be hesitant to suggest changes since it looks "done."
Make no mistake, it's not trivial to get it all down on paper, but that's where I would start. Probably start with just a section or two of the application. And make sure somebody with good people skills and/or explaining skills is there to walk the user through it. Have a second person on-hand to take notes. Try to ask open-ended questions, etc.
For a hallway test, I would test with NONE of the functionality implemented.
Test against designs done on a whiteboard or on paper. You'll be surprised at how much you find out in these minimal mockups. And they are very inexpensive to make!
Functional prototypes are for later. If you give your usability subject a functional interface, they are much less likely to question whether you've implemented the right set of features in the first place.
I would make the UI functional, so that the user can really play with it, it will be much better than a static image. People can tell you whether they feel comfortable on the UI.
I would make sure everything in the UI works, or at least takes you to a clear, unambiguous message pointing out that the feature isn't implemented yet.
Showing prototypes to clients with a disclaimer up front about how feature X doesn't work yet will usually be ignored. They'll try out the prototype, click on featuree X and indignantly reply "Feature X doesn't work! This really needs to work in the final version! Why doesn't it work?". The client is confused and unhappy about the product, and it's frustrating for yourself because it overshadows the positive feedback. Besides, you told them it didn't work, why can't they use their imagination to envision how it would work in the final version?
Make it work, be it with a rough version, dummy data, or even a simple message saying "would show results sorted alphabetically now".
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I'm trying to establish more formal requirements and testing procedures then we have now, but I can't find any good reference examples of documents involved.
At the moment, after feature freeze testers "click through the application" before deployment, however there are no formal specification what needs to be tested.
First, I'm thinking about a document which specifies every feature that needs to be tested, something like this (making this up):
user registration form
country dropdown (are countries fetched from the server correctly?)
password validation (are all password rules observed, is user notified if password is too weak?)
thank-you-for-registration
...and so on. This could also serve as something client can sign as a part of requirements before programmers start coding. After the feature list is complete, I'm thinking about making this list a first column in a spreadsheet which also says when was the feature last tested, did it work, and if it didn't work how did it break. This would give me a document testers could fill after each testing cycle, so that programmers have to-do list, with information what doesn't work and when did it break.
Secondly, I'm thinking of test cases for testers, with detailed steps like:
Load user registration form.
(Feature 1.1) Check country dropdown menu.
Is country dropdown populated with countries?
Are names of countries localized?
Is the sort order correct for each language?
(Feature 1.2) Enter this passwords: "a", "bob", "password", "password123", "password123#". Only the last password should be accepted.
Press "OK".
(Feature 2) Check thank-you note.
Is the text localized to every supported language?
This would give testers specific cases and checklist what to pay attention to, with pointers to the features in the first document. This would also give me something to start automating testing process (currently we don't have much testing automation apart from unit tests).
I'm looking for some examples how others have done this, without too much paperwork. Typically, tester should be able to go through all tests in an hour or two. I'm looking for a simple way to make client agree on which features should we implement for the next version, and for testers to verify that all new features are implemented and all existing features are working, and report it to programmers.
This is mostly internal testing material, which should be a couple of Word/Excel documents. I'm trying to keep one testing/bugfixing cycle under two days. I'm tracking programming time, implementation of new features and customer tickets in other ways (JIRA), this would basically be testing documentation. This is lifecycle I had in mind:
PM makes list of features. Customer signs it. (Document 1 is created.)
Test cases are created. (Document 2.)
Programmers implement features.
Testers test features according to test cases. (And report bugs through Document 1.)
Programmers fix bugs.
GOTO 4 until all bugs are fixed.
End of internal testing; product is shown to customer.
Does anyone have pointers to where some sample documents with test cases can be found? Also, all tips regarding the process I outlined above are welcome. :)
ive developed two documents i use.
one is for your more 'standard websites' (e.g. business web presence):
http://pm4web.blogspot.com/2008/07/quality-test-plan.html
the other one i use for web-based applications:
http://pm4web.blogspot.com/2008/07/writing-system-test-plan.html
hope that helps.
First, I think combining the requirements document with the test case document makes the most sense since much of the information is the same for both and having the requirements in front of the testers and the test cases in front of the users and developers reinforces the requirement and provides varying view points of them. Here's a good starting point for the document layout: http://www.volere.co.uk/template.htm#anchor326763 - if you add: steps to test, resulting expectations of the test, edge/bound cases - you should have a pretty solid requirement spec and testing spec in one.
For the steps, don't forget to include an evaluate step, where you, the testers, developers, etc. evaluate the testing results and update the requirement/test doc for the next round (you will often run into things that you could not have thought of and should add into the spec...both from a requirements perspective and testing one).
I also highly recommend using mindmapping/work-breakdown-structure to ensure you have all of the requirements properly captured.
David Peterson's Concordion web-site has a very good page on technique for writing good specifications (as well as a framework for executing said specifications). His advice is simple and concise.
As well you may want to check out Dan North's classic blog post on Behavior-DrivenDevelopment (BDD). Very helpful!
You absolutely need a detailed specification before starting work; otherwise your developers don't know what to write or when they have finished. Joel Spolsky has written a good essay on this topic, with examples. Don't expect the spec to remain unchanged during development though: build revisions into the plan.
meade, above, has recommended combining the spec with the tests. This is known as Test Driven Development and is a very good idea. It pins things down in a way that natural language often doesn't, and cuts down the amount of work.
You also need to think about unit tests and automation. This is a big time saver and quality booster. The GUI level tests may be difficult to automate, but you should make the GUI layer as thin as possible, and have automated tests for the functions underneath. This is a huge time saver later in development because you can test the whole application thoroughly as often as you like. Manual tests are expensive and slow, so there is a strong temptation to cut corners: "we only changed the Foo module, so we only need to repeat tests 7, 8 and 9". Then the customer phones up complaining that something in the Bar module is broken, and it turns out that Foo has an obscure side effect on Bar that the developers missed. Automated tests would catch this because automated tests are cheap to run. See here for a true story about such a bug.
If your application is big enough to need it then specify modules using TDD, and turn those module tests into automated tests.
An hour to run through all the manual tests sounds a bit optimistic, unless its a very simple application. Don't forget you have to test all the error cases as well as the main path.
Go through old bug reports and build up your test cases from them. You can test for specific old bugs and also make more generalizations. Since the same sorts of bugs tend to crop up over and over again this will give you a test suite that's more about catching real bugs and less about the impossible (or very expensive) task of full coverage.
Make use of GUI and web automation. Selenium, for example. A lot can be automated, much more than you think. Your user registration scenario, for example, is easily automated. Even if they must be checked by a human, for example cross browser testing to make sure things look right, the test can be recorded and replayed later while the QA engineer watches. Developers can even record the steps to reproduce hard to automate bugs and pass that on to QA rather than taking the time consuming, and often flawed, task of writing down instructions. Save them as part of the project. Give them good descriptions as to the intent of the test. Link them to a ticket. Should the GUI change so the test doesn't work any more, and it will happen, you can rewrite the test to cover its intention.
I will amplify what Paul Johnson said about making the GUI layer as thin as possible. Separate form (the GUI or HTML or formatting) from functionality (what it does) and automate testing the functionality. Have functions which generate the country list, test that thoroughly. Then a function which uses that to generate HTML or AJAX or whatever, and you only have to test that it looks about right because the function doing the actual work is well tested. User login. Password checks. Emails. These can all be written to work without a GUI. This will drastically cut down on the amount of slow, expensive, flawed manual testing which has to be done.
I've started a new role in my life. I was a front end web developer, but I've now been moved to testing web software, or more so, automating the testing of the software. I believe I am to pursue a BDD (Behavior Driven Development) methodology. I am fairly lost as to what to use, and how to piece it together.
The code that is being used/written is in Java to write a web interface for the application to test. I have documentation of the tests to run, but I've been curious how to go about automating it.
I've been directed to Cucumber as one of the "languages" to help with the automation. I have done some research and come across a web site for a synopsis of BDD Tools/Frame works,
8 Best Behavior Driven Development (BDD) Tools and Testing Frameworks. This helped a little but then I got a little confused of how to implement it. It seems that Selenium is a common denominator in a lot of the BDD frameworks for testing a GUI, but it still doesn't seem to help describe what to do.
I then came across the term Functional Testing tool, and I think that confused me even more. Do they all test a GUI?
I think the one that looked like it was all one package was SmartBear TestComplete, and then there is, what seems to be, another similar application by SmartBear called, SmartBear TestLeft, but I think I saw that they still used Cucumber for BDDing it. There a few others that looked like they might work as well, but I guess the other question is what's the cheapest route?
I guess the biggest problem I have is how to make these tests more dynamic, as the UI/browser dimensions can easily change from system to system, and how do I go about writing automation that can handle this, and tie into a BDD methodology?
Does anyone have any suggestions here? Does anybody out there do this?
Thanks in advance.
BDD Architecture
BDD automation typically consists of a few layers:
The natural language steps
The wiring that ties the steps to their definition
The step definitions, which usually access page objects
Page objects, which provide all the capabilities of a page or widget
Automation over the actual code being exercised, often through the GUI.
The wiring between natural language steps and the step definitions is normally done by the BDD tool (Cucumber).
The automation is normally done using the automation tool (Selenium). Sometimes people do skip the GUI, perhaps targeting an API or the MVC layer instead. It depends how complex the functionality in your web page is. If in doubt, give Selenium a try. I've written automation frameworks for desktop apps; the principle's the same regardless.
Keeping it maintainable
To make the steps easy to maintain and change, keep the steps at a fairly high level. This is frequently referred to as "declarative" as opposed to "imperative". For instance, this is too detailed:
When Fred provides his receipt
And his receipt is scanned
And the cashier clicks "Refund to original card"
And the card is inserted...
Think about what the user is trying to achieve:
When Fred gets a refund to his original card
Generally a scenario will have a few Givens or Thens, but typically only one When (unless you have something like users interacting or time passing, where both events are needed to illustrate the behaviour).
Your page objects in this scenario might well be a "RefundPageObject" or perhaps, if that's too large, a "RefundToCardPageObject". This pattern allows multiple scenario steps to access the same capabilities without duplication, which means that if the way the capabilities are exercised changes, you only need to change them in one place.
Different page objects could also be used for different systems.
Getting started
If you're attacking this for the first time, start by getting an empty scenario that just runs and passes without doing anything (make the steps empty). When you've done this, you'll have successfully wired up Cucumber.
Write the production code that would make the scenario run. (This is the other way round from the way you'd normally do it; normally you'd write the scenario code first. I've found this is a good way to get started though.)
When you can run your scenario manually, add the automation directly to the steps (you've only got one scenario at this point). Use your favourite assertion package (JUnit) to get the outcome you're after. You'll probably need to change your code so that you can automate over it easily, by e.g.: giving relevant test ids to elements in your webpage.
Once you've got one scenario running, try to write any subsequent scenarios first; this helps you think about your design and the testability of what you're about to do. When you start adding more scenarios, start extracting that automation out into page objects too.
Once you've got a few scenarios, have a think about how you might want to address different systems. Avoid using lots of "if" statements if you can; those are hard to maintain. Injecting different implementations of page objects is probably better (the frameworks may well support this by now; I haven't used them in a while).
Keep refactoring as you add more scenarios. If the steps are too big, split them up. If the page objects are too big, divide them into widgets. I like to organize my scenarios by user / stakeholder capabilities (normally related to the "when" but sometimes to the "then") then by different contexts.
So to summarize:
Write an empty scenario
Write the code to make that pass manually
Wire up the scenario using your automation tool; it should now run!
Write another scenario, this time writing the automation before the production code
Refactor the automation, moving it out of the steps into page objects
Keep refactoring as you add more scenarios.
Now you've got a fully wired BDD framework, and you're in a good place to keep going while making it maintainable.
A final hint
Think of this as living documentation, rather than tests. BDD scenarios hardly ever pick up bugs in good teams; anything they catch is usually a code design issue, so address it at that level. It helps people work out what the code does and doesn't do yet, and why it's valuable.
The most important part of BDD is having the conversations about how the code works. If you're automating tests for code that already exists, see if you can find someone to talk to about the complicated bits, at least, and verify your understanding with them. This will also help you to use the right language in the scenarios.
See my post on using BDD with legacy systems for more. There are lots of hints for beginners on that blog too.
Since you feel lost as to where to start, I will hint you about some blogs I have written that talks a bit about your problem.
Some categories that may help you:
http://www.thinkcode.se/blog/category/Cucumber
http://www.thinkcode.se/blog/category/Selenium
This, rather long and old post, might give you hints as well:
http://www.thinkcode.se/blog/2012/11/01/cucumberjvm-not-just-for-testing-guis
Notice that versions are dated, but hopefully it can give some ideas as what too look for.
I am not an expert on the test automation but I am currently working on this part. So let me share some idea and hope it will help you at the current stage.
We have used selenium+cucumber+intellij for testing web application. We have used testcomplete+cucumber+intellij for testing java desktop application.
As to the test of web application, we have provided a test mode in our web application, which allows us to get some useful details of the product and the environment; and also allows us to easily trigger events through clicking the button and inputting text into the test panel under test mode.
I hope these are helpful for you.