Related
In other words, do you spend time anticipating errors and writing code to get around these potential issues, or do you write the code as you see fit and then work through any errors on an issue by issue basis?
I've been thinking a lot about this lately and I'm very much a reactive person. I write my code, give it a whirl, go back correct error and repeat until application works as expected. However a friend of mine offered that he spends time thinking how each line is interpreted and fixes errors before they occur.
I must point out that re-active is pure PRE-live. I definitely make sure my application is working before it goes live.
There should always be a balance.
Too many error checking is slow and leads to garbage code. Not enough error checking makes your program crash on edge cases which is not very good to discover after having it shipped.
So you decide how reliable some piece of code should be and implement error checking accordingly. Some test utility can be not very reliable - less error checking. A COM server meant to be used by a third party search service in deep background should be super reliable - much more error checking.
I think asking this in isolation is kinda weird, and very subjective, however there are obviously a bunch of techniques that permit you to do each. I tend to use these two:
Test-driven development (this would seem to be proactive)
Strong, static typing (reactive, but part of a tight iterative development cycle, as in, it's enforced by my ML compiler, and I compile a lot)
Very occasionally I swerve into the world of formal verification of programs. That's definitely "reactive", but if you think a little more up-front, it tends to make the verification easier.
I must also say that I value a lot of up-front thought in programming. The easiest way to avoid bugs is to not write them in the first place. Sometimes it's inevitable, but often a little more time spent thinking about the problem can lead to better-quality solutions, and then the rest can be taken care of using the kinds of automated methods I talked about above.
I usually ask myself a bunch of what-ifs when coding, like
The user clicks the button, what if they didn't select a date?
The user is typing in the search box, what if they try to type html in there?
My label text depends on a value from a shared drive, what if it's not mapped?
and so on. By doing this I've found that when the application does go live, there are a ton fewer errors and I can focus on fixing more obscure bugs instead of correcting conditions that should have been in place to begin with.
I live by a simple principle when considering error-handling: garbage in, garbage out. If you don't want any garbage (e.g. invalid input) messing up your software, you have to find all the points in your software where it can get in and handle it. Of course, the more complicated your software is, the harder it is to find every point of entry, but I feel that the more you do up front the less reactive you will need to be later on.
I advocate the proactive approach.
I try to write the code in that style which results in maintainable and reliable code
I use the defensive programming techniques to prevent stupid errors in code due to my loss of attention and similar
I design the database model according to the fortress principle, SQL code checking for results after each singular operation
I think of potential problems that can happen with that part of the code and I account for that. Not for every possibility but for major ones I can think of right now.
This usually results in software operating rather smoothly. At times it even surprises me but that was the intended goal, so here we are.
IMHO, the word "Error" (or its loose synonym "bug") itself means that it is a program behavior that was not foreseen.
I usually try to design with all possible scenarios in mind. Of course, it is usually not possible to think of all possible cases. But thinking through and allowing for as many scenarios as possible is usually better than just getting something working as soon as possible. This saves a lot of time and effort debugging and redesigning the code. I often sit down with pen and paper for even the smallest of programing tasks before actually typing any code into my editor.
As I said, this will not eliminate all errors. For me it pays off many times over in terms of time spent debugging. Another benefit is that it results in a more solid and maintainable design with fewer bugfixing hacks and special cases added on later. But in any case, you will have to do a lot of debugging after the code is done.
This does not apply when all you want is a mockup or rapid prototype. Also practical constraints such as deadlines often makes a thorough evaluation difficult or impossible.
What kind of programming? It's impossible to answer this in any general way. (It's like asking "do you wear a helmet when playing?" -- well, playing what?)
At work, I'm working on a database-backed website. The requirements are strict, and if I don't anticipate how users will screw it up, I'm going to get a call at some odd hour of the day to fix it.
At home, I'm working on a program ... I don't even know what it'll do yet. I can't deal with 'errors' because I don't know what 'an error' is in this context, because I don't know what correct behavior is going to be. The entire purpose of the program can and frequently does change on a timescale of minutes to hours, so even a couple minutes spent thinking about errors this early is a complete waste of time. (It's even worse than browsing SO, since error-handling adds lines of code.)
I guess the only general answer is "I do what makes sense in terms of saving time in the long term", which is, after all, the whole reason to use machines to do work for us.
In 2002 I did a pretty large VB6 app for a client. It used a lot of UserControls and a 3rd party menu control (for putting icons next to menu names). It had dynamically "splittable" panels, TreeViews with multi-state checkboxes, etc. A very rich UI. My total time on the project was about 500 hours, which the client graciously let me spread over a whole month. (Yeah, it was that kind of job.) They were very happy, though, and they paid the bill on time with no argument.
So after having no contact with them for years, they suddenly call and wonder if I can update the app to .Net for them. My initial reaction is just to decline, since I don't use VB.Net. And having read a bunch of posts on SO about the difficulties of porting, etc., etc., I'm even more inclined to decline, so to speak.
Still, before I tell them no I am interested in roughly quantifying the effort it would take. I would love to hear from anyone who has done this kind of thing and has a feel for how much work it is. Was it:
Significantly less than the effort you used on the original?
Somewhat less than the effort you used on the original?
The same as the effort you used on the original?
More?
A lot more?
Please only respond if you have actually done this kind of port. And the answer doesn't have to be exact, since I really am only trying to ballpark this. My feeling is that the effort will be at least as much as it took for the original, if not more. But I could be wrong. Thanks for any help.
I have done what you've been asked. In my case it was an (amateur) bowling tournament mgmt system: Member database mgmt (personal info, IRS/Winnings info, mailing lists etc etc etc) , tournament mgmt (player assignments, scoring, lane ticket generation, check register for winnings and side-pots, etc etc etc) as well as IRS EDI generation for all winners in a given year. Plus about a billion little items scattered across ~ 50 screens/sub-screens.
The key is CLIENT UNDERSTANDING -- You must be clear that they understand that this is not trivial; this is a new adventure for BOTH of you (particularly if you are a new-comer to VB.Net) If they liked your work before then they may very well give you the leeway/freedom to learn VB.Net on their nickel.
Reading some of the previous answers let me make a few suggestions (based upon 30 years experience as a software developer, the last 20 of which as a consultant....)
TAKE THEIR MONEY IF THEY OFFER IT. You need to bring your skillset into the 20th century; let them pay for it (again, if they agree.) They may be Magazine surfing and want "the latest stuff" for NO GOOD REASON -- but maybe they realize that they can extend the life of (your) application by this port. In essence they may have all kinds of goofy reasons for doing it... if they are NOT hiring you to DECIDE whether or not this port is a good idea. Then you may express any dismay you have about the decision PROCESS out of good client relationship building; BUT if they want to do this then it might as well be your job.
Take all this mumbo-jumbo re: C# vs VB.Net with a grain of salt. I have worked EXCLUSIVELY in VB.Net / ASPX.net (vs C#) since its inception and have yet to come across ANY functionality NOT attainable in VB.Net. There are some 'purists' out there that just view VB.Net as a toy. Well, I came from the days of writing in Assembler, then C, then C++ (And you can throw in Fortran, PL1 for good measure) then VB5, then 6, then VB.Net ... and NOW JAVA for Android. Its ALL GOOD FUN... and each has it merits and drawbacks. Remember that C# and VB.Net are essentially just GUIs to achieve a meta-language intermediate. You can write a TERRIBLE (as measured by efficiency or memory use or whatever metric you choose!) Program in C# and a great one in VB.Net (and vice-versa.) DO NOT EQUATE GOOD PROGRAMMING WITH LANGUAGE SYNTAX. (... C# is "superior" ???? Gimme a break.)
I chose to allow the Visual Studio do most of the heavy lifting for the first pass. Then you go through the gazzillion errors and clean it up. It goes pretty fast.
BUT you need to decide whether or not to take advantage of any framework benefits that you had hand-coded in VB6. E.G. looping through a string to locate a specific character(s) is now as simple as The_String.IndexOf("c") I found that in my case I went through the code several times and took better and better advantage of the Class (i.e. object orientation) as well as framework goodies as I became comfortable... this adds to your development time (see CLIENT UNDERSTANDING mantra) BUT your code WILL BE MORE Efficient then it ever could have been in VB6. You could simply port to get the errors out and not take ANY advantage necessarily from the framework.
I have not found any issue with 3rd party active-x controls. You can add a reference to FRAMEWORK objects, COM objects, etc. It may even be likely that the control vendor has a .Net (managed code) version... OR there may be suitable alternatives since you wrote the thing in VB6. (See CLIENT UNDERSTANDING mantra)
So if your still reading, then now I will finally tell you that the second attempt at my application in VB.Net CONVERTED/PORTED from VB6 was ~ 1/3 of the original time to get to a working model... and I was learning the framework as well. (If your confident in your skill set, have learned a few languages through the years you will get the gist of VB.Net quickly --- its the SUBTLETIES that take awhile.)
I must caution you that the thing that can REALLY kill you if you do not preach the CLIENT UNDERSTANDING well enough is if they want to make changes WHILE you're porting (and this is VERY LIKELY since they've been using it for awhile... I was very true in my case as well.
There is no hard and fast rule here. It could be that changes will actually HELP YOU get to a better understanding the framework faster OR changes could be a real pain. Only you can determine which flavor they might be. AND if they look to be the PAINFUL type -- you might ask to do the conversion first so that you have reliably reproduced the functionality -- THEN go back and review the code to make changes and take advantage of the framework as necessary. But, as I said, there is NO Hard and Fast rules here -- and don't let the purists tell you differently --- remember they are probably the same guys that said that PASCAL was going to take over the world!
So after having no contact with them for years, they suddenly call and wonder if I can update the app to .Net for them.
You need to ask why they want this done.
It's a Bad Plan(tm) for clients to be making technical decisions on a whim. Before applying any solution, thoroughly understand their needs and their problem. Only after you understand the problem as they do should you make recommendations.
It could be that they're infatuated with a buzzword and want to be using the latest thing, or it could be any of a million other things. The solution to their problem could be something really easy, but if you don't find out what their problem is, you will never know the best way to solve that problem.
I would plan on it taking about 50% of the time and effort it took to create it in the first place.
I have done EXACTLY what you are asking about for a commercial software product that consisted of roughly 500 kloc. We balanced the desire for refactoring with the desire to get something working and released as quickly as possible.
It took nearly the entire team about 1 full year to get it done...for a product that took 4 years to create in the first place. It was a gigantic undertaking..not to be underestimated.
We are doing that exact thing right now, however it's slightly different. Instead of one huge application, there are many smaller ones. However, there are a few bigger ones in the list. What we found out is that it was significantly less work than we originally thought. BUT...the biggest unknowns had to do with third-party controls we had. If you have a lot of those to basically redesign, you will probably be looking at more work.
The one good suggestion I have is to use Visual Studio 2008 for the conversion (not VS 2005). There were far less problems when using the built-in converter in VS 2008 than there was with VS 2005. Not sure why, it just was that way.
So, I can't say that you won't spend 500 hours again, but most likely not. Most of your time should be spent on testing to verify no functionality was lost.
In my opinion,
Visual Basic 6 and Visual Basic .NET are so different that you should forget about their coincidence in their name, and treat this as a migration to .NET =P
I think that you have one big advantage and one disadvantage:
Advantage: you allready know what the application has to do. Probably you´ll have some meetings with your clients, in order to add some modifications or new features, but you have the major part of the requirements very clear. You´ll even have the old working application to see while you, or your team, are coding.
Disadvantage: you´ll have to learn a new language. Saying how much time is going to take, is very subjective. I´ve allways thought that learning a new language is not the problem, the big problem would be if you have to learn to program!. But you do know to program. You´ll know what you want to do, and you´ll have only to search how to do it in the new languaje.
Since you have to learn .NET, If you have to estimate, I think that you should suppose that it will take you at least the same as the previous application, even more.
My advice is too take the chance to learn this new language. And if you are more inclined to decline the project, let me give you another idea... Maybe you could estimate a little module of your application, and tell the client that you´ll try to do that module, to see how much it takes to you. The client should pay this little module, even if you decide not to continue.
You can´t tell the client that you know all the requirements, so they are saving money with you, in some way. And that they should take this little risk (we are talking about a little module), because who better than you to make the project, you know the old application perfectly!
If the client is right with this, you´ll can take the decision with more arguments. And If finally you decide to continue, with this little module done, you can estimate the whole project better than before.
Sorry, my English isn´t very good, and probably I´ve made many gramatical errors.
EDIT grammar mistakes... =P
This is a huge topic.
You should take a look at MS' Free Book - Upgrading Microsoft Visual Basic 6.0 to Microsoft Visual Basic .NET.
Were you to rewrite from scratch, rather than simply try and port the previous application, it shouldn't take too long. VB.Net has some features that will make the new application build take shorter. As it seems that you don't know VB.Net already, what will take you the most time is learn how to do things the new way(tm).
I'd go for two months this time.
But seriously, that's probably not far off — or even low — because the client will have some reason for needing the port done. It won't be just a straight port - they'll want some "small little enhancement" that will blow the whole project up.
Additionally, I'd worry a bit about the third party controls. Most of the rest of it should convert okay, but there's sometimes an issue finding a good analog to a 3rd party control in .Net.
I'm also concerned that you don't use vb.net. If you're a c# user and wanted to re-implement that way you'd be fine, but otherwise this is a non-starter. VB.Net is just different enough from vb6 for you to get yourself into trouble.
Aside from all that, I would expect it to be less work this time than last, because you have a very explicit design laid out in front of you that you merely need to follow.
I've a question about a good way to protect a bit my cocoa app from piracy. I know that this is impossible!
So, in my app I've an isRegistered() method that runs every time the user launch the app.
This is called from the applicationDidFinishLaunching: App delegate. So if this method returns true, the app continues to execute the code, otherwise an Alert appears saying that the app is not registered and there are xx day to buy a license.
This is a good way? Because, I have no experience in this.
Thank you in advance for your help!
SOLVED
First of all, thanks to everybody! I think the same thing: any copy protection can stop the piracy. I'm trying only to solve this little bug, even if I know that someone will crack my app again.
However, it's true - the best thing is to improve the app and not waste the time to try make the piracy protection more efficient.
The solution you describe requires almost no expertise whatsoever to crack. It is trivial to change your isRegistered() function to always return true. Thus, the effort required to circumvent your protection is a tiny fraction of the effort you would have to spend implementinging all the infrastructure to support users purchasing registration codes.
In other words, you're not getting a good return on investment. There is some debate over whether the return on investment implementing piracy protection (rather than improving your product) is ever good enough (because you pit yourself against people who have nothing better to do than prove they're cleverer than you).
One good way to redress the balance of return on investment is to use pre-existing code such as AquaticPrime. That way, at least you won't have spent so much time chasing rainbows :)
I am not in shrink-wrapped software business but my friend is. And his observation after 10 years of selling his product was that it makes no sense to create too sophisticated protection because always some one will hack it. You are alone and world is infinite. It is better to invest time/money in improving your software than working on copy protection.
Also keep in mind that around 10% of will never steal and other 10% will always try. Just make sure that those 80% is able to buy your product without any other mayor obstacle. Than you could ignore those nasty 10%. Actually it is a quote from Joel Spolsky IHMO.
So your solution seems to be completely OK from technical point of view and just stay with it.
it's almost never worth implementing your own anti-piracy system, because you'll almost always spend a lot of effort on something which can then be broken very easily. Rely on a shared implementation - in this case a framework like AquaticPrime (lots of people on the macsb mailing list recommend that one) - and you're effectively relying on the framework being good enough to protect your own app as well as all the others.
The code signing framework on Leopard and later allows you to sign your code such that if it's ever tampered with, it will refuse to run - see the documentation of the kill option in the manpage.
This is a good question. Having read the answers, I think what BitDrink was really getting at was this: we know that an isRegistered() function is dead simple to hack. With the understanding that any protection system eventually will be hacked, what are some strategies for writing a function that's harder to hack than an isRegistered function that returns a boolean?
Fundamentally, any copy protection system will eventually have something that looks like this:
if (program is registered)
let the program continue
else
nagging message
end
Any hacker with a copy of GDB will eventually find that first line and write a tiny little patch to strip it out. Most copy protection systems focus on security through obscurity, i.e. making that line hard to find. You can also make this system more robust by signing the binary and checking the signature, but you'll just add another hoop for the hackers to jump through. They'll eventually find your public key and change it to their own public key so they can replace your signature. However, I believe this will significantly slow them down. Leopard offers a code signing utility, but I don't know if it can be used to prevent incorrectly signed applications from running at all.
There's no perfect solution to this problem, but there are two main things to remember:
your registration system will be broken. There is absolutely no way around this.
your reigstration system is a barrier between the user and your program. You should optimize for the (hopefully majority of) legitimate users and make this as easy to do as possible.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
We're neck deep in a project right now, schedules are tight (but reasonable). Our general strategy is to get a strong beta done, release it for testing, and get feedback from our testers.
Quite frequently, we're being hit by small things that spiral into long, time-costing discussions. They all boil down to one thing: While we know what features we need, we are having trouble with the little details, things like 'where should this message go' and 'do they need this feedback immediately, or will it break their flow, so we should hold off'?
These are all things that our testers SHOULD catch, but
a) Each 'low priority' bug like this drains time from critical issues
b) We want to have as strong a product as possible
and
c) Even the best testing group will miss things from time to time.
We use our product, and we know how our users use the old version...but we're all at a loss as to how to think like a user when we try to use the new version (which has significant graphical as well as underlying changes).
edit - a bit more background:
We're writing a web app used by a widely-distributed base of users. Our app is a big part of their jobs, but not the biggest (and, of course, we only matter to them when it doesn't work). Getting actual users in to use our product is difficult, as we're geographically distant from the nearest location that serves as an end user (We're in Ohio, and I think the nearest location we serve is 3+ hours away).
The closest we can get is our Customer Service team (who have been a big help, really) but they don't really think like the users either. They also serve as our testers (it really motivates them to find bugs when they know that any they DON'T find may mean a big upswing in number of calls). We've had three (of about 12 total) customer service reps back here most of the week doing some preliminary testing...they've gotten involved in the discussions as well.
Watching someone using the app is a huge benefit to me. Possibly someone who is not entirely familiar with it.
Seeing how they try to navigate, how they try to enter information or size windows. Things we take for granted after creating/running the app hour after hour, day after day.
Users will always try and do things you never expected and watching them in action might bring to light how you can change something that might have seemed minor, but really makes a big impact on them.
Read Don't make me think.
Speaking generally, you can't. There's not any way you can turn off the "programmer" part of your brain and think like a user.
And you're right about (c), testing groups don't necessarily catch all the bugs. But the best thing you can do is get a testing group comprised of real, honest-to-goodness end users, and value their feedback. Draw further conclusions from their general comments.
If you want to know how your users will see your system, the closest you can get is usability testing with real users. Everything else is just heuristics and experience, and is also subject to error. There's no such thing as a bug-free product, but you should be able to get a "strong" product with usability testing.
Buy a cheap, easy to use video camera and record your testers using the app. Even better, get some people unfamiliar with the app. to use it and video them. It's relatively cheap, and you'd be surprised what it will highlight.
I like policy of "eating your own dog food"("http://en.wikipedia.org/wiki/Eat_one's_own_dog_food). It brings you one step closer, because you become a user, although you might think like one.
Try to use your app when you are very hurry (e.g. you have someone who waits for a dinner).
You will see all this little things because you have to wait, you have to go back to the mouse of the keyboard, etc.
And also, make your wife use it. Or your mother.
Another useful test : help someone to use it, by phone. If he can't find the button with your directions, that's probably a bug.
The important thing is to get enough information that you yourself can become a "user". Once you do that you can answer most questions yourself.
The way I always do this is to go talk with them about what they need to do, what they typically do, and how they use their current tools to do it. Then (very important) sit with them while they do it. Make sure you get on with them well enough that you can come back to them with questions about how they handle edge cases you think of later (often the answer will be the appalling "we go around the system manually for that").
I will almost always notice something they are doing that is a royal PITA that they didn't bring up because they are used to having to do that and don't know any better. I will always notice that their %90 typical workflow isn't the easiest workflow the tools provide.
You can't really rely on plain old-fashioned requirements gathering by itself, because that is asking them to think like a developer. They generally don't know what is possible to do with your software, what is easy, and what is hard. Also they typically have no clue on GUI design principles. If you ask them for design input they will just tell you to put any new control on their favorite page, until the thing looks like a 747 control panel.
The problem is often that even the users don't know what they want until they are actually working with the software. Sometimes, a small oversight can be a big usability problem, sometimes a well thought out function that was requested by many users sees only little use.
My suggestions to decrease the risk of not implementing the right usability features:
Take a look at users actually doing their day to day work. Even if they use another software or no software at all. You will be able to determine the artifacts they often need to get their job done. You will see what data they frequently need. Concentrate on the artifacts, data and workflows most used. They should be the most usable. Exotic workflows may be a bit more time consuming for the users than often used workflows.
Use working prototypes of the GUI to let users work through a realistic workflow. Watch them and note what hinders them and what works well. Adjust your prototypes accordingly.
If an issue arises in an often-used part of your software, it is time to discuss it now and in details. If the issue concerns a seldom used part, make it a low priority issue and discuss it if you have the time. If issues or suggestions are low priority, they should stay low priority. If you can't determine if solution A or solution B is the best, don't run in circles with the same arguments over and over. Just implement one of the solutions and see if the beta testers like it. The worst thing you could do is waste time over tiny issues, while big issues need to be fixed.
A software will never be perfect, because the viewpoints of users differ. Some users will think that a minor problem breaks the whole application. Others will live with even severe usability issues. People tend to lend their ear to those who argue the loudest. Get to know your users to separate the "loud" issues from the important ones. It takes experience to do this, and sometimes you will make wrong decisions, but there is no perfect way, only one of steady improvement.
If you can, set aside a certain amount of usability development resources for the rollout phase of your software. Usability issues will arise when people start working with it in a real production environment. Sometimes it is not important to present the perfect software, but to solve issues quickly as they arise.
The flippant (yet somewhat accurate) answer to how to think like a user is put a knitting needle in your ear and push really hard.
The longer response is that we as programmers are not normal and I mean that in a good way. I scratch my head at the number of people who still run executables they receive from strangers in emails and then wonder how their computer got infected.
Any group of people will in time develop their own jargon, conventions, practices and expectations. As a programmer you will expect different things from an operating system than Joe User will. This is natural, to be expected yet hard to work around.
It's also why BAs (business analysts) exist. They typically come from a business or testing background and don't think like programmers. They are your link to the users.
Really though, you should be talking to your users. There's no poitn debating what users do. Just drag a few in and see what they do.
A usability test group will help.. tests not focused on discovering bugs, but on the learning curve of the new design, made by a group of users, not programmers.
I treat all users like malicious idiots.
Malicious because I assume all users are going to try and break my code, do stuff that is not allowed, avoid typing in valid data, and will do anything in their power to make my life hell.
Idiots because again I can't assume they will understand simple stuff like phone formats, will run away screaming if presented to many choices, and will not make any leap of faith on complicated instructions. The goal is to hold their hand the entire way.
At the same time, its important to make sure the user doesn't realize you think they're an idiot.
To think like a user, be one. But are these actually bugs that your testers are reporting? Or are they "enhancement requests"? If the software behaves as designed per requirements and they just don't like the way it operates, that's not a bug. That's a failure of requirements and design. Make it work, make it rock solid, make it easy to change and you'll be able to make it what your users want.
I see some good suggestions here, especially observing people trying to use you app. One thing I would suggest is to look at the order in which things are presented to the user on paper forms (if they use these to do data entry from) and make the final data entry page mimic that order as closely as possible. So many data entry errors (and loss of data entry speed) are from them having to jump around on the page and losing their place. I did some work for a political campaign this year and in every case, entering data was made much more difficult because the computer screen did things in a differnt order than the paper inputs. This is particularly important if the form is one that can't be changed (like a voter registration form, a campaign has to use what the state provides) to match the computer screen. ALso be consistent from screen to screen if possible. If it is first Name last name on one form, making it last name first name on the next will confuse people and guanteee data entry errors.
If you are truly interested in understanding users though I strongly suggest taking a course in Human factors engineering. It is an enlightening experience.
The 'right' way to do this is to prototype (or mock up) your new interface features, and watch your users try to use them. Nothing is as enlightening as seeing a real user try to use a new feature.
Unfortunately, given most projects time and resources, this is not possible. If that is the position you are in I would recommend you discuss in the team who has the best grasp of usability, and then make them responsible for usability decisions - but that person will need to regularly consult real users to make sure his/her ideas are consistent with what the users want.
I'd suggest doing some form of usability testing; I've participated in such in the past, and found them quite useful.
If you were writing a ticketing system, for example, bring up tasks, and ask questions like "how would you update this ticket" or "what do you expect to happen if this button is clicked".
You don't necessarily need a full application, either, in some places screen shots can be used.
You could take the TDD/BDD approach and get the users involved before beta, having them work with you on refining requirements as you write your unit tests. We're beginning to incorporate some of those trends into our current project, and we're seeing fewer bugs in the areas where we have involved the users earlier.
There is no "think like a user" technique, get your hands on someone who knows nothing of the project and throw what you have done at them.
It's the only way to see how the look + feel + functionality present themselves to the end user.
Once you shocked that person who knew nothing of the product, listen to all of their idiotic (or so you think they are) complaints, fix them, arrange every silly cosmetic thing they point out (either by fixing the UI or by improving whichever documentation you had)..
and after you have satisfied the person you chose to look at your app from zero knowledge on the subject first round, pick another ...and another... until they stop being shocked when they see it, and they don't get stuck on.. "ok.. what does this do?" kind of phases.
You (as a member of the project, be it the project manager, developer, etc) will never think like a user is my answer to that question.
Old saying: You can make something "fool proof" but you can't make it "Damn-fool proof".
Additionally: When you make something "idiot proof" the world invents a better idiot.
Other than that, I agree with what everyone else said.
Ask someone with absolutely no knowledge, insight or programming experience to use the program and try to figure out every function of the program.
People who would NEVER use such a program are most likely to find bugs.
See it as a new Safari user (or FF) who tries to put the URL inside the search field...
As a programmer you guess no-one would be that stupid (or, well.. unknowing), but people actually sometimes find themselves in these situations. As a programmer, we miss these things.
What are your best usability testing tips?
I need quick & cheap.
While aimed at web design, Steve Krug's excellent "Don't Make Me Think: A Common Sense Approach To Web Usability" features (in the second edition, at least), a great chapter entitled "Usability Testing On 10 Cents A Day", which I think is applicable to a much wider range of platforms.
The chapter specifically deals with usability testing done quick and dirty, in a low-budget (no money and/or no time) environment, and illustrates some of the most important considerations for getting an initial "feel" of the thing.
Some of the points I like in particular are:
You don't need to test with a huge number of people (a sentiment also echoed by Jakob Nielsen)
A live reaction is worth a lot; if possible, make sure the developers can see the reaction (perhaps using a video camera and a TV; it doesn't need to be an expensive one)
Testing a few people early is better than a lot later
Joel Spolsky is known for advocating "hallway usability testing", where you grab a few passing users and ask them to complete some simple task. Partly inspired by the "a few users yield the bulk of the results" philosophy, it's also relatively convenient and inexpensive, and can be done every so often.
Ask someone non-techy and unfamiliar with it to use it.
The archetypal non-technical user, one's elderly and scatterbrained maiden aunt. Invoked in discussions of usability for people who are not hackers and geeks; one sees references to the “Aunt Tillie test”.
The Aunt Tilly Test (Probably needs a better name in today's day and age, but that's what it's referred to)
You have to watch people use your application. If you work in a reasonable sized company, do some 'hallway testing'. Pull someone who is walking past your door into the room and say something like, 'Could you please run the payroll on this system for the next month? It should only take two minutes'.
Hopefully they won't have any problems and it shouldn't be too much of an imposition on the people walking past. Fix up any hiccups or smooth over any processes that are unnecessarily complex and repeat. A lot.
Also, make sure you know what usability is and how to achieve it. If you haven't already, check out The Design of Everyday Things.
Some good tips here.
One mistake I made earlier on in my career was turning the usability test into a teaching exercise. I'd spend a fair amount of time explaining how to use the app rather than letting the user figure that out. It taught me a lot about whether my applications were easy or hard to use by how puzzled they got trying to use the app.
One thing I did was put together a very simple scenario of what I wanted the user to do and then let them go do it. It didn't have step-by-step instruction ("click the A button, then click the B button") but instead it said things like "create a new account" and "make a deposit". From that, the user got to 'explore' my application and I got to see how easy it was to use.
Anyhow, that was pretty cheap and quite enlightening to me.
Quick and cheap won't cut it. You have to invest in a user experience framework, starting with defining clear goals for your app or website. I know it's not what people want to hear, but after supervising and watching a lot of user testing over the years, using Nielsen's discount usability methods is just not enough in most cases. Sure, if your design really sucks and have made huge usability errors, quick and dirty will get 80% of the crud out of the system. But, if you want long-term, quality usability and user experience, you must start with a good design team. And I don't mean good graphic designers, but good Information Architects, interaction designers, XHTML/CSS coders, and even Web Analytics specialists who will make sure your site/app is measurable with clear goals and metrics. I know, it's a lot of $$$, but if you are serious with your business (as I am sure most of us are), we need to get real and invest upfront instead of trying to figure out what went wrong once the whole thing is online.
Another topic to research is Heuristics for usability. This can give you general tips to follow. Here's another use of heuristics
If you don't know where to begin, start small. Sit a friend down at your computer. Explain that you want them to accomplish a task using software, and watch everything they do.
It helps to remain silent while they are actually working. Write everything down. "John spent 15 seconds looking at the screen before acting. He moused over the top nav to see if it contained popup menus. He first clicked "About Us" even though it wasn't central to his task." Etc.
Then use the knowledge you gain from this to help you design more elaborate tests. Tests with different users from different knowledge realms. More elaborate tasks and more of them.
Film them. A web-cam mounted on the monitor is a good way to capture where their eyes are moving. A video recorder coming over their shoulder at 45 degrees is a good way to capture an overview. Bonus points if you can time-sync the two. Don't worry if you can't do it all. Do what you can do.
Don't plan your test as if it's the last one you'll ever need and you want to get it perfect. There is no perfect. The only thing approaching perfection is many iteration and much repetition. You can only approach 100% confidence as the number of tests approaches the number of actual users of your software. Usually nobody even gets close to this number, but everybody should be trying to.
And don't forget to re-test people after you incorporated the improvement you saw were needed. Same people, different people, either is ok.
Do what you can do. Don't lament what you can't do. Only lament what you could have tested but didn't.
I am answering very late but I was thinking about asking a similar questions about some ideas. Maybe it is better to keep everything in this question.
I would say that:
Do not teach people about your app. Let them have fresh eyes.
Ask them to make some tasks and record their actions with a tool like camstudio http://camstudio.org/
After the test, ask them to answer so simple questions. Here is my list:
What was your first feeling when you accessed the app?
Can you define the key concepts that are used by the app?
What are the top-3 positive things about the application?
What are the top-3 negative things about the application?
What do you think about these ideas?