Related
Ok, I think this question is at the wrong place and I'll head over to https://softwareengineering.stackexchange.com/ to read/ask about it. Thanks all for your answers up to this point. :)
apologies ;) I'm sorry if this question is a little bit subjective, but I can not come up with a better title. I'll correct it if you know something better.
In my organization there is a lot of buzz about this whole automated testing and continuous integration thing, but one argument I constantly hear is this:
How should I develop good, clean, easy to maintain code and write unit tests, if the
deadline is already set and it is only half of my estimate?
I'm a developer myself, so I can understand this. But I always try to respond that not only the developers need a paradigm shift, but the management too.
If you are a developer and your estimates are cut half, no matter what you estimate, you are not going anywhere, no matter how complex or trivial your problems are. You need the backup of the management guys, the One Guy who is giving the money.
Conclusion?
Can you give me some help, may it be a good URL to read about this development/management conflict, a book or maybe a personal insight? Did you survive a large process shift like this in a Waterfall company that is now doing Lean development? Or do you know this argument and have a clever answer to it?
And please, help me rename or move this question. :-)
Update
Thanks for all the answers already! :) I think I have to make clear that my point wasn't the "do it twice as fast" statement from management. It's about the negative point of view that comes with this statement from a developer.
Is there anything I can do to help people to understand that this is not the default in software development? That the PM is not actively preventing writing good code and that maybe both sides need a bit more education about the pros/contras of clean code bases, good coverage and lots of automated tests?
One good example is Technical Debt. It's manager friendly. Imagine your credit card. If you accrue debt for a few weeks that can be helpful. You don't need to carry around cash for daily purchases and you pay it off at the end of the month.
This is like a crunch before a release. You take on some debt and then pay it back soon. If you keep charging things and never paying off that debt it starts to compound. That new feature you want it more difficult because the foundation you're building on is unsound. The debt you've accumulated is keeping you from acting quickly. If you're over your limit even typical small purchases won't go through.
You might also want to take a look at Facts and Fallacies of Software Engineering . It talks about estimates and the troubles they can cause when they're not reviewed as the project evolves.
It may sound defeatist to say this, but I've worked in a few shops that had this issue, and they never changed- or more accurately, I found that it was not possible to change the system from within.
The issue is that, from the perspective of the management that insists on this type of development, as long as the product is being released approximately on time, and the customers are buying it, goal accomplished. To put it another way, As long as you are making money, quality does not matter.
Now, you, I, and experienced management understand the long term cost of technical debt. It may be possible to explain to a rational manager the cost of technical debt, the compounding reduction in return on investment in programmer time (by far the most expensive part of a software project), and the fact that a clean, well designed, well tested code base means that new features can be implemented more quickly, and that more time can be spent on new features instead of fixing bugs- leading to a long term improvement in the mean time between releases.
It may be possible to explain this to your management, but every place I've worked that had these issues required a critical failure before they wised up. This usually involved a large portion of the team quitting from frustration, or a large drop in sales as quality diminished due to unrealistic scheduling (in turn leading to massive layoffs). Either way, although I've heard of organizations changing after the fact.
In short, try to explain the cost of technical debt, and the benefit of a clean codebase. Explain it in terms of sales, releases, and customer satisfaction, instead of from a technical perspective. If that doesn't work, start looking for a new job, because poor management leads to a poor product, and a poor product reflects poorly on you as a developer.
What do you mean your "estimates are cut half"? Do you mean you give an estimate, and management says, "No, do it in half that time"? That is unacceptable.
Someone must push back against management. (I say "someone" because I don't know your hierarchy.) There is no such thing as a free lunch. If they want it sooner, then they must make hard, painful tradeoffs. They must prioritize and drop lower-priority features.
If they say, "No. We need it all now. Do it sooner or else," hold the line. They may be surprised, and they may be upset, but you'll earn their respect. The changes will come when they start listening.
There doesn't need to be a conflict between management and development. The conflict is between management and time. It's not your fault it takes time to do things. It's their job to make the hard decisions to get the products out on time without overworking developers until they quit in exhaustion. Just saying "Wrong, do it in half that time" is not management. It's fantasy.
In reality, your management will probably continue to be foolish. If so, you can try to play their game: come up with a safe estimate that you feel is very safe with the automated testing and then double it. Complain loudly when they cut the hours by half, then sigh in resignation. Allow them to feel they are doing their job. Mission accomplished!
A good PM does not estimate. Ever! A good PM will get an estimate from the person who is going to do the work. They will not change it. They may try to coax the worker to change it but, since the worker is the one doing the job, they should be controlling the estimate.
If you have a PM who cuts your estimate in half, make sure your estimate was in writing and then use that to explain to him (sorry for the gender bias, English doesn't really have a good neutral pronoun) and hopefully his boss that the reason your work only seems half finished is because he was screwing around with your estimates.
Tactfully point out that, if they're not going to take your estimates seriously, they should leave you alone and just pluck any old number out of their derrière. That will have the same effect of missed deadlines and unhappy customers but without you wasting time providing numbers that are just going to be ignored anyway.
In any case, the cold war between bad PMs and smart developers will naturally lead to the situation where you should initially double your estimates so that the halving will have little effect :-)
It may sound obvious, but my answer to this question is:
Writing unit tests before the code will allow you to develop good, clean and easy to maintain code
I had a problem with my management who were concerned that developers would not be able to complete their tasks on time if they will be required to write unit tests - this is a common concern when trying to implement a TDD in a Waterfall company. So I made this statement and we had to prove it by writing tests before code and not missing the deadline :) Actually when you get used to it, it will allow to write even more code.
Typically, if you improve your programming practices and code quality, you'll almost certainly speed up your development, as you'll save more time debugging than you will in writing unit tests and trying to make everything right to begin with. Very few shops are in the position of spending more time on code quality than it saves.
Another danger is if management meddles in the process, rather than just serves up impossible deadlines. If they expect you to cowboy code in order to make the deadline, you really won't be able to use good practices.
Working at client (non-IT) place as a .net programmer (alone) and asked to develope a windows application. No project manager, no SRS, no technical people to lead..., etc.
Directly getting requirement from customer on-their-need basis. It keep changes and has lot of ambguity. As the client is not understaning need of freezing requirement, it becomes huge headache to deal with. Has to do self document of requirement, coding, testing, bug-fixing and delivering build, educating users for application use by myself only. Reporing to a Boss, who is non-technical guy and always not understanding these problems.
Now it becomes, single developer-to-do-all SDLC activities. How should I proceed with this work environment?
Start by making demands on your environment, and on what is asked of you:
Demand that requirements and deadlines are fixed and agreed upon, in writing, before you write a single line of code.
Demand that you are given enough time for testing and bugfixing in the development cycle.
Demand that you are given time to setup source control, automatic builds etc (whatever you feel like you need for your development environment to promote effective work).
Demand that you are given time to write documentation, so that you can spend more time writing code and less time doing application demos.
Continue with backing it up:
Document and show your boss some statistics on how you use your time. If it turns out you use much less on actually writing code, maybe he'll consider giving some of the less programming-related tasks to some other member of the department.
And finally, remember that this this is not the only company in the world:
Robert L. Lead has a very good point in his How to be a Programmer: A Short, Comprehensive and Personal Summary: under Recognizing when to go home, he simply states:
Quit if you have to.
This might not be a very compelling option, but should it come to it, leave the company for the greener grass on the other side. Even telling your boss you're ready to quit if your working conditions don't improve might help you actually get what you want. I doubt that your company want to be left with a software product that suddenly can't be supported or updated, because their only developer quit.
Count your blessings, I'd say. Usually all the people standing between the developers and the users are just getting in the way of making successful software.
I think it is a good idea to adopt some agile tooling to organize yourself, like a scrum whiteboard, and by defining sprint periods/iterations. That will allow to manage your boss's and users' expectations, and still give them control over what should get priority. Don't forget to schedule for SDLC tasks, so you can make them visible to your boss. You should feel free to consider agile tooling as a supermarket: take what you think is useful and keep the rest in mind for later consideration.
As far as requirements documentation is concerned, I'd keep it very high level. I would not mind skipping it altogether but I can imagine that it feels sloppy, and it is perhaps also a way to document your achievements.
A combination of educating both your customer and boss; and an agile approach could be helpful here. It depends on how this project is billed to the customer.
If the customer is getting a fixed price deal, yet is allowed to change the specs, then educate your boss (or whoever is accountable for the financial results of the project) about the implications of this project. It means that the customer gets to ask for whatever they want, without needing to pay more. If the project isn't time boxed, your boss is giving away unlimited developer time at a fixed price. Make that clear. If the project is time boxed, explain that changing means redoing and that there's only so much redoing before you run out of time. If he doesn't see this is a problem, document your time use.
It's the equivalent of going to a car-repair shop, agreeing a price and then pushing the mechanic to not only fix your airconditioning (the original scope), but also replace your oil, uprate your suspension and do a full engine overhaul. In the long run, expect the customer to be demanding that the car flies, solve world hunger and bring world peace.
If you're on a billable hours project, then you're in more trouble. Your boss may not have any incentive for the customer to make reasonable demands, he may just care about you being effectively contracted out to a customer and bringing in revenue. In that case take charge of the project by agreeing an agile methodology with the customer, so you can at least deliver something that will address some customer needs. Feel free to take charge, it seems you're the de-facto manager - just make sure you understand what the terms of the contract for this project and work within those boundaries. If the contract is a bad deal, alert your boss, but your company will need to ride it out or renegotiate.
Work in two week sprints, and show to both your boss and client the ratio of functionality/features delivered vs. overhead (rework) vs other work (training,...). It may become clear quite quickly that your project is under resourced, or the demands to high for the price agreed. Track in spreadsheet, or use a lightweight agile project management tool something like TargetProcess
If the customer is unworkable and your boss only sees you at somebody to pimp out, reconsider if you want to work in such a place and if there is any particular reason why you're spending your professional time at your current company over another company.
Keep in mind that you could be in a reasonably strong position to push for some change to improve the situation. If you're the only developer in a non-IT shop, and you quit, your company will struggle to fulfill its obligations to its customer - your boss, lest he's a halfwit, will be mindful of that. Of course, threating to quit, is the nuclear option, don't play that card lightly.
What I usually do in such situations — and these situations I come across much more than I'd like — is to demand a certain minimum. You can only demand something if you have something you can use to pressure your "Boss". In a single-developer-does-all-and-can-do-all situation, your means of pressure is yourself.
There are some countries in the world where employees are badly protected and you have to be careful. For any other country, this almost becomes a no-brainer: simply demand minimum working conditions.
This means: you make a short-list of things you need. Keep it simple. Keep it — almost — free. Don't come with all kinds of procedures. Use a simple bug-tracking system you can also use for planning, report, feature-tracking and new-development (Jira comes to mind). Both your Boss and your Client should be taught to use it. For yourself you probably want to add source control if you don't have it already.
Now comes the tricky part: for a short while, become very strict. Use the comment threads of your tracking system for communication. Your client will continue to call you or e-mail you. Let him. But copy everything to the comment threads and write your answer there. Send the guy a link to the thread as an answer.
You Boss may not like this because he things it will slow things down. Tell him it will become clearer what can and will be done. It will also become clearer where his money (i.e.: your time) went. Tell him that you want to keep an overview yourself, but that it's good for him too. If he's not convinced, tell him to give me a call and then propose to him to do it 100% your way for two months. He'll make it one month and then you have a deal.
It's a tough game out there. But it's doable. Propose a few simple steps towards bettering your environment, communication and tracking. If he refuses, you refuse to do anything more and he'll be stuck with an even worse situation.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
In many online chess lobbies, I've seen instances of 'engining', where a cheater would open a chess program at the same time as the main game window. He would then set it up so that the opponent's moves are relayed to the computer, then which he would copy the computer's moves, until he (almost always) wins.
As a game developer and moderator, what is there to do about this situation?
I can't see that there is anyway to prevent someone to using a chess engine to assist them, unless you can observe the player.
You might have some luck protecting against fully automated bots, though.
Online poker sites use anti-bot measures similar to what you're describing. I recommend the series of articles How I Built a Working Poker Bot for a good overview of how these systems work, and how they are defeated.
I agree with the others who said that there's not much you can do to stop the most dedicated cheaters, but you might be able to prevent casual cheating. (The problem with that, of course, is that then the dedicated cheaters will rule your site.)
Many chess computers work to formulas and end game books, so they will often play the same move in a particular situation. You could run users game history through a variety of chess computers and see if the users chosen moves after the opening moves have correlation with how the various chess computers play. This could be used to highlight users that are using chess computers.
Sites like chesscube monitors you for some time if you comes under the radar of suspicion. They monitor how much time you are taking for hard moves and relative simple moves. If there isn't some serious difference, they may conclude you are cheating. Also I believe they implement some method to check the shifting between windows, however I'm not sure about what they use for it. But I personally know guys who had been banned. So their method is pretty good.
I second what JesperE say, You have to monitor the guy for sometime before arriving on an opinion.
As a holder of a similar site, I would suggest just to let them be. If you are not intended to monetize the bets, the cheaters will move to their level of Chess program that plays for them, and fall off.
Best practice is to keep several player rooms according to level, thus cheaters will even be welcomed, allowing strong players to reach out to higher level, and adding practice to rookies.
Technically, there's nothing I can think of you can do.
Socially, there's a lot. For example, all of the online board game servers I've seen make very public the user's win/loss record, and compute the user's rank from that. Doesn't that just encourage people to want to win? Instead, I'd record all games, but not present a win/loss record anywhere (does anybody at a real chess tournament know how many games they've won/lost ever?). Make rank a user-entered number, used for the purposes of finding an appropriate partner only, so simply showing rating of 5000 is meaningless. If you need to have some kind of 'user rating', then add a commentary system, to let users comment on moves of other people's games, and then let other users rate the comments. Commentary is one thing I haven't seen computers do intelligently yet, so it's something you can probably assume comes from a real person.
I would suggest having them have a webcam behind them but slightly to the left so you could see if they were pulling up another window such as a chess engine, as a chess master (rated 5th in Canada) I was baffled at how I was losing against players so frequently on the internet (the high timed games, ironically whenever I beat an engine user I was immediately accused of cheating) yet I would never lose to anyone except those top players in Canada's country tournaments where the best of the best were there. The difference? Those people couldn't use a chess engine while I was staring them down as they made their move. All you people that cheat, I fail to see the point, you aren't winning, you aren't furthering yourselves in the games, all you are doing is wasting your time mimicing a computer, you aren't even analyzing the board! I only play 5 minute games and blitz because these cheaters can't efficiantly use their engines in such a short time period but this is not how chess is supposed to be played you are supposed to think about you moves.
Nothing effective.
Depending on how much access you have to the computer the user is playing on, you can scan his process list for known chess programs and kick him if you see one... but there is no guarantee that he is actually using it in the manner you describe, and he can always use it on a separate computer if he has duel displays or a KVM.
Cheaters will find a way to cheat.
The good news in this case, is that the computer programs for chess are reasonably beatable unless they are running on some serious hardware.
Good luck.
You can theoretically prevent the automatic relaying of moves (but doing it manually is not much of an obstacle unless you're playing speed chess), perhaps even prevent any chess programs to be run on the same machine. But IMO that's a waste of effort, because you'll never be able to prevent people from running a chess program on a different machine sitting next to them.
Im not very familiar with this enviroment. But maybe CAPTCHA would help stop automated robots. You could also generate statistics for your users (games won/lost/average speed to move, etc). The first movements should be fast, but later on the movements should be slower as complexity increases. so you can highlight cheaters, monitor them and maybe ban their accounts as Wikipedia does with some editors. You could even make a point based system as stackoverflow does, to whitelist known good/clean players.
I cant see any way of stopping this from happening - pretty much whatever you do the cheater will still be able to manually "copy" the move that the other player has made (to another computer if necessary).
How about somehow using social mechanisms to discourage these sorts of players? Cheating in this way is obviously in itself fairly unrewarding in the long term for the cheater - if you can find and eliminate / safeguard any potential gain (for example ranked tournaments with prizes) that the cheater might be able to use this to explit against, then you should at least be able to keep the percentage of cheaters down allowing most other users to enjoy "genuine" chess games.
I don't know the specifics, but I'm sure you could get statistics about the behavior of players that cheat this way--in other words, find things that the cheaters have in common (length of turns, consistency, etc) and have your application automatically find those and put a "red flag" on players that look suspicious. Then you can personally review them (or have someone else do it) and see if indeed they look suspicious. If so, ban them.
Other than that, there is really not much you can do unfortunately. The above suggestion is a lot of work, so unless you're willing to put in the hours to create such a system, I wouldn't even bother with it. Whatever barriers you put up, determined cheaters will get around them.
I have two accounts on chess.com. The first one i use to cheat. I have rybka deep 3 which is the most beastly chess program i know. On this account i have played 70games and lost 8times. 6 of those times are to time running out. The other two was from playing two GMs. I would never enter a tournament with it because thats just crossing the line for me but regular rated game i cheat like crazy. I don't do it because i want to win. I do it because i want to see who can beat this program. The two GMs that beat it. It was one of the greatest chess games ive ever seen. They never won after that and i played them a lot after that. I have another account which is my legit account that balances out my conscience. Im more in between beginner and intermediate. Anyways great players can tell when someone is using a comp program. Ive been accused like a dozen times for cheating because some of the moves rybka pulls are just straight godly. I have gotten banned once before on chess.com for cheating. It sucked cuz I had some epic games saved on there but on my new account that i have for about 3months now has not been banned. Maybe because the people i play see it as a challenge than them getting duped. IDK but ill soon start losing on purpose to fall under the radar xD. So if you want to catch a cheater ill say look for people with ridiculous stats like 80games, 9losses, 3draws with ratings of 2200+(If youre using the regular chess rating system).
There isn't much you can do to prevent cheating other than using correlative methods and a banhammer. You can make it very difficult for them to get a new account once they are banned or better yet just match them up against other cheaters transparently. Eventually they'll get bored. While it does depend on how much access you have, I've seen some java applets that will effectively create a hash of hardware profiles similar to apples UDID, and then theres cookies. Matching banned account email hashes to strings in other login cookies wouldn't be too difficult either. Taking it to the next level, if you had an app running locally you could also peek at the process list. Looks like this might be a bit dated by now though.
The only solution is to show that cheating does not do anything beneficial for humanity and to show to cheaters that humanity is more important then some ego based reward they get from cheating. You cannot detect a computer vs human in all cases without putting them in a isolated room with no way to cheat. There are very good players, are they cheaters?
The reason there are cheaters is because society has allowed cheaters to exist and perpetuate and benefit.
Ask yourself, why does a person cheat? Self-worth? money?
When a society doesn't fulfill the needs that every human being needs the human beings find ways to do so.
e.g., if you are poor and can cheat at something(e.g., politics), become "wealthy", get a hot wife, famous, etc... then you are likely to do that.
In a perfect society everyone is equally wealthy and everyone works to help each other. There are no famous people because that creates an imbalance and everyone realizes that. There are no ugly or hot people because everyone is what they are. There are no fat people either because everyone works out to be healthy and not to be a drain on society(which makes them look better).
America and Christianity(and just about every other religion) have created a huge inversion of morality. By focusing on materialism, imaginary scenarios of life after death(Santa Clause and Tooth Fairy stuff just wrapped up in a guy named Jesus and a book instead of a Tooth).
The problems are fundamental and much more than chess and to solve the chess problem one must solve the fundamental problem. Society and humanity must change. Since this won't happen any time soon and is too complex a problem for 99.9% people to grasp(most are too busy reading their bible for answers that never work) the best thing to do for any individual that understands the consequences of cheating is to simply not cheat and life with the cheaters the best they can.
Don't go to the dark side and things will eventually get better. Cheaters cheat... they cheat each everyone, themselves, and life and there are consequences.
e.g., suppose you could divide the world up in to two groups. Cheaters and non-cheaters. You could isolate both. The cheaters would eventually die out because they don't do anything worth while. They are effectively parasites and need a host to life. Remove the host and they die(it doesn't happen in the real world because they can thrive and do quite well, because the non-cheaters are generally ignorant of the fact and unwilling to make hard choices. (e.g., if a husband is a cheater and the wife isn't, she usually stays with him rather than leaving him... or better yet, never marrying a cheater in the first place(which is hard but easier if Christianity didn't make it so hard).)
I'll give you an example: I used to play counter strike(a first person shooter) when it came out. There was obvious cheating going on and I knew it(it's impossible for humans to do certain things, just physics). I did research and found some bots and started cheating myself. When I suspected cheating I would use the bot. Guess what?!?! Sometimes I would lose! Bots are nearly perfect and instantaneous... yet I would lose! That means there are other cheaters! I would call them out and tell them I am cheating and that they are too. Most would say they weren't but I had a few admit they were.
What did I do? I quit playing the game when I realized it was more than 25% cheaters. I knew that I got nothing out of it. Cheaters don't get better at what they do. They cheat because they suck and can't handle the fact that there are better people in the world. Which goes back to society, because society creates these feels(sports, for example... just turn on TV and you'll see someone talking about how X is so great because he did Y... and it subconsciously makes you feel like you are not great, even though you are because your in the top 0.00000000001% of intelligence on this planet and probably galaxy).
Games are suppose to be about learning. When you cheat you don't learn. Society has forgot that and instead make fun of people that are not as good at something as someone else... but this is almost solely due to time. People good at X put in a ton of time in it. Cheaters don't get that. They first think the other person must be cheating because "How could I lose??!?! I am a genius! They must be cheating! I will cheat too! I'm saved, I will go to heaven! So it's ok!".
Think about it!
If people were completely honest and cared about everyone else(because it matters), we wouldn't have: Murders, cheaters, scammers, captchas, passwords, rich/poor, iOS(ok, joking ;), etc...
It can be done but only through enlightenment. When people learn the world isn't flat, that it takes two to tango, etc then we will get beyond all this mess. Until then, we can only work on trying to make it better:
Don't cheat or you become like them and go to the dark side... which is only fun temporarily cause that's how it gets its teeth in you.
If you play any type of game with others expect that they will be cheating. If they are not cheaters, then they won't get mad. When someone accuses me of cheating I take it as a compliment because it means I am better than they are(well, that's ego but it's ok because I don't take it personally ;)
Playing with cheaters only makes you better. Imagine 99% of people on chess.com cheating... and you don't. You will get better because you will learn strategy. They can't beat you in the real world then. The real deal is always better than the fake synthetic crap that America and Christianity pushes down our throats.
Ultimately it doesn't matter. You are going on die. There is only one purpose in life and that's to transcend. Know that if you are against cheating you are already further along than cheaters. But it's not a game, help them transcend and the world and after life will be better.
Or you can move to a large deserted island like me and just allow the bad world to cheat itself to death! I'd invite you to stay but only if you can prove to me that you aren't a cheater! ;)
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
We're neck deep in a project right now, schedules are tight (but reasonable). Our general strategy is to get a strong beta done, release it for testing, and get feedback from our testers.
Quite frequently, we're being hit by small things that spiral into long, time-costing discussions. They all boil down to one thing: While we know what features we need, we are having trouble with the little details, things like 'where should this message go' and 'do they need this feedback immediately, or will it break their flow, so we should hold off'?
These are all things that our testers SHOULD catch, but
a) Each 'low priority' bug like this drains time from critical issues
b) We want to have as strong a product as possible
and
c) Even the best testing group will miss things from time to time.
We use our product, and we know how our users use the old version...but we're all at a loss as to how to think like a user when we try to use the new version (which has significant graphical as well as underlying changes).
edit - a bit more background:
We're writing a web app used by a widely-distributed base of users. Our app is a big part of their jobs, but not the biggest (and, of course, we only matter to them when it doesn't work). Getting actual users in to use our product is difficult, as we're geographically distant from the nearest location that serves as an end user (We're in Ohio, and I think the nearest location we serve is 3+ hours away).
The closest we can get is our Customer Service team (who have been a big help, really) but they don't really think like the users either. They also serve as our testers (it really motivates them to find bugs when they know that any they DON'T find may mean a big upswing in number of calls). We've had three (of about 12 total) customer service reps back here most of the week doing some preliminary testing...they've gotten involved in the discussions as well.
Watching someone using the app is a huge benefit to me. Possibly someone who is not entirely familiar with it.
Seeing how they try to navigate, how they try to enter information or size windows. Things we take for granted after creating/running the app hour after hour, day after day.
Users will always try and do things you never expected and watching them in action might bring to light how you can change something that might have seemed minor, but really makes a big impact on them.
Read Don't make me think.
Speaking generally, you can't. There's not any way you can turn off the "programmer" part of your brain and think like a user.
And you're right about (c), testing groups don't necessarily catch all the bugs. But the best thing you can do is get a testing group comprised of real, honest-to-goodness end users, and value their feedback. Draw further conclusions from their general comments.
If you want to know how your users will see your system, the closest you can get is usability testing with real users. Everything else is just heuristics and experience, and is also subject to error. There's no such thing as a bug-free product, but you should be able to get a "strong" product with usability testing.
Buy a cheap, easy to use video camera and record your testers using the app. Even better, get some people unfamiliar with the app. to use it and video them. It's relatively cheap, and you'd be surprised what it will highlight.
I like policy of "eating your own dog food"("http://en.wikipedia.org/wiki/Eat_one's_own_dog_food). It brings you one step closer, because you become a user, although you might think like one.
Try to use your app when you are very hurry (e.g. you have someone who waits for a dinner).
You will see all this little things because you have to wait, you have to go back to the mouse of the keyboard, etc.
And also, make your wife use it. Or your mother.
Another useful test : help someone to use it, by phone. If he can't find the button with your directions, that's probably a bug.
The important thing is to get enough information that you yourself can become a "user". Once you do that you can answer most questions yourself.
The way I always do this is to go talk with them about what they need to do, what they typically do, and how they use their current tools to do it. Then (very important) sit with them while they do it. Make sure you get on with them well enough that you can come back to them with questions about how they handle edge cases you think of later (often the answer will be the appalling "we go around the system manually for that").
I will almost always notice something they are doing that is a royal PITA that they didn't bring up because they are used to having to do that and don't know any better. I will always notice that their %90 typical workflow isn't the easiest workflow the tools provide.
You can't really rely on plain old-fashioned requirements gathering by itself, because that is asking them to think like a developer. They generally don't know what is possible to do with your software, what is easy, and what is hard. Also they typically have no clue on GUI design principles. If you ask them for design input they will just tell you to put any new control on their favorite page, until the thing looks like a 747 control panel.
The problem is often that even the users don't know what they want until they are actually working with the software. Sometimes, a small oversight can be a big usability problem, sometimes a well thought out function that was requested by many users sees only little use.
My suggestions to decrease the risk of not implementing the right usability features:
Take a look at users actually doing their day to day work. Even if they use another software or no software at all. You will be able to determine the artifacts they often need to get their job done. You will see what data they frequently need. Concentrate on the artifacts, data and workflows most used. They should be the most usable. Exotic workflows may be a bit more time consuming for the users than often used workflows.
Use working prototypes of the GUI to let users work through a realistic workflow. Watch them and note what hinders them and what works well. Adjust your prototypes accordingly.
If an issue arises in an often-used part of your software, it is time to discuss it now and in details. If the issue concerns a seldom used part, make it a low priority issue and discuss it if you have the time. If issues or suggestions are low priority, they should stay low priority. If you can't determine if solution A or solution B is the best, don't run in circles with the same arguments over and over. Just implement one of the solutions and see if the beta testers like it. The worst thing you could do is waste time over tiny issues, while big issues need to be fixed.
A software will never be perfect, because the viewpoints of users differ. Some users will think that a minor problem breaks the whole application. Others will live with even severe usability issues. People tend to lend their ear to those who argue the loudest. Get to know your users to separate the "loud" issues from the important ones. It takes experience to do this, and sometimes you will make wrong decisions, but there is no perfect way, only one of steady improvement.
If you can, set aside a certain amount of usability development resources for the rollout phase of your software. Usability issues will arise when people start working with it in a real production environment. Sometimes it is not important to present the perfect software, but to solve issues quickly as they arise.
The flippant (yet somewhat accurate) answer to how to think like a user is put a knitting needle in your ear and push really hard.
The longer response is that we as programmers are not normal and I mean that in a good way. I scratch my head at the number of people who still run executables they receive from strangers in emails and then wonder how their computer got infected.
Any group of people will in time develop their own jargon, conventions, practices and expectations. As a programmer you will expect different things from an operating system than Joe User will. This is natural, to be expected yet hard to work around.
It's also why BAs (business analysts) exist. They typically come from a business or testing background and don't think like programmers. They are your link to the users.
Really though, you should be talking to your users. There's no poitn debating what users do. Just drag a few in and see what they do.
A usability test group will help.. tests not focused on discovering bugs, but on the learning curve of the new design, made by a group of users, not programmers.
I treat all users like malicious idiots.
Malicious because I assume all users are going to try and break my code, do stuff that is not allowed, avoid typing in valid data, and will do anything in their power to make my life hell.
Idiots because again I can't assume they will understand simple stuff like phone formats, will run away screaming if presented to many choices, and will not make any leap of faith on complicated instructions. The goal is to hold their hand the entire way.
At the same time, its important to make sure the user doesn't realize you think they're an idiot.
To think like a user, be one. But are these actually bugs that your testers are reporting? Or are they "enhancement requests"? If the software behaves as designed per requirements and they just don't like the way it operates, that's not a bug. That's a failure of requirements and design. Make it work, make it rock solid, make it easy to change and you'll be able to make it what your users want.
I see some good suggestions here, especially observing people trying to use you app. One thing I would suggest is to look at the order in which things are presented to the user on paper forms (if they use these to do data entry from) and make the final data entry page mimic that order as closely as possible. So many data entry errors (and loss of data entry speed) are from them having to jump around on the page and losing their place. I did some work for a political campaign this year and in every case, entering data was made much more difficult because the computer screen did things in a differnt order than the paper inputs. This is particularly important if the form is one that can't be changed (like a voter registration form, a campaign has to use what the state provides) to match the computer screen. ALso be consistent from screen to screen if possible. If it is first Name last name on one form, making it last name first name on the next will confuse people and guanteee data entry errors.
If you are truly interested in understanding users though I strongly suggest taking a course in Human factors engineering. It is an enlightening experience.
The 'right' way to do this is to prototype (or mock up) your new interface features, and watch your users try to use them. Nothing is as enlightening as seeing a real user try to use a new feature.
Unfortunately, given most projects time and resources, this is not possible. If that is the position you are in I would recommend you discuss in the team who has the best grasp of usability, and then make them responsible for usability decisions - but that person will need to regularly consult real users to make sure his/her ideas are consistent with what the users want.
I'd suggest doing some form of usability testing; I've participated in such in the past, and found them quite useful.
If you were writing a ticketing system, for example, bring up tasks, and ask questions like "how would you update this ticket" or "what do you expect to happen if this button is clicked".
You don't necessarily need a full application, either, in some places screen shots can be used.
You could take the TDD/BDD approach and get the users involved before beta, having them work with you on refining requirements as you write your unit tests. We're beginning to incorporate some of those trends into our current project, and we're seeing fewer bugs in the areas where we have involved the users earlier.
There is no "think like a user" technique, get your hands on someone who knows nothing of the project and throw what you have done at them.
It's the only way to see how the look + feel + functionality present themselves to the end user.
Once you shocked that person who knew nothing of the product, listen to all of their idiotic (or so you think they are) complaints, fix them, arrange every silly cosmetic thing they point out (either by fixing the UI or by improving whichever documentation you had)..
and after you have satisfied the person you chose to look at your app from zero knowledge on the subject first round, pick another ...and another... until they stop being shocked when they see it, and they don't get stuck on.. "ok.. what does this do?" kind of phases.
You (as a member of the project, be it the project manager, developer, etc) will never think like a user is my answer to that question.
Old saying: You can make something "fool proof" but you can't make it "Damn-fool proof".
Additionally: When you make something "idiot proof" the world invents a better idiot.
Other than that, I agree with what everyone else said.
Ask someone with absolutely no knowledge, insight or programming experience to use the program and try to figure out every function of the program.
People who would NEVER use such a program are most likely to find bugs.
See it as a new Safari user (or FF) who tries to put the URL inside the search field...
As a programmer you guess no-one would be that stupid (or, well.. unknowing), but people actually sometimes find themselves in these situations. As a programmer, we miss these things.
What are your best usability testing tips?
I need quick & cheap.
While aimed at web design, Steve Krug's excellent "Don't Make Me Think: A Common Sense Approach To Web Usability" features (in the second edition, at least), a great chapter entitled "Usability Testing On 10 Cents A Day", which I think is applicable to a much wider range of platforms.
The chapter specifically deals with usability testing done quick and dirty, in a low-budget (no money and/or no time) environment, and illustrates some of the most important considerations for getting an initial "feel" of the thing.
Some of the points I like in particular are:
You don't need to test with a huge number of people (a sentiment also echoed by Jakob Nielsen)
A live reaction is worth a lot; if possible, make sure the developers can see the reaction (perhaps using a video camera and a TV; it doesn't need to be an expensive one)
Testing a few people early is better than a lot later
Joel Spolsky is known for advocating "hallway usability testing", where you grab a few passing users and ask them to complete some simple task. Partly inspired by the "a few users yield the bulk of the results" philosophy, it's also relatively convenient and inexpensive, and can be done every so often.
Ask someone non-techy and unfamiliar with it to use it.
The archetypal non-technical user, one's elderly and scatterbrained maiden aunt. Invoked in discussions of usability for people who are not hackers and geeks; one sees references to the “Aunt Tillie test”.
The Aunt Tilly Test (Probably needs a better name in today's day and age, but that's what it's referred to)
You have to watch people use your application. If you work in a reasonable sized company, do some 'hallway testing'. Pull someone who is walking past your door into the room and say something like, 'Could you please run the payroll on this system for the next month? It should only take two minutes'.
Hopefully they won't have any problems and it shouldn't be too much of an imposition on the people walking past. Fix up any hiccups or smooth over any processes that are unnecessarily complex and repeat. A lot.
Also, make sure you know what usability is and how to achieve it. If you haven't already, check out The Design of Everyday Things.
Some good tips here.
One mistake I made earlier on in my career was turning the usability test into a teaching exercise. I'd spend a fair amount of time explaining how to use the app rather than letting the user figure that out. It taught me a lot about whether my applications were easy or hard to use by how puzzled they got trying to use the app.
One thing I did was put together a very simple scenario of what I wanted the user to do and then let them go do it. It didn't have step-by-step instruction ("click the A button, then click the B button") but instead it said things like "create a new account" and "make a deposit". From that, the user got to 'explore' my application and I got to see how easy it was to use.
Anyhow, that was pretty cheap and quite enlightening to me.
Quick and cheap won't cut it. You have to invest in a user experience framework, starting with defining clear goals for your app or website. I know it's not what people want to hear, but after supervising and watching a lot of user testing over the years, using Nielsen's discount usability methods is just not enough in most cases. Sure, if your design really sucks and have made huge usability errors, quick and dirty will get 80% of the crud out of the system. But, if you want long-term, quality usability and user experience, you must start with a good design team. And I don't mean good graphic designers, but good Information Architects, interaction designers, XHTML/CSS coders, and even Web Analytics specialists who will make sure your site/app is measurable with clear goals and metrics. I know, it's a lot of $$$, but if you are serious with your business (as I am sure most of us are), we need to get real and invest upfront instead of trying to figure out what went wrong once the whole thing is online.
Another topic to research is Heuristics for usability. This can give you general tips to follow. Here's another use of heuristics
If you don't know where to begin, start small. Sit a friend down at your computer. Explain that you want them to accomplish a task using software, and watch everything they do.
It helps to remain silent while they are actually working. Write everything down. "John spent 15 seconds looking at the screen before acting. He moused over the top nav to see if it contained popup menus. He first clicked "About Us" even though it wasn't central to his task." Etc.
Then use the knowledge you gain from this to help you design more elaborate tests. Tests with different users from different knowledge realms. More elaborate tasks and more of them.
Film them. A web-cam mounted on the monitor is a good way to capture where their eyes are moving. A video recorder coming over their shoulder at 45 degrees is a good way to capture an overview. Bonus points if you can time-sync the two. Don't worry if you can't do it all. Do what you can do.
Don't plan your test as if it's the last one you'll ever need and you want to get it perfect. There is no perfect. The only thing approaching perfection is many iteration and much repetition. You can only approach 100% confidence as the number of tests approaches the number of actual users of your software. Usually nobody even gets close to this number, but everybody should be trying to.
And don't forget to re-test people after you incorporated the improvement you saw were needed. Same people, different people, either is ok.
Do what you can do. Don't lament what you can't do. Only lament what you could have tested but didn't.
I am answering very late but I was thinking about asking a similar questions about some ideas. Maybe it is better to keep everything in this question.
I would say that:
Do not teach people about your app. Let them have fresh eyes.
Ask them to make some tasks and record their actions with a tool like camstudio http://camstudio.org/
After the test, ask them to answer so simple questions. Here is my list:
What was your first feeling when you accessed the app?
Can you define the key concepts that are used by the app?
What are the top-3 positive things about the application?
What are the top-3 negative things about the application?
What do you think about these ideas?