Mechanical Turk without Masters (Categorization on Sandbox) - mechanicalturk

I'm trying to test out my HIT on Mechanical Turk Sandbox with the categorization layout.
Everything works fine and I can deploy my HIT, except that I'm unable to figure out how to remove the requirement that workers must be Masters to participate.
Thus I am unable to complete the HIT through the Worker Sandbox because my account doesn't have a Masters qualification. I tried applying for the Masters qualification to no avail (and unsurprisingly so). I've found quite a few workarounds to removing the Masters qualification that were posted about back in 2011-2013 period, but it seems that Amazon has quite significantly changed the webpage since then.
Any tips would be greatly appreciated, or if there is another way for me to try out my own HIT on Sandbox.

Thomas' answer is the correct answer. If you create a Project using the Requester Website (http://requester.mturk.com/) and select either of "Sentiment" or "Categorization" template then it chooses Masters and there is no way of disabling it. It's worth noting that even though I describe these as "templates" they are in fact considered "applications" -- that is, they are designed to remove a lot of the complexity and "heavy lifting" that a Requester would normally do. That includes making approvals simpler (the "application" will take care of reviews and approvals), making pricing easier (the "application" will help recommend pricing based on marketplace conditions), and Worker quality (this is where Masters are selected by default and are non-optional).
Thomas' suggestion is to create your own template for MTurk Categorization (or Sentiment Analysis). This is probably the best suggestion. Check out "Tagging of an Image" for a template that you could quickly rework (assuming your categorization is of an image). You'll want to convert the three text inputs into a dropdown that includes your categorization. If you need to get more fancy (multi-level categorization) you can check out some fancy instantiations of it at sites like this:
http://bootsnipp.com/snippets/featured/multi-level-dropdown-menu-bs3
Sorry, as I'm sure that's not the answer you wanted to hear. But hope that helps lessen some of the time you're no-doubt spending to investigate this.

Related

Mechanical Turk

I am trying to post a typical psychology experiment as a HIT on mechanical turk. I am presenting a set of a dozen word pairs and I need to randomize/counterbalance the order of the word pairs presented. On the typical HTML forms there seems to be no way to do this. Is there a way to do this using the API or Command Line interface?
Update 5/20/2016: Go with PsiTurk instead. See the answer by #John McDonnel
Update 6/25/2013: This answer is quite old. See Solomon Messing's blog post on MTurk in Social science for a starting point.
Given the amount of psych research that is starting to appear on AMT, you'd think they would add this as an option!
To the best of my knowledge, you can't do this with the web interface without:
Creating a bunch of templates (form A, form B, form C, form D).
Running the first form (form A).
Paying the workers and then blocking all of them.
Running the second form (form B).
Paying the workers and the blocking all from step 1 and step 2.
And repeating until you are done.
This is bad for a couple of reasons. The first being that you have to manually create the different forms. The second being that by blocking the workers who are doing good work, you are negatively impacting their reputation on the site. Your IRB will not likely look kindly on this.
The "best" way to solve this is to not use the web interface, but use the command line tools (CLT) to create ExternalQuestions. That way, you write a webapp that can handle the details of the randomization, give the workers an informed consent without relying on a qualification, and "politely block" workers who have already done the work. (The workers who have already participated would get a short message asking them to return the HIT.)
If you are interested in taking that path, I have some details on my blog about how I've done it.
if you decide to go the external question route listed by #Nathan VanHoudnos, I have built a framework for running Psychology experiments on Mechanical Turk using the external question interface. You can check it out here. The code runs a simple stroop experiment, hopefully you could change it to run whatever you want. Let me know if you want any help setting it up!

What process does professional website building follow?

I've searched for a while, but I can't find anything related on Google or here.
Me and some friends were debating starting a company, so I figure it might be good to do a quick pilot project to see how well we can work together. We have a designer who can do HTML, CSS and Flash, enjoys doing art, but doesn't like to do HTML and CSS... And 2 programmers that are willing to do anything.
My question is, from an experienced site builder's perspective, what steps do we do - in chronological order - to properly handle a website? Does the designer design the look and feel of the site, then the programmers fill in the gaps with functionality? Or do the programmers create a "mock-up" of the site with most of the functionality, then the designer spices it up? Or is it more of a back-and-forth process?
I just want to know how a professional normally handles it.
Update:
A recap taking some of the notes from each post.
Step 1: Define requirements. What will your site/application do?
Step 2: Use cases. Who will use the application, and what will they do with it? This doesn't have to be done with a bunch of crazy UML diagrams, just use whatever visual aids you think work best for you. Find a CMS vendor, or a search vendor, or both. While planning, maybe do some competitor analysis, and see how those in similar fields have done theirs.
Step 3: Visual proof-of-concept. This is done by your designer, NOT your programmers... Programmers are notoriously bad at UI. Use an image program like Photoshop, not an HTML editor. Leave it fluid and simple at first. Select the three-color theme for the site (two primaries and an accent.) Get a sense of how you want to lay things out, keeping in mind the chosen CMS and/or search functionality. Focus hard on usability, add pizzaz later. Turn the created concept into JPEG mock-ups, or create a staging site to allow the client to view the work. A staging site will allow for future releases to be tested prior to moving it to production.
Step 4: Once the site is conceptualized by your designers, have your HTML/CSS developer turn it into markup. He/she should shoot for XHTML compliance and test on as many major browsers as you can. Also a good time to set up versioning/bug tracking/management systems, to keep track of changes, bugs, and feedback.
Step 5: Have your programmers start turning your requirements into software. This can and should be done in parallel with Step 4- there's no reason they can't be coding up the major pieces and writing tests while the UI is designed and developed.
Step 6: Marry up the final UI design with the code. Test, Test, Test!!
Step 7: Display end result to client, and get client sign-off.
Step 8: Deploy the site to production.
Rinse, Repeat...
Step 1: Define requirements. What will your site/application do?
Step 2: Use cases. Who will use the application, and what will they do with it? This doesn't have to be done with a bunch of crazy UML diagrams, just use whatever visual aids you think work best for you.
Step 3: Visual proof-of-concept. This is done by your designer, NOT your programmers. Use an image program like Photoshop, not an HTML editor. Leave it fluid and simple at first. Select the three-color theme for the site (two primaries and an accent.) Get a sense of how you want to lay things out. Focus hard on usability, add pizzaz later.
Step 4: Once the site is conceptualized by your designers, have your HTML/CSS developer turn it into markup. He/she should shoot for XHTML compliance and test on as many major browsers as you can.
Step 5: Have your programmers start turning your requirements into software. This can and should be done in parallel with Step 4- there's no reason they can't be coding up the major pieces and writing tests while the UI is designed and developed.
Step 6: Marry up the final UI design with the code. Test, Test, Test!!
Rinse, Repeat...
There is no one universal way. Every shop does it differently. Hence, a warning: gross generalizations follow.
Web development typically consists of much shorter release cycles, because it's so simple to push out a release, compared to client-side software. Thus the more "agile" methods are more frequently used than the "waterfall" models encountered in developing client software.
Figure out what, exactly, you're building.
Take care of all the legal stuff (e.g. what business entity you'll be forming, how will each team member be compensated for their work, will there be health benefits, etc).
Mockups. I suggest having the designers do the mockups since programmers are notoriously bad at UI design.
Set up some sort of bug tracking / case management system so that you have a centralized place for all your feature requests and bug reports.
Start coding.
Once you have a simple version of your app, get some people to test it out to make sure you're on the right path.
???
Profit!
As a first step, I'd recommend doing a bit of up-front design using an approach such as paper prototyping, to lock down what it is you want your website to do, and roughly how you want it to look.
Next up, read up on the Agile approach to software development and see if you like the sound of what it suggests. It tends to work best with smaller, well-motivated teams.
Figure out the minimum amount of functionality you can create that you can deliver as a product so that you can get user feedback as soon as possibly. Then expect to iteratively add functionality to the product over time.
The Web Style Guide provides a pretty detailed overview of the process.
You should mix and match the lists provided here for your needs.
I just want to make sure you know one thing...
Customers are "stoopid" when it comes to web design.
You will have to claw, scrape, drag, gnash, rip, and extricate every requirement from their naive little souls. If you fail to do so? Guess who gets the blame?
The road you now look down is a hard one filled with competition, stress, and risk. It requires endurance, faith, patience, and the ability to eat ramen 5 of 7 days a week.
To add (or repeat) Dave Swersky's list.
Gather requirements from clients
Do some competitor analysis. Gather
screen shots of competitor sites.
Build a sitemap /wireframe - What is
the structure/content of the site?
Get designers to create JPG mockups.
They may use the screen shots for
"inspiration"
Get feedback from
clients based on JPEG's
Create HTML
mockups from JPEG's
Get feedback
from clients. Go back to step 4 if
necessary
Implement HTML using
technology of choice
Unit test the site
UAT and obtain sign off.
Deploy to live
client feedback is critical, they should be involved in every step to ensure a successful implementation.
Hope this helps
In addition to the steps outlined in other answers, I'd add this (to be added somewhere near the end of the "cycle"):
x. Once you have a more or less end to end solution, set up a staging site.
y. Get client sign off on staging site.
z. Deploy to production site.
Celebrate! But not too hard, there's almost always going to be a few iterations of changes, because users rarely know exactly what they really want the first time around.
So, when (not if), the client asks for changes, you can work on the changes and promote them to the staging site first! This is important because a) it gives clients a chance to preview changes before the whole world sees them b) if the integrity of the data on the production site is important, you can hopefully weed out any issues on the staging site before they impact production data.
Just to give something on the other side of the coin. Where I work, we have for the past couple of years, worked on a redesign of the company's website. Here are some highlights of the process:
Identify vendors for various functions that will be needed. In this case that meant finding a Content Management System vendor as well as a Search vendor.
Get a new design for the site that can be applied to what was selected in the first step.
Using system integrators and in-house developers, start to build some of the functionality for the site and take the flexible, customizable software in 1 and make it useful for the organization. Note that this is where a couple of years have been spent getting this working and some business decisions ironed out.
Release a preview site to verify functionality and fix bugs, add enhancements as needed.
Note that in your case you may not have the same budget but there are various CMS frameworks out there to select as well as how much integration do you want to have for the site? Does it have to talk to a half-dozen different systems? In the case I mentioned above there are CRM integrations, ESB integrations, search integrations, and translation integrations to give a few examples of where things had to be wired up correctly.
In response to the comment, be sure you and the client know what is meant by "simple" as if there is any e-commerce functionality, forums, or personalization these are examples where it can be important to know what is needed now and have an idea of what is needed down the road as there can likely be a ton of things that customers may want but you have to figure out some of the nitty-gritty details at points in the future. For example, some people may think that Google is simple, and from an end-user perspective it is though how many computers does Google have running how many different applications doing how much processing 24/7? Quite a bit, I'd imagine. Simple is good, but sometimes making something look simple can be incredibly hard to do.

How can I think like a user? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
We're neck deep in a project right now, schedules are tight (but reasonable). Our general strategy is to get a strong beta done, release it for testing, and get feedback from our testers.
Quite frequently, we're being hit by small things that spiral into long, time-costing discussions. They all boil down to one thing: While we know what features we need, we are having trouble with the little details, things like 'where should this message go' and 'do they need this feedback immediately, or will it break their flow, so we should hold off'?
These are all things that our testers SHOULD catch, but
a) Each 'low priority' bug like this drains time from critical issues
b) We want to have as strong a product as possible
and
c) Even the best testing group will miss things from time to time.
We use our product, and we know how our users use the old version...but we're all at a loss as to how to think like a user when we try to use the new version (which has significant graphical as well as underlying changes).
edit - a bit more background:
We're writing a web app used by a widely-distributed base of users. Our app is a big part of their jobs, but not the biggest (and, of course, we only matter to them when it doesn't work). Getting actual users in to use our product is difficult, as we're geographically distant from the nearest location that serves as an end user (We're in Ohio, and I think the nearest location we serve is 3+ hours away).
The closest we can get is our Customer Service team (who have been a big help, really) but they don't really think like the users either. They also serve as our testers (it really motivates them to find bugs when they know that any they DON'T find may mean a big upswing in number of calls). We've had three (of about 12 total) customer service reps back here most of the week doing some preliminary testing...they've gotten involved in the discussions as well.
Watching someone using the app is a huge benefit to me. Possibly someone who is not entirely familiar with it.
Seeing how they try to navigate, how they try to enter information or size windows. Things we take for granted after creating/running the app hour after hour, day after day.
Users will always try and do things you never expected and watching them in action might bring to light how you can change something that might have seemed minor, but really makes a big impact on them.
Read Don't make me think.
Speaking generally, you can't. There's not any way you can turn off the "programmer" part of your brain and think like a user.
And you're right about (c), testing groups don't necessarily catch all the bugs. But the best thing you can do is get a testing group comprised of real, honest-to-goodness end users, and value their feedback. Draw further conclusions from their general comments.
If you want to know how your users will see your system, the closest you can get is usability testing with real users. Everything else is just heuristics and experience, and is also subject to error. There's no such thing as a bug-free product, but you should be able to get a "strong" product with usability testing.
Buy a cheap, easy to use video camera and record your testers using the app. Even better, get some people unfamiliar with the app. to use it and video them. It's relatively cheap, and you'd be surprised what it will highlight.
I like policy of "eating your own dog food"("http://en.wikipedia.org/wiki/Eat_one's_own_dog_food). It brings you one step closer, because you become a user, although you might think like one.
Try to use your app when you are very hurry (e.g. you have someone who waits for a dinner).
You will see all this little things because you have to wait, you have to go back to the mouse of the keyboard, etc.
And also, make your wife use it. Or your mother.
Another useful test : help someone to use it, by phone. If he can't find the button with your directions, that's probably a bug.
The important thing is to get enough information that you yourself can become a "user". Once you do that you can answer most questions yourself.
The way I always do this is to go talk with them about what they need to do, what they typically do, and how they use their current tools to do it. Then (very important) sit with them while they do it. Make sure you get on with them well enough that you can come back to them with questions about how they handle edge cases you think of later (often the answer will be the appalling "we go around the system manually for that").
I will almost always notice something they are doing that is a royal PITA that they didn't bring up because they are used to having to do that and don't know any better. I will always notice that their %90 typical workflow isn't the easiest workflow the tools provide.
You can't really rely on plain old-fashioned requirements gathering by itself, because that is asking them to think like a developer. They generally don't know what is possible to do with your software, what is easy, and what is hard. Also they typically have no clue on GUI design principles. If you ask them for design input they will just tell you to put any new control on their favorite page, until the thing looks like a 747 control panel.
The problem is often that even the users don't know what they want until they are actually working with the software. Sometimes, a small oversight can be a big usability problem, sometimes a well thought out function that was requested by many users sees only little use.
My suggestions to decrease the risk of not implementing the right usability features:
Take a look at users actually doing their day to day work. Even if they use another software or no software at all. You will be able to determine the artifacts they often need to get their job done. You will see what data they frequently need. Concentrate on the artifacts, data and workflows most used. They should be the most usable. Exotic workflows may be a bit more time consuming for the users than often used workflows.
Use working prototypes of the GUI to let users work through a realistic workflow. Watch them and note what hinders them and what works well. Adjust your prototypes accordingly.
If an issue arises in an often-used part of your software, it is time to discuss it now and in details. If the issue concerns a seldom used part, make it a low priority issue and discuss it if you have the time. If issues or suggestions are low priority, they should stay low priority. If you can't determine if solution A or solution B is the best, don't run in circles with the same arguments over and over. Just implement one of the solutions and see if the beta testers like it. The worst thing you could do is waste time over tiny issues, while big issues need to be fixed.
A software will never be perfect, because the viewpoints of users differ. Some users will think that a minor problem breaks the whole application. Others will live with even severe usability issues. People tend to lend their ear to those who argue the loudest. Get to know your users to separate the "loud" issues from the important ones. It takes experience to do this, and sometimes you will make wrong decisions, but there is no perfect way, only one of steady improvement.
If you can, set aside a certain amount of usability development resources for the rollout phase of your software. Usability issues will arise when people start working with it in a real production environment. Sometimes it is not important to present the perfect software, but to solve issues quickly as they arise.
The flippant (yet somewhat accurate) answer to how to think like a user is put a knitting needle in your ear and push really hard.
The longer response is that we as programmers are not normal and I mean that in a good way. I scratch my head at the number of people who still run executables they receive from strangers in emails and then wonder how their computer got infected.
Any group of people will in time develop their own jargon, conventions, practices and expectations. As a programmer you will expect different things from an operating system than Joe User will. This is natural, to be expected yet hard to work around.
It's also why BAs (business analysts) exist. They typically come from a business or testing background and don't think like programmers. They are your link to the users.
Really though, you should be talking to your users. There's no poitn debating what users do. Just drag a few in and see what they do.
A usability test group will help.. tests not focused on discovering bugs, but on the learning curve of the new design, made by a group of users, not programmers.
I treat all users like malicious idiots.
Malicious because I assume all users are going to try and break my code, do stuff that is not allowed, avoid typing in valid data, and will do anything in their power to make my life hell.
Idiots because again I can't assume they will understand simple stuff like phone formats, will run away screaming if presented to many choices, and will not make any leap of faith on complicated instructions. The goal is to hold their hand the entire way.
At the same time, its important to make sure the user doesn't realize you think they're an idiot.
To think like a user, be one. But are these actually bugs that your testers are reporting? Or are they "enhancement requests"? If the software behaves as designed per requirements and they just don't like the way it operates, that's not a bug. That's a failure of requirements and design. Make it work, make it rock solid, make it easy to change and you'll be able to make it what your users want.
I see some good suggestions here, especially observing people trying to use you app. One thing I would suggest is to look at the order in which things are presented to the user on paper forms (if they use these to do data entry from) and make the final data entry page mimic that order as closely as possible. So many data entry errors (and loss of data entry speed) are from them having to jump around on the page and losing their place. I did some work for a political campaign this year and in every case, entering data was made much more difficult because the computer screen did things in a differnt order than the paper inputs. This is particularly important if the form is one that can't be changed (like a voter registration form, a campaign has to use what the state provides) to match the computer screen. ALso be consistent from screen to screen if possible. If it is first Name last name on one form, making it last name first name on the next will confuse people and guanteee data entry errors.
If you are truly interested in understanding users though I strongly suggest taking a course in Human factors engineering. It is an enlightening experience.
The 'right' way to do this is to prototype (or mock up) your new interface features, and watch your users try to use them. Nothing is as enlightening as seeing a real user try to use a new feature.
Unfortunately, given most projects time and resources, this is not possible. If that is the position you are in I would recommend you discuss in the team who has the best grasp of usability, and then make them responsible for usability decisions - but that person will need to regularly consult real users to make sure his/her ideas are consistent with what the users want.
I'd suggest doing some form of usability testing; I've participated in such in the past, and found them quite useful.
If you were writing a ticketing system, for example, bring up tasks, and ask questions like "how would you update this ticket" or "what do you expect to happen if this button is clicked".
You don't necessarily need a full application, either, in some places screen shots can be used.
You could take the TDD/BDD approach and get the users involved before beta, having them work with you on refining requirements as you write your unit tests. We're beginning to incorporate some of those trends into our current project, and we're seeing fewer bugs in the areas where we have involved the users earlier.
There is no "think like a user" technique, get your hands on someone who knows nothing of the project and throw what you have done at them.
It's the only way to see how the look + feel + functionality present themselves to the end user.
Once you shocked that person who knew nothing of the product, listen to all of their idiotic (or so you think they are) complaints, fix them, arrange every silly cosmetic thing they point out (either by fixing the UI or by improving whichever documentation you had)..
and after you have satisfied the person you chose to look at your app from zero knowledge on the subject first round, pick another ...and another... until they stop being shocked when they see it, and they don't get stuck on.. "ok.. what does this do?" kind of phases.
You (as a member of the project, be it the project manager, developer, etc) will never think like a user is my answer to that question.
Old saying: You can make something "fool proof" but you can't make it "Damn-fool proof".
Additionally: When you make something "idiot proof" the world invents a better idiot.
Other than that, I agree with what everyone else said.
Ask someone with absolutely no knowledge, insight or programming experience to use the program and try to figure out every function of the program.
People who would NEVER use such a program are most likely to find bugs.
See it as a new Safari user (or FF) who tries to put the URL inside the search field...
As a programmer you guess no-one would be that stupid (or, well.. unknowing), but people actually sometimes find themselves in these situations. As a programmer, we miss these things.

Quick and Dirty Usability testing tips?

What are your best usability testing tips?
I need quick & cheap.
While aimed at web design, Steve Krug's excellent "Don't Make Me Think: A Common Sense Approach To Web Usability" features (in the second edition, at least), a great chapter entitled "Usability Testing On 10 Cents A Day", which I think is applicable to a much wider range of platforms.
The chapter specifically deals with usability testing done quick and dirty, in a low-budget (no money and/or no time) environment, and illustrates some of the most important considerations for getting an initial "feel" of the thing.
Some of the points I like in particular are:
You don't need to test with a huge number of people (a sentiment also echoed by Jakob Nielsen)
A live reaction is worth a lot; if possible, make sure the developers can see the reaction (perhaps using a video camera and a TV; it doesn't need to be an expensive one)
Testing a few people early is better than a lot later
Joel Spolsky is known for advocating "hallway usability testing", where you grab a few passing users and ask them to complete some simple task. Partly inspired by the "a few users yield the bulk of the results" philosophy, it's also relatively convenient and inexpensive, and can be done every so often.
Ask someone non-techy and unfamiliar with it to use it.
The archetypal non-technical user, one's elderly and scatterbrained maiden aunt. Invoked in discussions of usability for people who are not hackers and geeks; one sees references to the “Aunt Tillie test”.
The Aunt Tilly Test (Probably needs a better name in today's day and age, but that's what it's referred to)
You have to watch people use your application. If you work in a reasonable sized company, do some 'hallway testing'. Pull someone who is walking past your door into the room and say something like, 'Could you please run the payroll on this system for the next month? It should only take two minutes'.
Hopefully they won't have any problems and it shouldn't be too much of an imposition on the people walking past. Fix up any hiccups or smooth over any processes that are unnecessarily complex and repeat. A lot.
Also, make sure you know what usability is and how to achieve it. If you haven't already, check out The Design of Everyday Things.
Some good tips here.
One mistake I made earlier on in my career was turning the usability test into a teaching exercise. I'd spend a fair amount of time explaining how to use the app rather than letting the user figure that out. It taught me a lot about whether my applications were easy or hard to use by how puzzled they got trying to use the app.
One thing I did was put together a very simple scenario of what I wanted the user to do and then let them go do it. It didn't have step-by-step instruction ("click the A button, then click the B button") but instead it said things like "create a new account" and "make a deposit". From that, the user got to 'explore' my application and I got to see how easy it was to use.
Anyhow, that was pretty cheap and quite enlightening to me.
Quick and cheap won't cut it. You have to invest in a user experience framework, starting with defining clear goals for your app or website. I know it's not what people want to hear, but after supervising and watching a lot of user testing over the years, using Nielsen's discount usability methods is just not enough in most cases. Sure, if your design really sucks and have made huge usability errors, quick and dirty will get 80% of the crud out of the system. But, if you want long-term, quality usability and user experience, you must start with a good design team. And I don't mean good graphic designers, but good Information Architects, interaction designers, XHTML/CSS coders, and even Web Analytics specialists who will make sure your site/app is measurable with clear goals and metrics. I know, it's a lot of $$$, but if you are serious with your business (as I am sure most of us are), we need to get real and invest upfront instead of trying to figure out what went wrong once the whole thing is online.
Another topic to research is Heuristics for usability. This can give you general tips to follow. Here's another use of heuristics
If you don't know where to begin, start small. Sit a friend down at your computer. Explain that you want them to accomplish a task using software, and watch everything they do.
It helps to remain silent while they are actually working. Write everything down. "John spent 15 seconds looking at the screen before acting. He moused over the top nav to see if it contained popup menus. He first clicked "About Us" even though it wasn't central to his task." Etc.
Then use the knowledge you gain from this to help you design more elaborate tests. Tests with different users from different knowledge realms. More elaborate tasks and more of them.
Film them. A web-cam mounted on the monitor is a good way to capture where their eyes are moving. A video recorder coming over their shoulder at 45 degrees is a good way to capture an overview. Bonus points if you can time-sync the two. Don't worry if you can't do it all. Do what you can do.
Don't plan your test as if it's the last one you'll ever need and you want to get it perfect. There is no perfect. The only thing approaching perfection is many iteration and much repetition. You can only approach 100% confidence as the number of tests approaches the number of actual users of your software. Usually nobody even gets close to this number, but everybody should be trying to.
And don't forget to re-test people after you incorporated the improvement you saw were needed. Same people, different people, either is ok.
Do what you can do. Don't lament what you can't do. Only lament what you could have tested but didn't.
I am answering very late but I was thinking about asking a similar questions about some ideas. Maybe it is better to keep everything in this question.
I would say that:
Do not teach people about your app. Let them have fresh eyes.
Ask them to make some tasks and record their actions with a tool like camstudio http://camstudio.org/
After the test, ask them to answer so simple questions. Here is my list:
What was your first feeling when you accessed the app?
Can you define the key concepts that are used by the app?
What are the top-3 positive things about the application?
What are the top-3 negative things about the application?
What do you think about these ideas?

Getting developers to use a wiki [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I work on a complex application where different teams work on their own modules with a degree of overlap. A while back we got a Mediawiki instance set up, partly at my prompting. I have a hard job getting people to actually use it, let alone contribute.
I can see a lot of benefit in sharing information. It may at least reduce the times we reinvent the wheel.
The wiki is not very structured, but I'm not sure that is a problem as long as you can search for what you need.
Any hints?
Some tips:
Any time someone sends information by email that really should be in a wiki, make a page for that topic and add what they put in the email. Then reply "Thanks for that info, I've put it into the wiki here so that it's easier to find in the future."
Likewise, if you have information you need to share that should be in the wiki, put it there and just send an email with a link to it, rather than email people.
When you ask people for information, phrase it so that putting such documentation in the wiki should be considered the default or standard: "I searched in the wiki but I couldn't find it. Have you put that info up there yet?"
If you are the "wiki champion", make sure other people know how to use it, e.g. "Did I go through how to create a new page with you yet?"
Edit the sidebar to make sure it is relevant to your work.
Use "nav box" style templates on related pages for easier navigation.
Put something like {{Special:NewPages/5}} on the front page, or recent changes, so that people can see the activity.
Take a peek at Recent changes every few days or week, and if you notice someone adding information without being prodded, send them an email or drop by and give them a little compliment.
As I mentioned before, a Wiki is very unorganized.
However, if that is the only argument from your developers, then invest some effort to create a simple index page and keep it updated (either do it yourself or ask people to link their contributions to the index). That way, the Wiki might grow into a very nice and quite comprehensive collection of documentation for all your work.
We've been using a wiki in some form or another for a while now, but it does take a while for people to get on board. You might find that you will be the only one writing articles for some time, but bear with it, other people will come on board eventually.
If someone sends an email around that contains information related to the project then helpfully point them in the direction of the wiki - and keep doing that - they should get the hint.
We have a SharePoint portal and use the wiki from there - we customised it with our own branding so that it "looks the part" - I really feel this has helped to improve the uptake of it.
Make sure that everyone is aware that the wiki is even more informal than email.... because there will be a "fear factor" that people may think anything they add to the wiki will be over-analysed.
I think most of the answers so far are spot on - the more you plug away at it yourself, the larger the body of useful information will become, so slowly but surely people will naturally start to use it.
The other approach you could use is this: Suggest that every time someone asks another team member a question about the project, they should answer the question as normal, but also add the answer to a section of the Wiki. This may take a few minutes extra, but it will mean that the next time someone asks the same question (which they inevitably will), you can save time by pointing them at the Wiki. This, in turn, should help people to start using the Wiki as a first source of information and help overall up-take.
You can't force developers to do something they do not have an incentive of using for; unfortunately wikis, like documentation (well, in fact wikis are documentation) rarely have any "cool" value for developers. Besides, they're already deep into dev work -- could you really bother them with a wiki?
That being said, the people who pushed for the wiki (e.g., you) should be primarily responsible for updating it, and you really would have a lot of work cut out for you if you're serious about it.
You might also try the ff:
It's not very structured you say -- a lot of people get turned off from ill-structured (hard-to-search/browse) wikis. So maybe you can fix that first
Maybe you can ask lead developers/project managers to populate it with things that are issues for them: things like code conventions and API design for your particular project
Lead by example: religiously document your part of the system. Setting a precedent may encourage others to do the same
Sell the idea of using the wiki to the developers. You've identified some benefits, share those with the developers. If they can see that they'll get something of value out of it they'll start using it.
Example advantages from What Is a Wiki
Good for writing down quick ideas or longer ones, giving you more time for formal writing and editing.
Instantly collaborative without emailing documents, keeping the group in sync.
Accessible from anywhere with a web connection (if you don't mind writing in web-browser text forms).
Your archive, because every page revision is kept.
Exciting, immediate, and empowering--everyone has a say.
I have done some selling and even run some training sessions. I think some people are turned off by the lack of WYSIWYG editing and ability to paste formatted text from Word or Outlook. I know there are some tools to work around these, but they are still barriers.
There are some areas where the wiki is being used to log certain areas, but people who update those are not doing anything else with it.
I will use the wiki to document my specialised area regardless as it acts as a convenient brain extension. When starting a new development I use it as a notepad for ideas that I can expand on as it progresses.
It would help if management would give it some vocal support, even if it is not made mandatory.
I have a hard job getting people to actually use it, let alone contribute.
One of the easiest ways to get people to contribute to a wiki, is to actually have them provide contents in a wiki-suitable fashion, i.e. so that whatever they post using their usual channels of communications (newsgroups, mailing lists, forums, issue trackers, chat), is basically suitable for inclusion on the wiki.
So that others (users/volunteers) can simply take such contents and put them on the wiki.
This sounds more complicated than it really is, it's mostly about generalizing questions and answers, so that they are not necessarily part of a conversation, but can be comprehensible, meaningful and useful in a standalone fashion.
For example a question like the following:
how do I get git to clone a remote repository???
Can be answered like this:
Hello,
Just use git clone git://...
But questions can also be answered in a less personal style:
In order to clone a git repository, you will want to use the clone parameter to git:
git clone git://....
What I am trying to say is that most discussions in a project can and should be easily used to become documentation eventually. With this sort of mindset, your documentation can actually grow rather rapidly. You only need to get people to keep in mind that useful information should be ideally provided in a fashion that is suitable for wiki inclusion.
I have witnessed several instances where open source projects started to use this approach to some extent and while some people (largely new users) complained that answers were not very personal, the body of documentation was increasing steadily, because other people simply monitored such discussions and started to copy/paste such responses to the wiki.
Basically, this is one of the easiest ways to get people to contribute to a wiki, without requiring them to actually use it themselves, the only thing that's required of them is a shift in thinking.
If the developers still need to maintain 'real' documentation (s.a. Word documents), I see no way to meaningfully duplicate that on a Wiki.
It does not make sense for people to write twice
Any duplicated data is prone to get out of sync, soon.
What my current customer has done is move all this to Wiki. So I only document once, and I do it on the Wiki.
This is okay. Working with Wiki is more tedious than with Word, but at least the doc is online and others can mix-and-match with it.
Another working solution (imho) would be to store docs alongside the source, on subversion. But then the merging system needs to be able to cope with rich text etc. as well. I don't know, if any solution for that exists (other than using HTML or LaTex, which actually would not be bad picks).
Find "sticky" items (sub-3 pg. docs / diagrams / etc) something that the team seems to be creating again and again & post it on the wiki. Make sure everyone has access to the wiki and knows its there - set up a notification mechanism if possible. With some luck, the next time they have to access, rather than dig it out of version control or their machines - they should hit the wiki.
If they still don't, try to see if the team has enough slack to actually use the wiki - Subtler issues may lie beneath their reluctance.
Take a look at the advice at http://www.ikiw.org/ Grow your Wiki
Just to add to some of the excellent advice being offered here...
As a dev in a small company that does largely gov't contract work in the 6-24 month range, I find that my time is often split between development and writing status reports (right up there with writing documentation, only worse!) Having a wiki to slap down unorganized thoughts and notes as we go along has made report-writing a lot less painful (not pain-LESS, but better all the same).
Further, if you're already in the Mediawiki world, you might want to look at SemanticMediawiki. It allows you to take the organization of your data to another level by semantically tagging it. That doesn't mean a lot on its own, I know, but I can tell you (for example) that it can drastically improve the relevance of the data returned from searches. It is definitely worth a look.
Generally good advice here. I'd like to add:
You really need a champion - someone pushing this to developers and management (without being pushy - that's a challenge!) and providing support & tutorials when possible. This person also needs to be a peer (so a fellow developer, not someone in a remote IT department) and really customer focused i.e. ready to make changes when requested.
Speaking of changes, some people here say wikis are unstructured. I disagree. Our MediaWiki installation is structured using categories, particularly with two extensions:WarnNoCategories (to require users to add a category when saving a page) and CategoryTree to show how all the categories fit together (this can be linked to from the sidebar). I've got more tips on how we keep this low threshold, if you're interested.