Posted by: bbannan | April 2, 2009

Usability – Debbie

As we have been working on usability testing this semester I have realized how much better many websites would be if usability testing had been done. Another office where I work is implementing a new site which no one in the office likes. I asked if any usability testing had been done and the answer was no. Big mistake!

I thought it would be interesting to see what information was out there on testing websites. I found an article called “OJR’s ‘five guide’ to do-it-yourself website usability testing” by Laura Ruel and Nora Paul. (They also have a link to the step-by-step usability guide which is a flowchart for usability.) Ruel and Paul believe that you can do usability with only five people in five hours and in five steps. They say that five people can uncover 80% of problems. It’s a quick article to read and touches on much of what we have been doing this semester. Do you think that usability testing can really be done well using the ‘five guide?’ What do you think you have to have to make this work? Is it missing anything?



  1. Debbie, as we have all had the pleasure of learning this semester, the rule of five is completely plausible . . . if you happen to select the right five. When looking for verification of a concept, design, or result, one of the first steps is to decide why you want to do the testing in the first place. When you come up with your best answer to this question, the rest of the questions become easier. A sample population used in testing has to have relevance toward what you are trying to determine. While it does seem to follow that a perfect sample of five can answer most of the questions you may need to know, the chance of finding that perfect sample is not always possible. One of the reasons we try to involve as large a sample as can be reasonable worked with is to overcome the percentage of the sample population that will, due to selection error, skew the results. Very simply, the larger the sample, the greater chance you have of obtaining useful material.

    Now, having said that, I have to admit that sometimes it is simply impossible to obtain as many volunteers (although rewards promised helps) as we would like to have to ensure our concept-design-result is indeed what we want it to be. With that in mind, I do have to agree with Ruel and Paul. I am inclined to be somewhat skeptical about their claim of 80% return. If this is to occur, I would think you would have to be very attentive in the selection of the five.

    My take on all of this is simple. Obtain as many qualified participants as you are able. Be extremely careful in how you test them, using more than one tool to ensure you have several approaches to results comparison. As for the five hours . . . well, if it is my project, I think it is not a matter of how short a timeframe I will use; rather it is the point that I think I have obtained as successful a result as possible.

  2. I appreciate the simplicity of this article and think that it can indeed help the novice designer get started in usability testing. I especially like the suggestion that for data analysis one should make a list of at the top three things that should not change and the top three things that should. Doing so will help the designer focus and prioritize redesign efforts and is in keeping with the iterative nature of instructional design.

    I contend that usability testing can be done well using this guide as just that, a guide. The five steps put forth by the authors of the article are fundamental but there’s a lot of substance to these fundamentals that needs to be addressed. For example, how does the designer make sure that the tasks the user is asked to complete will actually provide the information needed? And, how do you phrase open-ended questions so that they will elicit the type of feedback you need? I also agree with Ed that the success of conducting a usability test in this manner is highly dependent on the population participating in the testing. Extreme care must be taken to make certain that the sample used during usability testing is representative of the target audience. Thus, despite Jakob Nielsen’s chart, I would prefer to sample from 10 potential users instead of 5 to help ensure accurate representation of my target audience.

    I am looking forward to conducting my first usability test in just a few days. This article reassured me that my team is taking the right approach and served to reinforce my understanding of the basic methodology behind usability testing.

  3. I think the article is fascinating! It would be great if only 5 people can identify 80% of the problems. I really would like to get a statistician’s opinion on this as I learned way back when in my statistic’s class you had to have a large enough sample size to truly represent your population. However, I think Ed makes a really good point that if you select the five target users well they may indeed give you the key input you need. The advantage of only having to use five people would also enable you to potentially conduct additional usability testing throughout the process rather than just conducting one test with a lot more people. Very interesting article and I am glad you shared it with us!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: