0321702832.pdf Read online

  you test and your audience. I favor this approach for three reasons: professionals from a specific domain, like cardiologists for instance. I like to So what would

  Also, since you will have fixed the problems you uncovered in the first round, in I think I’d

  you do next?

  darned!

  these files are very valuable because they’re much easier to review quickly than some others.

  offer people a little more than the going rate, since (a) it makes it clear that I I think I’d

  click here…

  the next round it’s likely that all three users will uncover a new set of problems,

  > We’re all beginners under the skin. Scratch an expert and you’ll often find click here…

  videotape and they’re very easy to share over a network. I recommend that you value their opinion, and (b) people tend to show up on time, eager to since they won’t be getting stuck on the first set of problems.

  someone who’s muddling through—just at a higher level.

  always use a screen recorder during user tests.

  TWO TESTS WITH 3 USERS

  TOTAL PROBLEMS

  participate. Remember, even if the session is only 30 minutes, people usually Testing only three or four users also makes it possible to test and debrief in the FOUND: 9

  > It’s usually not a good idea to design a site so that only your target have to block out another hour for travel time. Also, I’d rather have people who same day, so you can take advantage of what you’ve learned right away. Also, First test: 3 users

  Second test: 3 users

  audience can use it. If you design a site for accountants using terminology are curious about the process than people who are desperate for the money.

  Who should do the testing?

  Three users may

  But in the

  when you test more than four at a time, you usually end up with more notes than that you think all accountants will understand, what you’ll probably discover not find as many

  second test,

  > Keep the invitation simple. “We need to have a few people look at our Web Almost anyone can facilitate a usability test; all it really takes is the courage to try anyone has time to process—many of them about things that are really “nits,”

  problems in a

  with the first

  is that a small but not insignificant number of accountants won’t know what site and give us some feedback. It’s very easy, and would take about forty-five it. With a little practice, most people can get quite good at it.

  which can actually make it harder to see the forest for the trees.

  single test.

  set of problems

  you’re talking about. And in most cases, you need to be addressing novices as fixed, they’ll

  minutes to an hour. And you’ll be paid $___ for your time.”

  well as experts anyway, and if your grandmother can use it, an expert can.

  Try to choose someone who tends to be patient, calm, empathetic, a good listener, In fact this is one of the reasons why I’ve almost completely stopped generating find problems

  they couldn’t

  > Avoid discussing the site (or the organization behind the site) and inherently fair. Don’t choose someone whom you would describe as written reports (what I refer to as the “big honking report”) for my expert

  > Experts are rarely insulted by something that is clear enough for have seen in

  beforehand. You want their first look to tell you whether they can figure out

  “definitely not a people person” or “the office crank.”

  reviews and for usability tests. I finally realized that for most Web teams their the first test.

  beginners. Everybody appreciates clarity. (True clarity, that is, and not just Test subject (A) sits in front of computer monitor (B),

  Meanwhile, cable (F) carries signal from

  what it is from a standing start. (Of course, if they’re coming to your office, ability to find problems greatly exceeds the resources they have available to fix something that’s been “dumbed down.”)

  while facilitator (C) tells him what to do and asks ques-

  camcorder to TV (G) in a nearby room where

  they’ll have a pretty good idea whose site it is.)

  tions. Camcorder (D) powered by squirrel (E) is pointed

  interested team members (H) can observe.

  them, so it’s important to stay focused on the most serious problems. Instead of Who should observe?

  The exceptions:

  at the monitor to record what the subject sees.

  written reports, nowadays I report my findings in a conference call with the

  > Don’t be embarrassed to ask friends and neighbors. You don’t have to feel Anybody who wants to. It’s a good idea to encourage everyone—team members, entire Web team, which may last for an hour or two. By the end of the call, we’ve Recruit loosely and grade on a curve

  > If your site is going to be used almost exclusively by one type of user and like you’re imposing if you ask friends or neighbors to participate. Most people You can use the video cable to run the signal from the camcorder to a TV in people from marketing and business development, and any other stakeholders—

  all agreed which problems are most important to fix, and how they’re going to fix it’s no harder to recruit from that group, then do it. For instance, if your enjoy the experience. It’s fun to have someone take your opinion seriously and another office—or even a cubicle—nearby so everyone on the development team When people decide to test, they often spend a lot of time trying to recruit users to attend.

  them.

  audience will be almost entirely women, then by all means test just women.

  get paid for it, and they often learn something useful that they didn’t know can watch without disturbing the user.

  who they think will precisely reflect their target audience—for instance, male about the Web or computers in general.

  When people ask me how they can convince senior management that their accountants between the ages of 25 and 30 with one to three years of computer

  > If your audience is split between clearly defined groups with very The camcorder needs to transmit what the user sees (the computer screen or the organization should be investing in usability, my strongest recommendation experience who have recently purchased expensive shoes.

  divergent interests and needs, then you need to test users from each group designs on paper, depending on what you’re testing) and what the user and the doesn’t have anything to do with things like “demonstrating return on at least once. For instance, if you’re building a university site, for at least one facilitator say. In a usability lab, you’ll often see a second camera used to show the The best-kept secret of usability testing is the extent to which it doesn’t much round of testing you want to recruit two students, two professors, two high observers the user’s face, but this isn’t necessary: The user’s tone of voice usually matter who you test.

  4

  2

  There are a number of screen recorders available, but I’m partial to Camtasia, made See Jakob Nielsen’s March 2000 Alertbox column “Why You Only Need to Test with 5 Users”

  school seniors, and two administrators. But for the other rounds, you can conveys frustration pretty effectively.

  by TechSmith, the same company that makes the screen capture program SnagIt at www.useit.com for a good discussion of the topic.

  For most sites, all you really need are people who have used the Web enough to choose any mix.

  You can buy the camcorder, TV, cable, and tripod for less than $600. But if your (http://www.techsmith.com). It’s very reliable and has a number of extremely useful 3 If you’re hiring someone to do the testing for you and money is no object, you might as well know the basics.

  budget won’t stretch that far, you can probably twist somebody’s arm to bring in features, and it costs
about $300. For $1,000 more, they have a product called Morae test six or eight users since the additional cost per user will be comparatively low. But only if specifically designed for capturing usability tests—sort of like Camtasia on steroids—which a camcorder from home on test days.

  it won’t mean you’ll do fewer rounds of testing.

  allows observers to view the test live on a networked PC, eliminating the need for a camcorder.

  It was very gratifying how many people

  [ 138 ]

  [ 139 ]

  [ 140 ]

  [ 141 ]

  [ 142 ]

  [ 143 ]

  u s a b i l i t y t e s t i n g o n 10 c e n t s a d ay c h a p t e r 9

  u s a b i l i t y t e s t i n g o n 10 c e n t s a d ay c h a p t e r 9

  u s a b i l i t y t e s t i n g o n 10 c e n t s a d ay c h a p t e r 9

  As a rule, you’ll always get more revealing results if you can find a way to observe users doing tasks that they have a hand in choosing. It’s much better, INTRODUCTION

  If you have questions, just ask. I may not be

  BACKGROUND QUESTIONS

  It’s important to mention

  Do you have any favorite Web sites?

  REACTIONS TO THE HOME PAGE

  for instance, to say “Find a book you want to buy, or a book you bought this, because it will seem

  able to answer them right away, since we’re

  Hi, Janice. My name is Steve Krug, and

  This whole first section is

  rude not to answer their

  recently” than “Find a cookbook for under $14.” When people are doing made-interested in how people do when they don’t

  Yahoo, I guess. I like Yahoo, and I use it all

  First, I’m just going to ask you to look at this

  The browser has been open,

  the script that I use when I

  I’m going to be walking you through

  questions as you go along.

  Before we look at the site, I’d like to ask you

  up tasks, they have no emotional investment in it, and they can’t use as much but minimized. At this

  conduct tests.5

  have someone sitting next to them, but I will

  You have to make it clear

  I find it’s good to start

  the time. And something called

  page and tell me what you think it is, what

  this session.

  just a few quick questions. First, what’s your

  with a few questions to get

  point, I reach over and

  of their personal knowledge.

  before you start that (a)

  try to answer any questions you still have

  Snakes.com, because I have a pet snake.

  strikes you about it, and what you think you

  click to maximize it.

  it’s nothing personal, and

  occupation?

  a feel for who they are and

  when we’re done.

  how they use the Internet.

  would click on first.

  As you begin designing your own site, it’s never too early to start showing your (b) you’ll try to answer

  them at the end if they still

  It gives them a chance to

  Really? What kind of snake?

  Don’t be afraid to digress

  design ideas to users, beginning with your first rough sketches. Designers are You probably already know, but let me explain

  I always have a copy in

  We have a lot to do, and I’m going to try to

  want to know.

  I’m a router.

  loosen up a little and gives

  and find out a little more

  front of me, and I don’t

  For now, don’t actually click on anything. Just

  wrote to tell me that they started doing their

  often reluctant to show work in progress, but users may actually feel freer to why we’ve asked you to come here today. We’re

  keep us moving, but we’ll try to make sure that

  you a chance to show that

  about the user, as long as

  hesitate to read from it,

  you’re going to be listening

  A python. He’s about four feet long, but

  tell me what you would click on.

  comment on something that looks unfinished, since they know you haven’t got testing a Web site that we’re working on so we

  you come back to the topic

  but I find it’s good to ad lib

  it’s fun, too.

  I’ve never heard of that before. What does a

  attentively to what they

  he should get to be eight or nine when

  before long.

  as much invested in it and it’s still subject to change. Also, since it’s not a polished can see what it’s like for actual people to use it.

  a little, even if it means

  router do, exactly?

  say—and that there are no

  You may have noticed the camera. With your

  At this point, most people

  he’s fully grown.

  And again, as much as possible, it will help us

  design, users won’t be distracted by details of implementation and they can focus making mistakes. When the

  wrong or right answers.

  I want to make it clear right away that we’re

  users see that I’m

  will say something like, “I’m

  permission, we’re going to record the computer

  if you can try to think out loud so we know

  on the essence and the wording.

  comfortable making

  not going to end up on

  Not much. I take orders as they come in,

  testing the site, not you. You can’t do anything

  screen and what you have to say. The recording

  Don’t hesitate to admit

  Wow. OK, now, finally, have you bought

  what you’re thinking about.

  mistakes, it helps take the

  America’s Funniest Home

  and send them to the right office.

  Later, as you begin building parts of the site or functioning prototypes, you can wrong here. In fact, this is probably the one

  your ignorance about

  pressure o◊ them.

  will be used only to help us figure out how to

  Videos, am I?”

  anything on the Internet? How do you feel

  anything. Your role here

  begin testing key tasks on your own site.

  place today where you don’t have to worry

  improve the site, and it won’t be seen by anyone

  about buying things on the Internet?

  Good. Now, roughly how many hours a week

  is not to come across

  about making mistakes.

  except the people working on the project. It

  as an expert, but as a

  I also recommend doing what I call Cubicle tests: Whenever you build a new would you say you spend using the Internet,

  also helps me, because I don’t have to take as

  good listener.

  I’ve bought some things recently. I didn’t

  kind of page—particularly forms—you should print the page out and show it to We want to hear exactly what you think, so

  including email?

  many notes. There are also some people

  do it for a long time, but only because I

  the person in the next cubicle and see if they can make sense out of it. This kind please don’t worry that you’re going to hurt

  watching the screen in another room.

  couldn’t get things delivered. It was hard

  of informal testing can be very efficient, and eliminate a lot of potential problems.

  our feelings.6 We want to improve it, so we

  Oh, I do
n’t know. Probably an hour a day at

  Notice that she’s not sure

  how much time she really

  to get things delivered, because I’m not

  need to know honestly what you think.

  If you would, I’m going to ask you to sign

  Give them the release and

  work, and maybe four hours a week at

  spends on the Internet.

  A sample test session

  non-disclosure agreement

  home during the day. But now one of my

  something for us. It simply says that we have

  home. Mostly that’s on the weekend. I’m

  Most people aren’t. Don’t

  As we go along, I’m going to ask you to think

  (if required) to sign. Both

  neighbors is home all the time, so I can.

  your permission to record you, but that it will

  too tired at night to bother. But I like

  worry. Accurate answers

  Here’s an annotated excerpt from a typical—but imaginary—test session. The site out loud, to tell me what’s going through your

  should be as short as

  aren’t important here. The

  only be seen by the people working on the

  possible and written in

  playing games sometimes.

  is real, but it has since been redesigned. The participant’s name is Janice, and mind. This will help us.

  7

  main point here is just to

  plain English.

  And what have you bought?

  project. It also says that you won’t talk to

  get her talking and

  she’s about 25 years old.

  anybody about what we’re showing you today,

  How do you spend that time? In a typical day,

  thinking about how she

  uses the Internet and to

  Well, I ordered a raincoat from L.L. Bean,

  since it hasn’t been made public yet.

  for instance, tell me what you do, at work and

  give you a chance to gauge

  and it worked out much better than I

  at home.

  what kind of user she is.

  Do you have any questions before we begin?

  thought it would. It was actually pretty easy.

  Well, at the office I spend most of my time

  No. I don’t think so.

  OK, great. We’re done with the questions, and

  checking email. I get a lot of email, and a

  we can start looking at things.

  lot of it’s junk but I have to go through it