0321702832.pdf Read online

  You’ll

  need Internet access for screen sharing and to get to whatever you’re testing if it’s online.

  The screen recording software is used to capture a record of what happens on the screen and what you and the participant say. The best

  [ 65 ]

  chapter 8

  part of having a recording is that you don’t have to feel pressured to take detailed notes. By dragging the “thumb” on the scroll bar in the video player, you can usually find any point in the recording within seconds.1

  The recordings can also come in very handy at the debriefing session in case there’s a disagreement about what the participant actually said or did.

  In theory, people who can’t come to the sessions can watch the recordings later, but in the universe I live in, it almost never happens.2 I consider this fortunate since I’d much rather have people come to the tests.

  I use Camtasia ($300 for the PC, $150 for the Mac). Even though there are less expensive screen recorders available (including one—CamStudio—

  that’s open source), I don’t know of another one that has as many useful features as Camtasia, including a built-in video editor that makes it easy to extract clips, add titles, and much more. I’ve relied on it for years, and it has never let me down. (Looking around for some wood to knock on.) The screen sharing software allows the people in the observation room to see and hear the test. There are many options to choose from; some require a paid subscription and some are free, and almost all of them have free trials. If you work for a large company, they may already have a license for one of them. (WebEx seems to be a corporate favorite.) Personally, I use GoToMeeting. It’s extremely user-friendly and reliable, it works on Macs and PCs, and it has a lot of very useful features. It costs $49

  a month for unlimited use, with up to 15 computers connected at a time.

  And it has VOIP capability built in.

  It’s also particularly good for doing remote testing (Chapter 14), which you’ll probably want to do eventually.

  b) A monitor and a keyboard. If you’re using a laptop, you should arrange to have a monitor and an external keyboard. A 17" monitor will make it easy for you to see the screen without having to sit too close to the participant.

  1 Believe it or not, this action actually has a name: “scrubbing” the video.

  2 To quote Yogi Berra again, “In theory, there is no diff erence between theory and practice.

  But, in practice, there is.”

  [ 66 ]

  mind reading made easy

  The screen resolution should probably be 1024 x 768, unless you know that a large majority of your users will have it set higher or lower. (If the participant says something like “I have a lot more showing on my screen,”

  you can increase it to a higher resolution, like 1280 x 1024.)

  c) A “plain vanilla” mouse. Don’t make the participant use an exotic trackball, a laptop touchpad, or one of those eraser-things that stick out from the middle of the keyboard that don’t seem to have a name. Some people will find anything other than a mouse difficult to use.

  d) A USB microphone. Getting good-quality audio into the observation room is crucial. Straining to hear what the user is saying can be very tiring, and eventually people will take out their BlackBerrys or just leave.

  Good audio also allows you to hear the “body language” of the user’s voice so you can easily sense whether they’re feeling comfortable or frustrated, for instance.

  I recommend using VOIP (voice over IP) instead of a speakerphone if possible, because the audio quality tends to be much better. GoToMeeting includes VOIP service, or you can use a service

  like Skype.

  You’ll need a microphone to use VOIP. My favorite is the

  inexpensive (about $25) Logitech USB Desktop Microphone.

  Even if the microphone built into your laptop is good, having

  an external mike allows you to position it so the observers

  can hear both the participant and the facilitator clearly.

  e) A

  speakerphone.

  Even if you use VOIP, you should have speakerphones

  available in both rooms as backup. (Make sure that you have the phone number for the observation room.)

  [ 67 ]

  chapter 8

  Pre-Test Prep

  Allow 60 Minutes

  At the beginning of the day, you need to make sure that everything in the test room is ready. Try to start doing this an hour before the first participant arrives so you’ll have enough time to fix anything that’s not working, plus a few minutes to catch your breath when you’re done.

  �Test the screen recorder. Make a short recording and play it back.

  The microphone volume setting should be turned all the way up since you and the participant are going to be relatively far away from the mike. In my experience, turning the recording volume all the way up won’t distort the audio.

  If you’re using a laptop that has a built-in microphone, make sure that you’re recording from the right source by tapping the external microphone during your test recording. (It should sound like you hit it with a hammer when you play it back.)

  �Test the screen sharing. Ask someone to step into the observation room for a minute and then start a screen sharing session and make sure that they can hear you clearly and see the screen.

  �Make the cursor larger than normal. This will make it easier for you and the observers to follow what the participant is doing.3

  �Turn off any software that might interrupt the test. Email, instant messaging, calendar event reminders, and scheduled virus scans are the most likely culprits.

  �Make sure you have bookmarks for any pages you’ll need to open during the test. You don’t want to spend valuable test time typing URLs.

  3 I’ll give details about how to do this—and the rest of my standard settings for Camtasia and GoToMeeting—at the book Web site.

  [ 68 ]

  mind reading made easy

  �Try using what you’re testing. It never hurts to have one last look at your site or prototype to make sure that your Internet access is working, your server hasn’t crashed, and some dedicated developer hasn’t done some last-minute tweaking without telling you. It’s much better to find out now than when the participant is in the room.

  �Reset everything. If you’re using sample data, make sure that you’ve reloaded a clean set. And clear the browsing history in your Web browser, so visited links don’t give the user “clues.”

  �Touch base with your Designated Greeter. Make sure the person you’ve asked to welcome the participants when they arrive is ready for them.

  [ 69 ]

  chapter 8

  Welcome

  4 Minutes

  You begin each session by

  reading the first part of the

  Hi, ___________. My name is ___________, and I’m going

  script,4 which explains how

  to be walking you through this session today.

  the test is going to work.

  Before we begin, I have some information for you,

  Some people like

  and I’m going to read it to make sure that I cover

  to improvise these

  everything.

  instructions from an

  You probably already have a good idea of why we

  outline so it will sound

  asked you here, but let me go over it again briefly.

  more natural, but I

  We’re asking people to try using a Web site that

  recommend using the

  we’re working on so we can see whether it works

  script and reading it exactly

  as intended. The session should take about

  as written. Even though

  an hour.

  I’ve been doing testing for

  twenty years, every time

  The first thing I want to make clear right away is
/>
  I give in to the temptation

  that we’re testing the site, not you. You can’t do

  to depart from the script,

  anything wrong here. In fact, this is probably the

  odds are 50-50 I’ll say

  one place today where you don’t have to worry

  something that gives the

  about making mistakes.

  participant the wrong

  As you use the site, I’m going to ask you as much

  idea (using a word like

  as possible to try to think out loud: to say what

  “opinions” or “feedback,”

  you’re looking at, what you’re trying to do, and

  for instance). Don’t

  what you’re thinking. This will be a big help to us.

  improvise.

  Also, please don’t worry that you’re going to hurt

  our feelings. We’re doing this to improve the site,

  so we need to hear your honest reactions.

  4 You’ll fi nd the entire script—complete with “stage directions”—on pages 147–152, and you can also download it from the book Web site.

  [ 70 ]

  mind reading made easy

  Read the script exactly as

  If you have any questions as we go along, just

  written.5

  ask them. I may not be able to answer them right

  Reading it may feel a little

  away, since we’re interested in how people do

  uncomfortable because

  when they don’t have someone sitting next to

  we don’t often read things

  them to help. But if you still have any questions

  aloud—at least not to other

  when we’re done I’ll try to try to answer them

  adults. But no one will

  then. And if you need to take a break at any

  mind because the script

  point, just let me know.

  explains why you’re doing

  You may have noticed the microphone. With

  it, it’s only about three

  your permission, we’re going to record what

  minutes long, and they’re

  happens on the screen and our conversation. The

  probably curious about

  recording will only be used to help us figure out

  what’s going to happen.

  how to improve the site, and it won’t be seen by

  anyone except the people working on this project.

  And it helps me, because I don’t have to take as

  many notes.

  Also, there are a few people from the Web design

  team observing this session in another room.

  (They can’t see us, just the screen.)

  If you would, I’m going to ask you to sign a

  simple permission form for us. It just says that

  we have your permission to record you, and that

  the recording will only be seen by the people

  working on the project.

  Do you have any questions so far?

  5 If you fi nd that each time you read it you’re stumbling over something that doesn’t sound natural to you, you can make very minor changes that don’t aff ect the meaning. For instance, if you think “If you have any questions during this session” sounds more natural to you than “If you have any questions as we go along,” then edit your copy of the script so you say it the same way every time.

  [ 71 ]

  chapter 8

  Just try to relax while you read it and connect with the participant: Make eye contact. Print the script out in large type so you can read it without having to stare too intently at the page, and try to look at the participant after every few sentences.

  Don’t mumble. The participant needs to hear what you’re saying.

  Don’t race through it, but don’t drag it out.

  Don’t read it in a monotone or a sing-song voice. Try to put a little life into it but don’t make it into “The Midnight Ride of Paul Revere.”

  [ 72 ]

  mind reading made easy

  The Questions

  2 Minutes

  Usability test participants

  are often asked a series of

  OK. Before we look at the site, I’d like to ask you

  pre-test questions at the

  just a few quick questions.

  beginning of the test and

  First, what’s your occupation? What do you do

  post-test questions at the

  all day?

  end.

  Now, roughly how many hours a week

  I only ask a few simple

  altogether—just a ballpark estimate—would you

  questions, and they serve

  say you spend using the Internet, including Web

  three functions:

  browsing and email, at work and at home?

  Get the participants

  And what’s the split between email and

  comfortable talking.

  browsing—a rough percentage?

  Everyone can come up

  with answers to these

  What kinds of sites are you looking at when you

  questions, so it gets

  browse the Web?

  them started talking

  Do you have any favorite Web sites?

  about themselves. This

  makes it easier when they have to start thinking aloud.

  Show them that you’re going to be listening to what they say.

  Knowing that you’re actually listening and not just getting what you need to fill in a form tends to make the participants more invested in the process and increase their comfort level. But to have this effect, you need to actually listen.

  [ 73 ]

  chapter 8

  Feel free to ask them follow-up questions. I usually ask at least one question about their job, like what their title means or what their company does. And if you don’t understand something they say (“We broker

  irreducible energy credits”), don’t pretend that you do. Ask them to explain it.

  Get the information you need to grade on a curve. By the time the participant has answered these questions, you’ll have a pretty clear idea of (a) what they do for a living and (b) how computer-savvy and Web-savvy they are. This—plus the sense of the extent of their domain knowledge that you get from their reaction to the Home page (coming up next)—is usually all you need to decide how this person compares to your target audience.

  [ 74 ]

  mind reading made easy

  The Home Page Tour

  3 Minutes

  I always start a Web

  OK, great. We’re done with the questions, and we

  site test by having the

  can start looking at things.

  participant look around

  the Home page and tell

  First, I’m going to ask you to look at this page and

  me briefly what they

  tell me what you make of it: what strikes you about

  make of it.

  it, whose site you think it is, what you can do here,

  and what it’s for. Just look around and do a little

  The point is to see if the

  narrative.

  nature of the site is clear:

  Can users figure out what

  You can scroll if you want to, but don’t click on

  this thing is? As I’ll explain

  anything yet.

  later,6 more often than

  you’d expect, the answer is a surprising—and revealing—“No.”

  Having them do this “narrative” also gives you some idea of what they already know about the site, the organization behind it, and the subject matter—their domain knowledge.

  Note that you’re not asking them for their opinion of the Home page. The script doesn’t say “Look around the Home page and tell me what you t
hink of it.”

  The instruction is carefully worded so they actually have a specific task to do: Figure out what this site is. This is a realistic (and important) task, one that people do on their own whenever they come to a new site. You’re just asking them to verbalize it.

  And it doesn’t use much time, since most people will run out of things to say in two or three minutes. You don’t want to let it go on longer than three minutes anyway.

  The script tells them that they can scroll, but they shouldn’t click on anything yet. If they click on a link anyway—and some people will—step in right away and ask them to go back to the Home page. Just say, “For right now, I just want you to stick to the Home page. Could you go back?”

  6 The Big Bang Theory of Web Usability on page 122.

  [ 75 ]

  chapter 8

  The Tasks

  35 Minutes

  The tasks are the “meat”

  Now I’m going to ask you to try doing some specific

  of the test.

  tasks. I’m going to read each one out loud and give

  At the beginning of each

  you a printed copy.

  task, you’ll hand the

  I’m also going to ask you to do these tasks without

  participant a copy of the

  using Search. We’ll learn a lot more about how well

  scenario and then read

  the site works that way.

  it aloud, word-for-word,

  exactly the way you

  And again, as much as possible, it will help us

  wrote it.

  if you can try to think out loud as you go along.

  Why not just let the participant read it? If you do, some people won’t read it carefully enough and they’ll end up wasting time based on some misunderstanding of the task. If you read it to them, at least you’ll know that they’ve heard every word of it.

  Once they start a task, try not to interrupt any more than necessary. Basically, just keep them focused on the task and thinking aloud until it’s time to move on to the next task.

  How do you decide it’s time to move on?

  Have they completed the task? If they have, hand them the next scenario and start the next task. (If they think they have, but haven’t really, you can ask if they could try doing it again another way, which will usually lead them to realize their mistake.)

  [ 76 ]

  mind reading made easy