Don't Make Me Think, Revisited: A Common Sense Approach to Web Usability Page 10

  * * *

  * * *

  * * *

  Background Questions

  Before we look at the site, I’d like to ask you just a few quick questions. First, what’s your occupation? What do you do all day?

  I’m a router.

  I’ve never heard of that before. What does a router do, exactly?

  I take orders as they come in and send them to the right office. We’re a big multinational company, so there’s a lot to sort out.

  OK. Now, roughly how many hours a week would you say you spend using the Internet, including Web browsing and email? Just a ballpark estimate.

  * * *

  I find it’s good to start with a few questions to get a feel for who they are and how they use the Internet. It gives them a chance to loosen up a little and gives you a chance to show that you’re going to be listening attentively to what they say—and that there are no wrong or right answers.

  Don’t hesitate to admit your ignorance about anything. Your role here is not to come across as an expert, but as a good listener.

  * * *

  Oh, I don’t know. Probably four hours a day at work, and maybe eight hours a week at home. Mostly that’s on the weekend. I’m too tired at night to bother. But I like playing games sometimes.

  What’s the split between email and browsing—a rough percentage?

  Well, at the office I spend most of my time checking email. I get a lot of email, and a lot of it’s junk but I have to go through it anyway. Maybe two-thirds of my time is on email and one-third is browsing.

  * * *

  Notice that she’s not sure how much time she really spends on the Internet. Most people aren’t. Don’t worry. Accurate answers aren’t important here. The main point here is just to get her talking and thinking about how she uses the Internet and to give you a chance to gauge what kind of user she is.

  * * *

  What kinds of sites are you looking at when you browse the Web?

  At work, mostly our corporate intranet. And some competitors’ sites. At home, game sites and some shopping.

  Do you have any favorite Web sites?

  Well, Google, of course. I use it all the time. And something called Snakes.com, because I have a pet snake.

  Really? What kind of snake?

  A python. He’s about four feet long, but he should get to be eight or nine when he’s fully grown.

  OK, great. We’re done with the questions, and we can start looking at things.

  OK, I guess.

  * * *

  Don’t be afraid to digress and find out a little more about the user, as long as you come back to the topic before long.

  * * *

  * * *

  * * *

  The Home Page Tour

  First, I’m just going to ask you to look at this page and tell me what you make of it: what strikes you about it, whose site you think it is, what you can do here, and what it’s for. Just look around and do a little narrative.

  You can scroll if you want to, but don’t click on anything yet.

  * * *

  Until now, the browser has been opened to Google so there’s nothing distracting to look at.

  At this point, I reach over and open a tab with the site we’re testing and give the mouse to the participant.

  * * *

  Well, I guess the first thing I notice is that I like the color. I like the shade of orange, and I like the little picture of the sun [at the top of the page, in the eLance logo].

  Let’s see. [Reads.] “The global services market.” “Where the world competes to get your job done.”

  * * *

  In an average test, it’s just as likely that the next user will say that she hates this shade of orange and that the drawing is too simplistic. Don’t get too excited by individual reactions to site aesthetics.

  * * *

  I don’t know what that means. I have no idea.

  “Animate your logo: free.” [Looking at the Cool Stuff section on the left.] “Graphic design marketplace.” “View the RFP marketplace.” “eLance marketplaces.”

  There’s a lot going on here. But I have no idea what any of it is.

  If you had to take a guess, what do you think it might be?

  Well, it seems to have something to do with buying and selling...something.

  [Looks around the page again.] Now that I look at the list down here [the category list halfway down the page], I guess maybe it must be services. Legal, financial, creative... they all sound like services.

  * * *

  This user has been doing a good job of thinking out loud on her own. If she wasn’t, this is where I’d start asking her, “What are you thinking?”

  * * *

  So I guess that’s what it is. Buying and selling services.

  OK. Now, if you were at home, what would you click on first?

  I guess I’d click on that graphic design thing. I’m interested in graphic design.

  * * *

  * * *

  The Tasks

  OK, now we’re going to try doing some specific tasks.

  And again, as much as possible, it will help us if you can try to think out loud as you go along.

  Can you think of some service that you need that you could use this site to get help with?

  Hmm. Let me think. I think I saw “Home Improvement” there somewhere. We’re thinking of building a deck. Maybe I could find somebody to do that.

  So if you were going to look for somebody to build your deck, what would you do first?

  I guess I’d click on one of the categories down here. I think I saw home improvement. [Looks.] There it is, under “Family and Household.”

  So what would you do?

  Well, I’d click.... [Hesitates, looking at the two links under “Family and Household.”]

  * * *

  Now I give her a task to perform so we can see whether she can use the site for its intended purpose.

  Whenever possible, it’s good to let the user have some say in choosing the task.

  * * *

  Well, now I’m not sure what to do. I can’t click on Home Improvement, so it looks like I have to click on either “RFPs” or “Fixed-Price.” But I don’t know what the difference is.

  Fixed-price I sort of understand; they’ll give me a quote, and then they have to stick to it. But I’m not sure what RFPs is.

  * * *

  As it turns out, she’s mistaken. Fixed-price (in this case) means services available for a fixed hourly rate, while an RFP (or Request for Proposal) is actually the choice that will get her the kind of quote she’s looking for. This is the kind of misunderstanding that often surprises the people who built the site.

  Well, which one do you think you’d click on?

  Fixed-price, I guess.

  Why don’t you go ahead and do it?

  * * *

  From here on, I just watch while she tries to post a project, letting her continue until either (a) she finishes the task, (b) she gets really frustrated, or (c) we’re not learning anything new by watching her try to muddle through.

  I’d give her three or four more tasks to do, which should take not more than 45 minutes altogether.

  * * *

  * * *

  * * *

  Probing

  Now that we’re done with the tasks, I have a few questions.

  What about these pictures near the top of the page—the ones with the numbers? What did you make of them?

  * * *

  While the participant is doing the tasks, I’m careful not to ask leading questions because I don’t want to bias her.

  But I always save some time at the end specifically to ask probing questions so I can understand more about what happened and why it happened.

  * * *

  I noticed them, but I really didn’t try to figure them out. I guess I thought they were telling me what the steps in the process would be.

  Any reason why you didn’t pay much attention
to them?

  No. I guess I just wasn’t ready to start the process yet. I didn’t know if I wanted to use it yet. I just wanted to look around first.

  OK. Great.

  * * *

  In this case, I ask this question because the site’s designers think most users are going to start by clicking on the pictures of the five steps and that everyone will at least look at them.

  * * *

  * * *

  That’s really all there is to it.

  If you’d like to see a more complete test, you’ll find a twenty-minute video on my site. Just go to rocketsurgerymadeeasy.com and click on “Demo test video.”

  Typical problems

  Here are some of the types of problems you’re going to see most often:

  Users are unclear on the concept. They just don’t get it. They look at the site or a page and either they don’t know what to make of it or they think they do but they’re wrong.

  The words they’re looking for aren’t there. This usually means that either you failed to anticipate what they’d be looking for or the words you’re using to describe things aren’t the words they’d use.

  There’s too much going on. Sometimes what they’re looking for is right there on the page, but they’re just not seeing it. In this case, you need to either reduce the overall noise on the page or turn up the volume on the things they need to see so they “pop” out of the visual hierarchy more.

  The debriefing: Deciding what to fix

  After each round of tests, you should make time as soon as possible for the team to share their observations and decide which problems to fix and what you’re going to do to fix them.

  I recommend that you debrief over lunch right after you do the tests, while everything is still fresh in the observers’ minds. (Order the really good pizza from the expensive pizza place to encourage attendance.)

  Whenever you test, you’re almost always going to find some serious usability problems. Unfortunately, they aren’t always the ones that get fixed. Often, for instance, people will say, “Yes, that’s a real problem. But that functionality is all going to change soon, and we can live with it until then.” Or faced with a choice between trying to fix one serious problem or a lot of simple problems, they opt for the low-hanging fruit.

  This is one reason why you can so often run into serious usability problems even on large, well-funded Web sites, and it’s why one of my maxims in Rocket Surgery is

  FOCUS RUTHLESSLY ON FIXING THE MOST SERIOUS PROBLEMS FIRST

  Here’s the method I like to use to make sure this happens, but you can do it any way that works for your team:

  Make a collective list. Go around the room giving everyone a chance to say what they thought were the three most serious problems they observed (of the nine they wrote down; three for each session). Write them down on a whiteboard or sheets of easel pad paper. Typically, a lot of people will say “Me, too” to some of them, which you can keep track of by adding checkmarks.

  There’s no discussion at this point; you’re just listing the problems. And they have to be observed problems; things that actually happened during one of the test sessions.

  Choose the ten most serious problems. You can do informal voting, but you can usually start with the ones that got the most checkmarks.

  Rate them. Number them from 1 to 10, 1 being the worst. Then copy them to a new list with the worst at the top, leaving some room between them.

  Create an ordered list. Starting at the top, write down a rough idea of how you’re going to fix each one in the next month, who’s going to do it, and any resources it will require.

  You don’t have to fix each problem perfectly or completely. You just have to do something—often just a tweak—that will take it out of the category of “serious problem.”

  When you feel like you’ve allocated all of the time and resources you have available in the next month for fixing usability problems, STOP. You’ve got what you came for. The group has now decided what needs to be fixed and made a commitment to fixing it.

  Here are some tips about deciding what to fix—and what not to.

  Keep a separate list of low-hanging fruit. You can also keep a list of things that aren’t serious problems but are very easy to fix. And by very easy, I mean things that one person can fix in less than an hour, without getting permission from anyone who isn’t at the debriefing.

  Resist the impulse to add things. When it’s obvious in testing that users aren’t getting something, the team’s first reaction is usually to add something, like an explanation or some instructions. But very often the right solution is to take something (or some things) away that are obscuring the meaning, rather than adding yet another distraction.

  Take “new feature” requests with a grain of salt. Participants will often say, “I’d like it better if it could do x.” It pays to be suspicious of these requests for new features. I find that if you ask them to describe how that feature would work—during the probing time at the end of the test—it almost always turns out that by the time they finish describing it they say something like “But now that I think of it, I probably wouldn’t use that.” Participants aren’t designers. They may occasionally come up with a great idea, but when they do you’ll know it immediately, because your first thought will be “Why didn’t we think of that?!”

  Ignore “kayak” problems. In any test, you’re likely to see several cases where users will go astray momentarily but manage to get back on track almost immediately without any help. It’s kind of like rolling over in a kayak; as long as the kayak rights itself quickly enough, it’s all part of the so-called fun. In basketball terms, no harm, no foul.

  As long as (a) everyone who has the problem notices that they’re no longer headed in the right direction quickly, and (b) they manage to recover without help, and (c) it doesn’t seem to faze them, you can ignore the problem. In general, if the user’s second guess about where to find things is always right, that’s good enough.

  Alternative lifestyles

  Here are two other ways to do testing that have distinct advantages:

  Remote testing. The difference here is that instead of coming to your office, participants do the test from the comfort of their own home or office, using screen sharing. Eliminating the need to travel can make it much easier to recruit busy people and, even more significantly, it expands your recruiting pool from “people who live near your office” to “almost anyone.” All they need is high-speed Internet access and a microphone.

  Unmoderated remote testing. Services like UserTesting.com provide people who will record themselves doing a usability test. You simply send in your tasks and a link to your site, prototype, or mobile app. Within an hour (on average), you can watch a video of someone doing your tasks while thinking aloud.5 You don’t get to interact with the participant in real time, but it’s relatively inexpensive and requires almost no effort (especially recruiting) on your part. All you have to do is watch the video.

  5 Full disclosure: I receive some compensation from UserTesting.com for letting them use my name. But I only do that because I’ve always thought they have a great product—which is why I’m mentioning them here.

  Try it, you’ll like it

  Whatever method you use, try doing it. I can almost guarantee that if you do, you’ll want to keep doing it.

  Here are some suggestions for fending off any objections you might encounter:

  The Top Five Plausible Reasons for not Testing Web Sites

  Larger Concerns and Outside Influences

  Chapter 10. Mobile: It’s not just a city in Alabama anymore

  WELCOME TO THE 21ST CENTURY — YOU MAY EXPERIENCE A SLIGHT SENSE OF VERTIGO

  [shouting] PHENOMENAL COSMIC POWERS! [softly] Itty-bitty living space!

  —ROBIN WILLIAMS AS THE GENIE IN ALADDIN, COMMENTING ON THE UPSIDE AND DOWNSIDE OF THE GENIE LIFESTYLE

  Ahh, the smartphone.

  Phones had been getting gradually smarter for years, gath
ering in desk drawers and plotting amongst themselves. But it wasn’t until the Great Leap Forward1 that they finally achieved consciousness.

  1 Introduction of the iPhone June 2007.

  I, for one, was glad to welcome our tiny, time-wasting overlords. I know there was a time when I didn’t have a powerful touch screen computer with Internet access in my pocket, but it’s getting harder and harder to remember what life was like then.

  And of course it was about this same time that the Mobile Web finally came into its own. There had been Web browsers on phones before, but they—to use the technical term—sucked.

  The problem had always been—as the Genie aptly put it—the itty-bitty living space. Mobile devices meant cramped devices, squeezing Web pages the size of a sheet of paper into a screen the size of a postage stamp. There were various attempts at solutions, even some profoundly debased “mobile” versions of sites (remember pressing numbers to select numbered menu items?) and, as usual, the early adopters and the people who really needed the data muddled through.