0321702832.pdf Read online

  Rocket Surgery Made Easy

  the do-it-yourself guide to finding

  and fixing usability problems

  Steve Krug

  opening remarks

  Rocket Surgery Made Easy:

  The Do-It-Yourself Guide to Finding and Fixing Usability Problems Steve Krug

  New Riders

  1249 Eighth Street

  Berkeley, CA 94710

  510/524-2178 510/524-2221 (fax)

  Find us on the Web at: www.newriders.com

  To report errors, please send a note to [email protected]

  New Riders is an imprint of Peachpit, a division of Pearson Education Copyright © 2010 by Steve Krug

  Editor: Nancy Davis

  Production Editor: Lisa Brazieal

  Copyeditor: Barbara Flanagan

  Design and production: Allison D. Cecil

  Illustration: Mark Matcho

  Notice of Rights

  All rights reserved. No part of this book may be reproduced or transmitted in any form by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. For information on getting permission for reprints and excerpts, contact permissions@

  peachpit.com.

  Notice of Liability

  The information in this book is distributed on an “As Is” basis, without warranty. While every precaution has been taken in the preparation of the book, neither the author nor Peachpit shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the instructions contained in this book or by the computer software and hardware products described in it.

  Trademarks

  Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and Peachpit was aware of a trademark claim, the designations appear as requested by the owner of the trademark. All other product names and services identified throughout this book are used in editorial fashion only and for the benefit of such companies with no intention of infringement of the trademark. No such use, or the use of any trade name, is intended to convey endorsement or other affiliation with this book.

  It’s not rocket surgery™ and The least you can do™ are trademarks of Steve Krug.

  ISBN-13: 978-0-321-65729-9

  ISBN-10:

  0-321-65729-2

  call me ishmael

  To my Aunt Isabel

  (Sister Rose Immaculata, O.P.),

  who has prayed for me

  every day of my life,

  My brother, Phil

  who worked his whole adult life

  as a Legal Services attorney, keeping

  families from ending up out on the street,

  And all the other people like them

  who spend their lives

  making sure that things work out

  for the rest of us.

  [ iii

  [

  ]

  iii

  opening remarks

  opening remarks

  Call me Ishmael

  2

  How this book came to be, some disclaimers,

  and a bit of housekeeping

  FINDING USABILITY PROBLEMS

  chapter 1

  You don’t see any elephants around

  12

  here, do you?

  What do-it-yourself usability testing is, why it always

  works, and why so little of it gets done

  chapter 2

  I will now saw my [lovely] assistant in half

  20

  What a do-it-yourself test looks like

  chapter 3

  A morning a month, that’s all we ask

  22

  A plan you can actually follow

  chapter 4

  What do you test, and when do you test it? 30

  Why the hardest part is starting early enough

  chapter 5

  Recruit loosely and grade on a curve

  38

  Who to test with and how to find them

  chapter 6

  Find some things for them to do

  50

  Picking tasks to test and writing scenarios for them

  chapter 7

  Some boring checklists

  56

  And why you should use them even if, like me,

  you don’t really like checklists

  chapter 8

  Mind reading made easy

  62

  Conducting the test session

  chapter 9

  Make it a spectator sport

  90

  Getting everyone to watch and telling them what to look for

  [ iv ]

  call me ishmael

  FIXING USABILITY PROBLEMS

  chapter 10

  Debriefing 101

  102

  Comparing notes and deciding what to fix

  chapter 11

  The least you can do™ 110

  Why doing less is often the best way to fix things

  chapter 12

  The usual suspects

  120

  Some

  problems

  you’re

  likely to find and how to think

  about fixing them

  chapter 13

  Making sure life actually improves

  128

  The art of playing nicely with others

  THE ROAD AHEAD

  chapter 14

  Teleportation made easy

  134

  Remote testing: Fast, cheap, and slightly out of control

  chapter 15

  Overachievers only

  140

  Recommended

  reading

  chapter 16

  Happy trails / to you

  144

  A few final words of encouragement

  Sample test script and consent form

  146

  Acknowledgments

  154

  Index

  158

  [ v ]

  opening remarks

  opening remarks

  Call me Ishmael

  how this book came to be, some disclaimers,

  and a bit of housekeeping

  [ 2 ]

  I love deadlines. I love the whooshing noise

  they make as they go by.

  —DOUGLAS ADAMS, AUTHOR OF THE HITCHHIKER’S GUIDE

  TO THE GALAXY, WHO WAS NOTORIOUSLY

  LATE DELIVERING MANUSCRIPTS

  I knew I wanted to write this book nine

  September 2000

  years ago, right after I finished writing

  Don’t Make Me Think.

  Without meaning to, in the process of writing

  it I had ended up convincing myself of

  three things:

  Usability testing is one of the best things people can do to improve Web sites (or almost anything they’re creating that people have to interact with).

  Since most organizations can’t afford to hire someone to do testing for them on a regular basis, everyone should learn to do it themselves. And…

  I could probably write a pretty good book explaining how to do it.

  There was just one small problem, though:

  I

  hate

  writing.

  Actually, I don’t hate it so much as I find it, well, probably the most accurate word is agonizing.

  And not “Should I buy the white iPhone or the black iPhone?” agonizing.

  More like red-hot-pokers-in-your-eyes agonizing. I’ve always said that writing is the hardest work I know of and that I can’t understand why anyone would do it unless someone was holding a gun to
their head (which, of course, is what deadlines are all about).

  As it turns out, though, it was probably a good thing that I wasn’t motivated to write this book right away, because one of the nicest side effects of the first

  [ 3 ]

  opening remarks

  book was that it gave me the opportunity to teach workshops, which suit my 1

  nature much better than writing or consulting.

  For the first five years,

  September 2001

  …and that’s why I

  my workshop was a

  think consistency

  September 2002

  combination lecture-

  is overrated.

  September 2003

  September 2004

  demo format, where I’d

  September 2005

  do brief expert reviews of

  attendees’ sites to show

  them how I thought

  about usability problems.

  I wanted to teach people

  how to do their own testing, but I couldn’t figure out how to fit it into a one-day workshop.

  Then three years ago,

  September 2006

  Why do they

  What are

  after a lot of pondering, I

  need my zip code

  you thinking?

  September 2007

  to send me email?

  finally figured out how to

  September 2008

  do a workshop that would

  teach people to do their

  own testing—including

  some hands-on practice—

  in one day. I changed the

  format so the whole day

  was about the topic of this book: doing your own usability tests.

  After teaching this new format for a few years, I understood a lot more about what people needed to know. (It’s true: if you really want to learn how to do something, try teaching other people how to do it.) And having watched a lot of people learn to do it, I was even more convinced of the value of do-it-yourself testing.

  1 With workshops, you can’t procrastinate: you either show up in the morning or you don’t.

  And there’s no homework. At the end of the day, you’re fi nished. Period. The fi rst time I taught a workshop, when everyone had gone home I remember having this very odd feeling that my work was actually done—something I hadn’t felt in all my years of consulting. I highly recommend it.

  [ 4 ]

  call me ishmael

  Finally, last year, in a moment of weakness, I

  September 2009

  gave in and signed a contract (and acquired the

  necessary deadline/gun) to write this book. After

  all, there are only so many people who can afford

  a day-long workshop. I like to think that reading

  this will be a pretty good substitute.

  Does the world really need another

  book about usability testing?

  I didn’t invent any of this. Usability testing has been around for a long time, and a lot of people—Jakob Nielsen being the most vocal and influential—have been advocating “discount usability testing” for at least twenty years.

  And there are several excellent books available that explain in detail how to do a usability test. I strongly suggest that you read at least one of them after you’ve had a chance to start doing some testing. 2

  But this book is a little different, in two important ways:

  It’s not comprehensive. This book assumes that usability is not your life’s work and probably not even part of your official job description. Since it’s not, there’s a limit to how much you really need to know and how much time you can afford to spend learning about it. As with Don’t Make Me Think, I’ve tried to keep it short enough to read on a long plane ride. 3

  The purpose of this book is not to make you a usability professional or a usability testing expert; it’s just to get you to do some testing. Some of you will get really interested in it and go on to learn everything there is to know.

  Chapter 15, Overachievers Only, is meant for you. But you don’t need to learn more than what’s in this book to get enormous value out of testing.

  2 You’ll fi nd a list of my favorites in Chapter 15.

  3 If you actually are going to read it on a plane, you should probably download the demo test video fi le to your laptop before you leave home, so you can watch it when you get to Chapter 2. You’ll fi nd it at www.rocketsurgerymadeeasy.com.

  [ 5 ]

  opening remarks

  It’s not just about finding the usability problems. Unlike the other books about testing, this one is about finding and fixing the problems.

  Chapters 10 through 13 explain how to decide which problems to fix and the best ways to fix them. This hasn’t really been covered in much detail before, and it’s kind of, well…important.

  Call Me Irresponsible

  Some people in the usability profession believe that it’s irresponsible to tell

  “amateurs” that they should do their own testing. These are smart people, and I don’t take their opinions lightly. Their two main arguments seem to be Amateurs will do a bad job and as a result, they’ll (a) make the thing that they’re testing worse instead of better, and (b) convince people that usability testing isn’t valuable.

  Amateurs will do a good job, which will take work away from professionals.

  Before I try to address these concerns, let me make one thing perfectly clear: If you can afford to hire a

  usability professional to do

  your testing for you, 4 do it.

  There’s no question: a good usability professional will be able to do a better job of testing than you will. In addition to having experience designing and facilitating tests, a professional will have seen the same usability problems many times before and will know a lot about how to fix them.

  Besides, it always helps to have a fresh pair of eyes looking at what you’re building. And for the price of the testing, you tend to get an expert review thrown in for free, because the professional will have to use the thing to figure out how to test it.

  4 …and it’s not going to consume your entire usability budget doing only one round of testing...

  [ 6 ]

  call me ishmael

  And then there’s objectivity: being an outsider, a professional may be in a better position to point out unpleasant (and important) truths, like the fact that you’ve created a product that doesn’t work or one that no one needs.

  The problem is, though, that the vast majority of Web sites can’t afford to hire a professional—at least not for more than one round of testing. And even if 5

  they could, there aren’t enough professionals to go around.

  Even more important, I don’t think amateurs will do a bad job. I haven’t seen it happen personally. And for years now I’ve been asking for anecdotal evidence of cases where someone has made something less usable as a result of doing 6

  some usability testing, and I haven’t gotten any to speak of.

  Not that I think it can’t happen, just that I think it rarely does. And in most cases, I suspect it would be the result of someone pretending to do unbiased usability testing while actually manipulating the process to push a personal agenda.

  And I also doubt that testing by amateurs will take work away from professionals. For one thing, it’s not the kind of work professionals really should be doing.

  Jakob Nielsen explained it perfectly in a speech about his vision for the 7

  future of usability at the UPA’s annual conference in 2001. He said that everybody should be doing what he called “simple user testing (debugging a design),” while professionals should be doing things that require more skill and experience, like quantitative tests, comparative tests, and tests of new technologies. Senior professionals, he said, should be doing really sophisticated things like international testing and developing new 5 Best estimate
s seem to be that there are roughly 10,000 people worldwide who would identify themselves as usability professionals, and only a fraction of them do testing for a living, while there are, at last count, umpteen billion Web sites. You do the math.

  6 In fact, I’ve been so impressed by the lack of response that I’ve thought about off ering The Krug Prize: ten million Indonesian Rupiah (10,000,000 RIA, or roughly $1,090.16 US) split among the fi rst ten people who submit reasonable proof of such cases.

  7 The UPA is the Usability Professionals Association ( www.upassoc.org ). If you end

  up deciding to really pursue usability, I highly recommend their annual conference. It’s usually held in June, in someplace that’s ungodly hot. But it’s an excellent conference; the sessions are very practical (not academic), and the people are very friendly.

  [ 7 ]

  opening remarks

  methodologies (i.e., thinking deep thoughts and hobnobbing with their fellow wizards).

  In my experience, people who have been exposed to testing almost always end up convinced that it’s valuable. So I would argue that if more people are doing their own testing (and more people are observing those tests), there will end up being more work for professionals, not less.

  Personally, if I had some money to spend on usability, I’d hire a professional to do an expert review and then do the testing myself. Or I’d hire a professional to do an initial round of testing who was willing to teach me how to do it myself.

  Not present at time of photo

  There are a number of things you won’t find in this book:

  Different testing methods. There are many kinds of usability testing—

  qualitative, quantitative, summative, formative, formal, informal, large sample, small sample, comparative tests, benchmarking tests, and on and on—and they’re all valuable for different purposes.

  I’ll discuss some of these variations at the beginning of the next chapter, but you need to know that this book is only about one particular kind: simple, informal, small-sample, do-it-yourself usability testing (sometimes known as discount usability testing).

  Ways to test instrument panels for nuclear reactors or air traffic control systems, or any systems where people can be injured or lives lost if someone gets confused while using them. The kind of testing this book describes is not for making things foolproof to use; it’s just for making them easier to use. For life-or-death situations, you want exhaustive, carefully designed, quantitative, large-sample, reproducible, scientific studies that produce statistically significant results. Or at least I do.