0321702832.pdf
Rocket Surgery Made Easy
the do-it-yourself guide to finding
and fixing usability problems
Steve Krug
opening remarks
Rocket Surgery Made Easy:
The Do-It-Yourself Guide to Finding and Fixing Usability Problems Steve Krug
New Riders
1249 Eighth Street
Berkeley, CA 94710
510/524-2178 510/524-2221 (fax)
Find us on the Web at: www.newriders.com
To report errors, please send a note to errata@peachpit.com
New Riders is an imprint of Peachpit, a division of Pearson Education Copyright © 2010 by Steve Krug
Editor: Nancy Davis
Production Editor: Lisa Brazieal
Copyeditor: Barbara Flanagan
Design and production: Allison D. Cecil
Illustration: Mark Matcho
Notice of Rights
All rights reserved. No part of this book may be reproduced or transmitted in any form by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. For information on getting permission for reprints and excerpts, contact permissions@
peachpit.com.
Notice of Liability
The information in this book is distributed on an “As Is” basis, without warranty. While every precaution has been taken in the preparation of the book, neither the author nor Peachpit shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the instructions contained in this book or by the computer software and hardware products described in it.
Trademarks
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and Peachpit was aware of a trademark claim, the designations appear as requested by the owner of the trademark. All other product names and services identified throughout this book are used in editorial fashion only and for the benefit of such companies with no intention of infringement of the trademark. No such use, or the use of any trade name, is intended to convey endorsement or other affiliation with this book.
It’s not rocket surgery™ and The least you can do™ are trademarks of Steve Krug.
ISBN-13: 978-0-321-65729-9
ISBN-10:
0-321-65729-2
call me ishmael
To my Aunt Isabel
(Sister Rose Immaculata, O.P.),
who has prayed for me
every day of my life,
My brother, Phil
who worked his whole adult life
as a Legal Services attorney, keeping
families from ending up out on the street,
And all the other people like them
who spend their lives
making sure that things work out
for the rest of us.
[ iii
[
]
iii
opening remarks
opening remarks
Call me Ishmael
2
How this book came to be, some disclaimers,
and a bit of housekeeping
FINDING USABILITY PROBLEMS
chapter 1
You don’t see any elephants around
12
here, do you?
What do-it-yourself usability testing is, why it always
works, and why so little of it gets done
chapter 2
I will now saw my [lovely] assistant in half
20
What a do-it-yourself test looks like
chapter 3
A morning a month, that’s all we ask
22
A plan you can actually follow
chapter 4
What do you test, and when do you test it? 30
Why the hardest part is starting early enough
chapter 5
Recruit loosely and grade on a curve
38
Who to test with and how to find them
chapter 6
Find some things for them to do
50
Picking tasks to test and writing scenarios for them
chapter 7
Some boring checklists
56
And why you should use them even if, like me,
you don’t really like checklists
chapter 8
Mind reading made easy
62
Conducting the test session
chapter 9
Make it a spectator sport
90
Getting everyone to watch and telling them what to look for
[ iv ]
call me ishmael
FIXING USABILITY PROBLEMS
chapter 10
Debriefing 101
102
Comparing notes and deciding what to fix
chapter 11
The least you can do™ 110
Why doing less is often the best way to fix things
chapter 12
The usual suspects
120
Some
problems
you’re
likely to find and how to think
about fixing them
chapter 13
Making sure life actually improves
128
The art of playing nicely with others
THE ROAD AHEAD
chapter 14
Teleportation made easy
134
Remote testing: Fast, cheap, and slightly out of control
chapter 15
Overachievers only
140
Recommended
reading
chapter 16
Happy trails / to you
144
A few final words of encouragement
Sample test script and consent form
146
Acknowledgments
154
Index
158
[ v ]
opening remarks
opening remarks
Call me Ishmael
how this book came to be, some disclaimers,
and a bit of housekeeping
[ 2 ]
I love deadlines. I love the whooshing noise
they make as they go by.
—DOUGLAS ADAMS, AUTHOR OF THE HITCHHIKER’S GUIDE
TO THE GALAXY, WHO WAS NOTORIOUSLY
LATE DELIVERING MANUSCRIPTS
I knew I wanted to write this book nine
September 2000
years ago, right after I finished writing
Don’t Make Me Think.
Without meaning to, in the process of writing
it I had ended up convincing myself of
three things:
Usability testing is one of the best things people can do to improve Web sites (or almost anything they’re creating that people have to interact with).
Since most organizations can’t afford to hire someone to do testing for them on a regular basis, everyone should learn to do it themselves. And…
I could probably write a pretty good book explaining how to do it.
There was just one small problem, though:
I
hate
writing.
Actually, I don’t hate it so much as I find it, well, probably the most accurate word is agonizing.
And not “Should I buy the white iPhone or the black iPhone?” agonizing.
More like red-hot-pokers-in-your-eyes agonizing. I’ve always said that writing is the hardest work I know of and that I can’t understand why anyone would do it unless someone was holding a gun to
their head (which, of course, is what deadlines are all about).
As it turns out, though, it was probably a good thing that I wasn’t motivated to write this book right away, because one of the nicest side effects of the first
[ 3 ]
opening remarks
book was that it gave me the opportunity to teach workshops, which suit my 1
nature much better than writing or consulting.
For the first five years,
September 2001
…and that’s why I
my workshop was a
think consistency
September 2002
combination lecture-
is overrated.
September 2003
September 2004
demo format, where I’d
September 2005
do brief expert reviews of
attendees’ sites to show
them how I thought
about usability problems.
I wanted to teach people
how to do their own testing, but I couldn’t figure out how to fit it into a one-day workshop.
Then three years ago,
September 2006
Why do they
What are
after a lot of pondering, I
need my zip code
you thinking?
September 2007
to send me email?
finally figured out how to
September 2008
do a workshop that would
teach people to do their
own testing—including
some hands-on practice—
in one day. I changed the
format so the whole day
was about the topic of this book: doing your own usability tests.
After teaching this new format for a few years, I understood a lot more about what people needed to know. (It’s true: if you really want to learn how to do something, try teaching other people how to do it.) And having watched a lot of people learn to do it, I was even more convinced of the value of do-it-yourself testing.
1 With workshops, you can’t procrastinate: you either show up in the morning or you don’t.
And there’s no homework. At the end of the day, you’re fi nished. Period. The fi rst time I taught a workshop, when everyone had gone home I remember having this very odd feeling that my work was actually done—something I hadn’t felt in all my years of consulting. I highly recommend it.
[ 4 ]
call me ishmael
Finally, last year, in a moment of weakness, I
September 2009
gave in and signed a contract (and acquired the
necessary deadline/gun) to write this book. After
all, there are only so many people who can afford
a day-long workshop. I like to think that reading
this will be a pretty good substitute.
Does the world really need another
book about usability testing?
I didn’t invent any of this. Usability testing has been around for a long time, and a lot of people—Jakob Nielsen being the most vocal and influential—have been advocating “discount usability testing” for at least twenty years.
And there are several excellent books available that explain in detail how to do a usability test. I strongly suggest that you read at least one of them after you’ve had a chance to start doing some testing. 2
But this book is a little different, in two important ways:
It’s not comprehensive. This book assumes that usability is not your life’s work and probably not even part of your official job description. Since it’s not, there’s a limit to how much you really need to know and how much time you can afford to spend learning about it. As with Don’t Make Me Think, I’ve tried to keep it short enough to read on a long plane ride. 3
The purpose of this book is not to make you a usability professional or a usability testing expert; it’s just to get you to do some testing. Some of you will get really interested in it and go on to learn everything there is to know.
Chapter 15, Overachievers Only, is meant for you. But you don’t need to learn more than what’s in this book to get enormous value out of testing.
2 You’ll fi nd a list of my favorites in Chapter 15.
3 If you actually are going to read it on a plane, you should probably download the demo test video fi le to your laptop before you leave home, so you can watch it when you get to Chapter 2. You’ll fi nd it at www.rocketsurgerymadeeasy.com.
[ 5 ]
opening remarks
It’s not just about finding the usability problems. Unlike the other books about testing, this one is about finding and fixing the problems.
Chapters 10 through 13 explain how to decide which problems to fix and the best ways to fix them. This hasn’t really been covered in much detail before, and it’s kind of, well…important.
Call Me Irresponsible
Some people in the usability profession believe that it’s irresponsible to tell
“amateurs” that they should do their own testing. These are smart people, and I don’t take their opinions lightly. Their two main arguments seem to be Amateurs will do a bad job and as a result, they’ll (a) make the thing that they’re testing worse instead of better, and (b) convince people that usability testing isn’t valuable.
Amateurs will do a good job, which will take work away from professionals.
Before I try to address these concerns, let me make one thing perfectly clear: If you can afford to hire a
usability professional to do
your testing for you, 4 do it.
There’s no question: a good usability professional will be able to do a better job of testing than you will. In addition to having experience designing and facilitating tests, a professional will have seen the same usability problems many times before and will know a lot about how to fix them.
Besides, it always helps to have a fresh pair of eyes looking at what you’re building. And for the price of the testing, you tend to get an expert review thrown in for free, because the professional will have to use the thing to figure out how to test it.
4 …and it’s not going to consume your entire usability budget doing only one round of testing...
[ 6 ]
call me ishmael
And then there’s objectivity: being an outsider, a professional may be in a better position to point out unpleasant (and important) truths, like the fact that you’ve created a product that doesn’t work or one that no one needs.
The problem is, though, that the vast majority of Web sites can’t afford to hire a professional—at least not for more than one round of testing. And even if 5
they could, there aren’t enough professionals to go around.
Even more important, I don’t think amateurs will do a bad job. I haven’t seen it happen personally. And for years now I’ve been asking for anecdotal evidence of cases where someone has made something less usable as a result of doing 6
some usability testing, and I haven’t gotten any to speak of.
Not that I think it can’t happen, just that I think it rarely does. And in most cases, I suspect it would be the result of someone pretending to do unbiased usability testing while actually manipulating the process to push a personal agenda.
And I also doubt that testing by amateurs will take work away from professionals. For one thing, it’s not the kind of work professionals really should be doing.
Jakob Nielsen explained it perfectly in a speech about his vision for the 7
future of usability at the UPA’s annual conference in 2001. He said that everybody should be doing what he called “simple user testing (debugging a design),” while professionals should be doing things that require more skill and experience, like quantitative tests, comparative tests, and tests of new technologies. Senior professionals, he said, should be doing really sophisticated things like international testing and developing new 5 Best estimate
s seem to be that there are roughly 10,000 people worldwide who would identify themselves as usability professionals, and only a fraction of them do testing for a living, while there are, at last count, umpteen billion Web sites. You do the math.
6 In fact, I’ve been so impressed by the lack of response that I’ve thought about off ering The Krug Prize: ten million Indonesian Rupiah (10,000,000 RIA, or roughly $1,090.16 US) split among the fi rst ten people who submit reasonable proof of such cases.
7 The UPA is the Usability Professionals Association ( www.upassoc.org ). If you end
up deciding to really pursue usability, I highly recommend their annual conference. It’s usually held in June, in someplace that’s ungodly hot. But it’s an excellent conference; the sessions are very practical (not academic), and the people are very friendly.
[ 7 ]
opening remarks
methodologies (i.e., thinking deep thoughts and hobnobbing with their fellow wizards).
In my experience, people who have been exposed to testing almost always end up convinced that it’s valuable. So I would argue that if more people are doing their own testing (and more people are observing those tests), there will end up being more work for professionals, not less.
Personally, if I had some money to spend on usability, I’d hire a professional to do an expert review and then do the testing myself. Or I’d hire a professional to do an initial round of testing who was willing to teach me how to do it myself.
Not present at time of photo
There are a number of things you won’t find in this book:
Different testing methods. There are many kinds of usability testing—
qualitative, quantitative, summative, formative, formal, informal, large sample, small sample, comparative tests, benchmarking tests, and on and on—and they’re all valuable for different purposes.
I’ll discuss some of these variations at the beginning of the next chapter, but you need to know that this book is only about one particular kind: simple, informal, small-sample, do-it-yourself usability testing (sometimes known as discount usability testing).
Ways to test instrument panels for nuclear reactors or air traffic control systems, or any systems where people can be injured or lives lost if someone gets confused while using them. The kind of testing this book describes is not for making things foolproof to use; it’s just for making them easier to use. For life-or-death situations, you want exhaustive, carefully designed, quantitative, large-sample, reproducible, scientific studies that produce statistically significant results. Or at least I do.