Testing
UX Prototyping
24 Mar 2026
Week TEN
Definition of usability testing
definition
usability testing: the process of learning about users from users by observing them using a product to accomplish some specific goals of interest to them, from Barnum (2010), page 6
definition
usability: the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use, from ISO 9241-11, quoted in Barnum (2010), page 11
Formative and Summative Testing
formative
- exploratory
- early in process
- used on lofi prototypes
- more communication between moderator and participant
- focus on user perceptions rather than task completion
- more likely to think aloud
summative
- more formal
- evaluating design choices
- used on hifi prototypes
- less likely to think aloud
- metrics
- quantitative measurements
Testing Low Fidelity Prototypes
We’ve discussed lofi prototypes
Recall Buxton’s characteristics
- explore
- suggest
- question
- propose
- tentative
Lofi purpose drives lofi testing
- to initiate a conversation
- to help people envision one alternative
- to stimulate ideas
How to initiate a conversation?
- show paper pictures of artifact
- ask participant where they might “click”
- offer participant a pointer to “click” with, e.g., a pen
- provide a different piece of paper as the result of the “click”
Encourage think-aloud
- ask the participant to reveal thoughts
- ask what do you want to do next
- ask what is holding you up
Watch the response
- Watch for body language
- Observe pauses—they may indicate confusion
- Indecision about where to point may highlight ambiguity
Don’t be afraid of pauses
- but don’t get sidetracked
- maintain focus on the test
- but let the test “breathe”
- pauses may be a natural part of the process
Put the participant at ease
- remind the participant that they are not being tested
- thank the participant
- assure the participant that what they are doing is valuable
- make sure they don’t feel you are too invested in the prototype
Aids
- Scripts, forms, advice, and checklists from the book Rocket Surgery Made Easy
- Available at http://sensible.com/downloads-rsme.html
- Also watch Steve Krug at https://youtu.be/VTW1yYUqBm8
Amateur and Professional Testing
how serious are you?
Here are two books: one is for amateur usability testers and one is for professional usability testers. (You can still be a UX professional and be an amateur usability tester. They catch many problems.)
How many problems can you catch? As many as you can fix, plus any that don’t matter
first, the amateur’s book
Most professionals of my acquaintance have read the amateur’s book, by the way. Maybe they want to be sure that they are not missing anything an amateur would catch.
The book is called Rocket surgery made easy, the do-it-yourself guide to finding and fixing usability problems by Steve Krug, 2010.
Steve Krug
Krug is best known for the book Don’t Make Me Think, Krug (2005). I’ve always resisted reading that book because the title gives a questionable command.
I was taught the goodness of thinking as an axiom.
Steve Krug’s maxims
- Start earlier than you think makes sense
- A morning a month, that’s all we ask
- Recruit loosely and grade on a curve
- Make it a spectator sport
- Focus ruthlessly on a small number of the most important problems
- When fixing problems, always do the least you can do
A morning a month, that’s all we ask.
- three users, debrief over lunch;
- morning means half a day;
- monthly sets expectations;
- keeps it simple;
- tester spends a couple of days prepping
Start earlier than you think makes sense.
Even test a similar product before you’ve done any design work at all!
Recruit loosely and grade on a curve.
It’s more important to test frequently than to get the right participants or more participants.
Make it a spectator sport.
Seeing is believing. Observing makes you realize how different users are from you. More observers are better.
Focus ruthlessly on a small number of the most important problems.
How many experience the problem and how severe is it for those who do?
When fixing problems, always do the least you can do.
Do something and don’t try to do everything. Tweak, don’t redesign. Take something away.
Krug’s common problems
- Getting off on the wrong foot
- Failure to shout
task-level measurements
Make a list of tasks!
Make each task into a scenario.
The scenario adds context, e.g., you are …, you need …, and supplies information, e.g., username and password, but doesn’t give clues!
Pilot test the scenarios.
〈 pause to watch a Steve Krug usability test 〉
the professional’s book
Usability testing essentials: ready, set… test! by Carol Barnum, 2010.
After you digest the amateur’s book, it’s time to tackle this one, especially chapters 5 through 7, describing planning, preparing, and conducting a test.