Usability testing is a cornerstone of the design thinking process. There are, however, common pitfalls that reduce the value of your test sessions. Here, we’ll go through four common issues you’ll encounter when you test your prototypes, as well as how to avoid each of them. At the end, we’ll also give you a cheat sheet of things to say and do during your test sessions. Let’s get started!
Usability testing is not as simple as it might seem. That’s because people—for various reasons—might not give us the honest feedback that we need. For instance, they might not want to hurt your feelings. Or they might think you’re testing them rather than the design, and thus try to avoid “committing” errors.
Here are four things you should take note of, as well as how to solve them to make your test sessions more valuable.
1. People Don’t Want to Hurt Your Feelings
We all like to please others. That’s a great thing—it keeps society in order and makes us friendly even when we interact with strangers. However, in a test session, this basic human instinct might prevent your participants from giving you their honest (and harsh) feedback. After all, they know you must have spent a lot of time to create the design. What if their critique of a feature ends up hurting your feelings?
You might think this is a trivial pitfall, but you’d be mistaken! Time and time again, designers have found that participants give overwhelmingly positive feedback unless they’ve been correctly prepped before the test session.
Solution: Brief Your Participant for the Test Session
You should brief your participant so that they understand that they can—and should—provide their honest reactions and feedback.
One simple way is to deflect any perceptions about your personal involvement. You might de-emphasize how close you are to the work, or underscore your distance from it if you are from another team or company altogether. Some flexibility with the truth might be warranted, but be careful about barefaced lies denying how involved you are — seeking honesty from a test participant by being deliberately deceitful can backfire. The intention is to provide room and permission for them to be honest without feeling they need to approve or avoid negative comments that might hurt your feelings.
Another method is to tell your participant that you genuinely appreciate negative feedback because it allows you to improve the design. This way, you reframe negative feedback as learning opportunities and make it less likely that people will avoid mentioning anything that’s negative.
2. People Often Assume that You are Testing Them
Most test participants see a usability test and think “Oh, they’re testing to see whether I’m competent enough to complete the task!”. As designers, we might forget that most of our participants are not designers and thus don’t know what a usability test is about.
When a participant thinks you’re testing them (rather than the design), they tend to behave differently. They might be overly careful when they do tasks because they don’t want to appear stupid. This means that your results won’t be accurate, because the average user will likely be less alert when they use your product!
Participants also tend to give different feedback if they assume that you’re testing their abilities. For example, rather than say something like “This sentence is confusing and I’m not sure what to do next,” they might say, “I’m sorry that I blundered at this point; I wasn’t paying enough attention!” This makes it difficult to spot usability problems, as you’ll then need to read between the lines to find out what exactly went wrong.
Solution: Let Them Know You’re Testing the Design, Not Them
Start your test session by explicitly stating “We are not here to test you. We are here to test the design. We know this design might have some issues, and we need your help to identify those issues.”
This way, you clearly communicate that you’re not testing the participant. You’ll also help participants get into the mindset of testing the design. On top of that, when you let your participants know you’re asking for their help, you make it more likely for them to be forthcoming with their thoughts. They become exam invigilators rather than the students taking the exam.
Make it clear that a usability test is not a test of the participant! Let participants know that you are testing your design.
3. People Often Focus on Completion Rather Than the Journey
We naturally love to complete tasks. It fulfills us and gives us a sense of satisfaction. In the context of a usability test, this means participants tend to focus on completing their tasks instead of the route they took to get there.
One implication is that participants might adopt a “complete at all costs” mindset and carry out their tasks unnaturally. For example, a participant might be so focused on completing a task that they simply click on every link to check if it does the job. This means they’ll complete the task quickly, but since an average user is not likely to click on every link to see what it does, it means that your test results are not going to be accurate.
Another problem with this is that when people feel good after they’ve completed a task, they tend to provide nothing but positive feedback. Essentially, participants might forget all the bumps in the road—bumps which you want to smooth out—and tell you the journey was perfect because they’ve achieved their goal.
Solution: Focus on the Journey, Too
This sounds philosophical, but we’re actually talking about very practical matters here! You’ll need to put in some effort to counter our very human tendency to focus on end results.
One way to focus on the journey is to ask participants to think out loud. This means they should vocalize every single thought that comes to them while they complete your tasks. This is unnatural, and your participants might need to practice before they get good at it. Tell your participants something like this: “May I request that you think out loud? This means that you should vocalize everything that you’re thinking. For example, when you click a button, you can say something like ‘I want to click this button because I think that’s how I purchase this item.’”
Remind your participants to think out loud from time to time. If you realize they’ve stopped speaking, gently prod them with a “What are you thinking right now?” or “Can you please tell me what you’re thinking?”.
When you conduct a test, note down strange behaviors. For instance, if a participant clicks on a button and rapidly presses the “back” button, write it down. Later, at the end of the task, you can walk them back through the process to ask them why they behaved that way. Refer to your observations and bring up any issues that you saw them encounter on the way through.
4. People Can Show You More, If You Let Them
You’re the expert—and that’s why you’re conducting the tests. But this assumption of your own expertise can lead you to presume that you’ve picked up all the pain points through observation. This makes it less likely that you’ll ask your participants to show you more.
People have so much more to offer! If you end the usability test after the participant completes all their tasks, you are missing out on a huge opportunity to gather feedback and ideas.
Solution: End with a Post-Test Interview
At the end of each test, set aside some time to conduct a short interview. Ask your participant questions such as:
“On a scale of 1 to 5, how easy did you find it to complete the last task: 1 being most difficult and 5 being very easy?”
“Do you have any feedback or ideas on the product? They can be totally unrelated to our test session.”
Your participants are not designers, so it’s likely that their design idea won’t make it into the final product. However, that’s not really the point. When you ask people such questions, you help yourself understand what they think about the product, as well as areas that they’re concerned about. Sometimes, you might find their responses interesting or unexpected!
Ask your participants for additional feedback at the end of each of your test sessions.
Putting It All Together: How to Get More Honest Feedback in Usability Tests
Below, we’ve arranged all our solutions so that you know what to do before, during and after a usability test.
Before a Session
Let participants feel comfortable giving negative feedback. Some things you can say include:
“Hi! I’m here today because I’m hired by [company] to test their design. Since I’m not related to [company] at all, you don’t have to worry about hurting my feelings! Feel free to give feedback, good or bad, so that we can learn and improve the design.”
“I’m here today to test a product that my colleague has created. Don’t worry, I won’t feel hurt if you say something negative about the design. In fact, I love hearing negative feedback because it lets me improve my colleague’s design!”
Let them know that you’re here to test the design and not them. You can say:
“Today, we’re not here to test you. We’re here to test the design. We know this design might have some issues, and we need your help to identify those issues.”
Politely request that they think out loud:
“May I request that you think out loud? This means that you should vocalize everything that you’re thinking. For example, when you click a button, you can say something like ‘I want to click this button because I think that’s how I purchase this item.’ Or, when you’re looking for something on a page, say something like ‘I’m trying to find the shoe category in this website.’”
During the Session
Remind the participant to think out loud if they fall silent. You can say:
“What are you thinking right now?”
“Can you explain what you’re doing right now?”
“Could you please think out loud, so I can follow what you’re doing?”
Note down any strange behaviors or questions that you’ve formed in your head while observing your participant.
Ask them either at the end of the task or at the end of the entire test session.
After the Session
If you’ve written down notes, ask them about these at the end of the test.
Ask them for more feedback:
“Do you have any feedback or ideas on the product? They can be totally unrelated to our test session.”
Download our template on how to get more honest feedback:
The Take Away
Usability tests can give you key insights that let you improve your design. However, it takes a conscious effort to extract better feedback out of participants during a test session. There are four common pitfalls you should take note of, and their solutions:
People don’t want to hurt your feelings.
Solution: Brief your participants before the test.
People often assume that you’re testing them.
Solution: Let them know you’re testing the design, not them.
People often focus on completion rather than the journey.
Solution: Focus on the journey, too.
People can show you more if you let them.
Solution: End with a post-test interview.
References and Where to Learn More
If you want to learn more about usability testing, check out our course on Conducting Usability Testing.
