Many people, who are new to usability testing, struggle with the idea of only testing with five, ten, or even fifteen people. It doesn’t seem like there’s enough people to be able to make some decisions on.
But this is where usability testing and user research is different to regular research. And the key part of this is that we’re not overly focused on what people think or what their opinions are. The priority of the work that we do is observational. We want to see people engaging with user interfaces, particularly websites or apps. As we watch them using the website, we’ll be talking with them, we’ll be listening to what they’re saying. Participants’ views are still important.
But often what we see is that what people say or what people think they think they want is often quite different to what they actually do when they use the website, when they start learning from the process that they’re going through. So their natural user behaviour is really important and the observation piece is really important.
How many people do you need to see struggling?
Well, when we’re looking at observational research, we’re looking at where people are maybe hesitating, getting frustrated, getting stuck, going back to Google because they’re fed up and they want to try somewhere else. This is about user behaviour and user journeys – looking to see where users struggle on the website.
Now, once we’ve seen three, four people struggle at the same point in a process on the same website, we know there’s some kind of issue with the user interface. A UX Researcher does not need to sit through tens or hundreds of people struggling at the same point to know there’s an issue there.
The double door conundrum
I often liken it to the double doors where the ergonomics of the handles aren’t overly helpful as I approach them – which of the two doors do I aim for, and do I push or pull?
If you were to sit and observe, you’ll see people turning up and grab one door to pull it, then they have push it but nothing happens. So they have to grab the other one and they pull it. Nothing. A final push on door two and eventually it opens.
You don’t need to see hundreds of people do that same thing to know there’s a problem with the design and signage of the doors. Once you’ve seen a few people struggle with it, you know there’s an issue.
The website conundrum
It’s the same with a website, when we see people struggling to interpret what a button might mean, or to actually find the button in the first place, or they’re not quite sure what this website is about. Once we’ve seen a few people come into the website and have the same struggles and same challenges, we know there’s an issue there.
And of course, across the test, across the 10 people we might be testing, we’ll notice whether that particular point is a problem that most of our participants are struggling with, or maybe just a few. And that will influence the prioritization of our findings that we deliver back at the end of the test.
How many participants is enough?
Usability testing and user research is highly qualitative, and that’s because it’s observational. It’s a better use of investment to not do too many people, enough to see that there’s an issue for multiple people, but too many and we’re just witnessing repeated behaviour.
If you have a generic user base, 8 to 10 participants is a great starting point. If, however, your audience is more specific, then five of each audience type will provide plenty of insight and direction. If you are testing regularly and iteratively, can afford less per test, to make quick decisions and be ready for the next test.
Clients will often have two or three groups of customers to focus on, so getting a balance is important. In most cases, even if the customers are categorised differently, they’ll often share many tasks and needs when using a website, so there is quite an overlap. We’ll often help to prioritise the user groups and recruit six each of the higher priority groups, and three each of the lesser priority groups, for example.
This questions is asked time and again
This is something we’ve been doing for a long time but this question still arises often with new clients. But when people get engaged and they watch usability testing, they see this and they realise what value it brings them.
Recently a client had to convince his COO (and organisation) to agree to sign off the budget for a usability test. After presenting the final report of findings and recommendations, the COO thanked our client for being persistent and pushing for the budget to get the project signed off. Otherwise he’d never have seen this level of insight.
The final number of participants can also be budget related, or restricted due to how difficult they might be to reach. So if you are embarking on a usability test for the first time, aim for a handful of participants. If any of your colleagues doubt the number, invite them to watch, and see the penny drop.
You might also be intersted in reading:
