Choosing a supplier you like and trust can take a lot of time and effort. Especially when a lot of potential suppliers all seem to say the same things. With the wealth of UX agencies in the digital industry, it can be difficult to work out which of them regularly conduct usability tests, and which do them occasionally. We’ve seen a lot of people say they do usability testing, only to find that they did it once, three years ago.
Of course, we’d like you to choose us as your usability testing agency, but we understand that you’ll probably be shopping around before deciding which company to work with. So we put together these questions to help you assess whether you are talking to a high quality experienced usability company or not.
1. Can I see a sample usability report?
It’s important for you to fully understand the output of a usability test. Anyone can point out problems with your website, but our experience and expertise ensures all our reports include actionable recommendations to every usability issue we find.
When you see a sample report from the company you are speaking to, make sure you understand the problem they have found, why it is a problem for users, and what change they recommend to resolve the problem.
2. Can I observe the usability tests?
In our opinion, it is essential for you and members of your team to observe users interacting with your website. We always offer observation as standard and our clients often enjoy the day out to snack on chocolate biscuits and indulge in some innocent people watching.
Watching users interacting with your website is where the real value of usability testing lies, so make sure the company you speak to offer you the ability to watch the tests, ideally in person, or at least remotely.
3. Will I receive videos of each usability test?
It’s useful for you to have a record of the usability tests. All our projects include a full video of each test and we offer a video highlight DVD on request. It’s important that you have evidence that your tests have taken place, particularly if you are unable to observe.
Ask to see a sample video clip from a previous test to make sure you are happy with the quality of their output. This is also a good opportunity to assess the quality of the questions they ask and the professionalism they demonstrate when facilitating the tests.
4. How many users do you recommend? Why?
We see a lot of companies proposing unnecessarily large amounts of users for a test. Of course each project is different, but typically our tests are with smaller amounts of users (5 or 6) to identify core usability problems. Typically we help fix those problems then test again. We wrote an article explaining this in more detail: Only five users?
If the usability testing supplier recommends large numbers of users, make sure their reasons are fully justified and not a way to charge more. There are projects where we will test larger numbers of users, but only when it is absolutely necessary.
5. How will you recruit users?
It’s essential that you are clear on who will be tested and where these people are recruited from. We have our own database of thousands of participants to choose from which we have built up over the 7 years we’ve been in business. However, if you need more specific or harder-to-find people, we use a professional research recruitment partner with recruiters all over the country.
When discussing your requirements for test participants make sure the company you speak to have a credible plan for how to recruit participants. It is important to understand how they would qualify the participants before they arrive, and how they will ensure participants will turn up as planned.
6. How will you plan the usability tests?
It’s essential that your usability test provider works closely with you to understand your needs and to tailor a test design for you. It’s no good simply asking users what they think of your site. We design our test scripts specifically to meet our clients’ needs. We spend time understanding their business objectives, the previous user research they have conducted, and any team disagreements or discussions that have taken place about the site that we can try to shed light upon. We then work hard to design a usability test that will encourage users to fully engage with realistic scenarios when using the website.
Make sure that the company you talk to, has a clear process for designing a test plan tailored to your needs.
7. How long before I receive your report?
No doubt you are working to tight timeframes, so it’s important for you to get a realistic view of how long your project will take. A typical usability test for us will take three weeks from sign-off to report delivery and workshop. Bear in mind that it usually takes at least seven working days to recruit test participants. A professional usability testing company will have a clear understanding of how long the test will take to plan, conduct, and report on.
If your potential supplier seems unclear or vague in their timeframe estimates, you should ask for a breakdown of the key steps involved in the process.
A typical usability testing project should look something like this:
1. Project briefing and test objectives
2. Recruit participants
3. Test plan workshop
4. Test preparation and technical set-up
5. Usability testing
6. Analysis of findings
7. Report of recommendations
8. Presentation and feedback workshop
8. Who will be conducting the user research?
It’s important that the person facilitating your usability tests is competent and experienced. A skilled usability testing professional will know how to ensure users act as naturally as possible, whilst picking up on any hesitations and concerns participants may display in their non-verbal communication. We provide short bios of all our usability testers along with a list of recent projects they have worked on.
Ask specifically who will be conducting the user testing and what previous experience they have. Then check them out on LinkedIn to make sure they really do have the experience they say they do.
9. Where will the research take place?
If you are observing the tests then you’ll be more confident that the tests will take place in an appropriate environment. If not, we recommend asking for photos of the test environment as this can have a big effect on user behaviour. In addition, you’ll want to know where in the country the tests take place. We have professional test facilities throughout the major cities in the UK but tend to do most of our tests in Bournemouth and London. We often find London is the best location to ensure that we can easily recruit participants to meet the criteria required for the test. Make sure that your supplier has a rationale for where they are testing.
When discussing the best location to conduct the tests. Make sure you select a test location this is most likely to have the highest concentration of people that meet the criteria for the test.
We hope that these questions will help you to assess the quality of the usability testing provider you are considering. We’d be happy to answer any questions you have about how to select the best usability testing company. Please get in touch if you’d like to know more about how we plan and conduct our usability tests.
