You’re going to need to conduct usability studies at some point during your design work. That may be for a product or for an information visualization but if you don’t know whether your outputs are usable – you cannot begin to guarantee a user experience. Usability testing is so important that even presidents get involved (below you’ll see Barack Obama testing the federal government’s IT dashboard).
© Pete Souza, Public Domain.
This article is concerned with recruitment of users only; usability methods (often observations, interviews or focus groups) are down to the needs of the researchers. However, for each type of usability study that involves users – you will need to determine an accurate sample size to be recruited. There are links in the references section of this article that can help you determine the right sample size for your project.
There are 5 main methods for recruiting users to your usability studies. Each of these has its advantages and each has its drawbacks.
General Notes on Recruiting Usability Study Participants
Before you begin recruitment you should:
Know what it is you want to study in terms of usability and why
Have determined the methodology you will use to study this
Be able to describe to potential participants the objectives of the study and how the study will be conducted
Be able to explain to users how to carry out their part in the study
Be able to explain to users how their data and any data generated in the study will (and will not) be used
Know how many users you need to recruit to give a representative sample
Know what type of users will best represent your user base (user personas)
Know what (if any) reward you will offer to participants
The 5 Methods for Recruiting Users to Usability Studies
Once you have the information above; you can begin to recruit participants. Let’s look at each method now:
Hallway or Guerrilla Testing
This is the simplest, theoretically, method of recruiting users. It consists of asking anyone passing by to participate. These could be folks in the office, friends, people in your local café, anyone. If you’re going to recruit like this your studies should be short (ideally less than 15 minutes) and it’s nice to reward participants with a coffee or small token for taking part. Don’t bully people into participating though – they’re not likely to be very helpful if you do.
The downsides of this method are that the users you recruit are unlikely to perfectly represent your user base and there’s only so much of a sample you can take. If you need a very large sample – this kind of recruitment almost certainly won’t cut it.
It’s best used when you want to find out a little bit about the most obvious issues in a design before developing that design further. It’s also important to remember that if participants aren’t your typical users – their feedback may not be 100% helpful when compared to the feedback of a more representative group.
© Pexels, Free to Use
Existing Users
For those carrying out usability testing on existing products or products that extend an existing portfolio – one of the easiest sources of users is the existing user base.
You can’t assume that existing users will take part. You need to ask them to do so. You can do this by any (or all) of the following:
E-mail requests
Pop ups (or pop unders) on your website
Requests in your social media groups
Asking sales people to reach out to certain customers
Asking customer services to ask customers at the end of a call
This method often provides very solid and useful feedback on usability. However, in many cases it’s not possible to get existing users to give up large chunks of their time (even if incentivized to do so) and for more complex studies you may need to examine additional recruiting methods.
Online Service Recruitment
There are three main online services (and many more that you can find using Google) that you can use to find participants for user studies:
Craigslist
Usertesting.com
Mechanical Turk (Amazon.com’s crowd sourced network)
Craigslist allows you to post specific adverts to solicit participants in research. You will want to be very clear about the purposes of your research in your advert and highlight any requirements that you have of participants (both in terms of demographics and where they will be expected to take part and what they will do) to get the best results from this. Expect to spend between $25 and $300 per participant (yes, you’ll need to pay participants) depending on the length of research, travel required, etc.
Some of the drawbacks of Craigslist are that you’re unlikely to find specialists and high income individuals in large amounts there. Most Craigslist users are looking to make a few bucks on the side.
© CoolKid1993, Public Domain.
Usertesting.com is geared to remote testing with both video and audio inputs. It has a large panel of users and you can pay to recruit from specific demographics or for users to answer a short survey to match specific criteria you require. It’s a good place to get quick results on short usability tests. It’s not good for long involved testing or lab testing recruitment. Expect to pay about $50 per user for a 20 minute test.
Mechanical Turk is Amazon’s crowdsourcing platform. You can recruit users for almost any price you want to offer. The downside is that these users tend to be very much low level users and you can’t expect much in the way of deep insight. It’s best used for quick testing. It’s also worth noting that some users make use of bots to complete tests – be on the lookout for them.
Panel Agencies
Panel agencies maintain databases of thousands (or more) users willing to take part in Unmoderated user testing. They have a huge amount of demographic information and allow you to target your testing to very specific audiences. You’ll want to Google for an agency that best suits your needs. It’s not particularly expensive to use these services – expect between $20 and $50 per response. However, the downside is that not every requested participant will respond and it may still be a struggle to get enough users if you require a very specialist audience.
Market Research Recruitment
The best, but most expensive, way to recruit for user research is to use a market research recruitment company to do it for you. You can find these via Google and the recruiter can help you draw up a good recruitment profile and then find people to take part. It’s important to note that it’s likely to cost $250 (ish) for each person they recruit and you will then need to compensate participants to the tune of $100-$400 per response. Because of the expense involved – you’re going to want to be very careful about the research you conduct like this; it needs to be deep, involved research with very clear objectives that are likely to result in big business benefits.
© On Device Research, Fair Use.
The Take Away
Recruiting participants for usability studies is relatively easy. However, it can be both expensive and time consuming. While guerilla/hallway studies allow for rapid recruitment – their results may not be highly representative of your actual user base. The other forms of recruitment can take weeks or months to get right and can start to cost substantial amounts of money; it’s best to be very clear about what you want from a study and what the benefits are before investing the time and money into these activities.
References & Where to Learn More:
Course: Conducting Usability Testing
How to determine population and sample size.
A handy calculator for determining sample sizes.
Hero Image: © Dave, CC BY-NC-ND 2.0
