Just as I was starting up as a freelance consultant, a colleague commented, “A guidance on writing briefs or request for proposals for user study recruitment would be so nice …” and voilà! here are my thoughts on working with agencies for recruiting study participants for user research, usability testing and co-create sessions.
Start early so as to allow at least six weeks for recruitment, especially when working with a specialized or very specific user group. So starting around ten weeks before the actual study would give you enough time to review and update proposals, have clarifying calls, and sign contracts with the recruitment agency, etc. and still have enough time left for the actual recruitment :)! A fellow human factors consultant Greg Thay commented that they are able to bring down the time to a few weeks sometimes
Selecting a recruiter
Write a Request for Proposals or Quotes
The first step is to create an outline what do you need from the recruiter and when do you need it, a.k.a. Request for Proposals. For example, it could include:
- Details of the user test or study, e.g., Briefly describe what is the study about and why are you conducting it, when are you conducting it and where will it take place and how long will (each) session be?
- Detailed user profile: Who are the people they should recruit and for what incentive. In general think about things like Gender, Age, Role, Professional qualifications, Experience with specific technologies, star wars fans? Also, think about who shouldn’t be included, i.e., Exclusion Criteria, e.g., should not have participated in a user test for this system/app/product in the last X months… Here is a nice article on some guidelines for writing user profiles, including accessibility considerations.
Be very specific about user profiles, assume that anything that is not explicit will not be considered.
- What do you need from the recruiter, e.g., Do you want them to just recruit people, handle user payments/incentives for taking part in the test, should they arrange a testing location and/or technology for you, do you need people from their team to receive participants or to facilitate the study session, etc.
Contact the recruiter(s) you want quotes from
In case you already have a database of suppliers, great! In case not, some criteria I use to select recruiters are:
- Are they specialized in the domain of your product/service/users?
- If the test is in multiple countries, do you want to use one recruiter or multiple recruiters? In my experience, multiple recruiters add an overhead of communication and administration from your side as you need to talk to and align with multiple people then. On the other hand, sometimes using local recruiters is much more cost effective, so it depends on your priority.
- What services are they able to provide?
- Ask your colleagues or peers about recruiters they use as well, I have had good experiences with the recruiters that were recommended to me.
Look for partners instead of ‘suppliers’.
Review and select the quote you like
- While reviewing the quotes, you might need to have a call or email exchange with the recruiter to discuss updates.
- Once you have compared the quotes, let the selected recruiter know and also the others (it is polite, I remind myself :)).
- I believe in creating partnerships and not just one-off engagements, and would advise to communicate and select recruiters keeping this in mind.
Engaging with the selected recruiter
The kick-off call
Now that you have selected a recruiter, schedule a call to go through the request with them in detail, answer any questions they might have and also update the profile with any specifics based on their questions. Also, agree on how often you will have updates and what is your way of working and what do they need from you and by when.
User Profile vs. Screener
- This was a learning for me, as I used to think that a user profile is the same as a screener but I learned that recruiters convert the user profile into a questionnaire which they use in order to filter or screen the study participants according to your criteria.
- As an example,
User profile: “Should have some experience with roller skating” .
Screener: “Do you roller skate?” If YES, continue to Ques. #X. If NO, terminate.
Here, unless you define what ‘some experience’ means, you might get either only very experienced skaters and someone who owns skates but has never actually learned. Which could be fine, but being specific will help you cover the types of roller skaters 😉 or study participants you want and not leave it up to chance or interpretation.
- You can create this questionnaire yourself or ask the recruiters to create it (usually for a small fee). In any case, review the screener to make sure it captures your criteria completely.
Details to send to the recruiter
After the kickoff call, I would often send one or more of the following details to the recruiter:
- An information letter about the study for the participants to help with the recruitment
- NDAs and Consent forms for review by the users before the study
- Address of the user test location — directions and map of the location
- Who will receive/meet the users — contact details and phone numbers
- How and when will the users be paid?
- Who will pay them?
Once recruitment is done
During the user test
- Update the recruiter after every day of testing if all users showed up.
- Also, if anyone was not suitable for the test and if additional exclusion criteria need to be added for remaining sessions.
Debriefing after the user test
- Have a short meeting to discuss your overall experience and also the recruiter’s experience.
- In general, things can go wrong despite everyone’s best intentions, in such cases work with the recruiter to help solve the problem instead of finding the party to blame.
Good recruiters will take responsibility where due and also challenge you where needed :).
- If it went well, you might have found a good partner to collaborate with and came away with learning points for the next time.
What have been your experiences with the recruitment of study participants?