As a designer, when collaborating with the researchers, I am the one to communicate about my hypothesis, discuss the testing flows, and build the prototypes, etc. But I always wanted to be on the other side, I wanted to be that user researcher and conduct user testing on my own. Then I started to take the user research responsibilities. And I came up a guideline for user testing, and used it for 3 projects for now, all worked really well.
Here are the 5 steps guideline:
- Step 1: Ask Designer to Fill Out Questionnaire
- Step 2: Researcher to Draft Testing Plan
- Step 3: Researcher to Setup Testing
- Step 4: Researcher to Write Reports and Share Results
- Step 5: Help Designers to Refine the design
For smaller companies/organizations who doesn’t have dedicated user researcher, sometimes people had to take the both roles as the designer and the researcher. The key is to be objective and not get biased. Following is the detail explanation of how to accomplish each steps.
Step 1: Designer to Fill Out Questionnaire
This questionnaire is for researchers to understand about the design. It’s very important to fill out the hypothesis, it helps researchers to focus on where to probe during the user testing. For example: “My hypothesis is that for the new design, user can easily follow the flow to create an account. However, user may have difficulty to find where to create an account.”
Step 2: Researcher to Draft Testing Plan
First, listed out a few commonly used research methods that’s likely to fit into my company’s needs. “Hallway testing” is the most casual one, you go to a coworker and ask for their fresh eyes opinion. “Card sorting” is commonly used to figure out hierarchy. “Participatory Design” is good for ideations. There are other research methods available, check this book “Universal Methods of Design”, written by Bruce Hanington, my professor at CMU.
Based on the questionnaire, select the right testing method and the right user pool for your testing. Since our users are distributed globally, so I use moderated virtual testing. Also, we require the user testing participant to have some prior knowledge of the applications.
When drafting the testing plan, try to be task oriented! Give one or a few tasks for user to do. When the user is completing the task, you can observe how they are doing, any difficulty they encountered. For example, for ecommerce, give user a task to buy $30 kitchen paper from the site, then observe how user discover the kitchen paper, how do they add to cart, how do they proceed to checkout. Then ask the designer to create the prototype that covers all the pages and flows needed to complete the tasks.
Have a thoroughly written script helps tremendously on moderating the user testing, I found it very useful especially I am not a native English speaker. When writing the user testing scripts, again, try to be task oriented. Make sure the order of the tasks & questions are lined up. Also include a basic introduction what the user testing session looks like, so the participants know what to expect.
Step 3: Researcher to Setup Testing
You may need to recruit participants on your own. Just make sure the user testing participants matches with your selecting criteria. When schedule the user testing sessions with the users, make sure they know how to operate the virtual meeting, sit in a quiet environment, have a stable internet connection. Also if you plan to invite observers to the virtual meeting, make sure they are on mute during the whole session.
For the user testing session:
Schedule 30min-1hour each session
Do screen recording (I like Zoom!)
Bring printout scripts+ pen
Participants info charts(name, age, occupation, etc)
Bring NDA if needed
When moderating the testing, always ask the participant to think out loud. Make sure you are not asking leading questions. Do not direct users to answer whatever you want to hear, instead, let them speak up for themselves. For discoverability, if you want to know if a user has seen that button, you shouldn’t point it out to the user. Instead, observe how they completing the tasks and see if they notice the button by themselves. You may ask “Have you seen that button?” at the end of the testing after user finished all the tasks.
I found some questions work really well for any testing:
“What do you expect to see before you click on that?”
“What do you think that is?”
“If you can only keep 3 things on this page, what would you keep?”
At the end of the user testing, ask your observers if there are any extra questions need to cover. Also, always thank your participants sincerely for their time to participant in the user testing.
Step 4: Researcher to Write Reports and Share Results
You can get help from observers and ask them to take notes. You can ask observers to write down quotes or ideas on post-its. Have an organized way to take notes is always nice, I like using charts to take notes and collect data when I look back at the video recording.
In the following example, I divided the charts into tasks. We had 6 participants, “P1” means participant 1, “2/6” means 2 out of 6 participants, familiarity score ranges from 1to 5, 5 means very familiar. “2:30” means this moment happened at the 2minutes and 30 seconds in the video recording.
Note down the moments in video clips and play back in the user testing report presentation, this works really! Write down some nice quotes! For example:
“It’s powerful. I like that you can search and defined the filters and then perform actions against one or many of the results.”
For writing user testing reports, I created this template:
Step 5: Designers to Refine the design
Help designers to iterate the design based on the user testing feedback. Researcher should be able to further explain the testing feedback if designers has any questions. Beyond that, user testing report is also a great way to showcase value to the product manager, engineers and stakeholders.