So, how did I prepare?
• Created one scenario — very common scenario performed by internal users of the software on a regular basis.
• Small 4 tasks/steps to achieve the scenario and test usability and interaction.
• Printed each task on separate paper to be handed to the user on the test day — for example, A. Please go into a privacy setting and toggle into public view.
• Participants were gathered internally from the client service team who use the platform on a daily basis and handle customer queries — total of 5 recruited.
How did I conduct the test on the day?
• Setup the platform ready for participants, and found a quiet place to sit in with minimal disruption in an open office floor plan — this was hard to do in the afternoon, mornings were the best time.
• Before they started I briefed each individual about how the test will run — once happy the printed out tasks were given to them to go through at their own pace.
• Participants were given my laptop to complete the test.
• Observed individuals complete tasks identifying problem areas.
• Successful participants — If a participant completed a task without aid from me (the instructor), I would mark that task down as successful.
• Struggling participants —As a last resort I only hinted or aided if they asked for help or if they had been lingering on that particular task for some time, for example, “Try this area here”, “It is similar to how you would do [another task]”. This was particularly the case if the current task was linked to the next task. Although, I preferred to sit back and see how the participants handled tasks before stepping in to help; this opened up the floor to see what alternative ways/hacks users used to solve their problem, seeing first hand how they “shouldn’t” be using the platform vs how they “should”.
• Failing participants — If a participant cannot complete a task without aid from me (the instructor), I would mark that task down as failed.
• Record user actions, frustrations and comments while they perform the tasks — turned out to be harder than first thought, you have to write fast and keep up with your users. Here I found making notes in short-form very efficient and handy.