User interviews are one of the most common and accessible forms of user research methods. Interviews, when done right, reveal fascinating insights about people’s behavior, their motivations, fears, needs and desires. However, it takes a lot of careful planning and execution to ensure the information we get are consistent and remain ‘pure’. There are many things that can go wrong while doing user interviews. Here are some of the things that can be of help to avoid the errors that can otherwise corrupt or compromise the research data.
1. Talk to the right people
The kind of data we get depends on the kinds of people we interview. A business class flyer might offer a perspective that’s very different from an economy flyer. So if we’re designing for the economy experience, it would be a bad idea to interview a person who flies business class. Know who your users are and interview only those people that you want to solve the problem for.
2. Involve stakeholders and the product team
As much as possible, try to include members from the business and the product team when doing interviews and have them observe or better, take notes. Building products is a team effort and everyone involved in the process should be aware of who their users are, their pain-points, needs and motivations to make sure the team is solving the right problem. Also, having a diverse participation provides an opportunity to understand the problem from different perspectives.
3. Well communicated schedule and logistics
If you’re not the one scheduling the interviews, make sure you communicate the schedule (when and where the interviews should happen), duration of the interviews, the user group and the number of participants needed for the interviews with the person in charge of planning very clearly before the study. Also, conducting user interviews can be tiring. Be sure to provide sufficient buffer between interviews so that you don’t end up getting drained by the third or forth successive interview which compromise the questions you ask or the note taker’s efficiency/enthusiasm to take notes.
4. Number of interviews
When it comes to user interviews, more isn’t always better. A good rule of thumb is — talking to 5–7 people will uncover 80% of the information. Anything more than that, we start hear repeated information which doesn’t add a lot of value.
5. Don’t prepare the script & question alone
Work with the partners when building the script and questions to be asked. Especially when we, as a researchers or designers, aren’t the subject matter expert in the product space you’re interviewing at. Working with partners helps us understand what area of the problem to focus on, knowing what answers to possibly expect to hear and get a general context of what users might say. This planning should happen after the kick-off meetings where the assumptions, goals and hypothesis are generated.
6. Capturing consistent data
One of the goals of the interview is to ensure the data captured remains consistent so that when synthesizing we can see patterns and trends emerge. This can be done by making sure we ask all the questions to every participant. Avoid introducing new questions half way through a study, even if it seems interesting. Having a certain insight from a single person when the other participants did not get a chance to share their thoughts on the topic can create ambiguities during synthesis.
7. Be curious
Even if you’re very certain about the answer to a question, ask the question anyway — It’s not about what we know, it’s about trying to understand what the participant has to say about the question. We need to fight against our biases, assumptions and instincts and remain curious. This comes by practice. Also, don’t assume participants wouldn’t know the answer to a question and skip a question. Ask anyway and see what they have to say about it. If you start hearing useless information, you can move on to the next question. (Unless you’re constrained by time).
8. Be a good listener
Listening carefully helps us pick up what users are saying and dig deeper to learn more about interesting behavior and insights. What people usually say isn’t always what they actually do. It takes a bit of probing to know the whys behind what they’re saying. The better we listen, the better data we can gather. Attentive listening is also important because people take their time out to talk to us and it’s just considerate to give them our full attention and make them feel they’re being heard.
9. Ask more ‘Why’s
People are not always great at explaining the whys behind what they do and it’s our job to help them articulate their motivations or reasons behind their goals, actions. Listen carefully and whenever necessary, ask the participants to explain why they say/think something the way they did. It helps us avoid making assumptions about users’ behaviors.
10. Avoid leading questions
Avoid asking questions that make the participant tell you what you want to hear. For example, instead of asking “Do you agree at using feature X to view the operational efficiency data is a pain?”, which almost always gets a ‘yes’ as an answer, it gives a better understanding of their thinking when we break it down and ask “Can you tell me about a time you used feature X? What did you use it for? How did you feel? Why did you feel that way?”. We must always be aware of what questions we ask and how we influence the users’ responses. Biased information can be very misleading when synthesizing the research data and this can result in incorrect product decisions. Better questions yield better insights.
11. Repeat questions tactfully
It’s ok to repeat certain critical questions in a couple of different ways even if some of the participants might feel a bit annoyed by it. Knowing if they stay consistent with what they say helps reinforce the data we gather. Or perhaps asking questions in different ways helps people think differently about a problem and gives us more insights. In my experience, I have rarely seen people getting annoyed and they are always happy to answer the questions even if they have to repeat what they previously said.
12. Diligent note-taking
Create a note-taking template that provides enough space for the note-takers to take notes for each question. Ask your note takers to take as much notes as possible. It’s also a good practice to record all interviews so that you can revisit them later in case there is any doubt or ambiguity while synthesizing. ALWAYS make sure you take the users’ permission before recording them.
Pro tip: Taking notes on stickies during the interview makes it easier for synthesis.
13. Test your script before the study
Once the script and questions are prepared, have them reviewed by other researchers, designers and business partners in the team to make sure there are no leading questions and the questions are designed to validate or invalidate the assumptions and hypothesis.
14. Test all tech before the study
Be present in the room at least 30 minutes before a study to make sure all the necessary tech (audio/video recorders, screen sharing/recording software, phone lines etc) are all working. I once used Quicktime screen capture to record a set of remote usability studies. After all the studies were done, I opened them to check if I had got all the data. To my horror, the software had recorded only the screen and not the audio because I have to separately select the audio recording option. Do a dry run of all the tech you’re planning to use before the research study to make sure they’re working as expected.
15. Be mindful of the time
In an ideal world, we would have all the time in the world to dig deep and ask participants to explain the whys behind everything they say. But seldom we have such luxury in reality. Usually, time goes very fast during interviews and as an interviewer, it’s important to be mindful of the time. A good way to do this is by prioritizing the questions before the study. Make a list of critical questions that you must get answers to and a list of good to have questions that can be skipped. Don’t spend too much time digging into one question if it risks missing out on other important questions.
16. Cut people, politely
Most people generally do not get a chance to talk about their pains and concerns. When they get a chance, some people tend to use these interviews as their therapy sessions and pour out all their needs and concerns. As an interviewer it’s your job to ensure the interview doesn’t spin out of control and spend all the time talking about an interesting incident the participant experienced. When such digressions occur, interrupt politely and say ‘this is really interesting. Let’s return to this after the next set of questions’ and move on.
There are many more nuances to successful interviewing and data collection. However, in the interest of time and attention, I will stop here. I hope this was helpful. If you found this article interesting and would like me to write about anything else related to UX, please let me know in the comments.