There are a lot of ways to screw up the user research. Poor understanding of the process, bad preparing, lack of recording. They are not critical, because you can easily fix them and don’t repeat. But the most harmful things are inside of our brains. We need to keep them under control and know how to mitigate. Here you’ll find some ways which help you to refine the user research and to get more reliable results.
In fact, the human research is most tricky and hard to conduct. It’s extremely difficult to separate yourself from users and do the research unbiased.
We believe that we understand users because we are people and also being users. But that’s not true. Our own user experience doesn’t matter as well as fact that we are humans.
Every researcher (especially new) should keep in mind some of the common mistakes of the thinking process. I’m not going to describe every one of hundreds existing fallacies. My purpose is to highlight the biggest mistakes and propose some simple ways to avoid them.
#1: Users are rational
Of course, not. People waste a lot of time absolute irrational for example scrolling Instagram or Facebook despite a pile of important things to do. We don’t really understand some decisions we make. And that’s not completely our fault. I bet you’ve heard of “lizard brain” (if not you can read about it here and here) — the old part of our own brain that is responsible for survival, rage and fear. Ancient times have gone but “reptile” is still here.
The lizard brain is our irrational side we can’t fight to. For the product, it means that a user’s behavior can be unpredictable. At the same time, it provides an opportunity to benefit from irrationality. Everyone use red color for important elements, right?
What should we do
To admit it, do not try to square the circle and to get rid of your own prejudices. You should be a distant observer that records and analyzes. If something seems illogical and bizarre, follow research procedure and don’t try to find patterns where it’s not.
In a condition of user’s irrationality, it’s barely possible to find a universal way to create product customers will love. So embody ideas and explore user behavior all the time.
Always remember that you’re also irrational.
#2 They actually have answers we need
Yes, they have. But to get them we need to make an effort and be very careful with our favorite “Ask user”. Let me tell you the story.
A couple of years I was the product manager of new CRM that already had a certain amount of users. We decided to make a survey and ask them about the product. And it seemed like users had waited for it. We got a lot of different proposals which looked very meaningful. My team and I even added some of them to the backlog and implemented. And… nothing. No one even tried to use them. I am sure, that almost everyone had the similar experience.
Why? Our users irrational, remember? They barely can, right off the bat, list what is not enough for them. But if you ask, they’ll try to provide information which is necessarily true. Maybe just to cheer you up.
What should we do
Firstly, be honest with yourself. If you seek a confirmation the product is perfect, then you easily get it whatever results. But looking for real insights we need to remember two rules.
- Users really don’t know what product should be. That’s your job. They may think they know. No. That’s still your job.
- If you need true answers, observe behavior, rely on metrics and ask to show you, not to tell.
Researcher : “Is this statistic tool good enough for you?”
User : “Yes, absolutely!”.
Researcher : “So why you don’t use it?”
User : “I’m gonna use it/don’t have enough time” (anything not to hurt your feelings).
Researcher : “Show me how do you collect the statistic?”
Sometimes the product has experienced users that can share their ideas with us. To have them is a big luck because information from these users is invaluable.
But it comes to another fallacy.
#3: We can truly understand them
And everyone else as well. It seems ridiculous, but the language is not a reliable way to exchange the information. There are a huge amount of barriers that interfere with our communication. We can recognize words, classify them, explain, but the meaning may be completely different. Everything has an impact on perception and interpretation. Past experience, education, nationality, mood, etc.
Roughly it look like that.
Interferences appear from both sides, environment and language also contain them. We ignore these facts and unconsciously apply our mindset to the person we talk to.
How in this case can we conduct user research?
What should we do
First of all, remember that we all are different. Users don’t know what you know. We can mitigate the miscommunications only receiving more detailed feedback. Small chunks of information are easier to decode and harder to distort.
It works well if we ask the right questions and not try to interpret answers. Also find out more about users, their habits and even hobbies (if related to the product) .
So don’t ask users what your product should be. Better ask about them.
These simple rules help you to resist your own biases and to mitigate interferences.
- Prepare a quick questionnaire to fill by information about age, education, an experience of using the similar product. Add everything you need, but make it short.
- Don’t ask leading question because they’re based on your opinion.
- If something unclear to you, do not interpret. Ask to expand and explain.
Researcher : “Is there any difficulties with statistic tool?”
User: “Yes, it’s hard to get the necessary information”.
PM concludes that tool should be expanded. The user had problems with interpretations of graphs.
PM knows that user didn’t work with statistic tools before and his education also is not related to data, so statistic can be unclear. PM asks the user to explain step by step what is wrong exactly and records everything to recall after a while if needed.
Important! Don’t select only ones, that can handle everything. Try to randomize the selection even if the product has the certain audience. It makes the research reliable and avoid selection bias.
By the way, the next problem is also about selection bias.
#4 All data is available
That’s, probably, one of most common biases which profoundly affect eveything. I bet, you’ve heard a story about bombers that needed to be improved during the WW2. Yes, it’s about survivorship bias.
Despite almost everyone knows of it, we’re basically biased all the time! Just remember how many time you’ve read articles like 10 habits of successful people or “How to earn your first million in 20 years” which also could be named “10 habits of millions unsuccessful people as well” and “How to lose all your money in 20 years”.
So unconsciously we believe that there is enough data for making decision and conclusion. But the important part is hidden.
For user research, it means we measure the experience of survivors (who have become users) but leave behind people that didn’t succeed. And that’s a huge source of information.
Lack of information is important information as well.
What should we do
There is no one-size-fits-all approach. To reach lost users we need to mix different types of analyses.
- Firstly define, where is the bottleneck(s) of the product. The using metrics really makes sense.
- Explore experience of users that see the product for the very first time. They didn’t work with product, thus, can be either survivor or lost. Right questions, randomization of selection and repeated research ensure that your data is reliable.
- The product can have some critical bugs you don’t know about. User should be able to leave a feedback or report a problem from every part of your product easily. Again, we have to thoroughly analyze every feedback and reach the user to get more details.
So, if you‘re able to explore the experience of lost user, the research definitely will be more comprehensive.