As design research expert Erika Hall writes so eloquently, “Surveys are the most dangerous research tool — misunderstood and misused.” They can seem deceptively easy to administer at scale, and this accessibility can be good. However, getting quality data back from a survey requires a lot of skill in how the survey is designed and constructed.
Surveys rely heavily on people self reporting, and also often constrain answers to limited choices. Surveys don’t allow researchers to observe behavior, and so are best used to explore things like large-scale customer demographics. The method often doesn’t allow researchers to dig deep into why something might be happening — so, if used, it can be most effective when paired with a qualitative method of research, like interviewing, that allows for a deeper dive.
Reality: Surveys often don’t have a good ROI in terms of the quality and usefulness of the data.
Myth 3: We can’t get anything worthwhile from talking to such a small number of people
For organizations that skew heavily toward valuing statistically significant market segmentation or surveying, or perhaps analytics data, the validity and worth of qualitative research with smaller sample sizes is sometimes questioned. There can be an attitude that a quantitative research approach is better than a qualitative one (or, indeed, sometimes the opposite).
In reality, the most effective UX research approaches allow for triangulation across multiple methods. The different data types can tell us different things, and complement each other nicely. Qualitative methods can help us understand the “why” behind human behavior, and dive deep into underlying customer needs and desires, even at smaller sample sizes. The researcher’s job is to identify insights and patterns. Nielsen Norman suggests that, for usability testing, testing with five users will find up to 85 percent of the issues.
Reality: Qualitative methods and usability testing can reveal important patterns at small sample sizes.
Myth 4: Customers don’t know what they want anyway
The (potentially misattributed) Henry Ford quote, “If I asked people what they wanted, they would have said faster horses,” gets pulled out a lot as justification for not talking to users. The idea that customers don’t know what they want may have some truth to it. However, doing UX research is not only about understanding what customers want, it’s about diving deep into their needs, behaviors, and experiences.
Customers are experts in their own lived experiences and mental models, and will demonstrate surprising ways of using your products and services. Being able to understand the customer perspective will enable teams to build experiences that are more aligned to where the customer is at, and truly provide value and delight. The researchers job is to sift through qualitative data to uncover people’s underlying needs in what they share. In the faster horses case, for example, people’s underlying need is to get from A to B quickly and reliably.
Reality: Users are experts in their own lived experiences and mental models, and the researcher’s job is to uncover their needs and behaviors to inform the design process.
Myth 5: Doing testing with users at the end of the build will be sufficient
One of the more common forms of UX research is to conduct evaluative testing, which entails testing with users once a product or service has been built. This is sometimes seen as the last “check in the box” in a design-and-build process. While this is certainly valuable, and better than doing no research at all, there is sometimes a missed opportunity to incorporate UX research earlier in the process.
Generative research is about exploring a problem space and generating possible problem framing or solutions. Doing upfront generative research can increase the chances that a team is approaching a customer need from the right perspective before proceeding to ideation and developing solutions.
Reality: Incorporating different types of research throughout the process will lead to the best results for customers and the business.