Regardless of the type of goal that you set, it’s important to make it measurable. Without measurement, how can you tell where you started or, perhaps more importantly, how you have progressed towards your desired outcome. If it can’t be measured, how can you tell when you’re done? When do you decide that you should adjust your strategy?
If you can’t articulate where you began, or when you’re complete, have you really accomplished anything at all? Your guess is as good as mine!
Often in a business setting, a company doesn’t have the data that they need to effectively measure something. There are two ways that the company can react to this situation. In the best case, people will be so hungry to fill the gap that they will find ways to collect the missing data. More commonly, however, people are resigned to the fact that the data doesn’t exist and shrug their shoulders and move on.
This leads to my main thesis. Transitioning an organization from gut or qualitative based decision making to more data based quantitative decision making and outcome oriented thinking requires a cultural shift. Fundamentally, this is not a transition that can be solved merely through the application of technology. Instead, it requires a much deeper change in the thought process of the people making decisions and those executing on the projects to make measurement a required outcome.
So, why measure in the first place?
In product companies, we measure to predict and influence the future, provide decision support and to aid in the optimization of a product or service.
Fine, but what should we measure?
User adoption — Measuring user adoption of a product can tell you whether the user perceives it to add value. Often, the quicker that users get in to the system and try it out, the higher they perceive the value of the problem being solved. Adoption can be measured overall for the product, but it is usually more valuable and actionable to measure adoption at a feature level.
User engagement — Engagement measures how much value a user receives from a product. Unlike adoption, engagement provides an indication of the realization of value. Engagement metrics include frequency and duration of access for a user. Engagement metrics might include things like transactions per user per day. It focuses on how invested a user is in a product or future and how likely they are to keep coming back.
User happiness — Adoption and engagement alone provide little indication as to whether a user is happy with a product or feature. The best way to measure happiness is to ask the user directly. The closer that you can ask the user to the moment at which they performed the action, the more useful that metric will be. Asking users in annual survey about their satisfaction with a given feature will provide significantly different results than embedding a simple question inside of that same feature flow. One metric for user happiness with a product that everyone should know is Net Promoter Score or NPS.
User retention — Retention determines how likely a user is to continue making use of a product or feature. In the modern world of the recurring customer, retention is key. Retaining customers is significantly cheaper than finding and selling to new customers. Positively influencing your retention rates can significantly improve your Customer Lifetime Value or CLV. Churning users quickly makes it difficult to improve your CLV outside of selling them additional products or raising your initial purchase price.
Task completion success — Completion success metrics require that you know what a user is trying to accomplish and whether they were able to achieve their desired results. A user might set out to create a quote but fail along the way due to technical reasons such as missing information or due to psychological reasons like getting distracted.
Measurement is not a point in time
To maximize the value of measurement, information needs to be gathered consistently and looked at over time. Establish a current baseline and then conduct experiments to influence that baseline.
Indicators are important
Not all measures are equal. When trying to influence outcomes, it’s important to understand what time of indicator you’re dealing with.
Leading indicators — a leading indicator is an artifact that occurs prior to an event. They are the red flags that when identified early can be addressed to change outcomes. Leading indicators tend to be easier to influence by adjusting the inputs. Decreasing user engagement might be a leading indicator for churn.
Lagging indicators —lagging indicators are harder to directly influence as they tend to focus on outputs. Annualized recurring revenue or annualized customer retention are examples of a trailing indicators.
If you’re building a measurement culture and your desire is to use measures to influence outcomes, spending time to identify leading indicators will improve your decision making process.
Measurement has consequences
When rotating towards measurement, it is important to understand that what you measure will influence your organizational behavior. Places that use standardized education produce children who perform well on standardized tests. Unfortunately, these same children don’t do as well when asked to demonstrate that knowledge in a different way.
“Perhaps what you measure is what you get. More likely, what you measure is all you’ll get. What you don’t (or can’t) measure is lost” — H. Thomas Johnson
When approaching a measurement task, I like to think about what the outcomes of that measurement will be. How will people interpret this measurement and what behaviors will it drive? It can be a rewarding exercise to spend some time and think about the unintended consequences of measurement. I often ask myself the question of what if someone tries to take this to the extreme and game the metric? What can I do to prevent that?
I hope that you found this article useful. I’d love to hear feedback on how you approach measurement within your organization.