Collecting data is easy. There are lots of tools out there and ways to gather data about everything that’s happening with your business, from lead generation through to customer satisfaction.
But what are we supposed to do with all that data? How does it help us focus on the key challenges at hand, provide us insights into our next steps, and drive success?
The data you collect may be helpful at some point; but if you can’t cut out the noise, you’ll get buried. That’s why you should think about a single metric that’s most important for the stage of your company’s development, a single number that you want the entire company to focus on and improve upon. I call it the One Metric That Matters.
The One Metric That Matters (or OMTM) is a single number that you care the most about at the current stage of your startup. First, let’s understand a bit more about the OMTM, then talk about what makes a good metric, and finally how to pick the right number to focus on.
Four Reasons You Need the OMTM for Your Startup
As I’ve said, the OMTM is a single metric that you care about at a given point in time, for the stage of your startup. So the first thing to remember is this: the OMTM will change. It’s not a single number that matters throughout your startup’s existence. We’ll discuss how it changes and why later on. For starters, let’s understand why you need the One Metric That Matters in the first place:
1. It answers the most important question you have.
At any given time you’ll be trying to answer a hundred different questions and juggling a million different things. You need to identify the riskiest areas of your business as quickly as possible – that’s where the most important question lies. The OMTM is responsible and necessary for measuring and answering that question.
If you’re using a Lean Canvas (or Alex Osterwalder’s business model canvas) as a 1-page “business plan,” it becomes fairly easy to identify the riskiest areas of your business. I’d strongly encourage you to check out Ash Maurya’s Lean Canvas at https://leancanvas.com. And here’s another tool for digging into the real problems of your startup (which aren’t always obvious): the Problem-Solution Canvas.
2. It forces you to draw a line in the sand and have clear goals.
After you’ve identified the key problem you want to focus on, you need to set goals. You need a way of defining success. It’s very hard for most startups to draw a line in the sand. Let’s say conversion on your website to trial accounts is your OMTM, and it’s currently at 0.5%, which you know is too low. So you’re going to put your entire startup’s resources into improving that number. But what should it be? How will you know if conversion is high enough that you’ve been successful?
At this point you need to draw a line in the sand and pick a target. The line you’re drawing is in sand for a reason; you can shift it as you start experimenting with solutions and learning. Just stay honest with yourself about why you’re doing it – don’t set a high bar, miss it, and then lower the bar in order to say you’ve succeeded and move to the next step. The One Metric That Matters is a forcing factor for encouraging you to set targets and analyze your results honestly and transparently.
3. It focuses the entire company.
Focus is good. In fact, it’s better to run the risk of over focusing (and missing some secondary metric) than it is to throw metrics at the wall and hope one sticks (the latter is what Avinash Kaushik calls Data Puking.) Put the OMTM front and center, physically visible to everyone all the time.
4. It inspires a culture of experimentation.
The Lean Startup movement has shown us the importance of experimentation. It’s critical to move through the “Build -> Measure -> Learn” cycle (explained in Eric Ries’ book, The Lean Startup) as quickly as possible to generate enough learning so that you can start executing effectively in the right direction. You want to instill and inspire a culture of experimentation throughout your organization – the One Metric That Matters can help.
What Makes a Good Metric?
We’ve all heard about vanity metrics before, right? Vanity metrics are great at sucking you in and distracting you, but they’re not good metrics for running your business. Here are five rules of thumb for what makes a good metric:
1. A rate or ratio is better than an absolute or cumulative value. For example, “New users per day” is better than “Total users.” Absolute numbers tend to be vanity metrics.
2. It is comparative to other time periods, sites, or segments. “Increased conversion from last week” is better than “2% conversion.” The key here is cohort analysis, where you track a metric over different groups of people, typically over different periods of time. For example, you drive traffic to your site through Google AdWords and measure conversion for a week. But you also measure engagement and churn down the road for those users. Then you make a change (to your website, product, or something else) and track those numbers for another batch of people. Each group of people visiting and signing up is a cohort.
3. It is no more complicated than a golf handicap. A good metric has to be incredibly simple and easy to understand; otherwise, people won’t remember it and discuss it.
4. For “accounting” metrics you use to report on the state of the business (to the board, investors, media, etc.), a good metric is one that makes your predictions more accurate.
5. For “experimental” metrics you use to optimize the product, pricing, or market, choose something which will produce an answer/result that will significantly change your behavior. Metrics that don’t cause a change of behavior are likely vanity metrics and of little value.
How To Pick the One Metric That Matters
There are a number of factors that play into how you can pick the One Metric That Matters. The two most important factors are the type of business you’re in and the particular stage of the business. Let’s take a look:
First: what business are you in?
There are a few big business model Key Performance Indicators (KPIs) that companies track, and they’re dictated largely by the main goal of the company. Of course, most companies want to make money eventually; but the ways in which they get there – and what they want their users and customers to do – vary significantly. Here are six broadly defined business models; you should be able to identify where you fit quite easily:
1. Transactional: Someone buys something.
Transactional sites are focused on shopping cart conversion, cart size, and abandonment. These are primarily e-commerce companies, but this applies to subscription businesses as well. This is the typical transaction funnel, and anyone who’s used web analytics is familiar with it. To be useful today, however, it should be a long funnel that includes sources, email metrics, and social media impact.
2. Collaborative: Someone votes, comments, or creates content for you.
Collaboration is about the amount of good content versus bad, as well as the percent of users that are lurkers versus creators. This is an engagement funnel, which should look something like Charlene Li’s engagement pyramid.
3. SaaS: Someone uses your system, and the value they get means they don’t churn or cancel their subscription.
SaaS is about conversions (and the associated cost of acquisition) and churn. What’s it take to acquire a customer, engage them, and keep them? The Lifetime Value of a customer is key to understanding the scalability of your SaaS business.
Customers are capable and ready to measure their own ROI from the use of your SaaS product, which is critical in understanding whether they’ll churn or stay engaged.
4. Media: Someone clicks on a banner, pay-per-click ad, or affiliate link.
Media is about time on page, pages per visit, and click-through rates. That might sound pretty standard, but the variety of revenue models can complicate things. For example, Pinterest’s affiliate URL rewriting model requires that the site take into account the likelihood someone will actually buy a thing as well as the percentage of click-throughs (see also this WSJ piece on the subject.)
5. Game (and many free mobile apps): Players pay for additional content, time savings, extra lives, in-game currencies, and so on.
Game startups care about Average Revenue Per User Per Month and Lifetime Average Revenue Per User (ARPUs). Companies like Flurry do a lot of work in this space, and many application developers roll their own code to suit the way their games are used.
Game developers walk a fine line between compelling content and in-game purchases that generate revenue. They need to solicit payments without spoiling gameplay, keeping users coming back while still extracting a pound of flesh each month.
6. App (and many free or paying mobile apps): Users buy and install your software on their device.
While similar to the Game category, Apps is a bit broader. But there’s clearly a lot of overlap. With this business model (whether a free app with in-app monetization or a paying app), it’s about number of users, percentage that have loaded the most recent version, uninstalls, ratings, and reviews.
Engagement numbers are critical here as well: daily active users, monthly active users.
Of course it’s not that simple. No company belongs in just one bucket. A game developer cares about the “app” KPIs when getting users, and the “Game” or “SaaS” KPIs when keeping them; Amazon cares about “transactional” KPIs when converting buyers, and “collaboration” KPIs when collecting reviews.
Second: what is the current stage of your business?
Defining the business you’re in is usually quite easy, but deciding on its stage can be a bit more complicated. This is where founders tend to lie to themselves – where they believe they’re further along than they really are; and it’s where Lean Startup is so important.
Your One Metric That Matters will be significantly impacted by the stage of your business. Premature focus or optimization of things that don’t really matter is a surefire way of killing your startup. So let’s take a look:
- Problem validation: This is the first stage for any startup, answering the question, “Am I solving a problem painful enough that people really, really care?” The One Metric That Matters at this point is qualitative. It’s not a number you can point to and track; it’s an honest assessment of your efforts in interviewing prospects and getting their feedback. There are strong indicators that you can get from people in defining the severity of pain they’re feeling with respect to the problem you’re trying to solve, but you’re not looking at hard numbers.
One number you should keep an eye on is the number of people you’ve interviewed. How much have you “gotten out of the building?” If you haven’t spoken to more than 10 people before moving to the next stage, you should be concerned about rushing forward.
- Solution validation: The next step is to conduct solution interviews with people that demonstrated strong interest in the problem set you’re tackling. Again, the data here is qualitative. Of course, if you can get people to pay you on the spot (for something that doesn’t exist), that’s a great sign; but I wouldn’t use “dollars collected” as the One Metric That Matters. It might not even be relevant for the type of business you’re in.
Lane Halley has some great tips on how to interview customers effectively in this Slideshare presentation.
- Minimum viable product (MVP) validation: Once you’ve gone ahead and built your MVP and put it into people’s hands, it’s time to get into more quantitative, measurable metrics. Here, metrics like amplification (how much does someone tell their friends about it?) and Net Promoter Score (would you tell your friends?) and Sean Ellis’ One Question That Matters (from Survey.io – “How would you feel if you could no longer use this product or service?”) are useful.
Early on during the MVP validation stage, you’ll most likely be looking at some form of engagement – daily, weekly, and/or monthly active users. You’ll measure this by cohort to see if engagement goes up or down as you change things. Ultimately, you’re trying to figure out two things: (1) are people using the product as you expected, and (2) are they getting enough value out of it?
- Generating attention / channel development: As you’re making progress with your MVP validation, you can look toward increasing the number of people exposed to the MVP to further test the value proposition, messaging, and channel / user acquisition strategies. This “attention generation” will be focused on a conversion metric at the end of a “long funnel” that tracks which proponents, campaigns, and media drive traffic to you.
Over time, this will get more sophisticated. Instead of just looking at conversion from the various channels, you’ll want to understand the value of the users / customers. A particular channel may bring a lot of users, but they may be low value, churning at a higher rate than others. This is where you string metrics together; use the One Metric That Matters to focus on one thing first (in this example that would be the channels and conversion) and then move to the next step of value creation (engagement / churn / lifetime value.)
- Ongoing feature development: Don’t build features in a vacuum. Test and measure the value they create for your customers. For example, it’s fairly easy to track how often a feature is being used. This may not be your One Metric That Matters, but it feeds into the overall value you’re creating and whether changing or adding (or deleting!) a feature has an impact on key metrics, like conversion, churn, lifetime value, revenue, etc.
- Business model validation: At some point, you have to make money. Depending on the business, this may come much earlier in the process, or later. There are a number of ways you can test your business model, including how you charge, when you charge, and what you charge.
Measuring revenue here is easy enough, but that might not be the best way to really understand your business. Revenue may go “up and to the right,” but does it really indicate the health and scalability of your business? “Revenue per customer” might be better (it’s a ratio after all!), and there’s a lot more you can learn from that metric. For example, if revenue is going up but revenue per customer is going down, it tells you that you’re going to need a lot more customers to continue growing. Is that doable? Does it make sense? The ratio helps you focus on making real decisions for your startup.
Marrying Lean Startup and Analytics Together Gives Us What I Call Lean Analytics.
Lean Startup helps structure your progress and home in on what you should be focusing on at any given time. Lean Analytics is used to measure that progress, helping you ask the most important question and get clear answers quickly. And the One Metric That Matters is one of the key tools for simplifying the complex nature of analytics and arming startups with the focus necessary to succeed.
About the Author: Ben Yoskovitz is the co-author of Lean Analytics, a book he’s writing with Alistair Croll that will be published by O’Reilly in 2013. You can follow him on Twitter or read more on his blog
Comments (5)