Tomasz Tunguz is well-known in Silicon Valley as an avid blogger, specializing primarily in data-driven posts. He is a Partner at Redpoint Ventures, and co-authored the book, Winning With Data. So it’s both fitting and beneficial to all of us that his talk at last year’s SCALE conference focused on how to properly run a data-driven company.
You Can't Run a Rearward-Looking Business
Tomasz poses a hypothetical situation where you are the owner of a bustling restaurant. He then challenges the audience to come up with a way to measure your key metric: customer satisfaction. The most obvious metrics are surveys or measuring tip-to-bill ratios, which are all fine, but the problem with these, Tomasz argues, is that they are “rearward-looking,” and you can’t run a business like this.
The Impact of Latency in the Value Chain
Rearward-looking metrics are the primary cause of the “Bullwhip Effect,” discovered by an MIT researcher, which is the product of latency in communication between each sector of a specific supply chain. Tomasz uses ”The Beer Game” to explain this phenomenon. Essentially, there are four groups in order: Brewery, Bottler, Distributor, and Retailer. In each round of the game, each group has to predict, in order, how much they are going to produce. The professor feeds the data to the Retailer in each round telling them how much demand there is in the market. The Retailer adjusts their prediction accordingly, but what happens from one group to the next moving backwards is that they each overcorrect at an increasing rate, causing higher and higher volatility in the prediction at the other end of the value chain.
This same value chain exists in startups. If you translate it from the Beer Game groups it runs in the following order: Product, Marketing, Sales, Customer Service. The same level of latency in communication is passed in the same way. This is precisely why you can’t rely on rearward-focused metrics, you need to be able to solve your customers’ problems in real time. So what is the answer?
A simple and effective way to measure customer experience in the restaurant example is by looking at how full the water glasses are. Tomasz argues that if your glass is always full without you asking or even noticing the server filling your glass, you are likely having a positive dining experience. It is a simple, measurable, and actionable metric to track that allows faster iterations on the data you’re collecting. This is the secret to defeating the issue of latency in your business, by finding a proxy metric such as the water glass example.
Real-World Examples of Proxy Metrics
- Facebook’s proxy metric in it’s early days was, “if you could get a user to have ten friends in seven days, they’re hooked.”
- Looker’s proxy metric was engagement minutes. The more time someone spent looking at data, the more likely they were to convert to a paying customer.
- thredUP looks at cross-platform usage. Is the user engaging on both the mobile app and the web?
- Expensify determines if the user is going to be active by uploading and processing an expense report. The second proxy metric they use is when a user within that business encodes a policy for the budget and what needs approval accordingly.
Tomasz also shares a common predictive proxy metric used by sales teams: Pipeline/Quota Ratio. By looking at the percent of prospective customers to converted customers in the past, you can make a predictive analysis of which salespeople will meet their quota based on the number of prospects in their pipeline.
The Key Takeaway
Find your proxy metric! It takes three steps that are easier said than done. First, determine the goal. Do you want to sell more? Maximize customer engagement? You get the idea. Second, you have to sift through the data. You really need to spend time and meticulously analyze the data you’ve collected that directly relates to your goal. And third, look for the correlations. Remember, your proxy metric isn’t the only metric you should be looking at, rearward-focused metrics are important too. But your proxy metric will drastically help with a rapid feedback loop, which is crucial for a scaling startup.