Traffic and Product Market Fit Are Key Determinants for Your Product Strategy
I often have gotten the following question: How do you find the difference between optimization and innovation? How do you balance them?
A lot of the balance of innovation and optimization depends on the stage of your company and how many people are coming to your product. A lot of it also depends on what you are trying to build. If you’re trying to build something that is brand new, something that is a total departure from what your company has done historically, or you are starting a brand new company, then you’re going to be more innovation-focused because you need to understand what new problem you’re solving. And if you’re in an optimization phase, where you’re iterating on a growth funnel or you’re optimizing a growth loop, that’s going to be more conversion-focused.
I’ve tried to outline this in the above chart. The easiest way to think about this is:
- Less traffic = more time you need to spend validating concepts with customers
- Less product-market fit = figure out the key value proposition for your product and spend more time with customers
- If you are a larger company working on a new product line, you have the benefits of traffic that you can use to validate your concepts more rapidly. That is, however, no excuse for not doing customer research.
Company stage and the amount of traffic really matter. If you’re a very large company, the cost of optimization is lower because you have so many users visiting the website or mobile app, meaning it’s a lot cheaper to run tests. So it’s likely that the time needed to run a test to statistical significance will be much less. This means for companies like Booking.com, Facebook, Google, Netflix, Amazon, etc, the cost of a single test is significantly lower, meaning you can put more resources into optimization.
Generally, a good rule of thumb for testing and optimization is that 10–20% of your tests will be successful. Even the best test with the most well-formulated hypothesis is subject to this rule. Therefore, your velocity in running and validating tests is critical to how successful you can be during an optimization phase.
If your team is only capable of running and validating one or two tests a month, then it’s probably a better idea to spend your time focusing on innovation or deeper investment in a few key feature areas. You either don’t have the development velocity or the traffic numbers to enable success with testing. Therefore, it’s more important that you validate features with customers and put more investment into them before building them.
Your level of product-market fit is also very important. If you have a product where you know that users are engaged and likely to use the product, then entering a phase of optimization makes a lot more sense. It’s then important to determine do you need to build up traffic to have more opportunities to optimize? Or, do you have enough traffic and can you just focus on building the best growth loop or the best funnel for your business?
If you’re at an earlier stage or you have fewer users or data points, innovation is a lot more important. Talking to customers is a requirement, because every test is going to be more expensive and you might not have enough traffic to your product. It is also more likely that you are still working to achieve product-market fit, which requires actually talking to and understanding your users.
Incredibly, talking to customers and actually spending time with them to establish product-market fit and to build the right product has increasingly become a rarer part of product management. However, if you look at the best companies and the best products, they start by doing things that don’t scale to serve their customers (a key Paul Graham insight), and then they learn how to scale those key customer elements. If you are in an innovation phase, this is also the time when you should be experimenting and spending lots of time with your users.
