In the beginning of June 2020, we had enough cash to sustain ourselves for 2 years, and had decided to pivot. At the time, we weren’t sure if we needed to move away from our product, on which revenues were growing very slowly, or not. We spent 6 weeks in full exploration mode to identify what our next step was.

During this period, we went through different phases, which gave us several lessons that helped us identify our final vision to build an open-source data integration engine. This article lists these lessons. We’d love to know what you think about them and if you would have done differently.

In the end, a pre-product-market fit startup’s only job is to find product-market fit (PMF), and therefore to explore different solutions as smartly and efficiently as possible.

1. Exploring 2 directions at a time, max

We’re 5 people on our team, including 4 with an engineering background. The first rule that came to us was that everyone could contribute any ideas. Seems pretty obvious, but it actually had several unforeseen consequences:

• We could end up with a list of 10 ideas after a brainstorming session.
• Everyone has some bias towards the idea they contributed to the most.

At first, we would let any team member assign themselves an idea, based on their own preferences, while putting aside the ideas that we could invalidate on the fly. We might end up with 4 different ideas to work on. We thought this was okay, as we wanted to learn as fast as possible. However, we understood very quickly that having the team scattered like this with a person sometimes working alone on an idea would only lead to a very biased assessment of the ideas. Also, we needed to delve deeper into the ideas we wanted to explore, and to do that we needed more time on the ideas, and different perspectives.

Working on 2-or max 3-ideas at a time was a lot better. But even 3 was too much, so we ended up, naturally, exploring 2 ideas at the same time.

Why not focus on only 1? We tried that, too, and saw that we were overlapping ourselves on the assumptions to validate. Plus, we were not convinced by any one idea more than the others, so it felt like we were putting all our eggs in the same basket without valid reasons at the time.

2. Focusing on problems to solve vs solutions

Nowadays, we always talk about going back to the problem we solved, not just to the solution. Having a solution that doesn’t address a problem that is a big enough pain point will never become a must-have. And that is entirely true. But we had 2 different observations:

1. When evaluating a solution during interviews, we would always evaluate the problem and would not talk about our solution until at the very end. A really great resource for customer development interviews is the book The Mom Test.
2. However, when our initial idea was just about trying to address a problem, without first having a solution in mind, we had a much harder time evaluating the said problem and a potential solution. We needed to have some intuition first, from which we would then evaluate the problem and iterate on the solution.

Think of it as being in a dense blizzard. When you start from somewhere in the blizzard, you can take one step ahead and you will see much better around this step forward. Finding product-market fit is like trying to find a particular place in the blizzard, and you don’t know where it is. So you need to walk as fast as possible and hopefully in the right direction — that means while taking on the good signals and discarding the wrong ones.

But if you don’t have a place to start from, you don’t know exactly where to start. So you need an intuition of a solution to start with, so you can better focus on the actual problem you’re solving and iterate on a solution that would address it.

3. A common template to evaluate ideas

Different backgrounds in the team is a blessing when in exploration mode. But it also means we needed to have a common set of criteria across the different ideas that we were exploring. Here is the template we eventually wound up using. It would ensure that we were considering all aspects of a project, including:

• The project’s unique value: is about evaluating how the unique features and value seems interesting to the audience we would be targeting. It has both a solution and audience consideration.
• The audience reachability: is about our capacity to identify and reach out to our audience.
• The market size: is about our overall offer — does it suit our audience? Can it reach a good market size?
• Our market knowledge: is NOT about what we know now, but about the market knowledge we can grow quickly if we focus on it.
• The project’s defensibility: is about our capacity to create a big business.
• Our team’s strength: is about how this project would take advantage of our strengths.

Those were key criteria for us. We agreed that we wouldn’t pursue any project that was very weak on one of these points. This helped us tremendously in evaluating ideas faster and being more efficient in our exploration period.

4. Seeking indicators of potential Product-Market Fit, or avoiding better mousetraps

Another rule that was obvious to us was that we couldn’t validate an idea that we didn’t have any customer development interviews for. You can’t have actual indicators of product-market fit (PMF) if you don’t have some form of validation from the market side.

What do we mean by indicators of PMF?

• An unsolved problem that addresses a validated pain point that potential customers would be paying for
• A solution that is significantly different enough from existing alternatives that you can position it differently, and that it is perceived differently
• A solution on which you can have some kind of commitment from potential customers (POC, LOI, etc).

The issue we ran into most is that we identified opportunities that were not significantly different enough in perception from the potential customers. Building a better marketing analytics platform, for instance, could be appealing at first glance given the size of the market. However, it would be going into a “ better” mousetrap, which you don’t want to do.

Indeed, just building a better product that is perceived as solving the same problem the same way will not enable you to capture an interesting market share. It will take years for you to catch up in terms of features and surpass your competitors; they are several years in advance product-wise, with a much larger marketing budget. You can only disrupt incumbents if customers perceive you as different and unique.

5. When to start prototyping the MVP

3 weeks into our exploration, we were becoming edgy about starting to build one of our explorations. 4 of the 5 people in the team were engineers. At the same time, we didn’t want to work on a project without being convinced by it.

To make the decision easier, we defined what we expected from the prototype and its scope. The goal of prototyping was to validate some known unknowns in the project. We would try to fit the prototyping within the timeline of a single month, so we could take all the shortcuts needed and remained completely okay about discarding it. Prototyping was a complete part of the exploration, and was a two-way door decision.

It took us 6 weeks to start prototyping a project. Before then, we were not convinced enough to commit a full month from the whole team.

Please note that we did mockup prototyping for several projects during those 6 weeks. So we mean actual working prototype here.

6. How to react during stalemates

Another challenge that arises in this exploration period is the fatigue attached to not having a vision that you strongly believe in. At moments, we would arrive at a stalemate, having invalidated all the ideas we had. At that moment, we would do 2 things:

• Go back to the ideas we have had in the past few weeks and the reasons why we investigated them and invalidated them.
• Have another brainstorming session.

The first step is important, as it ensures that you have all the learnings you have accumulated for the brainstorming sessions.

This explains why some old ideas we investigated at the very beginning resurfaced with some small tweaks later on.

Should pre-PMF startups look like this in the end?

About 80% of startups don’t find product-market fit. This means for most of them that they haven’t identified the need to pivot and explore other approaches early enough. So you might wonder if a pre-product market fit startup should look like this!

This mental map shows all our explorations. At the far right, you will see what we’re currently building: an open-source data integration engine — the OSS ELT.

In the 1st month of investigating this idea, we did 21 customer interviews, and identified the same patterns across companies.

• Current cloud-based solutions — Fivetran and StitchData — don’t cover all their integration needs, so they still need to build their own integration pipelines, which are cumbersome to maintain.
• Their pricing indexed on volume limits how they can move their data.
• They were very sensitive to having full control over their data and its privacy, through a self-hosted solution.

We started prototyping after the 5th interview, as we saw those 3 patterns emerge. The 16 other interviews enabled us to validate our assumptions further and understand their exact requirements and needs, in addition to building up a list of prospects, once our MVP was launched.

We would like to finish on this note. If we were investors, we would have much more trust in a team that moves fast and iterates a lot, rather than a startup that only investigates one approach.

What do you think?

Originally published at https://airbyte.io on September 8, 2020.