How we used the Superhuman’s Product-Market Fit model to learn what to build and whom to target
A few months back I read an amazing Blog post by Rahul Vohra about how Superhuman was able to reach Product-Market Fit. The normal approach to this is “I’ll know it when I get there” but that doesn’t actually help you to get there (or work out how far along you are). In short, the blog suggests using a Survey with the goal (set by Sean Ellis) of:
Getting 40% of your users saying that they would be “Very Disappointed” if they could no longer use your product.
There are certain articles you read where you change your entire thought process, and this was one of them. At last, I had a real way to understand whereabouts I was. With Haystack, we were at a loss as to where we were… Yes, we got some users emailing us saying how much they loved it (or even better what would make them love it), but it was hard to know if these were signals in the wrong place.
In the article as well they segment groups so you can understand more about which people you should be targetting. In startups, there is often a focus on Ideal Customer Profile (ICP), but many consumer products struggle with this because they just don’t know who will love it first, and saying “Everyone” is a recipe for startup failure. This is how Superhuman went from 33 to 58%:
I also loved that using the additional questions, you can work out which area of the product to double down on and where to add additional features. Anyway, here is how we used the article to direct our latest version of Haystack:
Survey Setup
Collecting Emails
The first thing we needed was a collection of users email address. As a privacy-centric tool, this was a little hard as people don’t “Login” to create an account. However, when people attach a Cloud Provider account, we send a message to our servers with the email address they used. This means that we can be emailing a person a few times (if they attach multiple email accounts to Haystack), and we also miss people who don’t attach any, but it was the only way to do it.
Selecting a Survey Tool
We wanted a tool that would make it easy to upload the emails into and have a one-click answer to the first. I also know that most people drop off from surveys, especially sent over email, so I wanted one that didn’t require someone to get to the end of the results (like Google Forms).
We ended up using Iterate, which is an amazing tool (actually an Expa company as well) and has the ability, later on, to embed it directly into the tool as well.
To see an example of this survey check it out here:
https://iter.ly/9ptu2 (note: the first question appeared in the email)
Segmenting Users
Alas, without paying money for data access, segmenting users was done manually. I used Superhuman, to try to grab peoples LinkedIn or AngelList profiles, then took down their Job Role (as reported), the rough area they seemed to be in (i.e. Founder, Marketing, CX) and the number of employees at the company — this took me an hour or so.
Month 1 Results
Overall Results
Of those ~400 email addresses, we received 70 replies, who answered the question: “How would you feel if you could no longer use Haystack?”
- 31% would be very disappointed (Good)
- 43% would be somewhat disappointed (OK)
- 26% would not be disappointed (Bad)
Not terrible results at all! Only 9% off from the goal of getting past 40%.
Now we had these results, I looked at the breakdown by segments:
Very disappointed:
Marketing (26%), Creative (12%), Founder (9%)
Somewhat disappointed:
Founder (28%), PM (11%), Marketing (5%), VC (5%)
And these were not disappointed:
Couldn’t find LinkedIn (32%), Creative (24%), Founder (14%)
I also found that people that had over 1000 employees were less likely to be positive too.
After Segmentation
So removing those I couldn’t find on LinkedIn, Founders, Creatives (who I originally thought would love it) and people with companies over 1000 people, the results became:
- 47% Very Disappointed (Good)
- 46% Somewhat Disappointed (OK)
- 7% Not disappointed (Bad)
This was really interesting because I expected Founders and Creatives to be really positive to Haystack!
General Feedback
The next question after this was asking people why they answered how they did which gave the following word cloud across all segments and answers:
You see above, that quite a few people hadn’t yet used it, a few people had run into bugs and a few others reported that they wanted additional searching on their own computers.
How to make great things better
Next, using the group that reported they were very disappointed (i.e. they loved Haystack), I asked: What is the main benefit you receive from Haystack?
So from this, we can see that speed (Quick, quicker, quickly), and saving time is key for them, also easy of use.
How to work out what to work on next
Next, by taking the group that was ‘somewhat disappointed’, I looked at the answers to the question: How can we improve Haystack for you?
You can see that most people said nothing, many needed more time before they were getting value, but there was an ask for an App (both Desktop and Mobile).
Additional Questions
If you’ve used and been surveyed by the team at Superhuman, you’ll know that their questionnaire is much longer. By using Iterate, I could ask many more questions and if people dropped off, then no worries I had the key answers above anyway so no worries. So for we asked:
- Have you recommended Haystack to anyone? 59% Yes, 41% No
- What additional Integrations would you like? Most of those requested were already in the works but good to clarify further.
- What would feature would you pay for if it was launched tomorrow? Of which the people that loved it mainly said a Desktop search version (or not at all)
So what changes did we focus on?
Well, for those that loved the tool, ease of use was key, so we decided to make the UI cleaner. I was fortunate enough that the team over at Mev, nicely offered some UX feedback, and Philip Bell gave some great insight into these areas. From that for Haystack Version 2 (coming soon) the UI is going from:
To:
You see how we’ve removed the left & right bars, added an auto-completing text (right arrow) and removed the filtering UI. We’ve also added in the scroll wheel on the left. In truth, the Superhumans UI was quite an inspiration for this version, as they really clean out the traditional email UI. To paraphrase an old quote (that I can’t recall fully):
Remove everything that isn’t essential but nothing more
The next big focus was speed, so we’ve re-written a number of the search functions to optimise even further, so now Haystack is 2x faster in the majority of cases.
How to move people from like to love
The main issue for me, was that most people after 2+ weeks didn’t feel that used it enough, which means it hadn’t become a habit (or at least felt like it was). For most tools, that's not surprising, but as people spend 1–2 hours a day finding information this signifies that aren’t using us as often as they should. We have a number of ways to increase usage and engagement in V2.
The second issues were around availability and compatibility, so support a native Desktop Application is already in the works. Safari, Edge, and Firefox versions are also coming together, and we’re considering a mobile-specific solution too.
All of these to help move the ‘somewhat’ to the ‘very’ pile.
Key Takeaways
- You could be completely wrong about who your ideal/loving customers are
- Segmenting and focusing on the right groups can really help you focus on these ideal customers
- It’s a lot of work but the insights you get are worth it
- People are confused by the main question, I had to redo some of peoples answers because it was clear that they didn’t read it fully. I think there is definitely a better way to write it!
We are less than a month and a half since we launched Haystack into the world, so there is much more data that needs to be collected, but having a framework like this has been really helpful!
Thanks again to Rahul Vohra, the Iterate team, Philip Bell and everyone who completed the survey!
Any questions or thoughts, comment below.
PS If you don’t know what Haystack is, its a simple tool that lets you search all your cloud files, documents and work in one place, really quickly. It saves most people 20+ minutes a day, so definitely worth a go.
Here is a waiting list skipping access code for reading the whole blog post :).
