How to Use Product Analytics to Collect Targeted Customer Feedback
A key ingredient to build a better product is customer feedback. There are various ways to collect customer feedback that include customer surveys through sales executives, feedback forms, focused interviews, etc. In my opinion, these traditional methods followed by many companies are inefficient as they take long hours, are outdated quickly, have a low response rate, and are subjective. Using analytics, one can capture actual behavior of customers, ask targeted questions, collect accurate feedback, and repeat the process with much less effort next time. Hence this process is efficient, objective, and a lot faster than traditional survey methods. Also, once the analytics process is set up, it is easy to monitor unlike traditional surveys in which one needs to repeat the entire process from scratch.
This article provides analytical framework and tactical steps to collect targeted customer feedback to improve B2B SaaS products. The same approach with a few alterations can be replicated for B2C SaaS products as well.
Step 1: Identify the metric or feature that requires customer feedback
The metric or feature selection depends on what stage the product is at and what problems are being addressed such as how to acquire new customers, how to increase trial conversion rate, how to increase adoption of a new feature, what is the feedback on a particular feature — specifically newly launched feature, how to reduce the churn rate, how to increase overall product adoption, how to upsell/ cross sell to existing customers, etc.
The identification of metric is driven by the goal to be achieved, which is mainly guided by the product strategy and leadership within the organization.
Let me explain this with a B2B product example as there are not many B2B product related examples available: I want to understand customers’ usage behavior of a particular feature in a product to improve the product for better customer experience and identify business opportunities.
Step 2: Gather relevant data of all the customers for the identified metric/ feature that requires feedback
Now that metrics are identified, we need the relevant data. If an organization is matured and data driven, the data would exist, and we just have to curate that to the needs.
If an organization is still in the growth or early stage and does not have the relevant data, the first step is to gather the relevant data. How to gather relevant data? — This is a topic for another article.
Sometimes we might need to provide the business needs to get this data. This is why step 1 is crucial as it supports the case with rational explanation on why we need the requested data.
Please note that sometimes because the data volume is high, we will need some technical skillset to play with the data such as SQL. Large organizations generally have a data engineering team that will provide curated and quality data ready for analysis.
Continuing my example from Step 1: I gather data that provides the number of times the customers are using that feature in a day. I generally gather this day level data for the last 12 months to observe trends, growth over time, and historical behavior.
Step 3: Compare all the customers for that metric in a single view
Once we have the data, arrange it such that we have all the customers listed in rows along with the metric/s that are being measured.
Note that it will be meaningful and important to categorize these customers into cohorts, especially if we have a large customer base. For B2B products, the cohort could be revenue cohort*, number of users per customer cohort*, time period, etc. In case of B2C customers it could be a usage related cohort, that is, MAU/ DAU.
*Revenue Cohort: $10M+, $1M — $10M, $500k — $1M…$10k — $100k
*User Cohort: 1M+ users, 500k — 1M users.. 0–10k users
Example: Below screenshot provides a sample for this step
Step 4: Identify the outliers
Once the dashboard/ visual is ready as described in Step 3, you may notice a few outliers. First, confirm that the outliers are legit and remove any internal systems created outliers. Now, we can confidently reach out to the customers with targeted questions. For B2B customers, it is feasible and efficient to have a call for direct feedback. For B2C, one can send out targeted questions with some personalized incentives to the customers.
Depending upon the metric, we may notice that the number of outlier customers may vary. If there are a high number of outliers, we have got the key insight for the product. For instance, in one of my previous experiences, we were trying to increase the adoption of an automated phone calling feature in the product. Following these analysis steps 1–4, I was able to identify that a number of customers used the feature in the first 3 months, but the number and usage declined over time. On further analysis, I identified that a high number of dialed calls failed, and this was because the dialer was not intelligent enough to understand different phone formats. We made necessary changes to fix the phone format issue and worked with customers to try again. This increased the adoption of the feature by 22%, and that is a significant increase in adoption of a feature.
Step 5: Reach out to the selected customers with targeted questions
Once the outliers are identified, as a next step, we send a note to the B2B customers to set up a discussion to understand their product usage behavior.
Example of a Note: (Many times this note is sent by the Sales Account Owner on behalf of the Product Manager. Please check the protocols in your organization)
“From our Customer Success Analytics, we understand that you are a valuable customer since <datetime> with <some good successful metrics about this customer such as number of users, activation rate, etc.>.
We noticed that you heavily use this particular feature. We would like to understand more about your use case and get your feedback, if any, to improve your user experience.”
(Support this note with relevant data)
Step 6: Capture customer feedback and work with corresponding teams to resolve the problem.
Final Note
This is one of the analytical frameworks to gather customer feedback that works for me for all different analyses that include increasing the sales, improving product features, gathering feedback post product/ feature launch, increasing adoption of a feature/ product, increasing trial conversion rate, enabling governance on a product usage, increasing upsell/ cross sell, and the list goes on.
The outcome of the analyses mentioned earlier also feeds into building data driven product strategy and roadmap — this is my main goal for doing different analyses that I do in my current role as a Product Manager.
The biggest benefit is that we only need to set it up once and then regularly monitor to get into a feedback-improvise-feedback loop. Also, this becomes a multi-win when many other relevant teams across the organization also benefit from this readily available information.
If there is any product related goal that you are trying to achieve analytically, try out this framework. I would love to know your story on how you tried it and whether it worked for you.
