Product Manager Intelligence

Growing revenues is a challenge every product manager faces.  Should you focus on features? Creative packaging and pricing? How about sales execution? Product managers face these choices every day.  Usually, they rely on some partial data, gut instinct, and experimentation.  Product managers can combine MoneyWheel analysis and Win/Loss analysis to get unparalleled insights into what is going on n their markets, prospects, customers, and sales channels.

The Magic 8 Ball

Don’t you wish you had a Magic 8 Ball you could ask how to solve revenue growth problems? Unfortunately, there is no magic cure for revenue growth problems.  Product managers of products that have hit the Early Majority stage of the Technolgy Adoption  Life Cycle.  By this stage the market has been established, there are multiple competitors, product functionality has become standardized enough that customers can use Requests for Proposals (RFPs) to compare and select products.

There are some tools product managers can use to navigate these challenging times.  The MoneyWheel is a tool that can help product managers discover the repeatable patterns of sales transactions that have happened for a product.  Win/Loss Analysis is a qualitative research technique that helps understand why customers decided to purchase or not purchase a product.  When you combine MoenyWheel and Win/Loss Analysis you can develop so,some powerful fact-based insights into your products. The MoneyWheel tells you what happened, Win/Loss Analysis tells you why it happened.

What Is The MoneyWheel?

The MoneyWheel is a tool you can use to understand the repeatable sales transactions that occur in your product’s market.  This knowledge helps you make better decisions about feature prioritization.  You can also use it to make better decisions about investments in demand generation programs.

A MoneyWheel has two components: categories and spokes.  A MoneyWheel category is a broad type of sales transaction.  There are five broad categories:

  • Net New Customers.  These are customers that have not bought any of your products before.
  • Add on Sales.  These are sales of add on products to customers who have already bought a base product from you.
  • Expansion Sales.  Customers buying more units of a product they own, like more authorized seats or more transactions.
  • Competitive Migrations. A special type of new sale where the customer is migrating from a competitive solution to your product.
  • Financial.  Transactions that are specifically financially based and not on the features/functionality of your product.  Enterprise-wide licenses are one example.  Multi-year contracts are another.

Spokes are the specific types of repeatable sales transactions that occur within a MoneyWheel Category.  They are often triggered by an event that occurs at a customer or in a marketplace.  Spokes help explain why a customer purchased something.

For example, here are four spokes for the Net New Customer category for a Marketing Automation SaaS provider:

  • New Marketing Vice President.  When a new VP is hired, they often replace existing marketing automation solutions with vendors they have had success and experience with.
  • Merger/Divestiture.  When a merger occurs, the acquirer often replaces an existing marketing automation solution with the one the acquirer has standardized on.
  • Bad Quarter.  If the customer has experienced a bad quarter from a financial perspective, they will often consider a new marketing automation solution to boost revenues.
  • Competition.  If a customer’s competitors start performing significantly better, companies will often consider upgrading their marketing automation solution to match or exceed a competitor’s performance.

Spokes are unique to your product and market.  While MoneyWheek categories are common across most SaaS companies, most spokes are context-specific.

What Does a MoneyWheel Look Like?

Here is a table that shows a MoneyWheel for a marketing automation SaaS provider:

MoneyWheel
DevelopentCorporate

As you can see, sales to net new customers account for over 75% of the annual contact vale booked in the quarter.  Sales due to the arrival of a new VP or due to the customer experiencing a bad quarter financially account for over 50% of all bookings.  Converse;y, sales of the mobile version of the product or the sale of more seats to existing customers account for only 6% of the bookings but almost 25% of the sales transactions.  This type pf data can be extremely helpful for product managers.

Drill Down Into MoneyWheel Detail

You can extend this analysis to the sales region and even the sales rep level.  Consider this table that describes Q3 for just the Midwest region:

MoneyWheel
DevelopentCorporate

You can identify where the highest-performing reps are succeeding and where the lowest-performing reps are struggling. You can repeat this analysis for the entire salesforce.

You can also slice the data by relative deal size:

MoneyWheel
DevelopentCorporate

Not surprisingly, deals greater than $50K ACV account for over 80% of the ACV but only 54% of the deal count.  Deals below $25K ACV account for less than 5% of the ACV, but over 19% of the total deal count.

You can look at the difference between the top performers and others:

MoneyWheel
DevelopentCorporate

There are dozens of ways to slice data by Product, MoneyWheel category, MoneyWheel spoke, Revenue Tier, Sales Region, and even by Sales Rep.  The goal of the analyses is to give you granular insight into what it working and what needs improving.

It is critical to note that Product Managers should not judge or attempt to coach individual sales managers and sales reps based on MoneyWheel data.  MoneyWheel Analysis is a tool to understand what has happened.  There are a lot of different dynamics that explain relative sales performance that product managers rarely have insight into.  The message is stay-in-your-lane or you will end up alienating a lot of important people.

What is Win/Loss Analysis?

Win/Loss Analysis is a qualitative research technique to help you understand why you won or lost a particular deal. Interviews with people who participated in the decision-making process are the main focus of Win/Loss Analysis.  During the interviews, a series of open-ended questions are asked.  The goal is to understand why the prospect/customer acted the way they did.  The MoneyWheel can tell you what happened, Win/Loss Analysis helps you understand why it happened.

Look at Entire Sales Pipeline

Using the Marketing Automation example from earlier, let’s look at the entire sales pipeline:

Win/Loss
DevelopentCorporate

As the table indicates, the New VP and Bad Quarter spokes account for over half of the total pipeline and have 60%+ win rates.  The Steal Away spoke has low win rates and a small contribution to the overall pipeline.  The  Improper Use Audit.& Enterprise License spokes have decent win rates, but a small contribution to the overall pipeline.

You can extend the analysis by looking at sales region and sales rep performance:

Win/Loss
DevelopentCorporate

You can further look at sales rep / MoneyWheel Category / Spoke performance:

Win/Loss
DevelopentCorporate

There are all types of conclusions you can draw from this type of quantitative analysis.  But you cannot understand why.  This is where Win/Loss Analysis comes into play.

Win/Loss is a Qualitative, Not Quantitative Analysis

Win/Loss is a qualitative type of analysis, not quantitative.  In the case study for this article, there were 245 deals.  To obtain a statistically relevant result (<5% margin of error) you would have to conduct analyze at least 150 deals.  Here is a link to an online calculator for Student’s T-Test where you can calculate how many participants you would need out of a population of 245 deals to yield a statistically significant result (answer: 150).  As we will discuss later, recruiting 150 Win/Loss interviews is a daunting task.

The Win/Loss Process

Win-Loss Analysis is a market research technique companies can use to discover key learnings from customers and prospects. These learnings can then drive improvements in marketing and sales, resulting in more revenue and profits. A typical Win/Loss project consists of five major steps:

1. Planning

The first step of the process is to do all of the preparatory work required to ensure the success of the project. An important first step is setting research objectives. Setting clearly defined Research Objectives will help you to both target your Win/Loss research as well as set expectations for the success of the program.  And be sure to align your Research Objectives with the strategic goals of your organization. There is no sense chasing information from buyers that your organization has no interest in anyways. Research objectives may include:

  • Your Product-Market-Price fit
  • New market problems that your organization can solve
  • Your service levels
  • Persona refinement
  • Buyer purchase-decision process
  • Marketing channel effectiveness
  • Sales process
  • Communication style
  • A better understanding of your place in the competitive mix

In addition to establishing your Research Objectives, you should also define the questions that will be asked during the interviews. The questions should be open-ended and designed to encourage a free-flowing discussion.

Finally, you will need to design the process you will use to recruit potential interviewees, conduct the interviews, analyze and report the results you have learned. The plan should lay out the roles and responsibilities of each participant in the project. You should also develop a schedule and a simple status reporting mechanism so everyone can stay up to date. Create a project charter that contains the research objectives, interview questions, roles, responsibilities, and schedule. Conduct a review meeting with the participants to approve and commit to the plan.

2. Recruitment

The next step in the process is to recruit people for the interviews. Most companies target a mix of net new customers and existing customers who have upgraded or expanded their use of products. You are looking for a mix of won deals and lost deals. Most Win/Loss Analysis projects with a single research objective try to get 15 to 20 interviews.

A portfolio approach to soliciting interviews works best. The tactic that has the highest success rate is when a salesperson makes a personal outreach to a potential interviewee. Generally, 25% to 40% of these contacts will convert to an interview. The second most common approach is email. Organizations pull a list of candidates from their CRM or sales force automation system and then email them asking for participation in the project. This approach performs like most email campaigns – a 20% open rate and a 2% to 5% click-thru rate. 50% of those who click-thru convert to interviews. The final tactic is to do phone follow-up with contacts that opened the email but did not click thru. This tactic performs like telemarketing campaigns – a 1% to 2% success rate.

Most Win/Loss Analysis programs use incentives to encourage targets to commit to doing the interview. Companies offer a $25 or $50 gift card. This provides the interviewee with something tangible in exchange for their valuable time.

The biggest challenge many Win/Loss Analysis projects face is recruiting enough quality interviewees. Sending cold non-personalized emails to all of the CRM contacts associated with a customer tends not to be successful. What happy customer wouldn’t want to do a short interview and get a $50 VISA gift card? A lot fewer than you would expect. Experience has shown that a portfolio approach that combines warm introductions, multiple emails, and phone call follow-ups works best.

Sometimes when a Win/Loss Analysis project has problems in getting sufficient interviews to reach the research objectives. Teams will then increase the interview incentive to $75 or even $100. While this may spur more people to respond to the solicitation, often these individuals are only interested in the incentive payment. These individuals may only have limited exposure to the product and may not have been involved in the actual sales process. This risk can be reduced by selecting the best candidates to target with the solicitation. Sending an email blast to every contact from a company in your CRM system will not produce the quality results you are looking for.

3. Interviews

The core activity of the project is conducting interviews. Most interviews take 20 to 30 minutes to complete. Companies use internet meeting services that facilitate the interview scheduling process, but also allow the discussion to be recorded. This enables the interviewer to focus on the discussion instead of trying to listen and take notes at the same time. There are services that will transcribe the recordings for you into a Word document for about $1 minute.

Experience has shown that interviewees are more comfortable talking to an independent third party instead of a representative from the company. This results in a free and easy discussion. It also avoids making embarrassing or disparaging comments about their experiences or opinions.

A critical aspect of the interview process is asking why a customer believes certain things they say. Surveys and checklists are one way to get customer feedback, but they lack the ability to follow up on interesting statements. The real value of using experienced interviewers is that they can follow up and explore why a customer believes specific things. Often this is the most valuable outcome from the interviews.

4. Analysis

After the interviews are completed, the recordings are transcribed. Next, the team reviews the transcripts to identify common themes. These themes are analyzed and documented in a final report. The report contains a summary of the interviewees – company size, interviewee title, transaction type (new/upgrade and win/loss). For each theme or finding, specific quotes from the interviews are included. This lets the report’s reviewers hear, from the customer’s perspective and in their voice, the exact point they were trying to make. A meeting is held with all of the interested internal organizations to review the report’s findings and conclusions.

5. Action

The final step is to take action on the recommendations. Effective Win/Loss Analysis programs are really part of a cycle to drive improvements in the business. Win/Loss Analysis is a variant of the Six Sigma DMAIC (Define, Measure, Analyze, Improve, and Control) methodology. If action is not taken based on the recommendations from the interview analysis then an opportunity to fundamentally improve your business will be missed.

Win/Loss Themes

Themes are the learnings distilled from the Win/Loss interviews.  Here are some examples from the Marketing Automation Saas provider: For reference, the Marketing Automation SaaS provider had been in business for over eight years.  They had over 3,500 active customers.  Top line revenue was around $50 million.  They were VC-funded, having recently raised over $20 million in a Series C round.  Most of their inbound leads came from the web.  Their product was considered to be a market leader in terms of functionality.

Fifteen interviews were conducted from ‘win’ transactions.  5 were from net new customers, 10 were from existing customers.. 6 were from mid-market organizations with less than $100 million in revenue. * were from enterprises with revenues between $100 million and $2 billion.  The major themes included:

Positive Themes from Interviews

Thought Leadership.  The vendor was considered to be a thought leader in the industry.  Almost all interviewees cited the quantity and quality of thought leadership articles the vendor published.  The readers liked that the vendor did not push their products in these articles.

Functionality.  All the interviewees cited the depth and quality of the product’s features and functions.  Many felt that competitors fell short in a few key areas.

Sales Team Effectiveness.  Many interviewees cited the professionalism, responsiveness, and courtesy of the sales teams.  They did not feel pressured or subjected to typical software sales techniques (buy before the end of the quarter to avoid a price increase).

Analyst Reports.  Many interviewees cited the inclusion the Leaders Quadrant in Gartner Gruop’s Marketing Automation Magic Quadrant as being important.

User Reviews.  Some prospects/customers cited the multitude of reviews on user review sites like G2.com and Capterra

Negative Themes from Interviews

Price.  Most interviewees considered the price for the SaaS solution to be among the highest in the industry.  Many competitors had better pricing.

Packaging/Commercial Terms.  

The majority of the ten existing customers cited dissatisfaction with the product packaging.  Most were satisfied with the entry-level offering but were very dissatisfied with upgrade options.  Many felt that the upgrade packages included add-on products that they saw little to no value in.  The pricing was based on the number of modules purchased and a transaction limit.  The entry-level offering had three modules and 10,000 transactions a month.  Transactions above 10,000 were billed at  $0.05 a transaction.  The smallest upgrade was four times the cost of the entry-level package.  The upgrade included six additional modules and 25,500 transactions a month.  Customers rejected the upgrade 80% of the time.  They saw little value in the extra modules and rejected the effective per-transaction price increase.  Many customers moved to a solution that offered significantly lower transaction costs but also had lesser functionality.  They accepted the trade-off.

Change in Priorities.  Many loss transactions were due to a change in customer business priorities.  The need to focus on another issue was more important than investing in the vendor’s solution.

No Perceived Differentiation.  The prospect was unable to see a real difference in value between competing solutions.  While competitive solutions had clearly lower levels of features/functions, the prospects were unable to see any real difference.  Lower prices trumped what appeared to be better functionality.

Disruptive Events.  Events like mergers, acquisitions, divestitures, or departures of key executive sponsors often delayed or canceled planned purchases.

No Decision.  The most common reason cited fr lost deals was No Decision.  Either the prospect/customer never made a decision or the vendor was unable to determine why the sales process stopped.

Combining the MoneyWheel & Win/Loss Analysis

By combining the results of MoneyWheel Analysis with Win/Loss Data, product managers can get unparalleled insights into their markets and their own company’s performance. For example, you can look at the reasons why customers decided to purchase in specific categories and spokes:

MoneyWheel & Win/Loss
DevelopentCorporate

Or you could drill into specific deal sizes, MoneyWheel categories, spokes, and win/loss reasons:

MoneyWheel & Win/Loss
DevelopentCorporate

You can even drill down into individual sales reps:

MoneyWheel & Win/Loss
DevelopentCorporate

By combining MoneyWheel and Win/Loss Analysis you can develop very targeted recommendations to grow revenue.  

A challenge all product managers face is developing actionable recommendations to improve their products and the success of their companies.  This type of fact-based analysis can be a big help.  While it is qualitative in nature and not necessarily statistically accurate with a 5% margin of error, it can provide significantly better insights than most current approaches.

Summary

Growing revenues is a challenge every product manager faces.  The combination of MoneyWheel Analysis and Win/Loss Analysis can provide product managers with unparalleled intelligence.  It can help them make better fact-based decisions about feature prioritization, demand generation, and sales campaigns.


You can download the Excel file containing the base data and pivot tables for the case study here


Also published on Medium.

By John Mecke

John is a 25 year veteran of the enterprise technology market. He has led six global product management organizations for three public companies and three private equity-backed firms. He played a key role in delivering a $115 million dividend for his private equity backers – a 2.8x return in less than three years. He has led five acquisitions for a total consideration of over $175 million. He has led eight divestitures for a total consideration of $24.5 million in cash. John regularly blogs about product management and mergers/acquisitions.