Measure What Matters
How your measurement strategy can reduce OpEx
Nidhi, Geordie and I conducted a three hour tutorial at O’Reilly’s Strata London yesterday. Thanks to all of you who attended “Measure What Matters: How your Measurement Strategy can Reduce OpEx” — we enjoyed the facilitating the session with a very engaged audience.
It was an interactive session where we presented elements of Radical Product thinking and interleaved it with exercises so each person could develop a measurement strategy for their product/ organization, that was grounded in their vision and strategy. The session started with crafting the “source code” for your Vision and an RDCL Strategy that was aligned with that vision. Each person then translated one Initiative from their Strategy into execution by creating a cross-functional Strategic Roadmap for that one Initiative and then developing an Execution and Measurement model including Metrics and Hypotheses for that Initiative.
Slides and key learnings:
Many of you asked for the slides, so here is a link to the presentation. Below are your answers when we asked at the end, “What was your “Aha!” moment from this session?”:
“Even in a big company, I can bring product thinking to make it feel more like a startup”
One person shared that while his workplace felt like a bureaucracy, he felt like this Radical Product approach would make it more nimble. He was looking forward to bringing product thinking into his organization to align teams on a clear vision and strategy so they would make faster progress. The organization as a result would feel more like a nimble startup.
“I can run Lean and Agile and be very responsive, without compromising on the vision + strategy.”
A vision-driven approach and a fast feedback-driven execution are complementary. The responsiveness made possible by Lean and Agile approaches, works well when grounded in a clear vision and strategy. Lean and Agile used in the absence of vision and strategy looks more like the “move fast and break things” model and often leads to product diseases.
“I can cure Hypermetricemia and Narcissus Measurement Complex!”
We talked about the product diseases each of us have experienced at the beginning of the session: Strategic Swelling, Obsessive Sales Disorder and Pivotitis. In this session we also talked about two measurement-related product diseases:
Hypermetricemia: Obsession with metrics and analytics which usually manifests as measuring things based on popular metrics, rather than driving your metrics off your unique strategy.
This leads to incremental product improvements that don’t move the needle or getting stuck in local maxima.
Narcissus Measurement Complex: Metrics focused internally, often for vanity rather than actionability.
Some of the attendees who did a self-diagnosis exercise, shared that they were looking forward to curing these diseases by applying Radical Product thinking.
“I can use the shared language of Vision Debt to discuss priorities and push back when needed”
We talked about the Vision Fit vs. Sustainability prioritization rubric that teams can use to evaluate and prioritize opportunities and features. It introduced the concept of Vision Debt, the business counterpart to Technical debt. Where Technical Debt is a measure of shortcuts that deviate from building a technically robust product, Vision Debt measures mitigating short-term risks at the expense of deviating from the vision. You can occasionally take on vision debt to mitigate existential threats, but accumulating an excess of vision debt leads to confused customers, demotivated teams, and a directionless product. Discussions around prioritization and evaluating opportunities often strike at the heart of why everyone is in the building in the first place, and can be very emotional. Using this shared language of Vision vs. Sustainability you can help focus, clarify, and structure your internal discussions, collaboratively identify vision debt, and create a plan to pay it back.
“When metrics are based on vision and strategy, they measure progress towards achieving the vision”
When metrics are requested top down, often just before a board meeting, or there’s a policy of “We measure everything”, the team becomes disconnected from the rationale for tracking those metrics. This makes it easy to fall into the trap of making iterative product changes that don’t move the needle. Instead, when a team shares a vision by going through the Vision exercise, has a clear RDCL strategy and an Execution and Measurement Model (including hypothesis-driven metrics), the metrics become indicators of progress towards the vision for the team.
We were excited to see the crisp vision statements that resulted from this session and the enthusiasm to take this strategic product thinking back to your teams in high tech, research, non-profit and government. We look forward to hearing about your experiences as you apply Radical Product thinking in your organization!
Radical Product is a movement that provides a methodology for strategic product thinking, in a similar way that Lean and Agile provided a methodology for feedback-driven execution. Use the free and open source Radical Product toolkit to bring Radical Product thinking into your organization.
