Listen now on Apple, Spotify, and YouTube.
Brought to you by:
• Explo—Embed customer-facing analytics in your product
• Dovetail—The customer insights hub for product teams
—
Archie Abrams is the VP of Product and Head of Growth at Shopify, where he leads a 600+ person growth org across product, design, engineering, data, ops, and growth marketing. Shopify powers over 10% of e-commerce in the United States, with $235 billion in GMV in 2023 (roughly the size of Finland’s economy). He previously led Consumer product and growth at Lyft and was at Udemy for 8 years as SVP of Product having joined the company when it was 10 people. In our conversation, we discuss:
Why Shopify optimizes for churn
Why the core product team doesn’t use metrics-based goals
Why they keep multi-year experiment holdouts
How they structure their growth team
The benefits of not having a CMO
Lessons learned about integrating sales into a product-led growth model
The power of discounting as a growth lever
Much more
Some takeaways:
Shopify optimizes for getting as many new merchants as possible to start businesses, even if many of them fail. This approach works because the few successful merchants generate enough revenue to make up for the many that don’t succeed.
Shopify has found that 30% to 40% of experiments that show positive short-term results have no long-term impact. Initial lifts can be misleading, and some of your “losers” might actually yield unexpected long-term value.
Adopt a “hundred-year mindset” in your decision-making. Stop chasing short-term wins that feel good now but might sabotage your future. Every decision should be about building a product or service that can withstand the test of time. If it feels like a quick buck, it probably isn’t worth it.
Don’t shy away from shipping experiments that may have neutral impacts. If your intuition suggests that an idea is beneficial, validate it by launching it. Just because the initial data doesn’t show a positive lift doesn’t mean it won’t create value in the future. Let the market respond, and be open to adjustments based on real user feedback.
Regularly assess elements of monetary friction in your product—like trial lengths and incentive structures. Make sure these elements align with what users need to succeed, as they can greatly influence user behavior and long-term retention.
Shopify’s growth team is divided into two main groups: Growth R&D (product, design, engineering, data) and Growth Marketing (paid acquisition, SEO, email, content). The company also uniquely includes customer support within the growth organization. This clarity helps teams align their goals and understand their unique contributions to the overall growth strategy.
Where to find Archie Abrams:
• X: https://x.com/archieabrams
• LinkedIn: https://www.linkedin.com/in/archie-abrams-b6aa8b6/
In this episode, we cover:
(00:00) Archie’s background
(02:30) Shopify’s impressive growth
(06:17) Shopify’s unique approach to churn and retention
(08:43) Monetization model and success metrics
(11:08) Long-term experimentation and metrics
(23:00) Examples of big wins that Archie’s team has shipped
(26:42) Monetary friction
(27:14) Metrics
(29:47) Shopify’s growth team structure
(33:03) Goal setting and forecasting
(37:10) Examples of long-term results within Shopify
(41:36) Shipping neutral experiments
(42:05) Building a hundred-year company
(48:04) Why Shopify doesn’t use KPIs
(51:30) Shopify’s “Get shit done” framework
(54:30) Cross-team collaboration
(58:48) The importance of an opinionated founder
(01:01:12) Growth and sales integration
(01:06:42) Shopify’s marketing structure
(01:08:49) Insights on discounting from Udemy
(01:11:09) Lightning round
Referenced:
• Shopify: https://www.shopify.com/
• Tobias Lütke on LinkedIn: https://www.linkedin.com/in/tobiaslutke
• Gross Merchandise Value: Calculation and Best Practices: https://www.shopify.com/retail/gross-merchandise-value
• Brian Chesky’s new playbook: https://www.lennysnewsletter.com/p/brian-cheskys-contrarian-approach
• Glen Coates on LinkedIn: https://www.linkedin.com/in/glcoates
• Harley Finkelstein on LinkedIn: https://www.linkedin.com/in/harleyf
• Udemy: https://www.udemy.com/
• Scientific Advertising: https://www.amazon.com/Scientific-Advertising-Original-Claude-Hopkins/dp/1640954252
• Four-Minute Mile: https://www.amazon.com/Four-Minute-Mile-Roger-Bannister/dp/1493038753/
• The Sopranos on HBO: https://www.hbo.com/the-sopranos
• Suno: https://suno.com/
Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email podcast@lennyrachitsky.com.
Lenny may be an investor in the companies discussed.
I loved this conversation between Lenny and Archie—Shopify’s approach to experimentation is both fascinating and incredibly unique. I think it’s worth highlighting how Shopify’s dynamics likely mean that their high rate of false positives on tests wouldn’t apply to most SaaS companies.
Two key factors make Shopify’s situation so different:
1) High churn due to entrepreneurial failure rates: As Archie pointed out, Shopify’s mission to lower the barriers to entrepreneurship means that many merchants try but don’t succeed, regardless of Shopify’s value. This high churn introduces variability that can skew the results of experiments—short-term wins may just reflect temporary behaviors rather than lasting impact.
2) Power-law dynamics in cohort success: Shopify’s cohorts are driven by a few merchants who generate the majority of GMV and profits. Archie’s analogy to venture capital was spot on: these big winners create outsized results that can make short-term metrics unreliable.
For most SaaS businesses, this isn’t the case. They tend to have more consistent retention and less variability between their most and least valuable customers—particularly those customers that reach the "aha moment." Because of that, they’re less likely to see as many false positives from statistically significant tests.
I love how Shopify’s long-term holdout experiments align with their unique model, but for other companies, this level of scrutiny may not be as necessary. Thanks for such an insightful discussion—this really got me thinking about how growth strategies should be tailored to a company’s specific dynamics!
Thanks for sharing Archie. Loved the really long hold out and the 100 year vision! Go Shopify!
Love it that you optimise for helping entrepreneurs get started time and time again.
Also agree the 4 Minute Mile is a great book. My old office was next to where the track was in Oxford and you could feel the magic in the air!