Stepping up Your A/B Tests

Mind the Product

This is the first of two posts that deep dive on A/B testing, expanding on a talk I gave at Google Playtime 2016 in London. In this post I share some of the learnings we’ve had after running 60+ A/B tests at Peak , looking at each step of the A/B testing cycle in turn.

How to Hook Users in 3 Steps: An Intro to Habit Testing

Nir Eyal

Changing user habits isn’t easy — but understanding how to conduct Habit Testing will increase your odds of success. In this video, I provide a brief introduction to the three steps of Habit Testing.

Trending Sources

Why an MVP Needs to Be a Product and Not Just a Test

280 Group

The post Why an MVP Needs to Be a Product and Not Just a Test appeared first on 280 Group Product Management. Language is dynamic, and terms in product management are subject to the same forces. Meanings evolve, and this can lead to ambiguity.

How to Sell Your Boss on Roadmaps Without Timelines

Mind the Product

As product managers, we instinctively know that planning things out in advance is impossible, because our work depends on a constant stream of testing, data and feedback. Show Them What you Lose When you Give up Validation and Testing.

Using Experimentation to Drive Product – Stephen Pavlovich (CEO of

Mind the Product ran a test version of a product where they created the levels by usage rather than functionality. The participants don’t even realise they’re in a test and so biases that invalidate the outcomes can’t be established.

Level Up Your Product Game Using Empathy

Mind the Product

Jeremy takes some time out from our often highly data-driven testing culture, to show how employing empathy drives you to build products people love , not just stuff they use. A/B Tests and MVPs. Product Design ProductTank Video A/B testing empathy mvp personas user research

Q&A on User Testing

Under10 Playbook

Most product managers recognize the value of user testing. So it is often hard to fit in testing and design iteration before the sprint deadline. Steven Cohn , the Founder of Validately and a user testing expert, gives some practical advice on how to balance the time pressures of product management and conducting proper user testing. (SJ Why should a product team allocate its limited time to user testing? SJ: What are the different testing options?

Your survey data is flawed, and what to do about it

Product Beautiful

“…Among 1008 surveys, their test flagged 17% as likely to contain a significant portion of fabricated data. Fortunately, the researchers, Michael Robbins and Noble Kuriakose, point to the likely culprit: “The basis of the test is the likelihood, by chance alone, that two respondents will give highly similar answers to questions on a survey.

Sprint: Solve problems and test ideas in five days

Product Bookshelf

The process moves through identifying questions, creating solutions, storyboarding, prototyping, and testing with customers. Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days by Jake Knapp. Over the course of a week: ideate and flesh out, prototype, and test it.

The Ladder of Evidence: Get More Value From Your Customer Interviews and Product Experiments

Product Talk

But no matter how good you get at collecting specific stories, you still have to test those insights. Too often, when we are ready to test a solution, we put it in front of people and ask, “What do you think?” In this case, you can use existing solutions to test your idea.

Managing Product Teams for Success

Product Talk

The most important habits I like to see are a regular cadence of customer interviews , prototype tests, and assumption tests. Good product teams have a regular cadence of customer interviews, prototype tests, and assumption tests.

When should Product Backlog Grooming Take Place?

Roman Pichler

Note that this approach still assumes that you can collect the relevant feedback in the sprint review meeting, for instance, by carrying out a product demo, usability test, or solution interview. Option 1: In the Sprint Review Meeting.

How Two Companies Hooked Customers On Products They Rarely Use

Nir Eyal

It’s called “The Toothbrush Test.” Larry Page, CEO of Alphabet (the company formerly known as Google), has a quirky way of deciding which companies he likes. According to the New York Times, when Page looks at a potential company to acquire, he wants to know if the product is, like a toothbrush, “something you will […]. The post How Two Companies Hooked Customers On Products They Rarely Use appeared first on Nir and Far.

3 Best Practices for Adopting Continuous Product Discovery

Product Talk

They interview customers , run usability tests, and conduct A/B tests. Hypothesis Testing. And you can use the other half to test a couple prototypes. Because Frank’s team already had designs, they just started by prototype testing their concept designs.

Getting the Band Together: How to Get More Development Resources as a Product Manager


A common challenge that Product Managers face is trying to move their product and company forward with limited development and testing resources.

Andrew Harder – The Seven Best Ways to Screw Up Your User Research

Mind the Product

Andrew Harder, Senior Research Manager at Ebay, spoke to Product Tank San Francisco about how to avoid screwing up your user testing. Andrew spoke about his experience at Nokia: throughout user testing their customers said that text input was very important, leading Nokia to dismiss the first iPhone. Testing to get Blessed. Testing must be an integrated part of the whole product process. Test to get coached: use testing to experiment, learn, and iterate.

Product Manager vs. Product Owner

Roman Pichler

They would then hand off a requirements specification to a project manager who would work with development and test to deliver the product. As you may know, the product owner originated from Scrum, where the role is responsible for maximising the value the product creates.

How to Be Customer-Centric at Scale


Your world is all about interviews, a/b tests, and nailing a value proposition and user experience that resonates with the market. When you start building a product, you are obsessed with “the customer.”

Is Scrum Right for Your Product?

Roman Pichler

Scrum is a great fit at this stage: Your product will benefit from an iterative, cyclic process that allows you to quickly test an assumption, address a risk, generate new insights, and come up with new ideas. When is Scrum Most Helpful?

UX 102

The T-Shaped Product Manager

Roman Pichler

Taking my product management test helps you with this. Test competing offerings. Balancing Specific and Generic Skills. To do a great job as a product manager or product owner, you require two skills sets: product-specific and generic ones.

The PM’s Toolbox: Product Management Tools and the Problems they Solve


When it comes to studying user-experience and their interactions with your product, UserTesting comes highly recommended due to its robust set of usability-testing functions. It’s no secret that product managers wear a lot of hats.

10 Product Backlog Tips

Roman Pichler

Addressing uncertain items early on allows you to test your ideas, to fail fast, and to learn how to continue. Tip #1: Complement your Product Backlog with a Product Roadmap. Use a roadmap to sketch the overall journey you want to take your product on.

Coping with Product Manager Burnout

280 Group

I could actually run tests to see immediately if.[continue Vision Built on Uncertainty When I started my career as a young programmer I was tasked with bug fixing as a way to learn the code base….I I hated it. It wasn’t challenging enough for me.

Scaling the Product Owner Role

Roman Pichler

The feature and component owners manage their assets: they capture new ideas and requirements, test them using feedback and data, and collaborate with the teams developing the features and components. Scaling and the Product Life Cycle.

How to Prioritize Product Backlogs in Three Easy Steps

280 Group

Prioritize Product Backlogs in Three Easy Steps Using effective, time-tested techniques to prioritize product backlogs is an important step in getting a product to market rapidly that meets customer needs.

Transitioning from Scrum to Kanban

Mind the Product

Our teams were already experienced in Agile development and were working in two week sprints, with a regression test and proposed release at the end of each sprint, but the business stakeholders were still concerned about the speed of delivery.

Agile 93

The Evolution of Modern Product Discovery

Product Talk

Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days. If we build in usability testing earlier in the development cycle, we are able to answer, “Can customers use it?” It’s helping us test if our solutions deliver on our desired outcomes.

10 Writing Tips For Product Managers

PM Hardcore

Include the criteria for determining if the spec is satisfied, or the feature is “done” – i.e., acceptance tests. Tips for writing. (CC CC BY-SA 2.0 by jvleis). I’ve started working on an online course on “writing for product managers.”

Mindfulness Tips for Product Managers and Product Owners

Roman Pichler

Here is a simple test: Close your eyes and gently observe your breath. And we will struggle to run effective experiments and test new ideas, as we will be tempted to either reject any data that doesn’t support our view, or interpret the test results so that they confirm our opinion.

Why This Opportunity Solution Tree is Changing the Way Product Teams Work

Product Talk

It’s easy for teams to work along both dimensions of product discovery—working to understand their customer’s context and iteratively testing their ideas—but that doesn’t mean they know how to connect the two sets of activities. We don’t want to test every idea we have.

Ways to Define Success and Measure Value for your Product Features

bpma ProductHub

Erin McHugh Saif showed us how she used analytics and customer insights to help her tackle a large-scale business problem at TIBCO Spotfire and Sooyoung Kim shared how Rue La La is using Riskiest Assumption Testing to rapidly test ideas with zero engineering effort.

Why You Are Asking the Wrong Customer Interview Questions

Product Talk

Your first list probably includes things like interview customers, run usability tests, look at metrics, get feedback from stakeholders, a/b test it, release it only if it beats out the control. I recently asked a woman what factors she considered when buying a new pair of jeans.

The Playbook for Achieving Product-Market Fit

Mind the Product

My Lean Product Process guides you through how to articulate and test the key hypotheses by working your way up the Product-Market Fit Pyramid. Test your MVP with customers. We tested it with customers and it did not do well. Why Products Fail.

Agile at Scale – Outcome Driven (or Broken)

Tyner Blain

It was more than just incremental delivery, we really did incorporate agile principles like pairing and demoing, and engineering practices like automated build and test processes, code-reads, design for testing.

Agile 51

The Rise of Modern Product Discovery

Product Talk

Continuous integration reduces the risk of large code conflicts that used to take days or sometimes weeks of testing to resolve. Technology teams did little to test their assumptions about what the business needed, which resulted in solutions that didn’t quite work. Or a usability test?

How Minimum Viable Products & Features Helped Me Write My New Book

Roman Pichler

At the same time, I wanted to test that the product would create enough value for its readers before writing the actual book. A minimum viable feature (MVF) is similar to an MVP: it tests an idea and acquires relevant knowledge. Minimum Viable Product #1.

Books 36

Why You Aren’t Learning As Much As You Could From Your Experiments

Product Talk

I love that A/B testing has become so popular. Most teams make one of the following two mistakes: They either gate releases with an A/B test trying to understand whether or not the feature had the intended impact or they test variables blindly hoping to stumble upon a better variation.

Mental Models for Product Managers – Part 2

PM Hardcore

Brain Wiring (by Wellcome Images, CC licensed). In part 1 I introduced mental models and some reasons they are important. And I provided a few “general purpose” examples. In this part we dive into what you really came here for – product management-specific mental models.

What is a Digital Product?

Roman Pichler

Product. What is a product ? It might be tempting to say, something we can market or sell. But when it comes to digital products, this definition has only limited applicability. Take the search function on your company’s website. Is that a product? Or is the entire website the product?

The life of a Product Manager – Learning by Doing

Mind the Product

Not only did they have to challenge themselves why this was the case but they were also planning on using their internal employees as beta users to test their algorithms. They had tested MVPs but in safer environments and so faced unexpected challenges in Thailand. Test retention.