Stepping up Your A/B Tests

Mind the Product

This is the first of two posts that deep dive on A/B testing, expanding on a talk I gave at Google Playtime 2016 in London. In this post I share some of the learnings we’ve had after running 60+ A/B tests at Peak , looking at each step of the A/B testing cycle in turn.

How to Hook Users in 3 Steps: An Intro to Habit Testing

Nir Eyal

Changing user habits isn’t easy — but understanding how to conduct Habit Testing will increase your odds of success. In this video, I provide a brief introduction to the three steps of Habit Testing.

Tracking your A/B tests

Mind the Product

This is the second of two posts that deep dive on A/B tests, expanding on a talk I gave at Google Playtime 2016 in London. As mentioned in the first post, if you plan to run a lot of A/B tests then you should think about using a document to store test information, which I call the tracker.

The Importance of Accessibility Testing


Industry leaders learned that accessibility testing is an innovation driver for solutions to unanticipated problems Click To Tweet. How to run an accessibility test in UserZoom. The post The Importance of Accessibility Testing appeared first on UserZoom.

3 Tests for Great Product Messaging

280 Group

Fortunately Bill is here to provide us with three tests that.[continue The post 3 Tests for Great Product Messaging appeared first on 280 Group Product Management. In this new video and blog post we discuss something that is exceedingly important in the product world: messaging.

Mobile Testing on iPhoneX – What Developers Need to Know

Amir Rozenberg

Testing your app on the iPhoneX presents even more difficulties. If you cannot see the real-time website or app in your testing, you will find issues late, which will require you to redo the code you wrote, possibly undo code that’s built on top of it, and fix it all.

Why an MVP Needs to Be a Product and Not Just a Test

280 Group

The post Why an MVP Needs to Be a Product and Not Just a Test appeared first on 280 Group Product Management. Language is dynamic, and terms in product management are subject to the same forces. Meanings evolve, and this can lead to ambiguity.

5 Cross-browser Testing Mistakes You Might be Making Right Now

Amir Rozenberg

So, you’d like the recipe for cross-browser testing success? Let’s take a look at a handful of examples of mistakes we commonly see with browser testing- and solutions to deal with them. Engineering should no longer separate coding and testing between desktop and mobile.

4 Key Data Points A/B Testing Reveals

Generation Focus Blog: Perspective on Customer Dis

A/B testing is a crucial element in any startup foundation. Six months later an engineer saw the project, and noticing the low cost of writing the code for it, decided to do a bit of A/B testing. Thanks to the test, Microsoft went on to earn massive returns.

Multivariate Testing vs A/B Testing: Which Works Better?

UX Studio

Deciding between multivariate testing vs A/B testing without knowing their advantages and limitations can pose extreme challenges. When optimizing a website’s conversion rate, A/B testing usually comes up first. Sometimes testing more alternatives proves necessary, for which multivariate testing performs well as a method. So which to choose, multivariate testing or A/B testing? What is A/B testing? How can you set up a test?

How to Sell Your Boss on Roadmaps Without Timelines

Mind the Product

As product managers, we instinctively know that planning things out in advance is impossible, because our work depends on a constant stream of testing, data and feedback. Show Them What you Lose When you Give up Validation and Testing.

3 Things I Learned About Usability Testing By Looking at Customers in Their Underwear

Joe Cotellese

Back in my days with the apparel company, the inescapable fact was that I needed try our fit-testing body scanner on real humans. Today, working for an email marketing company, I rarely have to ask people to strip down to their underwear to test out a product. Still, that awkward experience from many years ago taught me several important things about usability testing. During our fit testing experiment, we had one guy come in who swore he had a 32” waist.

Rebooting Product Usability Testing for Much Better Insights

bpma ProductHub

Usability Testing – Something Has Always Seemed Off… Something has never seemed quite right to us about how product usability testing is done. No matter what is being tested, from software to cereal to screwdrivers, there is always a herd of elephants – in the room.

How to prepare your Think-Out-Loud Study in UserZoom


The Agile Usability Test Think-Out-Loud, or TOL for short, uses a task based structure to record a user’s screen and their audio. With Advanced UX Measurement you can test a variety of methods all-in-one! Helpful tips to start collecting think out loud feedback from participants.

Agile 28

How Do You Test Automated Data Entry for Mobile Web Apps? A Tip We Gave to Our Customer

Amir Rozenberg

An end-to-end approach for testing mobile web app imaging is made possible through leveraging preloaded images onto the device. This test automation, in turn, enables rapid feedback and scaling of code coverage.

Perfecto Automates Face ID Flows for iPhone X

Amir Rozenberg

All of these changes require integration of new APIs, rehash of layout (specifically considering the notch), and lots of testing. Let’s also keep in mind, developers have no time to conduct this testing: iPhone X adoption is happening ( Source ), and applications have to be ready.

The next generation automation testing for video streaming apps by Perfecto & Accenture S3

Amir Rozenberg

Teams need automated, stable and scalable methods to continuously test and monitor video services on every phone, tablet and browser and in any environment; at home, on the road, inside a plane, etc.

The Typology of Design Sprints

Mind the Product

At it’s core, a design sprint is a tool for answering a critical business question through design prototyping and testing with users. When possibilities seem too wide to move forward, a sprint can help prioritize and test out what one of the directions might look like “in action”.

UX 60

How UX Booth Redesigned Their Homepage With UserZoom


As the team discussed our rollout plan, we all agreed that testing the design before launch would be the right thing to do—we are, after all, focused on user-centered design. The problem was doing testing in a timely manner, while still gathering enough information to make sure our new homepage would provide a fantastic experience for our readers.” Testing a new design should be easy & provide valuable feedback.

UX 19

What Should Product Managers Do When Their Main Product Fails?

The Accidental Product Manager

They claimed that they had invented a product that with just a minute amount of blood from an individual could run a large number of health tests. This blood testing device was called Edison. The whole idea behind the company was to try to find a way to create a product development definition that would standardize the wide range of blood tests that are commonly performed by creating a novel new hand-held device.

Using Experimentation to Drive Product – Stephen Pavlovich (CEO of

Mind the Product ran a test version of a product where they created the levels by usage rather than functionality. The participants don’t even realise they’re in a test and so biases that invalidate the outcomes can’t be established.

Separate Quality Testing from Availability

Under10 Playbook

Quality testing identifies anomalies or defects that are not working as expected. “Alpha” is a testing phase where a release candidate (or potentially shippable increment) is tested on client systems (such as devices and browsers) and data by VENDOR employees.

10 Ways to Use Feature Flags

Mind the Product

By offering higher risk features to certain groups of power-users or opt-in users only, you can test your new feature on the groups who know and love you best. ProductTank San Francisco Video developers Product Design Product Development Software user testing

Why Data Science and UX Research Teams are Better Together

Mind the Product

Chris shares real-world examples and lessons learned as User Testing established its Product Insights team, a team which is made up of UX researchers, data scientists and data engineers. User Testing’s research and data science teams working together.

UX 30

Daniella Patrick – Innovation Lab Product Development

Mind the Product

At Accenture, Daniella is a part of a design team whose primary goal is to research, ideate, prototype, and test solutions to improve the experience of their candidates, employees, and alumni. Daniella’s team operates with a process of ideating, prototyping, testing, and learning. Testing means they select their population, define their success and failure, and carry out the experiment. As a Product Manager, Daniella’s first lesson she learned is to user test continuously.

4 Battle-Tested Ways to Improve Corporate Learning

Good Product Manager

Here are four battle-tested tips to improve corporate learning so your talent is ready for the technology-driven future that seems to be coming their way: #1. The post 4 Battle-Tested Ways to Improve Corporate Learning appeared first on LSA Global.

Tactics for Redesigning and Relaunching Legacy Products

Mind the Product

Test if you have time: Some people put out beta versions in advance or launch new versions to partial audience segments. Two examples of forgetting missing pieces are not stress-testing code, since new code is often not battle-tested, and not redirecting old links properly.

Naimish Gohil – Adapting Your Product for a Changing Market

Mind the Product

In order to get to this stage, they took the approach of any good product manager – they tested repeatedly to learn what worked best and iterated. Product Management Process User Experience (UX) User Research User Testing Video adapt B2B edtech education Emerging Business Models iterate London Product/Market Fit ProductTankNaimish Gohil , Founder of edtech software company Satchel , shares his experience of knowing your user and adapting your product with ProductTank London.

Level Up Your Product Game Using Empathy

Mind the Product

Jeremy takes some time out from our often highly data-driven testing culture, to show how employing empathy drives you to build products people love , not just stuff they use. A/B Tests and MVPs. Product Design ProductTank Video A/B testing empathy mvp personas user research

Q&A on User Testing

Under10 Playbook

Most product managers recognize the value of user testing. So it is often hard to fit in testing and design iteration before the sprint deadline. Steven Cohn , the Founder of Validately and a user testing expert, gives some practical advice on how to balance the time pressures of product management and conducting proper user testing. (SJ Why should a product team allocate its limited time to user testing? SJ: What are the different testing options?

UX 2

Your survey data is flawed, and what to do about it

Product Beautiful

“…Among 1008 surveys, their test flagged 17% as likely to contain a significant portion of fabricated data. Fortunately, the researchers, Michael Robbins and Noble Kuriakose, point to the likely culprit: “The basis of the test is the likelihood, by chance alone, that two respondents will give highly similar answers to questions on a survey.

Betterment Tested Three Performance Management Systems So You Don't Have To

First Round Review

As Founder and CEO of Betterment, Jon Stein realized startups can't just establish one way to measure employee performance — they have to continually iterate as they grow. Here's how to do it well. Continue reading at First Round Review »

Increase Test Automation with Touch ID & Fingerprint Authentication Simulations

Amir Rozenberg

The test flow needs to include a successful authentication,an unsuccessful one with limited retries, and eventual failure. Figure 2: fingerprint authentication drives test case growth It is estimated the increase in test cases with the new fingerprint authentication is between 10%-30%.

Product Management at General Assembly

Product Manager in Heels

Depending on where you are in the life-cycle of a product or feature, you could be doing market research, writing user stories or testing features before they head out into production.

How to Improve Your Experiment Design (And Build Trust in Your Product Experiments)

Product Talk

Experiment design, on the other hand, is the plan that a product team puts in place to test a specific hypothesis. My concerns are that the format encourages teams to test the wrong things and it doesn’t require that teams get specific enough to lead to sound experiment design. Test Specific Assumptions, Not Ideas. When we test an idea, we get stuck asking, “Will this feature work or not?” The best way to answer that question is to build it and test it.

The Ladder of Evidence: Get More Value From Your Customer Interviews and Product Experiments

Product Talk

But no matter how good you get at collecting specific stories, you still have to test those insights. Too often, when we are ready to test a solution, we put it in front of people and ask, “What do you think?” In this case, you can use existing solutions to test your idea.

Sprint: Solve problems and test ideas in five days

Product Bookshelf

The process moves through identifying questions, creating solutions, storyboarding, prototyping, and testing with customers. Sprint: How to Solve Big Problems and Test New Ideas in Just Five Days by Jake Knapp. Over the course of a week: ideate and flesh out, prototype, and test it.

Managing Product Teams for Success

Product Talk

The most important habits I like to see are a regular cadence of customer interviews , prototype tests, and assumption tests. Good product teams have a regular cadence of customer interviews, prototype tests, and assumption tests.

How Two Companies Hooked Customers On Products They Rarely Use

Nir Eyal

It’s called “The Toothbrush Test.” Larry Page, CEO of Alphabet (the company formerly known as Google), has a quirky way of deciding which companies he likes. According to the New York Times, when Page looks at a potential company to acquire, he wants to know if the product is, like a toothbrush, “something you will […]. The post How Two Companies Hooked Customers On Products They Rarely Use appeared first on Nir and Far.

When should Product Backlog Grooming Take Place?

Roman Pichler

Note that this approach still assumes that you can collect the relevant feedback in the sprint review meeting, for instance, by carrying out a product demo, usability test, or solution interview. Option 1: In the Sprint Review Meeting.