It’s a match! Optimal matching for marketplace startups, and the role of bias
Originally published on Mind the Product.
You just finished your dinner and are ready to head home. You see plenty of Uber drivers around, but the ride-sharing app matches you with a car that’s a whole 10 minutes away.
This just can’t make any sense?!
In reality, matchmaking is one of the core levers of any marketplace business, and data crunched by product teams at Uber, Airbnb and other established marketplaces has revealed that engineering an optimal marketplace can sometimes defy common sense.
Greedy and network-optimized matching
In the example above, the intuitive approach is to match each rider with the closest car, on a ‘first come, first served’ basis.
Such transaction-level optimization will lead to the shortest possible time for customers in high-trafficked areas, but can lead to a terrible experience for other riders, and suboptimal aggregate wait times.
A better long-term solution — taking into account the needs of the entire user base — is to predict imminent demand, and optimize matchmaking for a network-level variable such as aggregate wait time.
Consider the example below:
The figure on the left illustrates greedy matching, the intuitive expectation of most riders that find themselves in our scenario.
This approach leads to the minimal wait time of just one minute for rider A, but makes rider B wait an unbearable six minutes, for an aggregate wait time of seven minutes.
The figure on the right illustrates network-optimized matching, which is closer to how most rider-sharing apps assign riders to their pool of drivers.
This approach leads to a longer, but still reasonable wait time of 3 minutes for rider A, and a short wait time of just 2 minutes for rider B, for an aggregate wait time of 5 minutes, a 33% improvement over greedy matching!
Better yet, since 2017, if a better match becomes available after you book a ride, Uber will automatically swap your trip, a feature internally branded as Trip Swap. A feature that may be a nuisance to drivers when it first happens to them, but over the lifetime of their driving will earn them more money.
Fungibility and its impact on matchmaking
In a fungible marketplace, supply is interchangeable.
Ignoring premium options, one Uber driver is broadly equivalent to another. Similarly, most customers will care little whether their Instacart carrot delivery comes from Aldi or Costco; a carrot is a carrot.
In an non-fungible marketplace, each supplier provides unique value.
Airbnb hosts offer a very different experience from one another. Similarly, on Pona, my home-cooked food marketplace, some home chefs specialize in unique dishes that can’t be provided by other sellers on our platform.
In reality, fungibility is not black and white, and whether a supplier is interchangeable with another can depend hugely on market expectations.
Many businesses operate between these two extremes (think tradespeople on TaskRabbit, or business formation lawyers on LegalZoom).
Recognizing where your marketplace startup falls on the fungibility spectrum is crucial, and has major implications on how you approach matching, pricing, and other core aspects of your marketplace business.
To see how optimal matching changes with fungible supply, let’s first consider the recommendation engine of Airbnb.
Most (and especially high-value) customers won’t browse through pages of listings if the initial results fail to inspire confidence in the platform. It is therefore a priority for Airbnb to optimize the first 5–10 listings you see in your search results.
The greedy approach to ordering Airbnb listings would be to show properties that will result in an immediate booking by the user. Under this scenario, all customers would see broadly similar results with an exceptional cost-benefit ratio, and book them on a First-Come, First-Served basis.
By now, your intuition should suggest that there is a better solution.
Just as with our Uber example, Airbnb should, and probably does optimize their search results for network-level variables (Airbnb’s website only says that there are “more than fifty factors that contribute to each set of search results”).
In the example above, if a property rental website optimizes at a transaction level, the first customer to make a booking can end up with a better property than they expected. That’s all good and well, but it may mean that a more picky customer making a booking just minutes later is left with no acceptable options.
If the website optimizes at a network level, on the other hand, the search algorithm could nudge the first user to book a different property that’s equally satisfactory for them and keep the exceptional option available for the picky customer who makes a booking about the same time.
Rather than “will the user make a booking?”, the Airbnb product team might ask something closer to “will showing these five properties lead to the highest aggregate number of bookings in this city?”. They might even optimize for aggregate Lifetime Value (LTV), taking into account that some customers are less price-sensitive than others.
The impact of bias
You might be wondering why companies such as Uber and Airbnb don’t prioritize best-behaved, repeat customers in their matching.
Wouldn’t it be a good idea for Uber to reward their best customers with faster service? Or for Airbnb to highlight Superhosts above all other results?
It turns out, just as greedy matching, bias in matchmaking leads to suboptimal outcomes at a network level.
This is what we found out at Pona.
Most of our cooks specialize in regional, international, and experimental cuisine that is not sold in restaurants, which makes our supply less fungible than for other delivery apps.
Let’s say that we only have one Ethiopian home cook in our service area, and she happens to be one of our top-rated and best-selling chefs. Unlike a restaurant, she can only cook a limited number of dishes every day, and always sells out.
On the demand side, the greedy, unbiased approach would be to display her profile to all customers, ordering search results either randomly, or based on distance.
A better, network-optimized algorithm might nudge buyers to try food from a different chef each day, or take into account how picky each customer is about their food choices.
In either case, there is a strong temptation to introduce bias into the search results. For example, we could show our top chefs to high-value customers first, as a reward for their loyalty.
Bias can also creep in as a side-effect of a seemingly unrelated feature. For example, some of our customers have asked for a way to follow their favourite chefs, so they can see them at the top of their search results.
What seems like an obvious, and rather benign little feature has the potential to wreak havoc at the network-level. Just imagine if a dozen customers favourite our Ethiopian chef!
If we had surplus demand, as in the illustration above, our Ethiopian chef, and other top sellers would make all the sales, prompting new entrants to leave, and in turn reducing the variety of food available on the platform.
If we had surplus demand, new Pona users would never get to see our Ethiopian chef, or taste her cuisine, and would become banished to new and less exceptional sellers.
An unbiased approach allows us to ensure sales are more evenly distributed across all our chefs.
If you’ve ever wished for Airbnb to allow filtering based on user reviews, now you have a better idea as to why the feature is unlikely to be introduced any time soon.
To further complicate matters, Pona orders are made at least 24 hours in advance, and our chefs have no overhead costs such as rent or staff.
Most chefs would prefer to cook one large batch of soup, sell it all on a single day, and enjoy their free time for the rest of the week, rather than sell one portion every day of the week (even if it meant fewer sales overall!).
In a surplus supply scenario illustrated above, the ideal matching algorithm would cluster sales on a different day for each cook, which would be impossible if search results were biased by favourites, or a ratings-based ordering.
But I’m just a startup!
I know all this can be overwhelming, and as a startup with limited resources you can’t have a team of data scientists optimizing your every match.
Fortunately, top academics and product teams at Uber, Airbnb and other established companies have done a lot of the work for you.
Although each marketplace is different, and you will want to run your own simulations and A/B tests down the line, it’s worth keeping some of the considerations above in mind as you build your product.
Developing a matching algorithm? Think whether distance is the best variable to base it on. Launching a new feature? Consider how it might inadvertently bias your search results.
And as you grow, make sure to dedicate ample resources to these questions.
