Rising Uber Cancellations — What do you do as a Product Manager?
How do you figure out what the problem is...
Product Managers often deal with situations where they have to figure out what is causing an issue and figure out the possible steps to deal with the problem. This process, in general, takes a lot of collaboration with different individuals such as Data Scientists and Product Analysts to collect data and test our hypothesis. In this blog, I will consider a hypothetical scenario — The Driver Cancellations for Uber are up for the past week and I as a Product Manager at Uber have to figure out what is going on.
Also, I will assume this to be more of an interview scenario and so my approach will be how I would think about this in a relatively short 30–45 minutes call.
Note: In interviews, the interviewer is usually your go-to person for any questions that you have about data to diagnose the issue
1. Clarifying Questions
The situation that we are considering is pretty broad, to begin with. So, it is important that I first clarify a few important points that will help me narrow down the issue.
- What kind of cancellations are we talking about?
There can be cancellations from the drivers’ end, the riders’ end, or the admin. Let's assume that for the scope of this blog we are only facing the said issue on the drivers’ side cancellations.
- How exactly do you define a ‘canceled ride’? Have we changed the definition in the time that we are seeing the rise?
Let's assume that we consider a trip to be canceled if a driver accepts the ride request and then cancels it before picking up the rider. Also, there has been no change in the definition of ‘canceled rides’.
2. Categories of Inquiry Questions
It is important to ask a bunch of questions to figure out exactly where the problem is. These inquiry questions can briefly be put into four categories —
- Questions around the Marketplace — What’s going on in the physical world where the product is used?
- Questions around the technical ways used to analyze data — Different ways that are used to slice and dice the data to get to the root cause
- Questions around user-specific issues
- Questions about the competitive landscape
3. Inquiry questions to get to the root cause
i. Marketplace
- Did the change happen gradually or was it a stark change?
Let's assume it was pretty sudden, maybe over a week or so that could mean that something fundamental has changed over the past week to cause the change
- Are we noticing this rise in a particular state or a city or is it just across the country? (We are assuming that this is in the United States)
Let’s assume that we are seeing a rise across cities in the country.
- Were there any big events across the county in the time frame that we are seeing the rise?
Let’s say that there were no prominent events during the time frame.
ii. Technical Problems
- Is the problem being observed only on a certain operating system or is it on both Android and IOS?
Let’s assume that there is no deviation in numbers between the platforms
- Has there been a newer version of the app released recently?
Let’s assume that there has been no release in the time frame that we are considering.
- Are we seeing a rise in drivers that are using a particular version of the app?
No, there have been no noticeable patterns to correlate app version to the rise in drivers’ cancellation
- Any change in the backend, maybe around matching algorithms?
Let’s assume that there was no major update in the algorithms on the backend.
- Has the app been facing more bugs or has been crashing more than usual?
Let’s assume that neither of the two has been happening in the time frame that we are considering.
iii. User Specific Questions
Since our ‘issue’ is around the drivers, they will be the ‘users’ in this case.
- Are we seeing any clusters of cancellations in terms of the time frame between the driver accepting the ride and the driver picking up the passenger? (This is to figure out if the cancellations are happening soon after the driver accepts the ride or if they are happening a little later when the driver is closer to picking up the rider)
Let’s assume that the cancellations are happening more in the early parts of the process i.e. soon after the driver accepts the ride
- Is the distance from the driver to the pickup point longer than normal?
No, the distance of the rides has not changed noticeably, and neither has the duration of the ride.
- Has the number of drivers increased recently?
No, there has been no change in the number of drivers.
- Has there been anything in the news that could have caused the drivers to have a negative sentiment about Uber?
Let’s assume that there has been no such change.
- Are we seeing any like segment of drivers that are disproportionately canceling?
Let’s assume that it’s very slightly skewed towards people who have been on the platform a little bit longer
Check-In: So far, the only piece of significantly useful information that we have derived is that the cancellations are happening soon after the drivers have accepted the rides
iv. Competitive Landscape
Even though there are multiple competitors out there, the biggest one is Lyft so we will be focussing only on Lyft.
- Has Lyft launched anything significant recently?
There has been an increase in Marketing Campaigns by Lyft
- Some drivers are on Uber exclusively and then some are drivers for both Uber and Lyft. Has there been any difference in cancellations between these two segments of drivers?
Let’s say that there has been no change for drivers that Uber exclusive.
(This gives us some hint that the issue could be Lyft related, so we can dive deeper here)
- Is there a way to find out if there was a reason why drivers who use both platforms have been canceling more? Any user experience research data that can help us answer this?
Let’s say that the UX research team told us that the drivers have admitted that Lyft has been sending them additional notifications about monetary benefits if they use Lyft more than Uber. Lyft is providing drivers with some extra incentives when they leave the Lyft app and jump to Uber’s app and that is causing them to switch back to Lyft.
Aaaaand Voilà! We got it!
4. Summarize
Here, I talk about the line of questioning I followed. I started with a few time and geographical-related questions and kept digging until I got a significant clue. The first big hint was that the cancellations were happening more in the early part of the process. Digging more into that, I started inquiring about the competitive landscape and the happenings in that space. On more questioning, I found out that a probable cause of the issue is Lyft sending drivers additional notifications and incentives when they start leaving their app and jump on to Uber’s app.
