As you may have noticed, we recently re-launched our UX Wars campaign with a bit of a twist. We have included audience participation by running polls on social media. Currently, we hope to compare and contrast usability testing results with social media poll results and gain insights about why people prefer one app over the other. With this current round, we focused on rideshare apps. So without further ado, let’s dive into UX Wars: Lyft vs Uber!
Overview
For this round of UX Wars, we wanted to focus on rideshare apps. Selecting two apps to pit against each other was very easy, because there are two main apps that are prominent.
Uber is a global rideshare app and the first to take over the market over a decade ago. It has solidified itself as a household name, effectively pushing taxis to the side in the United States. Its primary competitor, Lyft is only available in North America. However, Lyft has been able to leverage itself as sizable competition by offering slightly lower rates.
Commonly, people have a preference for one over the other. Obviously, availability is a factor. But, the real question is, does the UX play a role in who is choosing which app? We decided to put this theory to the test, quite literally.
So, we ran a poll and usability testing to see which app has the best user experience. And the results we got were very interesting, to say the least. Let’s see what the people had to say about both apps.
UX Wars: Lyft vs Uber poll results
On social media, we had a very close run on all platforms. In the end, Lyft won with a score of 5 to Uber’s 4. As we have previously discussed, it’s quite possible that the results are influenced by the fact that most of our followers are based in North America. However, when we take a look at the usability testing results, we can see that perhaps there’s something to the social media poll results.
UX Wars: Lyft vs Uber usability testing results
As it goes with our UX Wars campaign, we set up user testing for both apps and asked users of a set demographic to navigate a series of tasks. From there, we evaluated the overall qualitative and quantitative data from each usability test.
Here, we have a closer look at how we set up the demographics and scenario for both tests, followed by a deep dive into the results of each.
Demographics
For this edition of UX Wars, we decided to focus on a younger target audience. Our main users are in the age range of 18-34 so we set that filter up for both tests. For Lyft, we set location up for the U.S. and Canada, since the app is only available in North America. With Uber, we set the location for the U.S. and North America to limit the geographic and cultural influences, because it is a more globally available app.
In this round of UX Wars, we decided to go with a sample size of 10 users for each iteration of the test. This allowed us to include UX Crowd data, which we share below.
In addition, we added a screener question that required users to already have an account with the app they were testing. This removes the added step and potential confusion of having users download the app and create an account.
UX Wars: Lyft vs Uber user testing scenario
For this round of UX Wars: Lyft vs Uber, we set up the usability testing scenario with the following parameters:
You have an early morning start tomorrow and want to schedule a ride to get coffee before you start the day.
While this scenario is simple, it gets to the point of what we want the users to accomplish. In this way, we can make sure that their mindset is appropriate as they navigate the respective apps.
So, now let’s have a look at each app and how they fared during the usability test.
Lyft usability testing
Usability testing tasks
The tasks we wrote for this particular test are as follows:
- Login to the app. Take a moment and describe what you see and how you feel about it.
- Using “Where are you going,” find the nearest Starbucks to you. If you cannot find a Starbucks, search for the nearest coffee shop you know of.
- After you’ve selected the Starbucks/coffee shop closest to you, select “Plan ahead.” Confirm the location, then set pick-up time for tomorrow at 8 am. Select “set pick-up time.”
- Choose Payment Method and confirm that the correct card is listed as default payment.
- Select “Schedule.” This should navigate you back to the home screen.
- Find the calendar icon within the app. Confirm that the ride is scheduled.
- Hit “Cancel ride.” Confirm by selecting “Cancel,” when prompted.
The outcome
An initial look at the UX Diagnostics dashboard shows us that Lyft scored an overall 89 on the System Usability Scale (SUS). It also received an A+ letter grade and is ranked in the 99.5 percentile for user experience.
Upon looking at the individual scores for the SUS questionnaire, scores largely fell at 90 or higher. However, there were a few outliers that pulled the average score.
The issues
With Lyft’s usability test, user didn’t encounter very many issues overall. The biggest concern expressed was with one user who wanted to navigate through the test without her information being shared publicly, which is understandable. Because of her searching for those options, her test took a bit longer than others.
For other users, there were some minor UX issues that weren’t common for everyone. One user encountered an issue where her scheduled ride showed 7 am, instead of 8 am. She was unable to directly edit the ride and had to cancel and start over. Another user ran into an issue where confirming the ride didn’t navigate her back to the home screen. And most users were initially apprehensive about being charged for the ride up front. However, the language Lyft used around scheduling the ride assured them.
The single ease question
The Single Ease Question graph, shows that a lot of our users had varying results when it comes to usability. The average tends to trend higher, meaning most users found the tasks easier to complete overall. However, there are several outliers. A couple of the users fall far below the average score, and a handful more had difficulty with at least one of the tasks. This means we will need to take a closer look at those users and the specific tasks they struggled with to evaluate what the issues are.
Task duration
Overall, it appears that users were able to complete tasks fairly quickly. We do see one very obvious outlier, as well as a couple of other users that took slightly longer on a couple of the tasks.
Task completion
It appears that users had no problem completing the first three tasks. However, with 4-7, they ran into some difficulties. Task 4 and task 7 had one user who did not finish, and with both 5 & 6, more than 25% were unable to complete. This means we will need to close pay attention to those tasks, when we are reviewing the users’ videos. In fact, it may benefit us to play those users’ videos at the exact moments they were having issues during the test.
The UX Crowd
With this UX Wars: Lyft vs Uber, we were able to tap into the UX Crowd, because we set our sample size at 10 users. Above shows the comments different users made about the app with votes from the other users. Here, we see there are very few negative remarks but lots of suggestions to make the app more user-friendly and bring in more customers.
Key reflections
Users had an overall positive experience with Lyft’s user interface. There were a few bumps for a handful of users, but everyone had positive remarks about the experience in the end.
One of the key takeaways with Lyft’s platform is that the scheduling feature felt intuitive and natural. As well, users liked that they could see how long the ride would take, which made it easier for them to plan ahead.
The main complaints about the platform are around pricing. Users feel like the pricing could be more transparent and breakdown why it costs more to go to one Starbucks over another, particularly a closer one.
As well, it appears that Lyft could work on making the ability to edit an existing scheduled ride a bit simpler, instead of forcing users to cancel and start over.
All in all, users felt very positively about the experience and interface, even when they encountered issues with the process. One user went so far as to say that Lyft is her favorite app.
Uber usability testing
Usability testing tasks
For the Uber usability test, we generated the following task list:
- Login to the app. Take a moment and describe what you see and how you feel about it.
- Using “Where to,” find the nearest Starbucks to you. If you cannot find a Starbucks, search for the nearest coffee shop you know of.
- Once you have chosen your coffee shop, on the next page, find the schedule icon and select it. Set up the ride to take place tomorrow at 8am and hit Set.
- Choose Payment Method and confirm that the correct card is listed as default payment.
- Select “Reserve Uber.” From there, select “Confirm pick-up,” then hit “Done.” This should navigate you back to the home screen.
- Navigate to the menu. Go to “Trips.” Select “Upcoming.” Confirm that the ride is scheduled.
- Hit “Cancel ride.” Confirm by selecting “Cancel reservation,” when prompted.
While the tasks are basically the same, we changed the language slightly to match what the app says exactly. This way, we don’t confuse users with rhetoric that doesn’t match what they see.
The outcome
An overview of the SUS UX diagnostics page shows us that Uber got an overall score of 81, with an A letter grade. They also scored in the 90.21 percentile for user experience.
The breakdown of individual scores shows us that there were a wide range of scores, ranging from the 30s to 100. This explains why the overall score is in the low 80s. There is a massive outlier that gave a score of 35.0, so we will have to pay close attention to that user’s test for further context and information.
The issues
With Uber, we found a few common issues among the users. The biggest issue we saw in the videos and among the quantitative data was hesitation or outright issues with the payment steps. Most users felt some anxiety around scheduling a ride with their payment method. They were afraid of being charged. Because of this, they often skipped the steps around scheduling the actual initially. After they realized they would be asked to cancel the ride, they went back and re-did the steps.
Another issue we encountered was some of the users had issues getting their payment method to work. They had a lot of confusion around this, as Uber did not offer any clarity in the error messages they sent. This kept those users from completing any tasks after the payment method task.
A final issue we encountered was a minor UI issue. One of the users was unaware that the menu function within the app is the profile picture icon. While this is a seemingly minor issue, a user brought it, up, which means it’s worth looking into.
Overall, users seemed to find the tasks easy and would have even completed the tasks they skipped if not for the payment method issues. Both of the related issues are very easy to fix, so Uber has the chance to step things up.
The single ease question
The results of the Single Ease Question are quite scattered, with several users scoring some of the tasks as very difficult to complete. It will be very important for us to review these user’s results more closely to understand what exactly went wrong.
Task duration
Overall, users took about the same amount of time completing each task, with most results middling in the same timeframe. There are a few outliers scattered throughout, particularly on the first task, and a couple around task 3 & 4.
Task completion
This graphs shows us that all users were able to successfully complete tasks 1-3 without any issues. However, for tasks 4-7, several users were unable to complete, with results showing nearly 50% incompletion rate for tasks 5-7.
The UX Crowd
The UX Crowd for this usability test yielded very different results from Lyft. Here, all users agreed that the navigation through the app proved difficult. It says that they had to use back buttons to navigate through, which can be quite time consuming and generally annoying.
Key reflections
Overall, users found the tasks to be largely easy to complete and were able to do so in a timely manner. However, the payment method UI issue created a lot of issues for users towards the end. With many users feeling hesitant about being charged up front for the scheduled ride, we feel it would be beneficial for Uber to add in language or a pop-up alerting them that they won’t be charged for the ride until the driver confirms. This alone would have changed the user experience and made it easier for users to complete the test.
Similarly, with some users getting an error message around payment issues, it would be helpful for Uber to explain exactly what the problem is. For some users, it seems like they didn’t allow them to use payment methods that were listed. Uber should probably not list or allow payment methods that users are not actively using or able to access. This would eliminate a lot of confusion and frustration.
Finally, for small UI fixes like the menu, Uber could instead use an icon that is more widely known as a symbol for a menu. When the menu expands, the account holder’s profile picture can be visible then.
Conclusion
In conclusion, both on social media and usability testing, Lyft wins. With a superior user experience and far easier user interface, they were able to win the hearts of the people in every way. While Uber still scored really well and was close in the social media polls, there are clearly some steps the UX design team can take to improve the user experience. This will put Uber ahead of Lyft in the future. We hope you enjoyed this edition of UX Wars: Lyft vs Uber!
Tyler Winstead