How do you get your team excited about contributing to research? It requires valuable time and effort to participate, and may not immediately benefit them. While user researchers get excited about each new project and opportunity, too often they are met with blank faced team members whose main question is, “how long will this meeting take?”. Vicki Tollemache knows all about this after 12+ years as a user researcher. She’s started standing research days at four different companies, but when she came to Grubhub two years ago, she took things one step further. What started as an opportunity to research new markets for Grubhub evolved into what is now Grubhub’s Parts Unknown research program.
Giving Parts Unknown its own name, specific branding, and particular requirements created a special branded research program that has helped get more people at Grubhub on board with research. Erin and JH chatted with Vicki about how Parts Unknown came to be, her tips for starting a standing research day, and how she gets the team excited about research.
Listen to the episode
Click the embedded player below to listen to the audio recording. Go to our podcast website for full episode details.
About our guest
Vicki Tollemache is the Director of User Experience Research at Grubhub. She's originally from Dallas, TX. She's recently moved to New York and loves learning about the food cultures of different cities through Grubhub's research program.
Building a branded research program
Vicki knew when she and her colleague had the idea to start Parts Unknown that getting people on board would take a little more than slapping a cool name on it. They had to lure people in with something fun. Since their research started out focusing on emerging markets, they chose fun, foodie cities to send people to, like New Orleans. To keep the program going, they needed a few other things to build a sustainable research brand.
Executive sponsor
The key to adoption for many research projects is getting key stakeholders on board. Grubhub is lucky to have not only a great Director of UX Research in Vicki, but their CPO Sam Hall is a huge advocate of the Parts Unknown research program. When he heard about the program, he wanted to participate. Now, he’s Parts Unknown’s biggest advocate.
Now, every Parts Unknown project is presented to Sam. And since teams are eager to impress the CPO, Parts Unknown has become something teams actually compete to participate in. It’s also easy to be sure that action will be taken on each project, since it’s backed by top leadership.
Name
The name of the program is also important to its success. Vicki said Parts Unknown felt like a good fit because, when they created the program, Anthony Bourdain’s smart, cool, and inventive show was a phenomenon. If the research team could channel even a fraction of that vibe, they knew they would have something magical on their hands. Parts Unknown the show was all about exploring new places, connecting with people, and eating great food. Parts Unknown the research program was hoping to accomplish some of the same things—explore new territories (both physical and in-product) and learn about the people that make Grubhub great (users, drivers, and restauranteurs).
Specific requirements
Parts Unknown started out as a project to encourage people to participate in research for expanding markets. But it’s evolved beyond that. Now, Parts Unknown projects can also examine in-product experiences that haven’t been explored in a while. It’s reserved specifically for discovery research that can help Grubhub move in new directions.
Vicki sets Parts Unknown projects apart from other research efforts by adhering to those specific requirements. Each presentation has a specific look and feel, and people within Grubhub always know when a Parts Unknown project is happening.
Start a standing research day
Vicki has also started a standing research day at Grubhub, as well as at 3 of her previous companies. Now, she has a specific methodology for setting up a standing research day as a new researcher in the company. She says it’s a great easy win for new researchers, as long as the company is on board. These research days are typically reserved for evaluative research.
1. Choose a day for research
Choosing the right day is the first step to a successful standing research day. Vicki advises against choosing Friday, since everyone’s itching to get out of the office, and it’s more likely participants will skip their sessions on Fridays. Mondays and Fridays can also get complicated if there’s a holiday coming up. Tuesdays-Thursdays are the best bet for a standing research day, and Grubhub’s happens on Thursday.
2. Choose a day for pitches
Next, you’ll need to pick a standing day and time for designers and product people to pitch their ideas for research to you. For the team at Grubhub, pitch day is Monday. Establish a clear structure for choosing which projects get pushed through to the actual research phase so no one is upset if theirs isn’t chosen. A clear structure also ensures that pitches go faster and that your team is not wasting time listening to pitches you can’t help with anyway. For Vicki, this includes things like priority of the project, it’s viability, and whether or not the team is capable of producing a rough prototype by research day.
3. Let the team know
Tell your team you’re doing a standing research day! Not that you’re thinking about it, or you maybe might want to someday, tell them you’re doing it. From there, you can work on getting stakeholders and teams involved and engaged with the results of your research. Vicki has done this by promising research results within 24 hours of doing the actual research. This means that she and her team sit down with designers and stakeholders Friday afternoon with a complete research report. This details their findings and allows teams to take action immediately. This moves projects along more quickly and helps teams make changes fast.
One of Vicki’s favorite success stories from these research days is a quick change to Grubhub’s onboarding sequence. They wanted to test if diners wanted to open Grubhub, provide some information about what kinds of foods they liked, then proceed to the ordering page against going straight to the ordering page and providing preferences at the end for their next order. After research day testing, they found that people really just wanted the food as quickly as possible. When users open Grubhub, they’re likely already hungry. Anything Grubhub does to stand in the way of getting food quickly isn’t going to go over so well.
Vicki also noted that since research day involves bringing participants to their office, they typically try to use that resource for bigger projects. The research team doesn’t want to waste valuable participant time on whether the button should be blue or green. Typically, Vicki and her team outsource those kinds of tests to quick remote usability testing sites.
Transcript
Erin: This is Erin May.
JH: I'm John Henry Forster and this is Awkward Silence.
Erin: Silences.
Erin: Hello everyone and welcome back to Awkward Silences. We are here today with Vicki Tollemache. She is the Director of User Experience Research at the Grubhub.
Erin: We are going to talk today about branding your research programs. So as we know, everyone wants to be as impactful as possible in user research that they're doing. And Vicki is found this kind of really cool way to make that a reality at Grubhub by doing something I'm really into on the marketing side, which is branding things. It's kind of getting the gang involved in the research they want to do on an ongoing basis.
Erin: So we're going to talk about what those programs are, how they came to be, and how that's working out for them at Grubhub. So thanks so much for joining us, Vicki.
Vicki: Yeah, thanks for having me on. Also, I failed to mention, I really love the name of the podcast. Awkward Silences.
Erin: Thank you so much. Yeah, truth in advertising: we will find one somewhere.
Vicki: I am like the queen of this, so... I was like, that's perfect. It makes me feel comfortable straightaway.
Erin: I mean the name was actually born because when we talked to user researchers, it came up a lot that, as kind of best practices, to not try to cover over every awkward... that's how you get the most juicy bits and we thought that that could fit nicely for us.
Vicki: Yeah. You got to sit with it sometimes and then people will come to things on their own. It's true.
Vicki: So branded research programs. Do you want me to dive into it? Do you guys have specific questions?
Erin: We do. We have specific and general questions. All sorts of questions. But why don't we start with a kind of direct and general question, which is how did you come to this idea of these branded programs? What led you to this?
Vicki: It's kind of interesting. I guess, I think all research teams within an organization struggle with how do you make research sticky; how do you get maybe your stakeholders to be more engaged with it. And one of the things we came up with is this idea of ... it had worked really well in the evaluative space. Like we set up a recurring weekly Research Day, where our designers would come in and pitch things they wanted researched. And by having it on a weekly cadence, it just meant they always knew that they had the opportunity to do research. And it kind of became this thing that we notice naturally. Everyone's kept referring to it as Research Day, and this and that. It kind of branded itself.
Vicki: And the other program that we do we call Parts Unknown, is generative research so it's more about going out into the field, and it requires a higher level of engagement, right? Like people are often having to travel to engage with those research projects.
Vicki: And so we felt like maybe we needed to do a more formalized approach to branding it in order to encourage engagement. And that's how we came up with name Parts Unknown. We were kind of ripping off Anthony Bourdain at that time, because everyone loves his shows. And we talked about what makes that specific show so engaging. And he's going into areas where maybe people aren't familiar with that city, and how they go through and learn about that city, and the food culture in that city. And so we kind of stole the idea from that for a generative program that kind of partnered with this evaluative Research Day program that we had kicked off. And had been very successful on.
JH: Awesome. How far... When you say 'branded', like how far do you take it? The name is obviously one thing. In how you refer [inaudible 00:04:19]. Brand's like a fuzzy word, right? So, is the way that the reports are made? Or is there a logo? Or what are all the components that make up the brand?
Vicki: Well, so, not every research project, not every generative research project is a Parts Unknown. Parts Unknown is specific to us shining a light on something that is essentially unknown, or new to our audience.
Vicki: Initially, it's kind of evolved itself. Initially, it began with us taking a look at emerging markets. We kind of learned that our platform is different, depending on geography. So it's taking people outside of New York and Chicago and giving them exposure to other cities.
Vicki: And then Parts Unknown also became about and kind of evolved into us taking a new look at a part of our experience that maybe we haven't cast or put a light on, right? So we hadn't really revisited our onboarding experiences. So Parts Unknown then kind of morphed into exploring a part of our product that we hadn't looked at in a long time. So considering that that way. So by branding it, it's not every project is a Parts Unknown. It has to kind of meet those specific requirements.
Vicki: When our new CPO Sam Hall came in, he really saw what we were trying to do, and he himself was really into the idea of this branded research program. He now kind of uses it as a research program that aligns with him. Even though action will be taken, that Sam's behind it as well, which kind of has given the brand more power... I don't know that power's the word, but kind of like more engagement, right? Because people know that our CPO is a part of every Parts Unknown. And at the end it will be presented to him and he will be looking for what action will we be taking from those findings, right?
Vicki: Every report has a certain format that looks different than our other formats. I don't want us to get in trouble for, I don't know, branding issues. We do have a[inaudible 00:06:16] Anthony Bourdain [inaudible 00:06:22] We did create this before he unfortunately passed away. And it was just a thing we noticed within the organization, that people really loved his program. And so we thought that it kind of fit together well with what we were doing.
Vicki: It is a specific thing. We sometimes don't even have to... like everyone knows when we kick off a Parts Unknown program, that it's being worked on. And people just come now when they hear that name. It means something. You hear people in the organization referring to it as like, "Oh, they have a Parts Unknown going on with that." As if it its own methodology, which it's not. It's the way we've branded... We use a number of methodologies, but most of them are like more generative research, which is about understanding an ecosystem and going into the field.
Vicki: But yeah, for most part, there is a look and feel to it. The name has kind of become part of the lexicon at our company. It has really helped as well that our CPO has obviously backed it. And everyone knows that he is a driving force as part of it, as well.
Erin: It's almost like, if you're going to have a potent brand, you need your brand influencers, right. So you've got your internal influencer, the executive sponsor's kind of attached to it. You need a good name. You've got that. It's a name that is what it sounds like, but also it's familiar. So you're not starting from zero. And has this other attachment that is totally on brand for you because food - food. So A plus. Well done.
JH: Did it click internally like kind of right away? Like, you announce it, and you start doing this and people are into it and engaged from the start? Or did it take some time and a couple of these reports to come out for it to kind of snowball and be something that became a part of the lexicon?
Vicki: Yeah. It took a little bit of time. When we first started, we had a problem where a lot of our designers and product organization are New Yorkers and have been New Yorkers for life or are... Our company's split between New York and Chicago. Or are from Chicago and have been in Chicago for life. Everyone in our organization orders food from our platform, right. So in some ways, they all consider themselves a diner.
Vicki: But they're diners--and that's what we call our users who are ordering food through us-- they're diners in two markets that are the most optimal markets to be diners in, right? They are 10th food market, there are lot of restaurants on our platform. And so initially what we wanted, I think, --and I think most researchers would agree-- that experience is often the best educator.
Vicki: So originally in Parts Unknown, I was often... I'm from Texas, originally, I'm not a native New Yorker. And I've only been in New York for two years, so I definitely understood what our platform, when I came here, looked like in Houston, where my family is from. Or Dallas, where I was living at the time. You know, it was vastly different. At 10 pm at night, if you came home from a night out with friends, maybe there'd be two restaurants on the platform at that time, right? That's not a lot of options. And here UI looks really different when you have two restaurants, compared to like, two hundred restaurants.
Vicki: And so the goal of it initially was to encourage people to come out into the field, and to encourage them to experience not just... understand the food culture, and how that's impacted by geography, but also understand how it's impacted by the experience our drivers are having in that market. And then also, how it's impacted by the way we partner with restaurants in that market.
Vicki: So what we were trying to do was to use that to encourage our product team members, or dev team members, or designers, to come into the market with us, to come into other markets, travel outside of New York. And go through... Essentially, like, going out into the field, and experiencing diners in their home ordering food, and understanding how food ordering fits into their lives. And then, driving with the driver, and watching his troubles and tribulations as he's trying to deliver food in the market. And then go into the restaurants to see the challenges that they're facing as well. And to do that, we wanted to make it... It had to be kind of fun, right? You're asking people to put everything they're doing on hold for two or three days, and travel. And so we knew by calling it Parts Unknown, there is some sort of like... I don't know if sexiness is the word, or maybe that's [inaudible 00:10:34]. But there is something kind of edgy, right? And engaging, and obviously, we all really loved Anthony Bourdain because of that. So we felt like by kind of associating it with him, like that added a little cache to it. Like it made it sound already like it was going to be fun and interesting because his show is fun and interesting, and he's engaging personality.
Vicki: And then we also knew that when we initially kicked it off, we couldn't invite people to markets they weren't interested in visiting, right? Like we had a list of markets that we wanted to explore, but we also knew that some of those markets were more appetizing, so to speak, than others.
Vicki: So, I think the first one we officially did was New Orleans. It's a big food market, it was considered an emerging market for us, and we knew that it would engage people. And that people found New Orleans, as a city, engaging. So that's kind of how we started it. To try to get people to engage with us and want to come with us, we tried to select markets that matched our requirements, but we also knew how to level... they were enticing. And also, we knew by branding it with Anthony Bourdain, it made it feel more engaging. We were guaranteeing these people, we're going to have great food, we're going to try the food in the area. As well as you guys get to come out and experience all these different aspects of our ecosystem in the wild. And that was successful. That initially got people to come out with us. And as we produced research, and did big presentations... The presentations... people also wanted to hear about what their team members did in New Orleans; what their team members did in Charleston; what their team members did in Portland, Maine. As we went through, because they were kind of becoming interested in these trips, and like people were coming back, and like, "I learned a lot. I ate a lot of great food. I had a lot of fun. It was really worth it, the two or three days I invested."
Vicki: We also saw that as an education, the experience part, it was helping. It was changing the way our product team members, and our design team members thought when they came back. They kind of were able to see outside of New York and Chicago. So all those things initially engaged people, but it was very grass-roots, right? And as I we built that up, then it got the attention of our CPO, who then saw how powerful it was. And he himself was like, "I don't know if this is just an education through experience, or if we can use this as a research methodology to actually educate the broader organization rather than just educate people as they come on the road with you.
Vicki: But then after it evolved, it actually changed. Now it has become a really good tool. Does that answer the question? I hope that answers the question.
Erin: Yeah. You covered a lot of ground. [inaudible 00:13:11] Now I'm excited to go to New Orleans on Friday, which I'm actually doing.
Vicki: Oh, are you really? Is it your first time?
Erin: No, it's my third time. I love New Orleans. People there are so nice that it kind of is always an ethnographic experience to just chat with the locals. So, I'm pumped.
Erin: But how long ago was that? You said that New Orleans was one of your first trips, and that this kind of evolved in a grass-roots capacity at first. So how long has this been going on?
Vicki: That was... Well... Maybe a year and a half ago?
Erin: Okay. And so you said it was grass-roots, so you... Was there a pain in the organization at the time? Or an opportunity? Or a... What was the grass-roots compelling moment in which Parts Unknown was kicked off in New Orleans?
Vicki: Well, I think I had just come in to the organization and like anyone, if you're in a manager or director position, maybe you come in and try to assess the state of the research team within the organization that it's supporting. What's been successful in the past, and what has been a challenge, and you know, you also kind of [inaudible 00:14:14] where people are within their UX practice, right? Like, how evolved is your organization.
Vicki: And I think, when I got... when I came in, I also hired another researcher that I had worked with on and off for about, I don't know what, seven years? And him and I just saw a lot of opportunities. Like we were at the... And like people were asking questions, essentially, at the end of a product life cycle, just to like take the box. Like, does this work? Or --a question I can't stand-- do users like it, which I don't think is a very valuable research question.
Vicki: And so what we saw... We also saw that a lot of people had a very New York mentality to the entire United States. And I think, had an assumption like, "What I experience here, everyone experiences." Which is a very human way to think, right. Like, my experience is just like everyone else's, which is generally not the way.
Vicki: So, we just saw this opportunity... I always think researchers are really educators, and we just saw that opportunity. How do we show this organization in a way that isn't accusing them, but more like, "come along for the ride" and let's show you some things. Let's show you what Grubhub looks like in these other markets. How do we do that in a way that engages them, and that we can help begin this education process.
Vicki: And so him and I kind of sat in a room and talked about it, and kicked it around. And he had just come off a project where we had gone to several markets to understand why we had diners that ordered from us once and then did not order from us again. And we saw the value that that research had provided by bringing a few people onto different markets outside of New York and Chicago. And we knew our growth opportunities are in our emerging markets. So we knew that there was kind of like this opportunity. We saw that people were lacking this kind of education or engagement. So him and I kind of came out with the idea, and we got buy-in from our director at the time, who was not me. Subsequently, I'd become director. And then it was just about getting people to engage with it. Which is why, you know, we started being... thinking like maybe we brand it, make it something that people already have a good association with. Maybe we target markets that we know people are going to want to go to. And so that's kind of how it started.
Vicki: So, it was kind of born from two people who were very new to the organization, and just saw opportunities from coming in and assessing where the UX research team kind of stood and what education efforts needed to take place in order to further advance us within the UX life cycle.
JH: When you say people are engaging with it... Maybe I'm just taking the name too literally? Like, I'm picturing them watching a slickly-produced TV show. I imagine that's probably not what they're engaging with in the end?
Vicki: Now, when I mean engaging, like, when you do research... I guess there's different types of engagement, right? You can do research studies, and ideally you want people to be there with you. You want them to watch sessions. You want them to be actively involved, right? But oftentimes... do you know everyone has a busy workload. People have... I mean, our company is like meeting-ridiculous. Like, so many meetings. Oftentimes what I see --and I see it in other organizations-- is people come for the readout. And I think you lose a lot of the granularity when you don't attend sessions, or you don't come out in to the field with the researchers. And I also think researchers are very good at picking up on a vast array of behaviors and findings, and enable to put together a cohesive story. But designers and devs, and product people they definitely see things through a different lens. And what we really wanted-- maybe I call it active engagement -- not that they're watching a TV show, but they're coming with us into the field. They're meeting users for themselves. They're getting out of the office, and they're actively going in to the field. And they're asking questions, are there observing, they're part of the [inaudible 00:18:24]
JH: Got it. So how many people are going on these trips? Like, how many people are getting in the field?
Vicki: So we don't have a super huge organization, right? So we were able to bring... I mean, you don't want to pile twenty people into a restaurant or bring ten people into "so-and-so". It would generally be one researcher and two team members. But we would do multiple sessions in the field, and those two team members who watched, came out to a diner's home and talked to them about how they ordered food, etc., etc., But then later in the day go and do a driver shadowing with one of our researchers who supports our driver product.
Vicki: So you would have a group of people, but they would kind of be... we would have scheduled time slots, and they would kind of be moving through different parts of our ecosystem with different users at different times.
Vicki: But generally, no more than one or two, as well as the researcher, who's kind of handling or managing the facilitation of the conversations.
JH: All right. A quick, awkward interruption here. It's fun to talk about user research. But you know what's really fun? Is doing user research. And we want to help you with that.
Erin: We want to help you so much, that we have created a special place. It's called userinterviews.com/awkward for you to get your first three participants free.
JH: We all know we should be talking to users more, so we went ahead and removed as many barriers as possible. It's going to be easy. It's going to be quick. You're going to love it. So get over there and check it out.
Erin: And then, when you're done with that, go on over to your favorite podcasting app, and leave us a review, please.
Erin: So we talked to a lot of researchers who are doing a lot of remote research, right? The barrier to entry is pretty low. You can do it pretty quickly. It's less expensive.
Erin: But you're going out in the field with a couple of people, a lot more to plan, and bigger budgets. Was it hard to kind of get the approval? Or do you just have like tons of money to play with? Or how did you go from... again, we have this initial idea, where you have this need in the organization to get folks exploring these Parts Unknown. How far ahead did you plan, or did you kind of pilot this in New Orleans?
Vicki: We piloted this in New Orleans. I think it takes us, from a logistics perspective, the big thing is to recruit, right? And we recruit our restaurant partners and our drivers, and those tend to be the harder recruits. The diners, we either used local recruiting vendors... To be honest, we actually started using you guys. Some of these, I know you guys helped us in Tucson, and you were even able to get us at that time... We wanted college students, as well, who had used the service called Tapingo, and you guys were able to get that for us, so it was amazing. But, so it's about three weeks, and the hardest part is really the recruit. And then, for my team, it's pretty easy to put those schedules together.
Vicki: From a financial perspective, I will say doing research at Grubhub has probably been the place that I feel like we have been the most supported. If the organization has a question, if they feel that the research is valuable, they will provide us the budget to do that research. So they are very research-motivated, research-forward. In some regards, to be a researcher here has been the most freeing experience I've ever had. Because they definitely have a lot of questions they want answered, and are very, very supportive in making sure that we answer those questions in a way that we think makes most sense. And sometimes that involves travel, right? And they definitely... I mean, educating your product partners and your design partners, and your dev partners and creating empathy, so they really understand the real world situation that our users exist in means that they make better decisions and are more informed upfront. Which, I think my organization sees as an investment and as cost-savings in the long run, right.
Vicki: So I think that's the way it's done. But yeah, I mean, we're traveling. That does cost money. I've worked at other companies where the budgets are incredibly tight, and you don't have that luxury. And you are... you do more remote testing. We do remote testing here, as well, but we also, on a regular basis go out in to the field.
Erin: So you're bringing these --I'm picturing you guys are in like a Volkswagen van-- going on this like caravan with these three people. You've got you field mix. So you're going on this trip. Three people... I don't know the size of your organization, but there's more than three people. How do you... Is there pressure, interest, in kind of scaling that? You were talking about readouts, right, and how they're great but can be kind of limiting. Is there now this huge backlog of people who want to go on these trips? Or how do you kind of disseminate this experience and this information across the product organization or the larger organization?
Vicki: Sure. And so just keep in mind there are three people per individual session. But we have three researchers running sessions. So that means that there are actually six people and maybe even more than that, because sometimes people will kind of phase in and phase out, and other people will work remotely while people are attending the sessions. So we might actually be bringing anywhere from six to eight people on to the field with us. On top of the three researchers that attend. It's a joint effort to kind of inform people. Not only about what's taking place in that market, but also what's taking place within our ecosystem, which involves our restaurant partners, because they're our customers; our drivers, our delivery people; and then as well of our diners, and how they kind of...sometimes the most interesting part is where those people intersect.
Vicki: So, it's more than just three people. It can be a fairly large group. But afterwards, yes, we have I thing on Fridays, we came in and there's a thing called Lunch-and-Learn. But it really seemed like it had just become "Lunch" when I first came here [inaudible 00:24:39] ago. So we hijacked that.
Vicki: I mean, they bring in lunch for everyone to eat. You have a captive audience when people are eating. We hijacked that Lunch-and-Learn, and what were trying to do as well is we would work with the employee who orders lunch for both Chicago and New York. And for example, in New Orleans, we learned that there's a heavy influence on Vietnamese... from Vietnamese culture there. People are very big fan of Vietnamese food, and there is, as well, kind of like, the Southern culture, and the Southern eating, and the Creole culture. So then we would theme the food on Fridays to match kind of like the city that we had just gone to. And then we would do a very large presentation.
Vicki: And so that kind of engaged people as well. They were excited to try the food. I mean, obviously, at Grubhub we have a lot of food-loving people. It's a food-culture here. And so that was one of the other ways we tried to like drum up engagement. I mean, generally, we eat Dos Toros on Fridays, so I think people sometimes [inaudible 00:25:40] kind of excited that there'd be a change in the food. But yeah, so everyone knew that we would do the readout on Fridays. We had a captive audience because everyone was having lunch already. And we would generally do the presentation. We have large cafeterias in all our offices that are equipped to handle presentations for when we do like all-hands or a company-wide meetings. So we would just use those facilities to do that, to bring food in. Which was very successful.
Vicki: And then also we would do break-out sessions after we presented in regards to how do we solve some of the problems that we found in the field.
Vicki: But also, keep in mind too, the program has subsequently changed. Now it's not always about going in to emerging markets. Now sometimes we theme them. The one we did in Tucson was around like, how do diners think about pick-up? And how does that fit into their lives? And then, the same perspective for restaurants: how does pick-up fit into their lives from the perspective of companies like ours, Grubhub, and DoorDash, and Caviar. How do they think about pick-up?
Vicki: And now, we've subsequently gone through and done, it's not just about emerging markets. We went and reviewed all of our onboarding processes for all of our partners. So for...We know... We think onboarding takes this long for restaurants. What is that experience actually like for a restaurant? Does it take as long as we think it does? Were there friction points in it, and how do they overcome those or not overcome those. And those have kind of what it's morphed to now. So it's still Part Unknown, but now it's not a geographical location. Sometimes it's unknown or unexplored parts of our own product offering, our own experiences that we haven't taken a look at in a long time.
JH: If you were thinking about starting this up in another organization, or a researcher comes to you and hears this and is inspired, and wants to get their own version going. What parts of this--it sounds like you kind of found your way into it somewhat organically, coming in as an outsider to an organization. What are the key components that you'd recommend somebody, you know, focus on early on to get some momentum here and try to make their own version of something like this?
Vicki: I mean, I definitely think we've had success early on. It's very easy to do this in regards to evaluative research, right? I talk with a lot of old colleagues, and people in the community of researchers that I have, and lots of people have started up like a Research Day. Which is kind of where we first started. We had this standing day of... We subsequently have rebranded that. Now it's not called Research Day. But initially, it was called Research Day.
Erin: Wait, what's it called now?
Vicki: Well, we played on all the food shows, so we lovingly call it D-3, which is a reference to that ...
JH: The triple D?
Vicki: The...Diners, Drive-Ins and Dives.
Erin: I thought you meant the graphic technology for data.
Vicki: I call it D-3. It's a horrible weakness of mine that I can't keep track of acronyms. What does this stand for again?
Vicki: So we kind of rebranded that one as well. Because we started there, and lots of people have a standing day of research because, like I mentioned earlier, I think before we were recording, people often push back on evaluative research, and research in general, because they think timelines to get the research done are so long; and we have a release date; we don't have time to do this... So if you have a standing day, it kind of means like you always have the possibility of doing research in a few days, right? And that was so successful.
Vicki: I think anyone can brand a Research Day, an evaluative research day. And you can do remote research. Maybe it's you using user testing, and just like going through and educating your design partners had to do something like that. But you can always start there, small with evaluative testing, which is fairly easy and a much more manageable program. And for us, because we did have that support, and that financing, to explore problems out in the field, we were able to then scale that to the Parts Unknown program. Which, in the end became wider-known across the organization as well. Because once you're out in the field, that has implications not just for a product organization, but oftentimes a lot of it is reflective of how our... A lot of the problems we find with these research efforts are reflective of how our organization as a whole comes together to work.
Vicki: It often has meanings for our ops team, who's in charge of like... Our ops team is kind of like our internal employees who are like the touch points with our restaurant partners. So people who are doing account management for the restaurants. It involves them; it involves marketing; it involves our product organization; it involves feedback for our design organization. And oftentimes it's reflective of how the organization kind of has come together to create solutions for our teams. And maybe some of the issues that we find that our users are struggling with are indicative of how our organization works together to solve problems. And that's why the Parts Unknown one, I think, became so well-known within our organization. Because those findings scale at a much larger level than just evaluative research.
Vicki: But I definitely think, if I were to go to another company, I... This is the fourth company I've set up a research... a standing day of research in, and had a Research Day, a program, where we just do usability testing every Thursday or Friday every week. Those are very easy to set up. You can definitely brand those. I have never walked into an organization where that's a difficult thing to do. And I've always seen it be incredibly well-received. It's like one of the things now that I know them, I'm like, oh, I'll just set that up. That's an easy one for me.[crosstalk 00:31:40]
Erin: So how do you do it? You've done it a bunch of times. Like, 60 seconds. Like, how do I do it? What do I need to do? So it's Thursday or Friday. Research Day. Give it some like...
Erin: Standing day.
Vicki: I feel... I prefer Thursdays, because I find that users sometimes don't show up on Fridays, because we all got things going on Friday. So I prefer Thursday. Right before holidays; right after holidays; Mondays and Fridays are hard. Tuesdays through Thursdays is the better day. So I would do a Thursday. So you pick a day. You communicate that to the organization. Then you pick a day earlier in the week, preferably Monday, for designers to come in and pitch their ideas based off priority, viability, a number of things. You choose the topic for the week. There is a winner. You use a service like userinterviews.com.
Erin: Thank you.
Vicki: You can turn around really quickly.
Erin: Three free! First three free. userinterviews.com/podcast
Vicki: We did a lot of... There is a thing where you kind of have to sell your programs, so it's also, you know, sharing that you're going to set that program up, and building excitement about it with stakeholders, in meetings, and what it's going to provide and offer people. And getting the designers engaged and reminding them. At first, you're going to have to do a lot of kind of like... you're essentially marketing this program. But it very quickly becomes engaged.
Vicki: So it's really just choosing the day that people pitch; the day that the research will be done. We do a 24-hour turn-around. That means they will have results by end of day Friday. Bulleted results that we will read out and discuss as a group. And so they can take action immediately, right? And it's just setting up that quick research: you pitch Monday; you test Thursday; you have results spoken to by end of day Friday.
Vicki: And we have a researcher. That's her job. She just runs that every single week, but it also creates a really set, tight schedule for her. And it's a very easy thing to pitch, and I have never seen an organization respond negatively to it. It is always well-embraced.
Erin: Fantastic. Can you share if you have any top-of-mind just one or two designs or things that have been kind of answered or resolved from these Research Days? Tangible...
Vicki: Sure. I have to be careful. Because I'm not...[inaudible 00:34:01] I always have to talk with our legal team before I do these. I [inaudible 00:34:04] anything that's considered private to our company. Let me think of one that might be a good example...
Erin: Or even just the kinds of things, right. Like, as a designer, I've got this version and that version, and they're totally different. Like, help me settle a bet. Or usability tests... What kind of stuff's happening?
Vicki: So one of the things that... especially if designers are going...if we're adding any sort of new aspect to our product, and designers are trying something that hasn't been done before by a designer before them within our organization, it is often like, we maybe have done research to understand how users think about something. They have taken that research, created a solution. We are then usability-testing it, essentially, to see, "does their translation of those needs into a design match our users' mental mindset, right?
Vicki: So we always do five users. I think it's one... Our session's five users. And then we just get their feedback. And we've learned all sorts of... I mean... Some of the fascinating things, we have this whole... we're talking about onboarding. We had this whole idea--I think I share this one--this whole idea of like, do diners want to come in and give you a whole bunch of information about themselves, and then they will see restaurants that might be more meaningful to them because they've let you know that they don't like spicy food, they're vegetarian, they only want to get food from a certain radius, all these things, right? And we know a lot of assumptions that yes, people want to take that time to give you all this information when they first came to our product. And the truth is, they don't. They just want to get food. And maybe they want to do that after work.
Erin: Right. Right.
Vicki: Like, when we know --we know this from lots of studies--but we know that when users come to us, they're already hungry and anything that we do to slow them down in getting food to them is not good.
Erin: Right.
Vicki: So, I mean those are some of the things we've created [inaudible 00:36:07] that was like...it was a heavy load upfront, and users were essentially like, "Let me get the food first. And then you can ask me all these questions."
Vicki: But, there's tons of stuff. I will say, some of those smaller stuff, if it's just like small UI changes and users are like, "Should the button look like this, or should the button look like that?" Something that small, we're going to take to like usertesting.com because it's a better way to handle that question. And if we're going to use the resources to bring users in, I think it needs to be bigger questions.
Vicki: So we're generally looking to test maybe a flow. Or maybe we're looking to do competitive stuff. Or maybe we're considering... like... we're doing a usability study of some of our competitors and how they handle things. Or maybe we're trying to understand how to add complexity to our menus and we're looking at menus on restaurants, or competitors who are already complex and how do users handle placing those orders. Where do they struggle and how can we solve that?
Vicki: I mean there's a lot of different ways we use it, but really small stuff isn't the best use for it. It's probably larger; it's going through a flow of some kind; giving users a task and seeing... our scenario, and seeing how they're able to accomplish that, and what we learn from it.
Vicki: And sometimes we want to give them a number of scenarios and have them go through two different UI's with the same scenario and see what they get from the comparison, right?
Vicki: But yeah, it's nothing super small. It's usually bigger questions. But then it's also not too big, because we only have an hour. If we've done like a redesign of the app, we're not going to possibly cover all [inaudible 00:37:44] sessions. Yes.
Erin: Right. What happens to the designers whose pitches don't get chosen? They just throw their ideals out in the wild and see what happens?
Vicki: No, we try to give everyone help. A lot of the designers come in, and sometimes they pitch too early. They're pitching on a Monday and they have to have a prototype to us, or even a paper prototype by Thursday. And so oftentimes it knocks a lot of people out, when we're like, "Are you going to have a prototype by Thursday?" And they'll be like, "Maybe next Thursday." So we're like, "Well pitch on Monday...next Monday, then. You're a little early."
Vicki: There are times, though, when we do have a few candidates. And those cases, there will be a winner, but we don't... We try not to turn people down. We will work with the candidates who didn't get chosen to provide research for them in some other way. Usually, if it comes down to that, it's down to priority, right. Is the designer going to be able to use that feedback? Will they be able to make changes? And then also, how... what kind of level of priority is this within our organization.
Vicki: We also have Research Days when no one pitches. And then when my team gets to choose something that they're curious about and go explore that next, yeah.
Erin: When Parts Unknown and DDD collide.
Vicki: Exactly.
JH: So they... Say I'm a user researcher but I don't like cooking shows. What do I do? Am I just stuck?
Vicki: You are stuck. I mean, it wasn't like Anthony Bourdaine... I mean, in the end, doesn't that transcend [crosstalk 00:39:11]cooking? Yeah. No, I'm teasing. Awesome.
JH: Cool.
Erin: Vicki, you were saying that you are hiring at Grubhub.
Vicki: Oh, yes. We are actively looking for a senior researcher in the New York office. And we are adding a quantitative researcher to our team, which is pretty new for us.
Vicki: They can be based in New York and Chicago, and we're hoping to hire at a senior level, as well. I also understand that those people might come from a marketing background or a CX background. That's absolutely fine. But you can come over to Product and diversify yourself.
Erin: Fantastic. What should those people do if they're interested in applying?
Vicki: They could reach out to me. Or they can look me up on LinkedIn. And I have a post for both of those positions on my LinkedIn profile. And, honestly, reaching out to me is the best way, and the fastest way to get through.
Erin: All right.
Erin: Thanks for listening to Awkward Silences, brought to you by User Interviews.
JH: Theme music by Fragile Gang.
Erin: Editing and sound production by Carrie Boyd.
.jpg)
.jpg)
.jpg)
.png)
