Topics
Artificial Intelligence and Business Strategy
In collaboration with
BCG
In collaboration with
Khatereh (KK) Khodavirdi is focused on using AI to create better customer experiences — a process she compares to creating an “AI Legoland,” in which various technology components fit together to build cohesive solutions for PayPal’s customers. This is an approach she is applying in her role as senior director of data science in the online payment systems company’s consumer products division, where she oversees data science teams for PayPal, its peer-to-peer payment app Venmo, and e-commerce coupon-finder Honey.
On this episode of the Me, Myself, and AI podcast, KK joins Sam Ransbotham and Shervin Khodabandeh to describe how PayPal’s various consumer products work together to help users have a seamless experience across its products. She also talks about AI’s role in further personalizing the customer experience across the company’s brand portfolio, data governance challenges following corporate acquisitions, and her approach to creating effective teams.
Read more about our show and follow along with the series at https://sloanreview-mit-edu.ezproxy.canberra.edu.au/aipodcast.
Subscribe to Me, Myself, and AI on Apple Podcasts, Spotify, or Google Podcasts.
Give your feedback in this two-question survey.
Transcript
Sam Ransbotham: What do Lego have to do with how PayPal thinks about AI? Find out on today’s episode.
Khatereh Khodavirdi: I’m Khatereh Khodavirdi from PayPal, and you’re listening to Me, Myself, and AI.
Sam Ransbotham: Welcome to Me, Myself, and AI, a podcast on artificial intelligence in business. Each episode, we introduce you to someone innovating with AI. I’m Sam Ransbotham, professor of analytics at Boston College. I’m also the AI and business strategy guest editor at MIT Sloan Management Review.
Shervin Khodabandeh: And I’m Shervin Khodabandeh, senior partner with BCG, and I colead BCG’s AI practice in North America. Together, MIT SMR and BCG have been researching and publishing on AI for six years, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build and to deploy and scale AI capabilities and really transform the way organizations operate.
Sam Ransbotham: Today, Shervin and I are talking with Khatereh Khodavirdi. She’s the senior director of data science at PayPal. Khatereh, thanks for joining us. Welcome.
Khatereh Khodavirdi: Thanks for having me. Great to be here.
Sam Ransbotham: You build and oversee giant data science teams for the many different entities within PayPal. A lot of people know PayPal, but probably some people don’t realize the extent of all the PayPal activities, so maybe let’s start there. Tell us a bit about what PayPal does and what all these different subentities do as well. How are they connected?
Khatereh Khodavirdi: Yes. Obviously, a majority of the people know PayPal through PayPal Checkout, which is a core product that we have, and the company started from that product. But we have a wealth of different products, especially on the consumer and merchant side. We have whole suites of products to provide people the capability on the merchant side to run their business, from invoicing [to] having the offline capability for financial services and then the online transactions, and on the consumer side, starting from peer-to-peer payment to financial services to different types of credit and “buy now, pay later” capability, and savings accounts. I can go on and on, but on the consumer side, we have a wealth of different products.
Over time, we also acquired many other companies to help us accelerate and also be incremental in terms of the value chain that we are creating for the consumer and merchant, such as on the consumer side, a couple of years ago, we acquired a company — its name is Honey — which actually helps you to find the best deal on the internet when you’re shopping online.
I started at PayPal on the consumer side, so I was helping the small-business group, helping accelerate and solve the problems for our customers through data and data science capability. And then, over time, I supported all the merchant side of our equation — all the enterprise merchants, channel partners, our relationship with Shopify, WooCommerce, Magenta — programmatically bringing a merchant for us.
And then, a couple of months ago, I actually switched completely to the consumer side of the house. So I’m new on the consumer side but super excited, because you can leverage a lot of AI and data science capability to solve a lot of interesting problems in this domain.
Shervin Khodabandeh: That’s great. So obviously, PayPal is a big, multifaceted company with different business and subdivisions, and you talked about those. But since you’re in the consumer group, let’s talk about that. Share with us a bit how AI’s being used in the consumer business to drive the themes you want with your consumers and what use cases is AI being used for.
Khatereh Khodavirdi: We can actually bring all the different accounts together, so we have a point of view around the user interactions with us. For example, if Shervin has a relationship to Honey with us, but at the same time, he’s using peer-to-peer or the Checkout product or our credit capability, how can we make sure that we can actually look at Shervin’s relationship with PayPal through one lens?
So that is, I would say, one of the fundamental problems we are trying to solve as a company. But the piece I’m personally super excited about is around the fact of understanding the customer journey with us and how we can leverage personalization and AI to actually solve customer use cases — that “What are the jobs they are coming to PayPal to get done?” and “How can we show them the personalized and relevant messages to help them?”
So think about it: Internally, we always said, “Hey, there are a bunch of happy paths with PayPal, and there are a bunch of sad paths with PayPal. How can we migrate people from one happy path to a happier path with us, and how can we avoid the sad path with the customer?” And I truly believe that this is basically an AI capability that we need to develop for customers. If you build that, it will unlock a humongous amount of value for our consumers and for us as a company.
Shervin Khodabandeh: Let’s talk about some of those paths. You said, “What kind of jobs are people trying to get done?” so let’s talk about that. Tell us how personalization could be helpful there.
Khatereh Khodavirdi: Think about it this way: Imagine Shervin is one of the people that is actually using PayPal Checkout to shop through different merchants throughout the internet as well.
As you can imagine, once we acquired Honey, we also had a whole wealth of the coupons that are [available] out there, and over time, if you get a better sense around what type of categories or what type of merchant Shervin is interested in, we can actually show Shervin the right deals at the right time.
For example, it’s back-to-school time, and we know that historically, at this time, you shop in this type of category. Currently, you know, Target or Walmart or some of the top merchants for back to school are running these deals, and then we remind you and show you relevant and personalized deals to actually drive activity with us.
Or you might use Venmo on a weekly or biweekly basis to send somebody who cleans your house some money. We can remind you that, “Hey, Shervin, it looks like you’re doing this,” and through just one-click touch, you go and do that. So basically, we become part of Shervin’s life and understand what type of activity Shervin is trying to do and make it much easier for him.
The other part of it that we are all super excited about is tracking. When you shop online to a different retailer, one part of it is, you want to track that order and see where your order is and when you are getting it. We also want to make it much easier — you can actually get the notification and see where your order is, and you can look at your whole commerce activity and financial activity in one place.
I would say for me, also, the other part is that, yes, I’m in a data function and we build models, and I look at the outcome of our model, but also the more important part for me is — I always call it … qualitative and quantitative. … I would say, “Hey, I want a sample of some of the customers in this group. I want to understand their journey, the activity they had with us,” or, you know, actually attend the user research studies that we are doing with customers, so firsthand, I actually hear the challenges and the problems we are trying to solve.
Because for me, the most important part is that I just cannot go and in isolation build the model and solve all the problems. It’s really doing the qualitative and quantitative aspects and learning from each other and improving it over time.
Shervin Khodabandeh: That’s very helpful. I have to imagine that being more of a tech company than, let’s say, a financial services company, that some of these challenges are actually a lot easier for you to deal with than it would be, let’s say, for a gigantic global bank that’s trying to personalize across products and business lines and all that. Share with us some of the challenges. What’s difficult about what you need to do?
Khatereh Khodavirdi: I think one of the biggest challenges we have is that some of the other companies that we acquired over time, each of them is using different data stacks, so basically migrating all of them to one data lake and having one kind of technology to use across the board and being able to [build] that common data layer platform … end to end, and understanding all the touch points and all the data points we have with the customer. So having that common data technology platform is one of the common challenges we have internally.
Sam Ransbotham: That’s huge, I think, for everybody. I remember my first experience with this. Back in my past life, I used to work at the United Nations, and I was working with databases, and I looked down and we had just literally dozens of databases we were supporting, and I asked, “Well, which is the standard?” And they said, “Well, this is the standard.” And, I said, “Well, what are all the rest of them?” “Well, those were the standards then.”
And so you’re in that same situation when you’re acquiring companies, where you’re pulling together … lots of technology stacks [but] you don’t want to just kind of rip them out and start over. How do you manage that process? How do you get those in a cohesive data science process?
Khatereh Khodavirdi: Yeah, so, I would say, Sam, you brought a very good question as well — that it’s not only the technology problem aspect of it. I call it the data governance aspect of it as well: … what the definitions are, and how different people define and look at different things differently, and how you can define that common language across the board internally within the company as well.
So that is why we are trying internally to develop the best practices that every new company that we have … here are the steps, one to end, that we are going through to incorporate them as part of the rest of the data assets we have for the company. But you can imagine it’s not an easy exercise. It’s a humongous task.
But that data governance aspect of is it also very important, because when you are bringing different components together, you need to take a step back and look at the definition from a different lens as well and see if those definitions still are relevant in the new construct or not.
Shervin Khodabandeh: Let’s talk about teams a little bit. You’re talking about a series of challenges with personalization, with other use cases, and then the capabilities that are required to get that done, from data coming together and identity resolution and many, many other things. Tell us about the team. Clearly, they have to have very strong technical capabilities, but what else do they need to have to work in that kind of an environment?
Khatereh Khodavirdi: I would say this area is one of the areas that is so multidisciplinary, and you can imagine the different types of problems you want to solve need different types of skill sets and these types of talent.
So I always said that in data science, AI, or the overall data field, one of the things that is really important is diversity of talent. And by diversity of talent, I don’t only mean diversity of gender or background, which is very important, but diversity of thought leadership, diversity of problem-solving, diversity of technical skill set, because at the different stages of the problem, basically, you need to practice a different type of muscle in order to get the desired outcome.
So, for example, one of the areas that I really feel [is] underappreciated in the data world is business acumen — people who can actually tackle the problem through a very structured framework and be able to synthesize the recommendation and the “so what” to the business.
Because the worst thing that can happen is that you look at your data team and you feel like they are building a bunch of black boxes for the rest of the organization, and you’re not investing in the last mile — that people are actually translating the “what” and the “why” and the “so what” to the business group, to the product group, and to the rest of the organization. So you would not get the adoption that you are hoping from the capabilities that you are building.
So the way I’m looking at it is that we are actually also building a product organization that supports the personalization and AI, because like any other product development cycle, you basically have a product strategy behind this — that we are not practically building the AI model to solve specific use cases. We actually take a step back, understanding our consumer personas — what are the jobs they are coming to us to get done? — and build a product road map and product vision around this and tackle this problem in a cross-functional fashion instead of just internally within the data group.
Sam Ransbotham: Is it harder to get that last mile with AI-type projects? Is that something the people have a harder time understanding? Is it something that is harder for people to relate to?
Khatereh Khodavirdi: It might be a little bit harder because of just the scale of the problems that you are trying to solve here, because people cannot relate to it when you’re talking about millions of customers. So for me, it’s how I can actually break down the problem and solve smaller use cases through AI to create that adoption and championship in the organization [so] that it will help me to solve the biggest problem. It’s a humongous task, because I’m not only talking about the product touch points that we have. I’m talking about every touch point that the customers have with us, either through customer service, through risk — through all the different functions within the company.
Rallying all of those cross-functional functions around solving this problem would be much harder, versus my approach, which is that I will first start with solving it within the product organization, understanding all the touch points with all the products, understanding how we can understand the product task, and personalizing that component. And then you can add an additional layer — like bring the risk component — with each of the product components, then you add the customer service.
I would think about it as more of a Legoland — that at the end of the day, we will have an AI Legoland for PayPal. But right now, the way I am attacking to solve this problem is to build each of the individual Lego pieces with the hope that I can orchestrate and build a Legoland and it won’t become a bunch of separate Lego that are not orchestrated to solve the common equation.
Shervin Khodabandeh: I love the Lego analogy, because my kids are totally into Lego and we have probably like 900,000 different pieces of Lego going on at any given time. And if you just look at it in isolation, you think, “OK, this is all we’re doing.” But, of course, you’ve got to start there, and then the pieces come together.
So then my question to you is, as with Lego, when I see my 12-year-old or 8-year-old building stuff, and I’m looking at it in isolation, I might not have a full sense of the vision [of what] the whole thing is going to be.
So, I might say, “Oh, this is nothing,” or “What are you building? It’s like a small piece. Didn’t you do something like this before?” And then they bring out the box with, like, 8,000 pieces that’s going to look like this. And then, I go, “Uh-huh.” How are you doing the big “uh-huh” here at PayPal so people don’t lose sight of the big vision and don’t get myopic about the little things that it takes to get there?
Khatereh Khodavirdi: You can imagine there is no shortage of individual use cases — that there are many individual AI, or whatever you name it, capabilities within the organization. But when you take a step back, you do not have that guiding principle to see how they can help you to actually build that Legoland.
And actually I want to tackle this problem in a reverse order; first, I want to take a step back and say, “Hey, what will the blue sky look like in terms of the AI capability and personalization for us?” So building that product strategy and vision around it, and then trying to solve it backward, and then breaking it down into smaller pieces of the component Lego and being very prescriptive around what are the key problems each of these Lego will try to solve, and why we are building each piece of the Legoland — what will be the “so what” to the organization.
Then you can actually build something … because if you show that whole vision to everybody, it might be a little bit too much for some of the people to absorb it, so it might really slow down your progress in the organization. [In comparison,] when you show the bigger vision, you have something to rally the whole organization around. But at the same time, you can break it down into more tangible components so you can start making progress while you’re actually keeping that energy and enthusiasm around the organization to the north star that you have.
Shervin Khodabandeh: The breaking down actually is quite critical. To build on the Lego analogy, usually you get these 5,000, 6,000 pieces of Lego, and they come in like 20, 50, 30 boxes or little bags, and so you first do this, and then you do that, but then you have the whole thing.
Well, I got one two days ago that had 3,000 pieces in 20 different bags, but all the bags are unlabeled, so you don’t know what goes with what. And so then, what we have is, like, 3,000 pieces, and we’re trying to build a piece, and the analogy I’m trying to draw here is, as you’re building these little capabilities that then come together and get stitched together to support your bigger vision, how do you make sure that these pieces are actually connecting rather than the organization is looking to find what piece goes where, or how does this connect, or do I have eight of these instead of five of those?
How do you avoid that? And that’s, by the way, something that goes on in a lot of other organizations, where there are silos of folks building things, and they don’t all come together. How do you approach that?
Khatereh Khodavirdi: You brought up a very good question that, Shervin, basically, hey, I cannot go to run a step from that vision and product strategy for the AI and personalization to be a tactical component of the puzzle that you have. But once you build that mental model around the common themes of the problem you are trying to solve, then I would say the other reality also is that I cannot solve all these problems by myself, [nor can a handful] of people solve these problems in the organization. You need to create that culture, and you need to rally your organization around this.
Sam Ransbotham: That makes sense, because I like the Lego analogy, but the world isn’t as simple as Lego. You don’t have that perfectly labeled bag that you know will fit together in the end. You’ve got to bring those people along to pull that in.
One of the things that we did early in the pandemic was sort our giant things of Lego and, when you have those Lego that are not in the little bags, it’s practically impossible to find the right piece to pull it back together. Shervin, it reminds me of when we were talking to Arti Zeighami at H&M, and he was talking about working with individual pieces …
Shervin Khodabandeh: On the wheel, right?
Sam Ransbotham: His analogy there was tightening each lug nut a little bit as you go around, and there’s some logic here, too, that what’s different, too, about Lego is that you could solve one bag and move to the next bag, but in reality, you’ve got a lot of people working on lots of different bags who are moving at different paces.
Khatereh Khodavirdi: No, totally, Sam, and I would say that is why it is very important that, in my view, in order to make progress on such a complex topic, it’s kind of a top-down and bottom-up approach, and you just need to have that check-in on a regular basis. … The top down, I would call it your blue-sky strategy around where you want to be with AI strategy, and the bottom up is just mainly like the tactical differences of the Lego that somehow exist in the organization, or there are different components that different teams are building, and how you can actually bring the two components together.
And you’re absolutely right that the speed of the development and making progress for some parts is more difficult than the others as well. So you also coordinate the different components together, so at the same time, you are making progress, you can rally the organization around that, but also be realistic that some other parts are more complex and it will take more time.
Shervin Khodabandeh: And the one thing that I think companies have that Lego pieces don’t is they still have P&Ls and targets and numbers. And so I think that’s probably why it’s such a nuanced approach, as you were saying, KK, that it’s like you’ve got to figure out where the most practical path is for your organization, given all the players and all the stakeholders and all the pieces, and maybe for Arti, it was a bit “one lug nut at a time” to get the whole thing going, and maybe here it is like, “No, we’ve got to get personalization perfect before we move on to risk or pricing.” And I think that’s the nuances of different organizations a little bit.
Khatereh Khodavirdi: Yeah, and I would say, Shervin, you brought up a really good point as well, that at the end of the day, all the organizations are very value-focused, both for the customers and also for the company … and the shareholder as well.
And for me, one of the biggest risks that people can do is just looking at the output and the outcome of the P&L, [whereas] for such big projects like this, like personalization, the reality is, it will definitely take some time for you to actually get the true benefit of this from the output standpoint of your P&L. But what are the leading indicators and the KPIs you can have to actually keep the team and the organization accountable, to make progress so you make sure that you are moving in the right direction, while it will take you more time to see the whole benefit as an output in your P&L?
Because the reality is that the worst thing that can happen is that all of us know that this is absolutely the right thing to do for the company, but because you wouldn’t … there is no magic, there is all this good work consistently over a long period of time. There is no magic that [lets] you see the output overnight, so how can you keep the team accountable, the working team, to make progress, but to the leading indicator that you know that eventually it will get you to the outcome that you are looking for?
Shervin Khodabandeh: For sure, right? As you’re saying, the worst possible thing would be to set a big outcome goal for the vision, but for the wrong time.
KK, this has been quite insightful. I think, Sam, we should move to the rapid-fire question segment. So this is a segment we do, KK, where we ask you a bunch of questions in rapid-fire style, and please tell us what comes to your mind first.
Khatereh Khodavirdi: Sounds good.
Shervin Khodabandeh: What has been your proudest AI moment?
Khatereh Khodavirdi: My proudest AI moment goes back to grad school, when I was at Carnegie Mellon, before this field was this much in demand, and as part of my graduate research studies, I was building capabilities and platforms for energy management — smart energy management for residential buildings. That was the most proud AI moment of my life.
Shervin Khodabandeh: Cool. What’s your favorite activity that involves no technology?
Khatereh Khodavirdi: Playing tennis, because it really helps me to focus on the moment.
Shervin Khodabandeh: What was the first career you wanted? What’d you want to be when you grew up?
Khatereh Khodavirdi: Probably, for my first career I wanted to become a professor or a pilot. I don’t exactly remember which one came first, because in our family, education is a big piece, and my mom actually had an educational career, but probably either a professor or a pilot.
Shervin Khodabandeh: What worries you about AI?
Khatereh Khodavirdi: I would say there have been a lot of conversations about responsible AI and bias, and I chatted about this earlier — that it’s qualitative and quantitative as well. I think it would be a huge mistake to assume that AI can solve basically all the problems without having the right checks and balances in place.
Shervin Khodabandeh: What is your greatest wish for AI in the future?
Khatereh Khodavirdi: I would say there are a lot of challenges that the humanities are facing, from climate change or a bunch of other stuff, so I really hope that more and more people actually play a role in using AI to solve those problems. A lot of my colleagues actually started investing more of their time and energy, and I really hope that eventually in my career, I also can play a role as well.
Shervin Khodabandeh: Great. Thank you very much.
Sam Ransbotham: So, KK, I think that, obviously, the Lego analogy is going to be interesting for people, but I also like what you were saying about things like governance that, I think, are fundamentally important for this, and the idea that you would come on and mention some of that importance of things like governance getting you to that scale that you need — I think that’s something that maybe is more widespread or universal. Thanks for taking the time to talk with us today. We really appreciate it. Thanks for joining us.
Khatereh Khodavirdi: Thank you guys so much for having me.
Sam Ransbotham: Thanks for listening. Join us next time, when we talk with Fiona Tan, chief technology officer at Wayfair. Please join us.
Allison Ryder: Thanks for listening to Me, Myself, and AI. We believe, like you, that the conversation about AI implementation doesn’t start and stop with this podcast. That’s why we’ve created a group on LinkedIn specifically for listeners like you. It’s called AI for Leaders, and if you join us, you can chat with show creators and hosts, ask your own questions, share your insights, and gain access to valuable resources about AI implementation from MIT SMR and BCG. You can access it by visiting mitsmr.com/AIforLeaders. We’ll put that link in the show notes, and we hope to see you there.
