Topics
Artificial Intelligence and Business Strategy
In collaboration with
BCG
In collaboration with
In this episode of the Me, Myself, and AI podcast, Prakhar Mehrotra, vice president of machine learning at Walmart, discusses his background and explains how it helped prepare him to lead an AI team at a $500 billion retailer.
Before joining Walmart, Prakhar led data scientists and developed stochastic models at Uber and Twitter, where he learned how to move quickly and scale AI. (Fun fact: He even drove for Uber to better understand the driver experience — a perfect example of the role empathy plays in AI.) Now tasked with using artificial intelligence to help with decision-making and enhance the business, Prakhar focuses on the technology that improves store merchandising, which includes pricing, inventory management, and financial planning.
Hear Prakhar share stories on rallying and educating teams on AI, the relationship between AI and business intelligence, and what it means to make big bets in an enterprise setting.
Read more about our show and follow along with the series.
Subscribe to Me, Myself, and AI via Apple Podcasts, Spotify, or Google Podcasts.
Transcript
Sam Ransbotham: When you work in an established, successful company, current managers already know a ton. Still, AI solutions can offer insights to even experienced managers, if you can get the humans and the AI to work together. In this episode, Prakhar Mehrotra describes some moments where human and AI efforts came together for Walmart. And even more fun, he describes the hard work that it took to make those moments happen.
Welcome to Me, Myself, and AI, a podcast on artificial intelligence in business. Each episode, we introduce you to someone innovating with AI. I’m Sam Ransbotham, professor of information systems at Boston College. I’m also the guest editor for the AI and Business Strategy Big Ideas program at MIT Sloan Management Review.
Shervin Khodabandeh: And I’m Shervin Khodabandeh, senior partner with BCG, and I colead BCG’s AI practice in North America. Together, BCG and MIT SMR have been researching AI for four years, interviewing hundreds of practitioners and surveying thousands of companies on what it takes to build, deploy, and scale AI capabilities and really transform the way organizations operate.
Sam Ransbotham: Shervin, I’m looking forward to kicking off our series with today’s episode.
Shervin Khodabandeh: Thanks, Sam. Me too. Our guest today is Prakhar Mehrotra, vice president of machine learning at Walmart. He’s joining us from Sunnyvale, California. Prakhar, thank you so much for speaking with us today. Could you introduce yourself and share a bit about what you do?
Prakhar Mehrotra: Hi, I’m Prakhar Mehrotra. I’m the vice president of machine learning at Walmart U.S. My responsibilities include building algorithms that will power the decision-making of our merchants into core areas like assortment, pricing, inventory management, financial planning — all aspects of merchandising. I lead a team of 80 people. They are data scientists. They’re a full-stack team [of] data scientists, data analysts, data engineers. That’s my role at Walmart.
Sam Ransbotham: We’re particularly interested in you because you’re a top expert in artificial intelligence. Can you tell us how Walmart is using artificial intelligence to improve their business?
Prakhar Mehrotra: Walmart wants to use AI to serve our consumers better. And so my role is to make that happen. My expertise that I gained from Uber and Twitter and my graduate studies helped me achieve that dream. The secret sauce that I realized was that AI will be successful in companies if we partner with business closely and take business stakeholders along on the journey. It’s not just about algorithms, it’s about business, because the eventual goal of AI is to improve the business. I am responsible for all the machine algorithmic developments for core areas of merchandising, which include how do you price something, how do you select the right assortment, replenishment strategies, forecasting, and planning. So all the core aspects of merchandising [are] what we are trying to use machine learning and AI towards.
Sam Ransbotham: So, Prakhar, how did you get started on your path in AI? What are some of the more challenging aspects of implementing AI in your work now?
Prakhar Mehrotra: I started my career at Twitter; I was a data scientist. I picked up all the fundamentals of scaling and engineering at Twitter. At Uber, it gave me a massive break where it was a juggernaut. It’s like you’re just rolling. What disruption is — is what Uber taught me. And then when I joined Walmart, I had learned something about AI. I had got some experience about AI management algorithms. I picked up on the fundamentals of AI.
The most challenging part about the work at Walmart on the store side is there are no labels in the data. There are no tags. When customers shop in our store, all we record is, or all we have information about, is that the transaction was made. Unlike social media or unlike an app where you, in a Netflix type or in a recommendation type of environment … can track the history of a consumer and you can learn from it, that environment is not present in the store side. We don’t know what items customers are picking up and when they’re making these choices. So the job of algorithms is actually a lot harder as they have to infer all this as opposed to directly learn from the data, right? And so, inference became a big part about Walmart and then translating that inference into actionable insight that’s something we can make a forward-looking decision on. And so that was the key challenge at Walmart.
Sam Ransbotham: How does that feel when you’re gone from a world where everything’s highly quantified to things where everything is abstract, but you’re still asked to make a decision?
Prakhar Mehrotra: Yeah, the honest answer is you actually feel AI is a bubble. [Laughs.] The power or the promise of AI that we have [is] that it can solve anything; I can deploy an algorithm and I’ll get a very quick answer tomorrow — that starts to get challenged because when you’re doing something like inference or when you’re trying to identify these hidden patterns that are not pretty obvious, there are multiple challenges. There are challenges for a scientist to learn them from the available data that we have, and then to explain to the end user why the algorithm is inferring what it is trying to infer.
I think my education and my graduate studies at Caltech really helped me think through — it taught us how to think of a problem. I still remember my candidacy where my advisor literally asked me a question that had nothing to do with my Ph.D. and wanted me to defend it. And this was a pretty routine thing in aeronautics at Caltech. And so that type of training helped me to move [into other] areas. So when I decided to change fields and venture out into Silicon Valley, social media, Twitter — that was a completely different ball game. [Chuckles.]
On the way, I met people who were, on day one, not judging me but were more like, “You know what? We’ll invest in you. And we’ll teach you.” And I was able to connect the dots between science and business.
Shervin Khodabandeh: Prakhar, you talked about the scientist’s job being more challenging in an environment where data’s not tagged and inferences have to be made. What are your observations around the attributes of a good scientist in business versus a good scientist in academia or in a research lab?
Prakhar Mehrotra: Yeah, that’s a good question. At the end of the day, if a scientist has decided to spend time in industry and work in companies like Walmart, like oil companies, or like … nontraditional software companies where the core business is different, the key element becomes that you should be able to explain what you are doing. You should take the business user on the journey. In data science, we usually say that a data scientist should be able to tell a good story. But the story has many parts, right? Like, when I first started working at Walmart, I actually spent the first three months of my time in stores just trying to understand the terminology. What does BPI mean? What are [the] core metrics like? How do we do things? And that basically got me credibility with the leadership.
Sam Ransbotham: That all sounds very easy. I think we can wrap from here. [Chuckles.]
Prakhar Mehrotra: One thing I also came to realize is that when you’re taking these big bets around moonshots in an enterprise setting, usually there is a very fair optimism on day one. But you have to deliver something quickly to retain that trust. And so, there’s a delicate balance.
That transition was very challenging for me, where in my previous roles, everybody was believing in data science. And yeah, yeah, we have to go do this. And it was more about execution and writing those codes and write faster. I mean, move fast, break things, right? That’s the mantra in Silicon Valley here. It was slightly different in my current role at Walmart, where I have to work as a thought partner and show them what is possible. And also tell them the risks. So the balance between overselling and showing what is possible — that was also a big challenge for me as a leader.
Sam Ransbotham: What’s the parallel difficult challenge at Walmart that’s not execution?
Prakhar Mehrotra: Explaining what is possible and deciding on the big bets, that this is the power of AI. I think it’s not only unique to Walmart, it’s unique to any industry like health care or wherever you have this human expertise, where [the] human has expertise because of the way we think or [the] way we are wired, and where we can deal with uncertainty or unforeseen situations, and where the cost of doing a mistake is very heavy. There’s a penalization cost.
In some sense, I was able to find this parallel with aerospace where, when your rocket is launched, [chuckles] it has to now land on Mars. There’s no coming back, right? And so, while most of the data science training that I had in my previous roles and jobs was like you can do “millions of experiments” [chuckles], that’s not the paradigm here. And your algorithm is not the only decision maker. A good example might be how we decide assortment or how we think about what items should go into the store. That involves financial planning, negotiation with the suppliers, cost, how do I price it? That’s not just an algorithmic decision.
Sam Ransbotham: On the other hand, I see the point here, too, that unless some of these sort of mistakes are being made, there’s a danger of slipping back into the execution. And then it slips back into a pure execution, pure refinement mode, versus, I think what I might call more of an exchange mode, where you’re exchanging experiences — so execution versus exchange.
Prakhar Mehrotra: It’s about the journey — it’s not about just end execution. It’s about the journey you take, right? And the journey involves the exchange of ideas. You can’t execute it if you have not taken people along. I think there’s also a difference between — it’s a journey from BI to AI, like business intelligence. That’s how it would correlate. So you have to take a journey from business intelligence to artificial intelligence.
Sam Ransbotham: Prakhar, you’re a finalist for the Edelman Award.
Prakhar Mehrotra: Yes.
Sam Ransbotham: For those of you who don’t know, the Edelman Award recognizes examples of outstanding operations research in practice — which is a big deal in the OR world. How does that feel? How did your team feel? How did you feel?
Prakhar Mehrotra: It was a proud moment. It was a proud achievement because not only was it a breakthrough for me because I’m coming from aerospace — and, oh, wow, the community’s recognizing the work that I’m doing — it was also a proud moment for me and Walmart, that, look, the work that we are doing is recognized by a broader community. So I feel very humbled. [Chuckles.] And it’s like, yes, you are doing something right — because my Ph.D., my thesis, is not in this. And so, when you’re running something for a Fortune 1 company, and the broader community recognizes you, it was like a stamp of assurance that, yes, you got it right — a moonshot might be possible.
For me, one reason why I chose Walmart as a place to work in was because a dollar or 10-cent price savings might not mean much for, at least, most other people in Silicon Valley, or at least where I was out of. But that 10 cents can mean a world to a consumer. And so that basically gave me a meaning to it. You know what? It’s about finding the 10-cent savings or 20-cent savings. And you do that across many items that we carry in our stores and across our merchandising network, and those 10 cents add up and make $10 and make $20.
Sam Ransbotham: I like the framing of — you can check my math on this — but I’m pretty sure if you save 10 cents 10 times, you’re going to have a dollar, and that you keep doing that over and over. And I like the way you framed it — from not taking the dollar, you’re saving the dollar out of the process. And I think that’s where your team has a lot of potential.
Shervin Khodabandeh: Tell us a bit about your role at Walmart. How much of that is science and technical management? How much of that is team and stakeholder management? How much of that is evangelism and inspiration and whatever else?
Prakhar Mehrotra: Yeah, I spend probably 30% of my time in management, which involves output management, trying to set expectations with the company, what is possible, what is not, acting as a thought partner to the leadership, both in the technology side and in the business side. Another 10% to 20%, because I am a firm believer that if you are an AI leader and you’re leading a team and you’re in management, you can’t just be a people manager.
Shervin Khodabandeh: What’s the funnest part of your job?
Prakhar Mehrotra: The funnest part of my job is …
Sam Ransbotham: Besides talking to us.
Prakhar Mehrotra: [Chuckles.] Of course stuff like this. … I think the most fun part is when you see somebody who is not a believer in AI start believing in AI. And that happiness that you get when you see people start to believe in something, the passion that you share, is amazing. Second is I’m just making the Fortune 1 company a better place. Walmart is an essential part of our life. I mean, during these difficult times in COVID, we have to keep our stores open. We have to do it. It’s a role in the society. And so you keep that running, you play a teeny, tiny part in that. And so it gives a meaning for me to come every day to [the] office. And then support from the leadership, right? Those are the best parts of my job. And then you build a team. You have a team of very smart people spread across who share the passion with me. And you’ll see them rising in their careers.
It’s working with young people and then looking up to this crazy wave that we are riding in AI. On one side, everything is possible. Then the other, you come to a job and you’re like, no, it’s not possible. There are ups and downs that you face. It’s a roller-coaster ride leading an AI team and AI workstream.
Sam Ransbotham: Where do I apply?
Prakhar Mehrotra: [Chuckles.] Walmart was never on my radar to join because it was like, why Walmart when you have Googles and Facebooks next to where you live? And what I realized was that you have to tell the story. Somebody had to make this connection between the awesomeness of retail and how it connects to daily life to the power of machine learning. And so I spend a lot of time there. And when I’m not at work, I’m at home playing with the daughter and figuring out life. [Chuckles.]
Sam Ransbotham: Well, we don’t want to keep Prakhar from his family any longer. Thanks for taking the time to talk with us, Prakhar.
Shervin Khodabandeh: Thank you so much. So Sam, let’s recap what we heard from Prakhar. He made a lot of good points.
Sam Ransbotham: Yeah. A lot of great points.
Shervin Khodabandeh: I certainly enjoyed the conversation with him a lot. There was a lot of passion, but there’s also a lot of understanding of deep practicalities of what it takes to actually transform a company at scale — a company like Walmart — because he’s talking about certain processes where you cannot be dogmatic about it and say, “Well, this is what the engine says, and therefore you should do that.”
And some of these things are the inventory management, or on-shelf assortment, or store operations and store labor and things like that, where he talked about this notion of exchange and bringing the business owners along for the ride and during the ride. And so, designing the solution with them in mind, so that by the time it’s done, they’re not surprised. And they’ve been not only involved but instrumental in its design and build and incubation and implementation. And that’s really, really critical.
Sam Ransbotham: The tough part, too, is that these things aren’t going to be perfect. I think I really heard that in his discussion, that he knew that they weren’t perfect on day one, and when they’re not perfect, he’s going to lose some credibility. And how do they build that trust and how do they build that credibility on an ongoing basis?
Shervin Khodabandeh: Yeah. And that’s a very good point too. I’m reading between the lines of what he said, but an admission of sort of vulnerability and willingness for the AI engine and for his teams to learn from those experts and setting the right expectations — that just because we’ve built a piece of technology, it’s not supposed to be 100% perfect. That is actually not how any learning system works.
Sam Ransbotham: Yeah. I like your word vulnerability there because it came across. I mean, he’s clearly smart. And he clearly knows what he’s doing, but he’s still willing to learn and listen to what other people said and recognize that his algorithms weren’t going to be perfect right off the bat. And that was a humility that came through.
Shervin Khodabandeh: I think, actually, that is a secret sauce of somebody like him, whether it’s at Walmart or another person like him in a different company, being successful in that role — the willingness to listen, the willingness to partner, the ability to admit vulnerability and a desire to learn and that passion that he has, that “Look, when this thing works, it’s fantastic. And when it doesn’t work, I’ve already set your expectations that it will not always work perfectly, but every day it will get better than the day before.”
And I think that humility and that willingness is a real characteristic of folks that are in these roles increasingly, because there could be some organ rejection [when] you bring an expert in AI from a different industry, different field like Silicon Valley, like Uber, into a traditionally brick-and-mortar company. Because there is a belief that “Hey, we’ve already done it. It’s the right way. You guys have to just listen to me.” And we know that won’t work. And so, the sort of EQ that comes along with a role like that is really, really super crucial. And he really demonstrated that too.
Sam Ransbotham: One of the things that was interesting about Prakhar was how much it aligned with what we found in our research this year. We found that only 10% of organizations are getting significant financial benefits from artificial intelligence, and Prakhar really shows why that’s so hard. Most of the things he talked about weren’t technical. You could see him almost wistful for the days of perfectly labeled data, but that wasn’t the problems that he was facing.
Shervin Khodabandeh: Yeah. The problems were a lot more organizational change management, bringing users along. That’s all the human aspect and not so much the tech aspect.
And I think part of his formula for the 10% is going in up front with the admission that AI is not perfect. And AI has a ton to learn from the process, from the experts, from the organization. And sometimes it will be right. And when it’s right, it will be a credence to the judgment of those people, and sometimes it will be wrong. And when it’s wrong, it has the ability to learn. And so, I think actually going in with a mindset that AI is perfect is surely a recipe for disaster. And any good AI practitioner knows that that’s not the case.
Sam Ransbotham: Exactly.
Shervin and I are really excited about our next episode with Slawek Kierner from Humana. Please join us.
Allison Ryder: Thanks for listening to Me, Myself, and AI. If you’re enjoying the show, take a minute to write us a review. If you send us a screenshot, we’ll send you a collection of MIT SMR’s best articles on artificial intelligence, free for a limited time. Send your review screenshot to smrfeedback@mit.edu.
Comments (2)
Elie GAKUBA
James Johns