AI Can Change How You Measure — and How You Manage

With apologies to Peter Drucker, it is no longer simply what you measure that determines what you manage. It’s how you discover what to measure that determines how you manage. In industry after industry, we see innovative measurement systems leading to innovative metrics and new organizational behaviors that drive superior outcomes. More organizations are recognizing that benchmarking and executive expertise don’t always determine the best key performance indicators (KPIs). These data-driven companies employ predictive analytics such as machine learning, along with leadership acumen, to identify and refine key strategic measures. More finely tuned measures lead to better alignment of behaviors with strategic objectives.

Increasingly, business leaders are asking their teams: Do our metrics fully capture what drives value creation in our business? How can we use technology to improve our system of measurement — that is, how we create, assess, and use our metrics — to better discern these drivers and identify better KPIs? They are rethinking their approaches to measuring success, to developing metrics, and building organizational alignment.

In the technology sector, for example, losing talent can constrain growth, and, conversely, retaining talent can enable growth. Identifying and addressing attrition risks is therefore a strategic matter.

For most of its long history, IBM relied on management intuition and HR data to assess attrition risks — until Diane Gherson, IBM’s chief human resources officer at the time, recognized that predictive analytics could do a better job of supporting the company’s retention efforts by helping managers focus on the talent the company needed most. Under her guidance, IBM created a machine learning algorithm that can better assess which employees are preparing to leave the organization and offer recommendations to managers about what to do to keep them.1 The algorithm analyzes dozens of variables and millions of data points to deliver an analysis that is far more accurate than pure management intuition. Managers use the algorithm to identify the individuals to target for development conversations about cultivating skills and careers within IBM. The algorithm delivers perspective to managers and workers alike about skills and career development opportunities that connect with strategic needs across the IBM portfolio, in areas like cloud computing, AI, and quantum computing.

What’s more, the tool has helped change the content, tenor, and effectiveness of manager-worker communications. Most managers now choose to accept the algorithm’s recommendations, a dramatic change from when it was first introduced. As of early 2019, the improvement in attrition had saved IBM nearly $300 million. In short, predictive analytics has changed not only how the company measures and manages attrition; it has also improved the alignment of behaviors (such as conducting development conversations) with IBM’s strategic objectives to improve skills and increase retention. IBM would not have improved its retention outcomes to the extent it did had it relied on the old way of measuring and managing attrition, via management intuition and HR data.

Predictive analytics can do more than improve the achievement of executive-determined objectives. It can also create new success indicators, transforming how companies define performance. Organizations that obtain substantial financial benefits from AI are 10 times more likely to change their KPIs because of AI than other organizations.2 A ride-sharing business offers a case in point.

Early on, engineers at the ride-sharing company designed an algorithm to maximize revenue by matching driver supply and customer demand. The algorithm looked at all the possible combinations of riders and drivers and picked the combination that would maximize revenue based on the ride being requested, where the driver was located, and all of the system dynamics.3 Then, as engineers began using AI to test other ways to maximize revenue, one model discovered that optimizing conversion rates — the percentage of times a user actually ordered a ride after opening the app — would deliver more ride requests in the future. More ride requests meant measurably more revenue than what resulted from the company’s former emphasis on matching supply and demand. Combining the best human insights with the computing prowess of machine learning, the company improved on a key strategic metric.

The company continued to use its method for matching drivers and customers, but its approach to maximizing revenue now included efforts to increase ride orders after users opened the app. It improved app usability, ran marketing campaigns to increase ride orders, and set new KPIs for ride orders. The company’s choice of measurement tools — such as machine learning — preceded its choice of metrics, not the other way around.

Developing, connecting, and pursuing a blend of analytically sourced and executive-determined measures represents a fundamentally new organizing principle for aligning behaviors with strategy. This approach, what I call predictive alignment, offers a fresh perspective on the changing nature of strategic measurement and aligning organizational behaviors with strategy. In what follows, I discuss several examples of predictive alignment, illustrate how this approach contrasts with traditional alignment approaches, and recommend several practical steps about how to advance predictive alignment in your organization.

Predictive Analytics Fuels Predictive Alignment

Predictive analytics can help identify new leading indicators of future customer behavior. With these new indicators, new perspectives on customer behavior become possible, enabling new ways to create value for customers and businesses alike. It can transform what business performance looks like. At Experian, the credit reporting agency, executives recognized that customer conversion — a traditional metric in its business — had become less effective at predicting desirable outcomes. Under the aegis of the chief marketing and revenue officer, the company began using predictive analytics to establish a new set of metrics for consumer intent, engagement, and loyalty.

While many Experian reporting mechanisms continued to measure conversion rates for transactions and orders, none of the new metrics pointed to a dollar amount or count orders. Adding these metrics required a fundamental change to the company’s culture. One executive told us that the company spent two years changing the culture from one that was chiefly transaction-oriented to one that focused on understanding consumer intent and consumer values. This shift prompted a new set of questions: How often do people come back? When they come back, what features do they use? How often do they lock and unlock their credit reports? The answers to these questions enabled Experian to prioritize investments among various features. The shift also transformed the role of analytics in the strategic measurement process from one that simply reported on lagging indicators to one that also discovered (and defined) leading indicators, such as customer engagement scores.

With new metrics and a deeper understanding of the factors driving performance, organizational behaviors align around these new insights, setting off a virtuous cycle of learning, KPI creation, organizational behavior shifts, increased learning, and so on. A case in point is a coaching app from HR services company ADP, which emails feedback to workers who choose to use it. This machine learning-based tool improved productivity among users at the company by an average of 10%.4 More than 130 companies currently use a version of this tool. Besides helping users, the coaching app provides managers with data about their direct reports’ use of the tool. If using the tool becomes a leading indicator of future performance, additional analytics can determine which managers encourage or support employees to use it effectively. Management might then be evaluated, in part, on whether they follow recommendations generated by machine learning about how to support these employees. The coaching app becomes the source of new measures that beget new management KPIs that enable new management behaviors.

What’s New About Predictive Alignment

To see the difference between predictive alignment and more traditional alignment approaches, consider Robert Kaplan and David Norton’s work on the Balanced Scorecard. Their approach offers what is arguably one of the most sophisticated analyses of the role of learning in strategic measurement.

Decades ago, they argued that effective strategic measurement systems like the Balanced Scorecard incorporate “second loop” learning to refine and adapt strategic, functional, and individual KPIs by questioning underlying strategic assumptions and detecting and correcting errors in these assumptions.5 For example, in Kaplan and Norton’s Balanced Scorecard approach, “strategic learning consists of gathering feedback, testing the hypotheses on which strategy was based, and making the necessary adjustments. … A strategic feedback system should be able to test, validate, and modify the hypotheses embedded in a business unit’s strategy.”6 This view is heavily weighted toward assessing the hypotheses underlying existing strategic goals. It is essentially backward looking. Plus, it doesn’t matter how hypotheses are tested and validated, just that they are.

This approach fails to recognize the potential for large discrepancies in the quality of methods for testing and validating strategic assumptions. A strategic measurement system that lacks a predictive analytics capability may not effectively test strategic assumptions, including hypotheses about what drives KPI outcomes. In the predictive alignment model, it matters how you test strategy-related assumptions: Simply having a strategic measurement system that tests, validates, and modifies strategic assumptions is not enough. With predictive alignment, if how you measure is unsophisticated, what you measure may be unsophisticated as well.

Consider marketing spend, the effectiveness of which can be hard to predict.7 To combat this challenge, GE Healthcare uses KPI outcome data to develop new, more predictive KPIs for optimizing returns on marketing investments.8 As one GE executive explained, “It’s actually boiling out the KPIs from the data rather than setting the KPIs to be measured. We’ll try to derive KPIs from the data and then use that in order to do analysis for targeting purposes, in order to drive commercial impact.” Less sophisticated analytics, or management intuition, would not have effectively tested the assumptions underlying GE’s marketing KPIs.

What’s more, the Balanced Scorecard approach explicitly states that KPIs are set by senior leaders. Predictive alignment is essentially a hybrid approach in which KPIs are both determined by senior leaders (ex ante) and emerge from analytical efforts after strategy is set (post facto). The strategic measurement activities associated with assessing KPIs set by senior leaders and with generating KPIs from analysis require coordination and proper investment. Thus, the two strategic measurement processes themselves require integration and alignment. If management focus and, for example, performance management systems are too rigidly bound to predetermined KPIs, this can subvert alignment around new KPIs (via delay and misallocation of resources). KPIs that emerge from predictive analytics applications require an entry path into an organization’s overall strategic measurement system.

Leading With Predictive Alignment

Predictive alignment has (at least) three significant implications for practice. Leaders taking this approach to strategic measurement should keep the following directives in mind:

1. Expand accountability for accountability. With predictive alignment, managers are accountable not only for their performance on a given set of metrics, but also for how well the strategic measurement system itself can test existing strategic assumptions and formulate new strategic hypotheses. Developing an analytics capability that can test and learn is itself an important objective, and it’s critical to support that with sufficient investment. The upshot: Leaders need to hold themselves accountable for developing strategic measurement systems that can deepen their understanding of factors driving KPI outcomes, generate new KPIs, and adapt to new KPIs. For legacy companies in particular, that requires (a) investing in new analytical capabilities; (b) designing new processes that coordinate analytical insights, KPI adjustments, and new operational behaviors; and (c) identifying managers who will lead these investments and create these processes.

2. Cascade analytics, not just metrics. Predictive alignment is difficult to achieve when different functions within an organization have unequal access to sophisticated analytics tools. The risk of organizational misalignment increases with unequal access to advanced analytics. A member of an analytics group for a professional baseball team who focused on stadium concessions bemoaned the unequal distribution of analytics talent and resources at his organization.9 Unsurprisingly, player analytics received the bulk of analytics investments. Unequal access to analytics can intensify cultural divides, foment worker dissatisfaction, and undermine alignment of worker behaviors with strategic outcomes. Just as traditional alignment approaches have focused on cascading and linking key metrics, predictive alignment encourages the distribution of analytical tools (and expertise) across organizational functions to measure progress toward (and reassess the utility of) these metrics.

3. Include data specialists when developing metrics. Leaders accustomed to sitting down with different levels of management to articulate metrics that advance a given set of goals should begin to include data specialists (such as chief data officers, business intelligence executives and analysts, or data scientists) in these conversations. A key challenge is to integrate data experts with business managers who independently set key performance metrics.

Oberweis, a midsized dairy operation based in Illinois, was looking to expand its high-end dairy delivery service business to the East Coast. As in past years, the CEO brought together his executive group — composed of veteran operations managers in charge of trucks, drivers, and milk transfer stations — to plan strategy and set revenue targets. The group came together believing that they would plan how to reach a familiar demographic — a group they called Beamers and Birkenstocks. However, unlike in past years, the CEO introduced the team to a business analyst in the marketing department who persuasively argued that a different customer segment would drive more revenue. The analyst’s predictions led to new revenue goals and KPIs.

The CEO’s presence in the room was essential to conferring credibility on the analyst’s counterintuitive findings and methodology. The group subsequently shifted its growth plan, adopting new indicators of success. Just as significantly, Oberweis’s veteran operations executives began to seek out the data team to help with other issues after seeing the benefits of its customer segmentation insights.10

Predictive analytics is a powerful new tool for reconsidering a wide range of strategy-related assumptions. Embracing predictive alignment changes strategic assumptions and metrics that realign organizational behaviors toward strategic goals. Including data specialists does far more than help companies identify the right data to include in leaders’ dashboards. Leaders need data specialists to help identify the right data flows and create governance systems so that data required for key metrics has a consistent meaning across organizational silos.

A company’s system of strategic measurement is typically built to support objectives determined by leadership. In this approach, how you measure is less important than what you measure. Ultimately, what you measure — whether it is tracked in dashboards or mapped in three dimensions — reflects how you compete and align behaviors to strategic objectives.11

With predictive alignment, however, how you measure also reflects how you compete and align behaviors with strategic objectives. This is not a subtle shift. Using predictive analytics can expand the purpose of strategic measurement: It enables the discovery of new ways to drive growth, in turn enabling new metrics and new behaviors. Customer lifetime value or attrition means something quite different, and is more valuable, if you measure either with machine learning and big data sets.

Alignment without predictive analytics is comparable to using a sextant to navigate the ocean instead of using GPS. You can do it, but you’ll miss out on a lot of relevant information. Divorcing alignment efforts from a machine learning or predictive analytics program risks not only poor strategic execution but also suboptimal planning; it can be a competitive disadvantage. Investing in how you measure can ensure that what you measure matters.