The Downside of Real-Time Data

If information is power, you might assume that getting information more often should be powerful, too. Not so fast, say two business school professors.

“In many situations, real-time data comes in on a continuous basis, and then you, as a decision maker, have to decide which data is information and which is pure noise,” says Jayashankar M. Swaminathan, Kay and Van Weatherspoon Distinguished Professor of Operations, Technology and Innovation Management at the University of North Carolina’s Kenan-Flagler Business School. “That’s not an easy task, which is what this study shows.”

Swaminathan and Nicholas H. Lurie, assistant professor of marketing at Georgia Tech’s College of Management, conducted studies of undergraduate students to investigate how the frequency of information reports affects decision making. The researchers discovered that receiving information more frequently led to worse decisions, particularly when there was more “noise” — that is, random fluctuations — in the data.

“Is Timely Information Always Better? The Effect of Feedback Frequency on Decision Making,” an article forthcoming in Organizational Behavior and Human Decision Processes, describes the first study and establishes the fundamentals. Seventy-six students played a game in which they were retailers stocking the shelves with a perishable hypothetical product. Different sets of participants placed orders for periods analogous to either every day, twice a week or once a week. The participants were then shown the actual demand during that period for the product, and the demand fluctuated randomly within a predetermined range. The students were also shown their resulting profitability figures for the period. After that, they were asked to place orders for the next period, over the course of two 30-day games. No unsold inventory could be carried over from period to period.

Lurie and Swaminathan found that participants who received reports and placed orders daily had lower profits than their peers who got reports once or twice weekly. To put it another way: Even though all the participants received the same granularity of information — daily sales — those who got the information every day, as opposed to every three or six days, made worse decisions. This was particularly true when there was a high variance in actual demand.

What seemed to be happening is that students with daily information were more likely to give too much weight to the previous day’s data in making their decisions, rather than looking at a longer time period.