Rebroadcast: How to cut through the 'noise' that hinders human judgment

This rebroadcast originally aired on June 1, 2021.

If you consult three doctors and get three different opinions, that’s an example of what Nobel Prize-winning psychologist Daniel Kahneman and his colleagues call “noise.” How do you decide what to do when professionals don’t agree? We’ll cut through the noise and exploring human judgment.  

Daniel Kahneman, Nobel Prize-winning psychologist. Co-author of "Noise: A Flaw in Human Judgment." (@kahneman_daniel)

Olivier Sibony, professor of strategy and business policy at HEC Paris. Co-author of "Noise: A Flaw in Human Judgment." (@SibOliv)

How do we define what noise is?

Daniel Kahneman: “We define noise as unwanted variability in judgments that should be identical. And that's the broad definition. So your three physicians made judgments about the same case, and we would expect them to give identical answers. The fact that they're variable is an indication that something is wrong with the system."

On how 'noise' differs from bias

Olivier Sibony: “There is actually a very easy way to think about it. And it's to think of an example of measurement as opposed to judgment, it's easier to figure it. So suppose you step on your bathroom scale every morning and on average your bathroom scale is kind. It tells you that you're a pound lighter than you actually are on average every day. That's a bias. That's an error of a predictable direction. And on average is the error that your skill is making. Now, suppose that you step on your bathroom scale three times in quick succession and you read a different number each time. That is random variability of something that should be identical. It is noise.

“Now apply this to judgment to see the difference between the bias, which is the average error. And the noise, which is the random variability in the judgments. Suppose that we're making a forecast of, say, where the GDP growth is going to be next year or something like that. If on average, all of us who are making these forecasts tend to be optimistic, that's the bias we overestimate. That's an average error. But each of us is going to make a slightly different forecast. The variability between our forecasts is noise. So it's really quite simple. Bias is a predictable error in a given direction, it's the average error of a number of people or a number of observations by the same person. Noise is the variability in those observations.”

What is a 'noise audit'?

Olivier Sibony:  “A noise audit is not a way to reduce noise. ... It's a way to measure noise. So when we gave the example of the underwriter's or the example of the justice system, these are noise audits where you get a feel for how large noise is. And the reason you need to do that is that ... we don't imagine that people see the world differently from how we see it.

"And therefore we can't imagine that there is as much noise as there is. Because if I am a judge, I never hear what another judge would have sentenced this particular defendant to, because each defendant is unique. And if I'm a doctor and I look at an X-ray, I never imagined that another doctor looking at the same X-ray would see something different from what I see. So a noise audit makes this visible and tells you exactly how much noise there is in your system.”

Are there things that individuals can do to reduce their contribution to noise?

Daniel Kahneman: “Well, there are. Our idea on this matter is really quite straightforward. And it applies even to individuals' decisions, not only to systems. It applies to singular decisions, to strategic decisions that people make. And our argument is quite straightforward. If decision hygiene procedures work in repeated cases, there is every reason to believe that they will apply as well to unique or singular cases. So decision hygiene recommendations are applicable to any judgment that people make. The argument, which actually Olivier, the one who had that phrase, that we are very grateful for, that the singular event is a repeated event that happens only once. So that everything that we say about noise, as repeated events, is actually applicable to individual judgments.”

Excerpt from "Noise: A Flaw in Human Judgment" by Daniel Kahneman, Olivier Sibony and Cass R. Sunstein. Copyright © 2021 by Daniel Kahneman, Olivier Sibony and Cass R. Sunstein. Reprinted with permission of Little, Brown & Company.

Wall Street Journal: "Good Moods Often Lead to Bad Judgments" — "We’ve all noticed that our own judgments can depend on how we feel—and we are certainly aware that the judgments of others vary with their moods, too."

New York Times: "Bias Is a Big Problem. But So Is ‘Noise.’" — "The word 'bias' commonly appears in conversations about mistaken judgments and unfortunate decisions. We use it when there is discrimination, for instance against women or in favor of Ivy League graduates."

Washington Post: "How to turn down the noise that mars our decision-making" — "A friend of mine was suffering such severe back pain that it was difficult for him to walk or stand. He consulted three doctors about the best course of treatment."