Noise

Daniel Kahneman, Oliver Sibony and Cass R. Sunstein

HarperCollins

Otago Daily Times, July 24th 2021

It sounds like the start of a bad joke: A psychologist, a business strategist, and a behavioural economist walk into a bar and try to stop the noise by writing a book about it. The ‘noise’ they are concerned with is not aural, however, but refers to the variability between judgements made by different people – or the same person on different occasions – when presented with an identical scenario, and its consequences are far from a laughing matter. Decisional noise is a significant source of error in everything from hiring decisions to criminal sentencing, and even in ‘hard’ disciplines such as forensics and medicine, even an experienced practitioner can be influenced by factors that have nothing to do with the situation at hand. When averaged out across multiple decisions, the results may converge on a ‘correct’ outcome (or not, depending on whether there is also bias in the system), but such variation is far from fair at the individual level and costly for companies, organisations, and society.

In Noise, Daniel Kahneman, Oliver Sibony and Cass R. Sunstein use a combination of real-world examples and thought experiments to describe the various types, sources, and consequences of judgemental noise, then suggest ways to measure and reduce it at both an individual and institutional level. As you might expect, the solutions are complex. Calling on the ‘wisdom of crowds’ in the shape of independent opinion can improve judgment, but groups can also amplify noise. Guidelines and checklists can be diagnostically helpful in medicine but perform poorly in psychiatry, which involves subjective rather than objective symptoms. A well-designed diagnostic formula or algorithm can outperform professional judgment in many cases. Still, at this point, such models are not that much better than clinicians in terms of the accuracy of their predictions, although the authors are optimistic that this will improve with the adoption of more complex models in the future.

Despite this, Kahneman et al. remain steadfast in their insistence that the unfairness and costs of judgement noise are so high that any improvement is better than nothing. In their view, there will be a noise-reduction strategy that could reduce unwanted variability in most cases without impinging on individual creativity, human dignity, or perpetuating pre-existing bias. I am not quite so optimistic, but I admire their idealism.

Although written in the chatty style common to popular science books, Noise is information-dense and, at times, complicated for the lay reader. In an attempt to simplify things, the authors have divided it into six sections, moving from causes to solutions so that readers can choose whether to explore the structural and psychological basis of (mis)judgement or move directly to practical applications of noise reduction. Every chapter concludes with a list of key points, and the book closes with a brief summary of each section, an appendix to help readers conduct their own noise audit, and a checklist to help detect bias in decision-making. The net result is decidedly ‘mansplainy’, but after reading it, I have become very aware of how strongly my mood or student’s writing style influenced my response to the content of their assignment and how helpful a checklist was for moderating my marking. I may not agree with everything Noise has to say, but it is a book I will retain for future reference.