The Quantification Bias

Numbers can’t be treated like words, but there’s clearly a need for both of them in journalism. We just need to find a better way for them to live side by side.

Is it possible for a data-driven journalist to tell a good story? If you’re calling an election or breaking down a sporting match, good, rigorous data analysis can shine. But tackling the rest of the news? Or for that matter, culture? Traditional journalists might be uniformly bad at maths, but does maths matter when crafting a narrative? In the backlash against FiveThirtyEight, a few themes have emerged. There’s the idea that all the data analysis in the world can’t always tell a good story – sometimes the numbers don’t offer any space for strong conclusions. There’s the idea that data can never tell the whole story – that the reader needs more to grab onto than the aggregate. Silver speaks of the difficulty in telling stories from “generalisations” that aren’t steeped in data; I’d argue it’s an enormous challenge to tell any story with only generalisations.

In fact, as the former JSOC drone operator recounts, tracking people by metadata and then killing them by SIM card is inherently flawed. The NSA “will develop a pattern,” he says, “where they understand that this is what this person’s voice sounds like, this is who his friends are, this is who his commander is, this is who his subordinates are. And they put them into a matrix. But it’s not always correct. There’s a lot of human error in that.”