#data-driven journalism

Knight News Challenge OI Engine: How the Platform Works from Knight Foundation on Vimeo.

The Knight News Challenge accelerates media innovation by funding breakthrough ideas in news and information.

By Jon Schwabish

A quick story: A few months ago, I was chatting with one of my former graduate school professors. I was telling him about a client who was concerned that his organization wasn’t selling their work enough; that their reports weren’t being as widely read as they had hoped. “See, that’s what’s great about being an academic,” he said, “I don’t have to worry about selling my stuff.”

I sighed, having heard this sentiment many times. What’s ironic is that so many researchers want their work to be widely read and, importantly, to be used by people in the policymaking process. Yet, many researchers (especially, academics it seems) don’t yet value the importance of “selling” their work. By that I mean thinking strategically about trying to get people—other than their colleagues and other researchers—to understand and use their analyses.

Changes in, and the evolution of, such fields as data visualization, presentation design, and data-driven journalism ventures are teaching us that people are willing to read data-rich writing and research. That a market exists for analysis that extends beyond traditional journalistic storytelling means researchers and academics potentially have a wider audience for their work.

One of researchers’ main advantages, then, is their ability to work deeply with data, understand its challenges, and conduct thorough statistic and econometric tests. Moving from the research stage to the communication stage requires some understanding of—and at the very least, some appreciation for—the needs of a wider audience. Does the audience want visuals (such as infographics) to accompany the work? Do they want stories to help them relate to the analysis? How can interactive data tools, interactive visualizations, videos, or podcasts help aid in the wider dissemination of that research?

It boils down to this: Less editorial judgement, more algorithmic programming.

Key passage:

Imagine, for example, someone who claims not to be interested in art and yet is twice as likely to click on an article if it has “art” in the title. A programmatic newspaper would notice a pattern like this and deliver more art news to the user. Or imagine that you stay on a page twice as long when a certain athlete is mentioned in an article. For a programmatic paper, that’s valuable data that can be used to serve you content about that athlete. Likewise, if you search a medical concept on Google, a programmatic paper can start to include articles on the topic on its home page or deliver them to you in your personalized emails.

An op-ed in yesterday's New York Times politely describes the effect Nate Silver had as "against the grain for some at the Times."

More like: "he was an antidote to everything that is wrong with today's mainstream political coverage."

Silver talks about it in his marvelous book, The Signal And The Noise, which aptly characterizes political punditry for what it is: shouting "predictions" with odds rarely better than a coin-toss and absolutely no accountability after the fact.

Reason, Not Blather

FiveThirtyEight was not just a brand that wonks and political noobs alike flocked to, it's one of the few voices of reason at a time when a Rally to Restore Sanity still sounds like a great idea. While cable-news channels are desperate to hold our attention, the Internet has long since drank that milkshake, and the era of data-driven journalism means blowhards on the left and right have a lot less to blather about.

Who cares who "won the day?" Who cares what any of these pundits think the American people feel? Sure it's entertaining, but give me five minutes of data-driven journalism followed by twenty-five minutes of cat photos and I'm coming out smarter than any of your thirty-minute yell-fest-segments.

In '08 and '12, whenever a grand pundit's prognostication conjured its way into my newsfeed, I'd calmly hop over to FiveThirtyEight and remind myself that it was just another desperate ringing of the bell to keep me salivating. It reminds me of MBAs creating and perpetuating special jargon just to keep people believing 'serious business' only happened to people who had just the right ingredients and access.

Putting On the Robe and Wizard Hat

And then it happened to me. The Washington Post asked me to call the '12 election along with a slew of other folks, some "professional" pundits, others amateurs. I was clearly the latter, so I pulled my electoral vote predictions from 538 (sorry, Nate). How'd it go?

"Alexis Ohanian, a co-founder of the social news site reddit.com, had the best electoral vote prediction."

But I ultimately lost because the Post asked a bunch of tiebreaker questions Nate didn't publish answers for, so I had to guess, err, you know, be a pundit.

The person who ended up winning was Matt Matros, whose career shouldn't surprise you: professional poker player.

Bet On Data, Please!

Ultimately, I wish Nate the best (and realize he'll have far more work year-round at ESPN than he does tracking elections), and hope the Times drafts or promotes someone to follow his lead. Especially because sites like Priceonomics are built on data from the ground up.

They'd better, because this is the future of journalism (especially political), and Tom Friedman's random sample of taxi drivers ain't gonna cut it.