NPR Pits Robot Reporter Against Journalist, When They Should Have Had Them Work Together
NPR’s Planet Money published a story today on the rise of automatically generated news stories, but unfortunately they didn’t push beyond the "oh, shiny tech" level of fluff.
You can listen to the story over on the NPR website, but the short version is this: Planet Money matched an experienced journalist against Wordsmith, an automated summation/writing platform from Automated Insights. The bot and the journalist were tasked with jotting down a short piece based on an earnings report.
Both stories are posted over on the NPR website for you to read. Needless to say, the bot was faster (< 2 minutes vs 8 minutes) but readers have expressed a preference (by a factor of twelve to one) for the story written by the human.
That is an interesting test, but I feel it misses the point.
Bots can write passable blurbs based on chunks of facts like earnings reports, yes, but we already knew that. Forbes is using a bot to sum up analysts predictions, the AP is using a bot for earnings reports, and a startup called BookStats has been using a bot to turn sports scores into news reports since 2010.
Bots can write basic stories, that much we already knew.
What we don’t know is whether partnering a bot with a human being will result in a better story. That is what is worth testing, but I don’t think it could fit into the 3 minute slot which Planet Money allocated for this story.
I’ve followed the topic of bot-driven reporting closely for the past year and I will continue to follow this topic because I expect that one day bots will be cheap enough and good enough that most journalists, nay writers, will use one to help write stories. The bot will join the spell check and grammar check as just one more tool in the toolbox.
In fact, bot-assisted journalism is already happening. The LATimes is using a program called quakebot to generate seeds of news stories. According to Slate, humans then take over and expand on the stub with additional reporting:
The algorithm’s goal, Schwencke says, is not to write a compelling or insightful story. That’s up to the LAT’s human staff. Rather, it’s to “get the basic information out” as quickly and accurately as possible. That way, “Everybody else can go out and find out: Was anybody hurt? Was anything damaged? What do the people at the USGS actually have to say?”
Which is just what the paper’s reporters proceeded to do. By noon Pacific time on Monday, Schwencke told me Quakebot’s post had been updated 71 times by human writers and editors, turning it from the squib above into this in-depth, front-page story.
See, that is what Planet Money should have tried. That is the real future of robot reporters.
No matter how many algorithms are used to define it, writing is still as much of an art form as it is engineering.
That’s why I see it as a tool rather than a threat. I’d like to see what a real journalist can accomplish when you hand him a new tool, not what happens when you make him compete against said tool.
Will the journalist be better at his job than he was before the bot?
I think some will learn to use the new tools, but as with every new thing not everyone will be able to adapt. But either way, that question is more interesting and more complex than the simple speed race that Planet Money reported on this morning.