Many journalists see numbers as ‘research rather than journalism’

That’s a paraphrase from my former colleague and good friend, Simon Rogers, the editor The Guardian’s Datablog. Simon is a true champion of best practise using data in journalism, and here is what he had to say to Chris Elliot, The Guardian’s Readers’ Editor:

First, I think there’s a cultural challenge. Many journalists are arty types who traditionally have thought of anything to do with numbers as ‘research’, rather than journalism. That’s combined with an unwillingness to ask difficult questions about data, or read the notes that get attached to spreadsheets [that journalists receive]. This doesn’t apply to everyone – the specialists know all about this.

I applaud Simon for his frankness. As Suw and I talk about frequently, many of the issues that we deal with in our work are relate to culture. I wrote recently how journalists’ identity is often a barrier to the adoption of technology, and in some ways, technology and statistics are lumped into the same bucket by a number of journalists (unless you’re a stats-junkie sports journalist).

Chris also interviews Ben Goldacre, author of the Bad Science column in The Guardian, for his column about statistics. Ben hits on one of the issues that drives me nuts about my profession, statistics inflation. Sure, we can shout to the hills about a 200%  CORRECTION 100% (thanks Vincent) increase in the number of children who have been killed by albino elephants in zoos, but if that dramatic increase is from one child to two children, it’s not really a story. As a journalist, I can spot statistics abuse from a mile off, and I tend to think that many readers can too. Big percentages are always a tip off, especially if the reporter obscures or leaves out the actual figures entirely. Ben also raiss the issue of dealing with relative risk.

Another issue raised in the article is basic innumeracy in journalism. It’s shocking to see how often journalists conflate mean with median or use mean when it’s actually not representative or skewed by outliers. Mean is a simple average whereas median is a the middle value in a set of values. Depending on your set of numbers, mean and median can diverge by quite a lot. It’s not a hard and fast rule that one is better than the other. It’s worth checking the distribution of values first to decide which one is more representative of the data, and reasonable people can disagree about which one is more representative as Chris Elliot points out in the piece. The problem is that too few journalists know the difference.

There is a comment on the article from a biological scientist that is worth reading:

May I just ask why Journalists don’t have to study a minimum level of statistics before they are employed?

Don’t you people have to have some common level of understanding of the world you live in before you describe it.

And the:-

“who traditionally have thought of anything to do with numbers as ‘research’, rather than journalism.”

I take (that) means (sic) your writers value writing more than they value understanding what they write about.

I can see why journalists score as well as they do on respect polls.

I’m a biological scientist, I have have had (sic) to study statistics and ethics as part of my training.

I have to take and pass courses on toxins/hazards, clinical ethics, animal ethics and quite a few other courses, every year.

If I were to treat choice of mean, medium or mode as a matter of personal choice I would be torn apart by my referees.

With the number of choices that people have for information, we journalists need to step up our game. We do need to do more to understand the world we live, ask tougher questions and be more serious about the flood of numbers that inundate us everyday. Again, as a journalist, I know when a reporter has written around a hole in their reporting, relating to numbers or not. It’s pretty easy to spot. (I’ve had to do it myself.) It’s foolish to think that our readers can be duped so easily.

6 thoughts on “Many journalists see numbers as ‘research rather than journalism’

  1. I think there are two problems here. It’s true that a lot of journalists (and quite a few bloggers) look at a graph and think “ah, so they’ve got proof!”: a graph or a table is a signifier of scienciness, as you might say. But the underlying problem is that some journalists, who do know their way around stats, still disrespect the figures enough to use them as a resource to be cherry-picked – a decrease in the rate of decline can make a nice positive upward slope on a graph, for instance. I don’t think it’s dishonesty in most cases, just a feeling that the figures don’t *really* mean anything – so you might as well use the ones that look best. The worst of it is that digging behind this kind of presentation of stats becomes a job of research. A recent example is discussed here and here.

    PS Hi Kevin, hi Suw. Haven’t commented here since I stopped working in taxonomy/folksonomy/ethnoclassification/etc, back in 2008. Two years later, the blog still looks interesting!

  2. Innumeracy is a huge issue in journalism. We still have journalists who will proudly say they’re not “numbers people;” imagine one of them trumpeting the fact that they’re “not good with words and meaning.” And we’ve condoned it for too long.

    Journalism education really needs to build in numeracy – everything from what numbers mean to basic statistics to understanding data – as a core part of any curriculum; and beyond that, it should be a core part of any reporting class, in the sense that journalists should be trained to think of what numbers or data would support (or disprove) their story, and then go out and look for it. It’s surprising how many journalism programs don’t. Or perhaps it isn’t.

    To Phil’s point, above, there’s not enough understanding – or respect – for information graphics that holds people back from distorting a graph to make a point. At the good outfits there is; but too often someone will want to play with the scales on a graph to make something look “better.” We need to educate journalists on that, too.

    And that just gets us to the starting gate; in this age of data, we need to learn many more skills to be able to acquire, analyze and understand numbers better if we’re to do our jobs properly.

  3. Hear, hear Reg. There are a lot of things that we have condoned in the business that we simply can’t any longer.

    I’m ashamed to admit that got through university without a proper statistics course, but I’m working on remedying that.

    I have heard too many journalists wear their ignorance of the ubiquitous technology of our age as a badge of honour. Of course, oddly, field journalists have always had to know a bit about technology to file their stories, at least during my career. It’s always struck me as odd when journalists are proud to admit a lack of knowledge. Of course, it’s not expected that they know everything. However, isn’t curiosity one of the prerequisites of the job?

  4. “Sure, we can shout to the hills about a 200% increase in the number of children who have been killed by albino elephants in zoos, but if that dramatic increase is from one child to two children, it’s not really a story.”

    Actually, this is a 100% increase. A 200% increase would be from 1 to 3 children. Oh, well 🙂

  5. I should say that I whole-heartedly agree with the points you make. I’m one of the literary types whose understanding of maths stops well short of trigonometry. But even I have often been shocked by the cluelessness of many journalists when it comes to numbers and the related reluctance to draw their own conclusions from looking at raw statistics, rather than just echo the numbers selectively quoted in press releases and other self-serving marketing materials.

    In general, the lack of numeracy is a problem for society at large. It would be a huge step forward to educate journalists, but as long as the general population can be misled by transparent errors or take meaningless numbers at face value, there will be a temptation to play to these vulnerabilities. There’s an in-built safety mechanism with sports, because end-users are experts, but sadly not with science or economics.

Comments are closed.