Scientists should not be given the right to fact check the press

Last week, Dr Petroc Sumner, Dr Frederic Boy and Dr Chris Chambers, all from the School of Psychology at Cardiff University, argued in the Guardian that “Scientists should be allowed to check stories on their work before publication”. The only sensible and rational response to that is “No, they should not.”

Sumner, Boy and Chambers argue that scientists should be given veto because:

  1. Scientific papers are peer reviewed so have already been scrutinised by independent eyes, implying that journalists therefore don’t need to be themselves independent when reporting such papers.
  2. There are no political parties in science, ergo there can be no ‘conspiracies’
  3. Scientists have nothing to gain and a lot to loose from exaggerated claims in the press
  4. It’s the only way to ensure accuracy

What utter tosh.

There is no doubt at all that quite a bit of science journalism is appalling, riven by inaccuracies, biases and sometimes just complete twaddle. You only have to read Ben Goldacre’s Bad Science to see what a mess journalists get themselves into. The problem is that Sumner, Boy and Chambers are engaging in special pleading on the basis of four flawed premises:

Firstly, scientific papers may well get peer reviewed, but that doesn’t mean that they are correct, it simply means that they have been looked at by some other scientists who either can’t find or won’t find fault. Papers get retracted when problems come to light later on, so peer review is not a guarantee that a paper is correct.

Secondly, the idea that there are no lines to toe in science is utter bunkum. There may not be political parties but there certainly are scientific orthodoxies, and that means lines to toe. The fact that something has become orthodoxy does not, in and of itself, guarantee that it is correct. Prevailing theories do get overturned when new evidence comes to light and, whilst those who make extraordinary claims need extraordinary evidence to back them up, science has a rich history of exactly that.

Point three only needs to be answered with one word: Ego.

And point four shows a spectacular misunderstanding of the journalistic process and the factors that cause errors and misinformation to propagate. Let’s take a quick look at some of these (I’m sure there are more, please do add them in the comments!):

  1. Good journalists on tight deadlines have little time and few resources to do comprehensive reading and research on a story, so mistakes,  misunderstandings and inaccuracies can easily creep in to the work even of the most dedicated.
  2. University and research institution press releases, and sometimes even scientists themselves, can be misleading. Sometimes those inaccuracies are picked up by the journalist, sometimes they make it through to press.
  3. Some hacks and editors don’t actually give a shit whether something is accurate, they simply want a shocking or outrageous story that they think will get them lots of readers.
  4. Some hacks have financial relationships with the companies whose “science” they are writing about, destroying any vestiges of impartiality they might once have claimed.

Which of these scenarios would actually be helped by adding an extra layer of “fact checking” by scientists? In the first case, the journalist doesn’t have time/resource so the fact checking just isn’t going to happen. In the second, the fact checking would be undermined by the very people doing it as they would only propagate their own inaccuracies. And again, in the third and fourth scenarios, facts are irrelevant, so why would they get checked?

There are things that would help, however:

  1. Additional time and resources for science journalism and appropriate training . This is frankly never going to come from the news organisations themselves because they are all struggling to survive and science is seen all too often as a minority sport. It might be that a well-respected science communications charity or NGO could fund training for journalists wanting to cover science, e.g. in how to interpret papers and how to understand statistics. They would also need to fund the journalist’s actual work, ensuring that they had the time and resources required to do a good job.
  2. News organisations need to take complaints about inaccuracies more seriously. Even the so-called quality newspapers don’t always pay any attention to readers who point out problems in science stories. Often, then will officially stand by the most egregious bullshit because they’d rather not have to deal with the fact that they got it wrong.
  3. Some sort of standards commission with real power should hold all news organisations to account, forcing them to make corrections and imposing significant fines for the most egregious misbehaviour. I’d say “Maybe the PCC could do this sort of thing”, but they’re a spineless, toothless waste of time. If there was any censure at all of misinformation in the media, some of which is actively harmful to the reader’s health, maybe this conversation wouldn’t be dragging on and on for years, as it actually is.
  4. Oh, and here’s an innovative one: Maybe university press departments should stop sexing up press releases, liaise more tightly with their own scientists to get their facts right, and provide any relevant photos, graphs, graphics and data in reusable formats with a clear and concise explanation of what they illustrate. Of the press releases covering scientific topics that I’ve had to work from, none gave me even half of what I needed.
  5. Scientists should be responsive to press enquiries, and should prepare FAQs, lists of relevant links and re-usable quotes about their research up front. Their papers should be available, along with a proper bio and even photos, just in case. Whilst most of the scientists I’ve dealt with have been very responsive, none have sent me a link to a website with background info and relevant resources on it that I could use to quickly bring myself up to speed before asking them specific question. It would have saved everyone’s time if they had done that. (Plus it’d be a great thing for me to link to in my articles!)

Science journalism isn’t actually a collaboration between scientist and journalist, it’s a process of interpretation which depends on both sides being independent of the other.

So what would happen if Sumner, Boy and Chambers got their wish?

Well, where there’s orthodoxy there’s the opportunity for new ideas and voices to be suppressed when they come into conflict with established bodies. Although science is supposed to be immune to this, it does sometimes – thankfully rarely – happen. If there are no independent science journalists, there’s no opportunity for new voices and evidence to be heard.

When scientists get it wrong, which they do, we need science journalists like Goldacre need to be able to criticise their methodologies, assumptions and conclusions free from interference. If Goldacre had to pass everything he wrote past the people he was writing about, he would get almost nothing published.

In short, we would see the wholesale marginalisation of dissent, and not just dissent from the journalist, but from opposing scientific voices too. It would be, in short, a disaster for science and science communication.

(Hat tip: Glyn Mottershead.)

7 thoughts on “Scientists should not be given the right to fact check the press

  1. Great post. I think points 2 and 3 of your things that can help are really important. Otherwise all the things that the scientists can be accused of go double for journos.

  2. In a curious twist, though probably far from a changing of the guard, it seems that the comments to that Guardian screed are more level-headed than the screed (Sumner et al.’s proposal) itself… Most issues with the presentation of research appear to stem from misunderstandings of the statistical significance of the work, which is not helped by the statistics behind the findings being dropped from any reportage (e.g., the dreaded University press release) outwith the original document.

    There are many problems within scientific publication, but I would suggest that peer review is far from the top (cf. hyper-inflated impact factors for journals, sizeable and ever-increasing pay-to-publish fees, and a never-ending glut of journals). And some of the basic premises of peer review – consider everything at the outset to be bullshit – could help the problems of sensationalism at the source and further down the communication stream.

    But problems of sensationalism will persist, and the engaged scientist will have to be persistent in rebuttal. Here’s one recent example:
    The only thing missing from the NYT’s posting of the rebuttal is that there is no link to/mention of it in the offending article’s page. As you point out (Points 1-3), with a framework for responding to disputes, this could be resolved. What sort of resolution, though? In the case above, the offending article’s basic premise was flawed (i.e., there was a misunderstanding of what innervation in particular areas of the brain can mean), so redaction might be the best tact. Other articles’ flaws may not require such strong measures.

    If Points 4-5 were heeded along with the first three, it may just be possible to spread a bit more rational thought in this so-called information age. Perchance to dream.

  3. I agree journalists should have no compulsion to vet their articles with the scientists whose work is discussed. Often, journalists run a patch of text past the relevant scientists when the discussion covers complex subjects, but the POV of the article should be the journalists alone.

    Basically, science articles are interesting for one of two reasons – either there is a new event or result or there is a dispute over a previous result. In the former case, the scientists nearly always overstate its impact, in the latter case the journalist has the role of mediating the discussion to lead it to a realistic portrayal of the current understanding.

    In either case, the writer as arbiter of the case must be independent of many of the people involved in the scientific turmoil.

    And frankly, if the writer fails to properly interpret events, their reputation will rightly suffer, and most off-target articles are ignored rather than causing great damage.

  4. Hey Andy, thanks! Yeah, i’d love to see proper press… well, regulation is the wrong word, but some sort of whip-cracking organisation that can hold journalists of all stripes to account when required. It’s clear self-regulation in the UK really isn’t working, which is a bit of a shame.

    Bill W, totally agreed that dropping the stats from the reportage or press release doesn’t help matters, but there’s also the issue with journalists understanding the stats, or in some cases, twisting them to sound more impressive. And it goes again with the concept of some sort of industry standards body that would provide guidelines for how rebuttals, apologies, and corrections should be dealt with, not to mention when. So often, inaccuracies are subtle but important, yet totally ignored when pointed out. Stats are actually a good example of that – the old “33% increase” which is actually referring to 3 becoming 4.

    Hey John, good to see you over here. I do sometimes run sections of text past my contacts in order to ensure that I’ve understood things properly, but I can only do that if I’m writing a feature, not if I’m doing news. If i’m working a news story, even if it’s really just an opinion piece, I just don’t have time to double-check. Personally, I find that very frustrating indeed as I like to do a good job, but the realities of the industry are such that time can’t be afforded. I also agree that there are subtle forces that influence the way that scientists will discuss their results, just as a journalist’s own prejudices influence the way that they will report them. Ultimately, there are things that can be improved on both sides of the fence, but independence is key.

  5. Nice posts. Your blog is now bookmarked – if only iCloud were synching my bookmarks as well as MobileMe was synching them last week.

    Not infrequently, journalists read me a few sentences covering the technical details on which they are unsure to ensure accuracy near the deadline. But yes, time pressure is paramount.

    I’m amazed how many scientists think they are objective even as they disagree with other scientists. And the bane of earthquake hazard estimation are specialists who think they are so obviously right that even as the only person who holds their views, their conclusions should dictate the hazard estimates produced for engineers. Actually, it would be fun to interview the scientists just to be continually amazed at our outsized egos and overconfidence.

  6. Thanks for your kind words, John! Though Kevin should take most of the credit – I’ve been kinda quiet here lately due to work, moving house and organising Ada Lovelace Day.

    Overconfidence is everywhere, sadly. Especially in my specialty (social technology) I see the fingerprints of the Dunning-Kruger Effect all over the place —

  7. Thanks for the official name and link. Good to see one’s intuition stand up to some experimental testing.

Comments are closed.