From scientific study to public opinion — is John Oliver right?
Science journalism may have its pitfalls, but there’s plenty we can do to make sure studies don’t get lost in translation
By Elisa Nelissen Posted on 13 May 2016
Is coffee good or bad for you? Do opposites really attract? Should you stretch before running?
Depending on who you ask, you’ll get a different answer. Every one of us is influenced, consciously or not, by science stories that are featured abundantly in the media. But those reports are often contradictory, and the findings may be oversimplified or taken out of context.
In the latest episode of the satirical news show Last Week Tonight, John Oliver – in his typical hilarious fashion – exposed the bumps on the road from scientific finding to public opinion. One of the examples he highlighted was a study published in an Elsevier journal which was widely covered in mainstream media, though when you compare the coverage with the actual study, discrepancies quickly reveal themselves.
(Readers outside the US who cannot view this video can watch it on the Last Week Tonight Facebook page.)
Funny as it may be, Oliver’s show understandably hit a nerve for many researchers and science journalists. So what’s the problem with science journalism? Turns out it’s a complex one. Nowadays, the media tends to favor brief, easy-to-digest content that quickly attracts people’s interest. This is necessary, because we are all drowning in a sea of information. On social media, content is slimmed down even further until what remains are short, sharable headlines.
Because the stream of information never stops flowing, journalists are under constant pressure to turn out new articles. As a result, many science journalists hardly have the time or headspace to plough through complex scientific studies. Science journalists are also not (always) scientists – and vice versa – and we can’t expect that their understanding of a subject is the same as the researcher’s.
This is where our newsroom comes in. Publisher and university press offices function as a bridge between scientific findings and the public. Their press releases inform science journalists about new, significant studies of public concern so that they can then report about them in the media. However, a press release should be an invitation to further investigation — a teaser for the science journalists to delve into the matter.
Unfortunately, this isn’t always the case. Journalists often base their stories on the content in the press release only, so the story becomes a translation of a translation. And with every translation, something always gets lost. As soon as a story is trending, the online media machine takes over, replicating the content in ever shorter formats until what remains is one line stating that “men get hangry, women get horngry” (unfortunately this headline was introduced by the word “science”).
Actually, the Elsevier study Oliver mentioned investigated how satiated women with and without a history of dieting responded to pleasant stimuli. Are you still interested to read on?
The researchers chose romantic images as an example of such stimuli. The women’s brain activity was measured while they looked at these images both in a state of hunger and after they had eaten. What they found was that both groups of dieters and non-dieters had increased brain activity while looking at romantic images, though their brains did react differently. The researchers reported a number of limitations to the study, including the small scope (only 20 women participated) and the young age of the participants. Unfortunately, many journalists didn’t include this context, or only noted it at the end of the story. However, the rare readers who kept their focus until that paragraph were likely already convinced by the fun opener.
Even though the full story doesn’t sound as catchy, it’s an important one to tell to avoid hasty conclusions.
But the problem doesn’t lie with science journalism alone, as Oliver also notes. The science world is heavily influenced by the current culture of “publish or perish”: researchers’ output is measured through quantitative metrics, based on which they receive recognition and rewards. Consequently, similar to journalists, publishers are under pressure publish as much as possible. This results in some researchers “playing around” with variables until they get a significant result, which is further meaningless.
Moreover, this system hinders the publication of negative results or replication studies. As Oliver proclaims, “There’s no Nobel prize for fact checking.”
Everyone who shapes, edits and publishes scientific content carries a responsibility to remain as unbiased and nuanced as possible. While a critical scientist may catch the flaws in a colleague’s study, someone with less specialized knowledge likely will not. As the world’s leading academic publisher, Elsevier does its share to tackle these issues. Our newsroom guards over the quality of Elsevier’s press releases and works with journal publishers and editors to ensure that findings are presented correctly. We also make sure science journalists can get access to the research they are reporting about by giving credentialed science journalists free access to ScienceDirect and Scopus. Finally, we actively support Sense About Science, a nonprofit that champions evidence, scientific reasoning and a public discussion of scientific issues.
Most people don’t think about the extensive scientific studies that hide behind these media reports, but that shouldn’t be a problem. Responsible science journalism brings important findings to the public so that they don’t have to.
Helping the public ask the right questions
By Tracey Brown
It’s hilarious. John Oliver raises all the right questions about getting an individual study into context and lands some well-deserved punches. But he’s cheated the audience of the most exciting part: how to ask those questions about how reliable a study is. How can you tell whether it supports the claims on the morning news or press release.
Our Ask for Evidence campaign has been pooling help for people who want to ask on everything from government waste disposal programs to fashionable diets, and we’re seeing a good deal more challenge from members of the public to overstated claims. That’s often not a matter of simple true or false answers but about the weight to give to findings.
People can ask about peer review, about the stats (with help from great outreach such as STATS.org if they need it), about what kind of study it is — a systematic review, a double blind trial, an exploratory study — and what kinds of bias and weaknesses these different studies are prone to. The good news is that you don’t have to be a gastroenterologist, or even a savvy, smart TV presenter, to start questioning claims that MMR causes autism, or to put those questions to the people making the claim.
Read something you’re not sure of? Ask for evidence! Some people might find the final sketch on Oliver’s TODD talks funny. I’m annoyed that such a fantastic setting out of the issues, with all the warnings about becoming cynical of science, ended up throwing its hands up in such a big shrug.
Tracey Brown is Director of Sense About Science.
It’s not just media hype – we all need to do better
By Ed Sykes, PhD
I really enjoy John Oliver’s way of puncturing the big stories. In his science piece, there’s a huge amount of information wrapped up in the comedy – from sample size and p-hacking to the issues of bias, pressure to publish, not explaining that it’s a study with animals and generally taking far too much from each single piece of research. I completely agree that TV breakfast shows love rolling out the puff pieces, and as consumers of the news, it’s really our own fault – we should be demanding more.
Where I would have loved to see this show develop (though I realise there’s only so much that can be done) is to talk about the role of specialist journalists. Whatever type of media, whether in-depth print or drive-time features, the quality of the journalist is often far more important than the outlet. For that reason, at least in the UK, it is generally a rule of thumb that looking at the by-line and seeing whether it is put together by the specialist science, health or environmental journalist will give you a great clue as to the robustness of the work; they do a great job and spend as much time keeping rubbish out of the news as they do writing about the stuff that matters.
Of course, the specialist label is no guarantee of quality, but the media shouldn’t be shouldering all the responsibility anyway; a lot of it falls on the scientists who do the research, the press officers who put together the press releases and the wider scientific community that responds to them. We shouldn’t, for example, be doing press releases and press briefings on poster presentations at conferences in the same way as we do with published journal papers, as happened recently with autism and folic acid supplements. With something of that impact, we should give the data a chance to do the talking by going through peer review, ensuring it’s published in a reputable journal and opening it up to scrutiny from other experts. That’s how science advances. The media has enough problems feeding 24-hour news machines, racing to publish, tight deadlines, deluges of press releases, slashed staffing and increased competition without the scientific community magnifying the problem.
In short, we all need to do better, whether as consumers or producers of the news, because as John so eloquently explains, a loss of understanding of science, cherry-picking of data and the confusion that can arise can have major consequences. Our news is full of debates on e-cigs, antidepressants, air pollution, fracking and diet, and if we want to make the right decisions, the public and policymakers need to be led by the evidence, not just the catchiest headline.
The Elsevier Newsroom: supporting science journalists with updates and access to research
As part of Elsevier's Global Corporate Relations department, the Newsroom is our company's main point of contact for the media. We inform science writers on the latest research papers published in Elsevier journals through press releases, newsletters and free access to ScienceDirect, and we link journalists with discipline-specific publishing experts and journal editors upon request.
Copies of research papers highlighted in our press releases and research alerts are immediately available to credentialed journalists. Those wishing to search for their own stories can do so with a free media code to ScienceDirect, giving journalists online access to all Elsevier journals. In addition, we partner with national associations for science writers to increase our global network of science communicators and reach our joint objective: share scientific outcomes as broadly as possible.
Elsevier Connect Contributor
Elisa Nelissen recently graduated with a master's degree in Book and Digital Media Studies from Leiden University in the Netherlands, with a specialization in Publishing Studies. After an internship with Elsevier’s Corporate Responsibility program in 2014, she now works as a Press Officer in Elsevier’s Newsroom.