.png)
.png)
.png){kind=logo}
By Anna SayburnDuring your career as a doctor, patients, friends, or family members will no doubt ask for your advice on the latest health claims made in the media. Does a certain food or lifestyle choice cause cancer? How effective is this vaccine? Will this procedure cut my risk of getting ill? The questions are endless. But often we can’t trust the headlines at face value, and the consequences can be serious if we do. To respond appropriately to medical stories in the media, you need to understand how medical research becomes news and to critically appraise a story to assess its accuracy.
Most medical research stories in the media originate from press releases. These are sent to journalists by the researchers’ institution, the medical journal that published the research, the sponsor (often a drug company), or a medical charity.Press releases are written by a press officer and agreed by the team that did the research. They summarise the newsworthy aspects of the research and present selected findings. Press releases often include a quote from a researcher. The original study isn’t always sent out with the press release, although some journals provide a link to the paper. Journalists may be offered telephone interviews with a researcher.Press officers know which medical studies will get picked up by the media. Emma Dickinson, press officer at The BMJ, says topicality and controversy are important, but one major criteria is how many people are affected by a paper’s findings.“Lifestyle papers usually go big; anything about diet or exercise, anything to do with sex or shift work. Things that affect a lot of people. Also commonly used drugs like aspirin or antibiotics. Anything on pregnant women, or kids and TV watching,” she said.In an ideal world, an experienced medical journalist would read the press release, appraise the clinical paper, then talk to the researchers and to other experts in the field, before writing a balanced assessment of the findings. Dickinson says health correspondents “really do try” to ensure they do this. However, time strapped journalists sometimes resort to “churnalism,” where they uncritically repeat the story in the press release, regurgitating the quotes and figures provided. This means key questions go unasked and unchallenged.
How can you tell whether a story is an unsubstantiated health scare, a puff piece by a press agency, or a genuine medical breakthrough? Here are five crucial questions to consider.1. Where’s it from?Emily Jesper, associate director of science watchdog Sense about Science, says: “Firstly, was it published in a peer-reviewed journal? That’s a really good way to cut through the noise.” If it hasn’t been published—for example, a report from a medical conference—it hasn’t gone through the peer review process and should be treated with caution.You should also think about conflicts of interest. “Ask where it comes from, particularly when it is a survey,” says Jesper. “So if it’s a story about tea, is it funded by the tea industry?” Rob Cook, medical director for the company that writes NHS Choices’ Behind the Headlines service, agrees. “If it’s got a campaign behind it, I get pretty wary.”2. What did the researchers do?The study design can help you assess whether the results are likely to be reliable. Check how the study was done, how many people were involved, and where it was carried out. A good quality randomised controlled trial with a statistically significant outcome can tell you whether an intervention is likely to work. Unfortunately, some press releases report positive results without admitting that they are not statistically significant. Watch out for the clinical significance of the results—a tiny change in cholesterol, for example, may not be enough to make a difference to people’s health.Many studies that hit the headlines are observational, which means you can’t be sure that the outcomes are caused by the factor that’s of interest, rather than a confounding factor. For example, a study that shows people who exercise daily are less likely to get cancer could be confounded by the fact that they are also less likely to smoke. Good studies should control for this. “Correlation is not causation is the main one to look out for,” says Cook.3. Where does it fit into the science?Cook says medical students need to be able to identify whether this is early, concept building research or a later clinical study that indicates a new drug is close to launch. “It’s important to be aware of the research pipeline, so you know animal studies are at the start, not the end point of medical research.”Ask whether the new study confirms something that other studies have suggested before, or is a surprise finding that contradicts previous research. Most research builds on previous work in some way, and good news stories explain this.4. Is it relevant?If it’s research done in animals, the chances are it isn’t directly relevant to humans. All we know is the effect on the animal tested. It might not be appropriate to extend the application of the results reported.Cook also warns that stories to do with diagnostics and screening are often reported in a misleading way. “For example, if a study [on tests to diagnose an illness] is done in hospital, and then extrapolated to the whole population,” then the risks and benefits of the test will be skewed, because the chances of benefiting from the test are much smaller in a general population of largely healthy people.5. Is it corroborated?Ideally, the reporter will have spoken to independent experts in the field to get their take on the research and where it fits into the current state of understanding. “Look to see if it includes expert comments from [for example] Cancer Research UK or the British Heart Foundation,” says Jesper. Independent experts will be able to comment on the context of the research and whether it changes the current state of knowledge.
Critical thinking is crucial when approaching medical news. Although journalists don’t set out to mislead, they often have to get across a complex story in a few words, which can lead to oversimplification. They also need to make the story sound exciting, so caution about study results may not be reflected.Doctors are often the first port of call for patients worried about headlines in the newspapers or with false hopes raised by overly optimistic reports. They need to be up to date with medical news and to be skilled at reading reports critically, so as to advise and reassure patients in an informed manner.
Nick Timmins, senior fellow at the King’s Fund, and former public policy editor at the Financial Times, warns: “You have to remember that the headline is just advertising to get you to read the story.” Headlines are usually written by a subeditor, who will not have researched or written the article itself.
Jeremy Laurence, former health editor at the Independent, says: “I often read the headline and then go straight to the end to see the quote from the Department of Health.” Often the reporter puts in any denials, caveats, or cautions right at the end.
Timmins says his first question is always, “Whose byline is it? You get to know the good journalists, the ones worth following.” Health and science correspondents build up expertise and develop a healthy scepticism for sensational claims in press releases. They are more reliable than general reporters, who are more likely to accept a press release at face value.
Source: Dunn W, Walters M. A randomised crossover trial of the acute effects of a deep-fried Mars bar or porridge on the cerebral vasculature. Scottish Medical Journal 2014Methods and results: 24 volunteers ate either a deep fried Mars bar or porridge, and were then tested by ultrasound to calculate breath holding index, as a surrogate measure of cerebrovascular reactivity. The results were not significant, although they reached significance when women were excluded from the study.Context: High sugar, high fat food is unhealthy, and impaired cerebrovascular reactivity is associated with stroke risk.Relevance: The study was carried out in healthy men (14) and women (10), average age 21 years.Corroboration: A campaigner for healthy eating is quoted, advising, “If you are going to eat a Mars Bar, don’t batter it first.”Sceptic scale: Medium. The story is based on research in a peer review journal and is not presented too seriously. But it fails to make clear that the overall results were not significant. The headline is misleading—no one in the study had a stroke.Original article: Eating a deep fried Mars bar could give you a stroke in minutes
Source: Spence AR, Rousseau M-C, Karakiewicz PI, Parent M-E. Circumcision and prostate cancer: a population-based case-control study in Montreal, Canada. BJU International 2014 (but the source was not given in the report).Methods and results: A case-control study matched 1590 prostate cancer cases with 1618 controls. The researchers calculated odds ratios between circumcision, age at circumcision, and prostate cancer risk. Circumcised men had a slightly lower risk of prostate cancer, but this was not statistically significant. The difference was significant in men circumcised before age 36 (OR 0.55).Context: Circumcision has been associated with a lowered risk of sexually transmitted infections, and some sexually transmitted infections have been linked to an increased risk of prostate cancer.Relevance: The study was carried out among the French speaking population of Montreal, in men aged mainly 50 to 79.Corroboration: No independent expert is quoted.Sceptic scale: High. No source is cited and the headline wrongly attributes causality (and gets the age wrong).Original article: "Circumcision before age of 35 'cuts risk of prostate cancer by 45%"
Source: Survey from British Chiropractic Association news release.Methods and results: The press release from the British Chiropractic Association does not say how many people were surveyed, by whom, how the survey was carried out, or what criteria were used (although the news story says 2000 people were questioned). The press release says 77% of people surveyed said they have back or neck pain now, or have had in the past, and many said it was worse after sitting using a laptop computer.Context: Back pain is a very common musculoskeletal condition. Poor posture while using a computer or laptop is a plausible cause of back pain.Relevance: Impossible to tell as we are given no information about the population surveyed.Corroboration: No independent expert is quoted.Sceptic scale: Off the scale. This is an unexamined survey from an association that has a strong interest in making people think back pain is common and linked to modern lifestyles. The Daily Mail says three quarters of us have back pain now, but the figure in the press release included people who had ever had back or neck painOriginal article: "Three in four of us have a bad back"
The researchers investigated what they said was a consecutive series of 12 children referred to their unit with a history of normal development followed by loss of language and other acquired skills, together with diarrhoea and abdominal pain. They underwent investigations including colonoscopy and lumbar puncture. The onset of symptoms was linked by the parents to measles, mumps, and rubella (MMR) vaccine in eight children. Investigations showed varied intestinal pathologies. The study concluded: “We have identified a chronic enterocolitis in children that may be related to neuropsychiatric dysfunction. In most cases, onset of symptoms was after MMR immunisation. Further investigations are needed to examine its possible relation to this vaccine.”
The researchers held a press conference, during which Andrew Wakefield claimed to have identified “a genuinely new syndrome.” He speculated that giving the three immunisations in MMR together could “overload” the body’s immune system and cause the bowel disorder. He suggested individual vaccines might be safer. http://news.bbc.co.uk/1/hi/health/1808956.stm
The media latched on to Wakefield’s suggestion that MMR was the cause of a joint syndrome of autism and bowel disease. Headlines in the Daily Mail, as it followed the story, included, “Scientists fear MMR link to autism,” “New evidence ‘shows MMR link to autism,’” and “MMR fears gain support.”
Widespread distrust of the vaccine led to a collapse in immunisation rates. The proportion of two year olds who received the vaccine fell from 92% in 1995 to 79% in 2003 (http://www.bmj.com/content/336/7647/754). Steadily accumulating studies showed that the link was false. The rate slowly recovered to its previous level, but several serious outbreaks of measles occurred in the United Kingdom in 2013.The 1998 paper was retracted by the Lancet in 2004, when the journal’s editors said they had been misled about the study. Andrew Wakefield was struck off the medical register in 2010 for “multiple separate instances of serious professional misconduct.”
This article was originally published in Student BMJ in November 2014 (Student BMJ 2014;22:g6725). Read the original article.
With thanks to Student BMJ for kind permission to republish this article.
Follow Student BMJ on Twitter: @StudentBMJ
By submitting your details you agree to receive email updates about the campaign. We will always keep your data safe and you may unsubscribe at any time.
.png)