‘Doctoring Data': How to sort out medical advice from medical nonsense
Jed Stuber · Nov 01, 2015
“It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of the New England Journal of Medicine.” Dr. Marcia Angell, author of The Truth About Drug Companies: How They Deceive Us and What to Do About It
Dr. Marcia Angell, former editor of the New England Journal of Medicine, reluctantly concludes much medical advice is unreliable, and British cardiologist Malcolm Kendrick, author of Doctoring Data, goes even further: “The sad truth is that most of the advice we are now bombarded with varies from neutral to damaging.”
Dr. Kendrick wrote Doctoring Data to help his patients be able to see through common fallacies and deceptive tactics in medical advice. He lays out “10 Tools for Establishing the Truth” (see below), devoting a chapter to each one.
In his 30-year career, Dr. Kendrick has seen the medical community come to various points of what he sees as absurdity. Medical advice has become ever more simplistic and strident, as if patients couldn’t handle any kind of qualified statement or counter evidence. “If you sunbathe you’ll die of skin cancer.” “If you eat fat you’ll have a heart attack.” Guidelines developed by the European Society of Cardiology put 95 percent of Norwegians in categories requiring cholesterol or blood pressure medicines, yet it is well-established that Norwegians are among the healthiest and longest-lived people on earth. The elderly patients he works with often come to him taking 10 or more drugs daily. There is constant pressure to have endless routine testing “as if good health is only really possible through constant monitoring by the medical profession.”
Dr. Kendrick offers this explanation of his contrarian stance against conventional wisdom: “The main reason I wrote this book is most definitely not to tell people what to do. In my view there are more than enough people doing that nowadays. My hope is that once you have read this book, you will have far greater insight into the daily bombardment of medical scare stories and misinformation. Then, having gained this insight, you will be able to decide for yourself what to do. I suspect that you can then happily ignore 95 percent of the health care advice that rains down.”
Chapter 1: Association does not mean causation
Most of us would quickly affirm the statement “association does not mean causation,” having been taught this bit of basic logic somewhere along the way in our schooling. But the writers of medical literature and news reports about the latest findings are masters of subtly blurring the distinction between the two. Most of the time science can only justify tentative speculation about correlation, but that just isn’t as interesting as claiming “X causes Y!”
Another problem is that many claims of causation are based on observational studies, not actual experiments. The problem with observational studies is that there is no control over the composition of the … uh, well, “control” groups, so there is no attempt to rule out other causal factors.
Dr. Kendrick says a notorious example is the notion that hormone replacement therapy protected women from heart disease. It all started with the fact that young women don’t get heart disease, but young men do. Then a single observational study of older women using hormone replacement who had less heart disease was cited to confirm the hypothesis. But years later it was realized that the women in the study were very health conscious, had very high incomes, were highly educated, smoked much less than average, and exercised much more. They weren’t anywhere close to a selection of average women, and those other variables were very important. Finally a controlled study was actually set up and funded. It was intended to confirm that hormone replacement prevented heart disease in older women, but the results showed just the opposite—a huge increase in heart attacks and strokes.
Another problem Dr. Kendrick explains is that our minds easily fill in gaps in logical chains of reasoning based on beliefs we already hold. A newspaper headline reads “Eating red meat dramatically increases the risk of death from heart disease.” Without realizing what they are doing, people quickly reason: “Meat contains a lot of fat. Fat contains a lot of cholesterol. Fat raises blood cholesterol levels. Cholesterol in the blood causes heart disease.” The problem is the study showed those who ate the most meat had the lowest cholesterol levels, a fact that is never mentioned in the news reports. Furthermore, every step in that chain of reasoning is disputed by some researchers based on strong evidence from controlled, randomized, interventional studies.
It gets worse. The group in this study that ate meat also happened to smoke most, exercise least, and have higher incidence of diabetes. No mention is made of those factors correlating with heart disease. Again, this study was merely observational, and while the study itself carefully only claimed correlation, the language about “increased risk” in the news story is read by most people as establishing causality.
According to Dr. Kendrick, observational studies and the weak correlations found by them almost always mean nothing at all. If you understand that and don’t even read the rest of the book, he says, you will have gotten his most important message.
Chapter 2: Lives cannot be saved; we’re all going to die
Consider these passages from a real press release: “Heart attacks can be avoided in people at risk of vascular disease by using statin drugs to lower blood cholesterol levels. … This is a stunning result, with massive public health implications. We’ve found that cholesterol-lowering treatment can prevent strokes as well as heart attacks. … In this trial ten thousand people were on a statin. If now, an extra ten million high-risk people worldwide go onto statin treatment, this would save about fifty thousand lives each year—that’s a thousand a week.”
The notions of lives saved and problems prevented are very misleading. After all, we can be sure we will all eventually die, and if something were actually prevented, that would mean it never occurs again. Also, multiplying the numbers actually studied by a ridiculous factor of one thousand also adds to the deception. In this study it was technically true that there were more people alive at the end who had taken the statin than the placebo. But the real questions scientists—and anybody reading medical advice—should be asking are “How many more? And how much longer did they live?”
Here are the results of this study stated more truthfully: For every two hundred people treated with statins for a full year, one extra person was alive in the statin group. All of them were elderly to begin with and those on the statin survived an average of three months longer than those on the placebo.
If your family doctor is pressuring you to take statin drugs, Dr. Kendrick encourages you to ask these questions: “How much longer am I likely to live if I take it? What is the average increased survival time?” Dr. Kendrick writes, “I can guarantee here and now that they will not be able to answer this question. They will either say they do not know, or just guess, and they will guess something like five to ten years. I know this because I have asked many doctors and nurses this same question. No one gets anywhere near the vanishingly small figure.”
Dr. Kendrick also recommends you read his first book, The Cholesterol Con, which counters the theory that cholesterol causes heart disease. (You can find a review of it and other related articles)
In this chapter of Doctoring Data, Dr. Kendrick goes on to explain that while average increased survival time is a reasonable measurement for people to consider, it is increasingly being used in haphazard ways by governments to determine how research is funded and which treatments are covered by welfare programs.
Chapter 3: Relative mountains are made out of absolute molehills
People are not rational about risk. There’s a minuscule chance of getting meningitis, but we’re afraid of it. The chance of dying in a car crash is significant, yet we drive around every day without a thought. Hippos kill more people every year than sharks, alligators, crocodiles, lions, and tigers combined. But mosquito bites kill more people than any other animal by a factor of several thousand.
The medical industry knows we’re just as irrational when it comes to medical advice, and exploits us, often with a very deceptive tactic called “relative risk.”
Here’s an example. One hundred people start taking blood pressure medicine and one hundred do not. At the end of a year, one person in the group taking the medication has died, and two in the group not taking medication have died.
The absolute difference in death is 1 person per 100 vs. 2 people per 100=1 in 100, or 1 percent. The relative difference in deaths is 1 vs. 2, or 50 percent. So the claim is made that if you take the medication, your risk of dying has been reduced by 50 percent! This tactic can be used no matter how large or small the group being studied.
Relative risk can also be used to scare people away from something the medical profession doesn’t approve of. For example, to claim moderate alcohol consumption causes heart disease or cancer. Based on one study, wildly distorted with tricky math. Dr. Kendrick argues that when you consider the preponderance of evidence, moderate alcohol consumption is actually good for your heart, and nothing at all can be proved about its relationship to cancer.
Dr. Kendrick says relative risk just isn’t sound science, and whenever you detect it, you’re better off to completely ignore what you are reading.
The relative risk tactic has come in for criticism in recent years, but unfortunately it’s still used often or is replaced with other deceptive tactics, such as Number Needed to Treat, another statistic governments have become enamored with.
Rest of the book
Believing it necessary, Dr. Kendrick devotes several more chapters to the dismal topic of statistics and how they are misused. Thankfully, he manages to lighten the mood with his British sense of irony and sarcasm. Here’s a sampling:
“Who shall guard the guardians?”
“Things that are high should be lowered. Things that are low should be raised. Yes, we have a drug for that … Kerching.”
“This is not science. This is the world of faith and belief, or perhaps the Spanish Inquisition.”
“This is eminence based medicine. Or, ‘Do you know who I am?’ medicine.”
“You can only die once of one thing.”
“Rule 1: The experts are, frankly, no more likely to be right about any given hypothesis than you. Rule 2: The angrier experts become the more likely they are to be wrong. Rule 3: When an expert is wrong, he, or she, is far less able to change their mind than you.”
The concluding chapters give Dr. Kendrick’s take on the politics, manipulation, and outright corruption he has seen in his experience working in the British medical system.
The book’s focus is on using basic logic to see through fallacies, and it ends on a positive note. Dr. Kendrick believes the medical system is desperately in need of change, and the way change can be accomplished is by informed patients demanding better answers.
One word of caution about the book. Dr. Kendrick uses a quote in the introduction that has foul language, and occasionally uses some colorful language himself.
10 tools for establishing the truth
- Association does not mean causation.
- Lives cannot be saved; we’re all going to die.
- Relative mountains are made out of absolute molehills.
- Things that are not true are often held to be true.
- Reducing numbers does not equal reducing risk.
- Challenges to the status quo are crushed—and how!
- Games are played and the players are…
- Doctors can seriously damage your health.
- Never believe that something is impossible.
- “Facts” can be, and often are, plucked from thin air.